Jozef Vivoda, SHMI
Juraj Bartok, Comenius University Bratislava
presented by Oldrich Španiel
1. Introduction
Aim of the study was to verify high resolution precipitation forecast provided by various implementations of the same LAM ALADIN over Slovakia (unmasked area on Figure 1). There was only one main scientific difference between those studied implementations - different horizontal resolution. ALADIN/LACE 12km (referred as ALA 12km in following text) and ALADIN/SLOVAKIA 7km (ALA 7km).
2. Used observations and predictions
We used raingauge network observations of SHMI. This network consists of nearly 700 stations (Figure 1.). Observations are provided daily at 7 hour local mean time, that is approx. 5:52 UTC in western part of the country and approx. 5:30 UTC in eastern part. If we neglect those time differences, we can compare model forecast valid from 6 UTC to 6 UTC against those raingauge observations. Average horizontal distance of stations is 8 km.
We verified prediction monthly from January 1999 to December 2000. 24 hour accumulated precipitation predicted by 00 UTC runs were taken, valid at 06 UTC following day.
3. Method of verification
Due to the fact that observation network is very dense, we did not verify prediction in "classical" way, the station against the closest model gridpoint. We rather decided to reanalyze observations and predictions to independent grid and to perform comparison of those reanalyzed fields.
Resolution of independent grid was studied (from 5km to 15km with one-km step) and almost no sensitivity of results to mesh size of independent grid was found. Therefore mesh size equals to 7 km was chosen.
We calculated contingency table for given threshold R (Tab. 1).
Predicted | |||
precip < R |
precip >= R | ||
Observed |
precip < R |
Z |
F |
precip >= R |
M |
H |
Tab.1 Contingency table
We consider that reanalized data represents state in whole grid box (as it is valid for model prediction) rather than in single point (as is considered usually for observation).
To evaluate prediction we used score BIAS, critical success index (CSI) and Heidke skill score (HSS) calculated using values from previous contingency table. Defined as:
1. BIAS:
,
BIAS=1 means that model predicted precipitation for the same area as was observed.
2. CSI (Critical success index):
3. HSS (Heidke skill score):
Contingency table was redefined to calculate relative operational characteristics (ROC). Only difference was that while changing threshold R for prediction we kept R=0.1mm (minimum observable amount) for observations. ROC is curve plotted in hit rate (HR) - false alarm rate (FR) plane (likelihood-base rate factorization (Stephenson, 2000)). Here are FR and HR definition:
1. HR (Hit rate):
,
HR represents probability that observed precipitation was also predicted.
2. FR (False alarm rate):
FR is probability that if there is no precipitation observed, it was predicted.
We used simple interpolation method to get reanalyzed fields.
,
Here 
is reanalysed
value in grid point of new independent grid,
is predicted or observed value, 
is
horizontal distance between grid point 
of
new grid and grid point 
(or station
position) of original model grid. There are taken into account only grid
points (or stations) with distance smaller than chosen critical distance 
.
( in our case 
).
Independent grid was defined as regular isotropic grid with mesh size 7km on tangent Lambert plane.
4. Results
Score BIAS, CSI and HSS
We studied first precipitation from point of view when model predicted for grid box more precipitation that given threshold and also more precipitation than given threshold was observed in grid box. We compared reanalyzed fields as it was already mentioned.
We defined threshold equals to 0.1mm to study only occurrence of precipitation without regard to precipitation amount since it is minimal measurable amount.
The results are presented on figure 2. We see from BIAS curves that both models precipitate on bigger area that is observed except during winter where it is contrary (curves for precipitation occurrence ALA 12km R>0.1 and ALA 7km R>0.1). This is not valid for threshold 5mm and 10mm where models underestimated area with precipitation over these thresholds. Models overpredicted stronger precipitation occurrence than general precipitation occurrence during January, February, September and December 1999. We can say that ALA 7km gives less precipitation on smaller area than ALA 12km, but still gives more than it is in reality.
HSS is better measure of overall precipitation prediction quality. It takes into account also events that could be predicted by chance keeping the same marginal distribution of observed precipitation. We can say from HSS curves than precipitation occurrence is predicted same in both model implementations. It is not true for threshold 5 mm and 10 mm, since ALA 12km was better from March to September 1999.
ROC
There is ROC curves on figure 3. Curves were plotted from contingency table calculated for thresholds 0.1 mm, 0.5 mm, 1 mm-20mm (with 1 mm step), 25 mm. and 50 mm. The aim of ROC is to evaluate prediction of precipitation occurrence when occurrence is considered only if more than given threshold is predicted. ROC is more users oriented and it gives better overview about way of model precipitation prediction interpretation. It is possible to choose threshold to predict precipitation with desired FR and HR regarding to user needs. It is difficult to say from curves which model is better since for example in July ALA 12 km has HR around 93% (93% of observed precipitation was predicted) having FR around 40% (40% of area, where no rain was observed, precipitated according to model prediction). So if one wished to have available prediction of rain occurrence with HR around 50 % but with smallest possible FR during august 1999, the best prediction would be ALA 7km with threshold 1 mm.
Maps of monthly accumulated precipitation
Maps of monthly-accumulated precipitation over verified area help to recognize and compare patterns of predicted and observed precipitation. We can see on figure 4, that observed precipitation is less forced by orography that it is in ALADIN model (here we do not deal with interpretation and the way how and where precipitation observations are performed since we believe that our raingauge network is sufficiently dense and representative for our purpose).
It is interesting that on maps we can recognize even difference between model orography in ALA 7km and ALA 12km. There is underestimation of precipitation in lowland in southwest part of Slovakia while overestimation in mountain areas. Model precipitates over its peaks, if precipitation has stratiform character (see figure 4 for December 1999).
5. Summary
We have investigated model ALADIN prediction. We aimed to compare two model implementation with different horizontal resolution. We found out that better resolution doesn't mean also better precipitation prediction.
Model gives more precipitation over mountains and less in lowland as it is in reality.
 |
Home |