30.10.2001
Stefan Greilberger (Stefan.Greilberger++at++zamg.ac.at)
Department of Synoptic Meteorology - ZAMG Vienna/AUSTRIA
ALADIN-VIENNA temperature and precipitation forecasts (for point Vienna) are verified for the synoptic station Vienna/Hohe Warte (WMO Station nr. 11035). Comparisons of forecasts for day 1 and 2 of ECMWF (ECM), ALADIN VIENNA (AVI) and MOS (Model Output Statistics, based on ECM data) have been made for the period 01/2001 - 09/2001. ECM minimum and maximum temperature is calculated out of the model values from 00, 06, 12 and 18 UTC each day, so it is no real model minimum/maximum.
To make precipitation statistics comparable to the meteorologists forecasts, the daily period of detection was chosen between 06 and 18 UTC. In this case the period for the model forecasts is from 06 - 18 UTC, too.
1. Temperature
In Table 1 the forecasts are shown in chronological order (Tmin and Tmax for 2 days). Generally, AVI forecasts for minimum/maximum temperature are better than ECM forecasts and worse than MOS forecasts. Except for maximum day 2, AVI forecast has the lowest RMSE.
Comparing models (AVI and ECM) only, AVI forecasts for minimum temperature are only worse than ECM for day 2 by 2.0 K RMSE. Maximum forecasts in AVI are always better with RMSE of 2.0 K on day 1 and 2.1 K on day 2.
So it is obvious that BIAS (systematic error) also generates some RMSE. For ECM forecasts the minimum temperature BIAS is about +0.5K, for maximum temperature -0.5K. By eliminating BIAS in AVI RMSE will also be reduced.
Table 1: RMSE values (in K) for minimum/maximum temperature forecasts for day 1 and day 2.
Table 2: Same as Table 1, but for BIAS
Table 3: Statistics for detection of precipitation (counting "yes"/"no" predicted / observed). For predicted / observed precipitation more than 0.5 mm (between 06 and 18 UTC) is detected with precipitation "yes", otherwise with "no". The table shows POD (Probability Of Detection), FAR (False Alarm Rate) and TSS (True Skill Score) for ECM, MOS and AVI.
2. Precipitation
Table 3 compares the precipitation forecast of ECM, MOS and AVI in terms of POD (Probability Of Detection), FAR (False Alarm Rate) and TSS (True Skill Score) also for the period 2001/01 - 2001/09. Detection "yes" for both observation and forecast mean more than 0.5 mm cumulated precipitation in the period of 06 to 18 UTC.
POD for AVI is with about 79 % a little bit better than ECM (~72 %), but worse than MOS. MOS has the best POD with 96,4 % to 100 % (=optimum), but on the other hand the highest FAR with about 65 % (optimum = 0 %). AVI has far away the best FAR with only 30 % for day 1 and 37 % for day 2.
In TSS, ECM is best with 52 % and AVI worst with 43 % on day 1. On day 2 no significant differences can be found, only MOS is with 46% a little bit better than ECM and AVI (both 45 %).
3. Summary
Comparing model forecasts like AVI and ECM (MOS is not a model ), minimum and maximum temperature forecasts of AVI have better RMSE values than ECM forecasts. Precipitation detection ("yes"/"no") of AVI is also better than ECM, if looking at POD and FAR at the same time.
MOS based on ECM has the best forecast generally, but at the same time it reduces FAR quality. Precipitation - in other words - is predicted too often, when it is not observed.
On the other hand TSS picks ECM as best model for day 1. MOS based on ECM has a lower TTS than ECM. On day 2 there are no significant differences.
So far only temperature minimum/maximum and precipitation detection have been verified. In the future, verification of ECM, AVI and MOS will be extended to additional parameters (precipitation amount, humidity, wind, cloud cover, etc...).
 |
Home |