Working group 4 : Verification

Participants :

Radmila Brozkova, Dijana Klaric, Neva Pristov, Samuel Westrelin

Report :

1 Need of a "mesoscale" verification versus the current status of the verification work

It is well recognized that the classical verification scores computed against the networks of SYNOP and TEMP stations do not provide a satisfactory response to validate the mesoscale forecasts. The problem is the insufficient density of these conventional observations. Therefore the imagery data (satellite and radar) should be used to complete the traditional scores.

However, the situation in the countries running ALADIN in the operational exploitation is far from being correct even for the case of the conventional scores. At many places the traditional verification is done only partially, for example only against the stations of the country, or not continuously in time, or the scores are simply computed and not really evaluated. The tools employed to compute the departure between the model forecasts and observations are also different from one country to another. This makes impossible to compare the scores between the countries since the differences in these tools might create fictitious errors.

It is clear that the conventional scores situation must become clean first, before going to the more sophisticated verification using the imagery data, for example. To achieve this is not easy, since the previous verification projects launched in the past always failed. First, it is necessary to analyse why the verification projects are unsuccessful. One of the obvious reason is a lack of manpower devoted to the verification. This is even worsened by the fact, that no control unit usually exists in the organization structure of the small met-services. The verification is then done either by the modellers themselves (which is mostly unpopular job) or by the forecasters (mostly subjectively) when they have some time left. On the other hand the model verification is a very important thing and must not be further neglected.

2. Exchange of data between the countries

The group agreed that the exchange of all the available local data (additional SYNOP-type of stations, rain-gauge hydrological stations, etc.) between the ALADIN countries would be of a great help for the verification (and also data assimilation) purposes in order to get more dense mesoscale networks over larger territories. However it is not always easy to achieve such agreement due to a lack of confidence in the front of a possible commercial misuse of such fine-scale data. For the needs of mesoscale modelling, the continuous political and technical efforts are necessary to get around this obstacle. One idea would be to exchange data in a special format which would considerably lower the risk of a possible pirating of data. Such idea was promoted within RC LACE group some time ago but it failed (mainly due to the lack of the technical support and that nobody became really responsible to push this action ahead).

3. Common points with data assimilation

It was stressed out that the effort on verification has a lot in common with the one on data assimilation. Therefore it would be very useful to achieve a synergy of these two goals. There are a few examples of the common points: computation of the forecast departure from the observation, quality control of data, removal of the systematic errors from the observed data, density of data networks and so on.

4. Conclusions & Recommendation

The situation regarding the verification of model forecasts even by the means of the conventional scores is far from being satisfactory within the ALADIN countries, which exploit the ALADIN model operationally. A possible first step to improve this would be to create the "verification" or "control" units at the services and within these units to adopt a reasonable verification strategy (agreement on the methods, tools, launching verification projects and so on). This step seems compulsory before starting to work seriously on the mesoscale verification using non-conventional (imagery) data. The effort to exchange mutually the local data between the countries should continue despite the failures in the past. The verification and data assimilation community should closely collaborate in order to solve together the common problems relevant to their activities.