Test how observations improve a forecast

A ground system receives instrument data, processes it into usable measurements, applies quality checks and archives the results. An observed signal, a retrieved physical product and a forecast are different stages. For example, a channel’s radiation measurement and a future rainfall forecast are not interchangeable descriptions of the same number.

Data assimilation combines observations with a model-based estimate of the atmosphere, accounting for uncertainty and the relationship between the model and the measurement. A suspect channel or biased scan position may need correction or exclusion. More measurements help only when the system can use their information appropriately; nearby observations can share errors.

A December 2023 ECMWF report compared forecasts with and without FY-3E MWHS-2 observations. Its principal comparison ran from October 2022 through February 2023, with a startup period excluded from assessment. The report found benefits that varied with region and forecast length, and describes the instrument’s operational inclusion on February 22, 2023.

This is evidence about a particular instrument, model and experiment, not a guarantee for every Fengyun product or forecast. The report also studied observation spacing: using more closely packed observations eventually degraded some forecasts when shared errors were neglected. Treat data selection and error modeling as part of the experiment rather than assuming that the largest observation count must win.

In an original exercise, two systems forecast the same cases at the same lead time. Their mean absolute temperature errors are 2.0 K without a new observation type and 1.8 K with it. The reduction is 0.2 K, or ten percent of 2.0 K. These invented numbers illustrate the calculation; they are not the ECMWF result.

Before declaring a benefit, ask whether other model settings were held constant, whether the cases represent the intended use and whether the difference is robust to sampling uncertainty. Check regional results as well as the overall average. A dependable observing program connects instrument performance to tested usefulness while preserving the limits of that evidence.

Test a forecast benefit fairly