Observation monitoring provides detailed statistical information on the quality and availability of the different components of the observing system monitored and used by ECMWF. The monitoring results are primarily produced to help improve the usage of observations within ECMWF’s data assimilation system, which provides the initial conditions for forecasts. They are also used to provide feedback to observation data providers. Additionally, published observation monitoring statistics enable comparisons with results from other numerical weather prediction (NWP) centres and help identify the sources of differences between observations and the reference weather forecasting model. ECMWF makes a comprehensive set of observation monitoring statistics publicly available, covering almost all observations processed by ECMWF’s data assimilation system (https://www.ecmwf.int/en/forecasts/quality-our-forecasts/monitoring-observing-system). This is also part of ECMWF's support for observation-related activities of the World Meteorological Organization (WMO). The monitoring website includes thousands of up-to-date products (time series, geographical maps, etc.), which are pre-plotted routinely (daily/weekly) and uploaded to the web. Unlike the regular access to forecast products, the access pattern to monitoring statistics is not regular and depends on each dataset and the interest of the user community. The objective of the work presented here is to replace the current framework with an on-demand system, enabling users to visualise statistics on the fly without the need to systematically pre-generate thousands of plots. Combined with an efficient caching system, the new framework will help optimise resources and will enable products to be refreshed more frequently. It will also offer more possibilities to access and inter-compare monitoring statistics.
Purpose of observation monitoring
Under the previous as well as the new arrangements, ECMWF makes available a wide range of statistical products on the quality, availability, usage status and estimated impact of satellite, in‑situ and oceanic observations assimilated or monitored by ECMWF (see Table 1 at the end of this article). The statistical products are presented in various forms to highlight temporal evolution, vertical distribution, and geographical dependency. Although the monitoring products reflect ECMWF’s usage of the data, the statistics are of benefit to the wider NWP community and data providers. It is, however, important to note that quantities based on departures from the background (a short-range forecast) and from the analysis (the best estimate of the current state of the Earth system) are affected by changes that are not necessarily related to the observations themselves. Such changes might be associated with variations in the accuracy of the reference forecasting model driven by atmospheric variability (short-lived signal), or with model or data assimilation upgrades (https://www.ecmwf.int/en/forecasts/documentation-and-support/changes-ecmwf-model). Therefore, a great deal of care is needed to interpret changes affecting long time series of departure statistics. Observation monitoring statistics are presented for selected data selection criteria to reflect the data characteristics and counts for different data sampling scenarios (All data, Used data, etc.). For a subset of observation types, statistics are computed and presented for different land–sea masks (sea, land, etc.) to highlight the sensitivity to the underlying surface.
Observation monitoring website
The observation monitoring website is organised by reference model (operational, the ERA5 reanalysis), data categories (satellite, conventional and oceanic), observation families (infrared radiances, all-sky radiances, atmospheric motion vectors, etc.), geophysical parameters (radiances, wind, temperature, bending angles, etc.), satellite sensors (Advanced Microwave Sounding Unit‑A – AMSU‑A; Microwave Humidity Sounder – MHS; Global Precipitation Measurement Microwave Imager (GMI), etc.); and data delivery streams (normal delivery; EUMETSAT Advanced Retransmission Service – EARS; etc.). Observation monitoring products were updated daily (time series, Hovmöller, scatter plots) and weekly (geographical maps of statistics). Most products covered the last two months.
Thousands of observation monitoring plots were routinely pre‑generated and uploaded to the monitoring website regardless of how frequently these products were accessed. The procedure involved the processing of pre-computed gridded statistics, the generation of plots, and the transfer of products to the web infrastructure. The content of plots (layout, titles, colours, scales, etc.) was prescribed via plotting configuration files. This procedure was expensive and lacked flexibility.
New on-demand web plotting
The new on-demand framework (Figure 1) aims to make the access to monitoring statistics more efficient and flexible. It also provides more possibilities to explore the statistics.
The underlying data are pre-processed every day using a set of gridded statistics computed by observation monitoring suites, which serve a range of operational and research requirements. The gridded statistics are converted to NetCDF format using a template compatible with the ECMWF web infrastructure. The conversion is controlled by a single configuration file per data type. This file defines the temporal resolution of statistics, the geographical areas of interest, the length of time series, the data selection to be considered, land–sea masks to be included, observation quantities to be added, and the overlay of statistics from different satellites. The conversion configuration files are, however, bound by the content of pre-computed gridded statistics. Converted NetCDF files are then pushed to web machines, where the data are indexed and integrated in a data store ready to be used by the on-demand plotting system. Housekeeping tools were integrated to control the retention period of statistics. The web plotting engine has been developed and integrated in the web front end. It is mainly based on the JSON user interface enabling users to make a plotting selection (data type, area, data selection, period, etc). The selection is then used to interrogate the NetCDF files and return the data to be plotted by executables based on ECMWF’s meteorological plotting software Magics (https://confluence.ecmwf.int/display/MAGP/Magics). The JSON user interface controls all aspects of the on-demand plotting. This includes:
- The structure and hierarchy of observation monitoring statistics
- The name of underlying NetCDF files to be used for each dataset
- The content of the plots: number and content of panels for time series; observation quantities to be included for geographical maps and Hovmöller diagrams; areas, data streams, flags, etc.
- Titles, scales, colours, fonts – default settings are used if nothing is specified
- Settings to overlay statistics from different sources
- Available lengths of time series
- Retention period for daily updates.
Figures 2 and 3 show various time series obtained using the new system.
Being integrated in ECMWF’s OpenCharts web framework, the user interface enables users to compare statistics for each data selection axis (available satellites, areas, channels, data selection, etc.). This facility can be of great interest. The Json files can be easily updated and pushed to the web interface, enabling immediate update of the layout without any need to touch the underlying data. The new framework benefits from a powerful cache system that enables fast access to plots frequently used. The performance remains good to access all products.
The on‑demand plotting of observation monitoring makes it possible to perform daily updates of all products (instead of weekly), enables plotting of time series at various temporal lengths (two months, 12 months, and the whole lifetime of the data), and makes it possible to overlay statistics from available satellites in one and the same plot (Figure 4).
Migration timeline
With thousands of products to migrate, it was decided to follow a stepwise approach to migrate individual data types to the new system. This fits well with the OpenCharts framework, in which the user interface is fed from static plots or on‑demand plotting. Infrared radiances, microwave radiances and geostationary radiances were the first to be migrated to the new framework. The process was complete by the end of 2023.
Conclusion and perspectives
The migration to on-demand web plotting of observation monitoring is now complete. The new framework leads to efficient access to up‑to‑date observation statistics and offers additional possibilities to compare statistics and access long-term time series. The direct interaction between web client and NetCDF files will enable more capabilities in the future, such as zooming, overlay with external fields, access to grid point values, and access to time series over custom areas defined by users.
