Currently, weather and climate forecasts are mainly made by public meteorological services at a national level, which in turn are coordinated at a global level and also by public institutions. These predictions are based on primordial equations that encode physical laws and include hundreds of millions of daily observations. However, the need for weather forecasts and climate projections with increasingly better spatial resolution and uncertainty quantification leads to considerable computational and energy overheads. The costs of such efforts are inevitably passed on to the public purse.

The rapid rise of deep learning methods and foundation models, driven mainly by private technology companies, is creating unprecedented momentum to replace traditional physics-based weather and climate forecasting methods and the entire utility infrastructure with a new type of weather and climate enterprise.

In a paper recently published in the Nature Reviews - Earth & Environment journal, a set of experts explore how this shift in momentum requires a symbiosis between public and private efforts so that the benefits of deep learning and physics-based models can be exploited to their full potential while preserving the credibility of the products by adhering to public research expertise in meteorology and climatology, as well as to protocols and quality standards agreed by the scientific community.

Among the experts signing the analysis is Francisco Doblas-Reyes, director of the Department of Earth Sciences at the Barcelona Supercomputing Center-Centro Nacional de Supercomputación (BSC-CNS) and ICREA professor. 

"There is an urgent need to rethink the economics of weather and climate forecasting, given the huge investments behind the effort to use mostly public data. It is also being driven by a strong push towards deep learning methodologies and boosted by effectively limitless computational and data analysis capabilities in the private sector," says Professor Doblas-Reyes. He adds: "Foundation models add a new dimension here, offering users unprecedented interpretation and communication capabilities.

Experts argue that private entities should invest primarily in applying deep learning methods and foundation models to reference datasets generated through transparent processes following agreed standards. Companies would benefit from developing deep learning datasets using reference physics, validation of results and quantification of uncertainty provided by public actors. The public and private sectors should investigate the most innovative solutions.

The analysis concludes that this symbiosis would help harness the full potential of the private sector's innovative spirit and economic power while ensuring that sufficient investments and inputs continue to be made in the underlying public resources. An essential ingredient of this symbiosis is democratising software and data to improve forecasting capabilities for all, especially those most vulnerable to climate change and extreme weather events.

Image: Decadal prediction of the surface air temperature anomaly for the period 2028-2032. Credit: BSC decadal prediction system (https://decadal.bsc.es/).

Reference:  Bauer, P., Dueben, P., Chantry, M. et al. Deep learning and a changing economy in weather and climate prediction. Nat Rev Earth Environ 4, 507–509 (2023). https://doi.org/10.1038/s43017-023-00468-z.