Previously published by Oak Ridge National Laboratory.

The Earth System Grid Federation, an international multi-institutional initiative that gathers and distributes data for top-tier simulations of the Earth’s climate, is preparing a series of upgrades that will make using the data easier and faster while improving how the information is curated. 

A new project called ESGF2, led by the U.S. Department of Energy’s Oak Ridge National Laboratory in collaboration with Argonne and Lawrence Livermore national laboratories, is part of international efforts to upgrade the global data system that is integral to simulations of future climate. These widely respected simulations are made by scientists working with the Coupled Model Intercomparison Projects for the World Climate Research Programme. 

“ESGF data are about the future of life on Earth,” said Forrest Hoffman, lead for the DOE-funded ESGF2 project and the Computational Earth Sciences group at ORNL. ​“By providing scientists easy access to the full collection of international models, ESGF enables them to construct the very best understanding about the potential future trajectories of our climate.”

A key ESGF mission is to support the data needs of scientists conducting global climate change research, and notably those who prepare the United Nations Intergovernmental Panel on Climate Change’s comprehensive climate assessments released every six to seven years. ESGF Earth system data underpin IPCC landmark reports such as the recent Sixth Assessment Report, AR6, and its working group findings. The data also inform IPCC special reports focused on climate vulnerabilities, adaptation scenarios and mitigation strategies.

Another important aspect of the ESGF’s mission is to ensure that scientific investigation is transparent, collaborative and reproducible, given its direct impact on worldwide climate research and potential use in government and commercial decision-making.

“Almost all of the Earth system model data that go into the IPCC reports are stored in the ESGF,” said Hoffman. These data are provided by more than 50 climate research institutions around the world. ​“The federation is a unique consortium of community-minded institutions that aims to get data into the hands of the tens of thousands of researchers and stakeholders who analyze it and compare it with observational data to constantly update our best projections of the future.”

In the new DOE ESGF2 project, computational scientists are working to improve data discovery, access and storage in direct collaboration with international federation partners. Their work will leverage the latest software tools, cloud computing resources, the world’s most powerful supercomputers and DOE’s Energy Sciences Network, or ESnet. ESnet currently enables 100 gigabit-per-second transfer rates among national laboratories and connections to other national and international networks, universities and research centers. An upgrade launched this fall will boost ESnet transfer rates to as much as 400 Gbps.

“Working with federation partners in the United Kingdom, Germany, France, other European Union nations, Australia and other countries, we will develop and deploy a modernized and cyber-secure system for providing access to model output data to the global scientific community,” Hoffman said. 

The federation operates a network of large computer nodes hosted in the United States and 17 other countries, functioning as one global data archive. ORNL, Argonne and LLNL are collaborating with these partners to improve the reliability and scalability of the system, providing a smooth data replication process that ensures the broader scientific community has access to data from all ESGF sites. ORNL and Argonne are hosting a dual backup of the more than 8 petabytes (and counting) of the most popular subset of ESGF data, taking advantage of the world-class computing systems operated at the labs.