Official websites use .gov
A
.gov website belongs to an official government
organization in the United States.
Secure .gov websites use HTTPS
A
lock (
) or https:// means you’ve safely connected to
the .gov website. Share sensitive information only on official,
secure websites.
The Earth Prediction Innovation Center (EPIC) unites those in government, industry, and academia, which opens simultaneous opportunities to improve NOAA’s operational modeling systems; serve others in the community; fund research, modeling, and compute initiatives; and develop innovative tools and applications. NOAA embraced these opportunities by advancing the Unified Forecast System (UFS), the source system for NOAA‘s operational numerical weather prediction applications, as a community model.
The devastating impacts of hazardous weather and other environmental events on life, property, and the national economy can be mitigated with accurate and reliable forecasts that are clearly communicated to decision-makers. However, the ongoing success of the nation’s forecasting tools depends on continuous and sustained contributions to scientific research and modeling from across the weather enterprise.
With our community partners, EPIC is joining forces with the community for the benefit of the nation. The new model development community has transitioned the UFS code to GitHub and is supporting code managers for the UFS applications and component models. EPIC will continue to build upon these community partnerships to foster innovation, advancing research innovations into NOAA’s operational systems.
NOAA’s newest hurricane model, the Hurricane Analysis and Forecast System (HAFS), was instrumental in providing forecasters with accurate predictions of Hurricane Helene and Hurricane Milton…
The fifth issue of UFS Insights, a collaborative effort between NOAA’s Earth Prediction Innovation Center (EPIC) and the Unified Forecast System (UFS) community, is available…
The Weather Program Office EPIC team continues their collaboration with the UK MetOffice to advance the Data Assimilation Science! Check out the new and updated…
Join us this July 22-26 at Jackson State University or online for the Unifying Innovations in Forecasting Capabilities Workshop 2024 (UIFCW24). The theme for this…
The Earth Prediction Innovation Center (EPIC) made the cover of the Bulletin of the American Meteorological Society (BAMS) February 2024 issue with EPIC Possibilities.
The Earth Prediction Innovation Center (EPIC) will advance weather modeling skills, reclaim and maintain international leadership in the area of Numerical Weather Prediction (NWP), and improve the transition from Research to Operations by supporting the development of a community-developed model called the Unified Forecast System (UFS). EPIC will make the UFS source code available to the public so that researchers from academia, private and public sectors can participate in the development of model code for National Oceanic and Atmospheric (NOAA) operational models. EPIC is outlined by seven core investment areas in the EPIC Vision Paper:
1. External engagement and community
2. Software engineering
3. Software infrastructure
4. User support services
5. Cloud-based high performance computing
6. Scientific innovation Management and planning
Making NOAA’s model code available to the public will allow external world-class scientists and researchers the opportunity to collaborate on new model improvements. This is a new way of thinking. Instead of keeping research and development inside of NOAA, the entire weather enterprise (i.e. public, private, and academics) will be able to work with NOAA to improve our modeling system, thereby accelerating advancements to our mission of protecting life and property.
The Unified Forecast System (UFS) Steering Committee defines the UFS as “a community-based, coupled, comprehensive Earth modeling system. It is designed to provide numerical guidance for applications in the forecast suite of NOAA’s National Centers for Environmental Prediction (NCEP). The UFS applications span local to global domains and predictive time scales from hours to greater than one year. The UFS is also intended to be an exploratory model that can include research elements that are not part of production suite applications.” For more information visit here.
UFS configurations that support particular forecast requirements are called applications. Each application combines a numerical model, data assimilation, post-processing, workflow, and other elements. Each distinct software element of an application is called a component. UFS provides guidance for a set of critical forecasts that targets weather and hurricane prediction. UFS can also be configured to provide guidance for sub-seasonal and seasonal outlooks, and for specialized forecasts such as air quality and space weather. Current UFS applications and the types of guidance they are expected to provide are listed below.
Short-Range Weather/Convection Allowing: Atmospheric behavior from less than an hour to several days
Medium-Range Weather (Weather): Atmospheric behavior out to about two weeks
Subseasonal-to-Seasonal (S2S): Atmospheric and ocean behavior from about two weeks to about one year
Hurricane: Hurricane track, intensity, and related effects out to about one week
Space Weather: Upper atmosphere geophysical activity and solar behavior out to about one month * Marine and Cryosphere o Ocean and ice behavior out to about ten days
Coastal: Storm surge and other coastal phenomena out to about one week
Air Quality: Aerosol and atmospheric composition out to several days. For more information, visit here.
According to NOAA’s National Centers for Environmental Prediction (NCEP), “Numerical Weather Prediction (NWP) data are the form of weather model data that we are most familiar with on a day-to-day basis. NWP focuses on taking current observations of weather and processing these data with computer models to forecast the future state of the weather. Knowing the current state of the weather is just as important as the numerical computer models processing the data. Current weather observations serve as input to the numerical computer models through a process known as data assimilation to produce outputs of temperature, precipitation, wind, and hundreds of other meteorological elements” from the sea to the sun.
EPIC will initially focus on making short and medium range weather forecast model code available to the public. Once the Unified Forecast System (UFS) is fully integrated into NOAA’s operational infrastructure systems, community members will be able to use EPIC to improve NOAA’s other modeling initiatives, such as climate and ocean models.
In 2018, Congress passed the National Integrated Drought Information System Reauthorization Act. This law added language to the Weather Research and Forecasting Innovation Act (WRFIA) that was passed in 2017.
The Weather, Water, and Climate Enterprise, also known as the Weather Enterprise for short, is comprised of three main sectors that contribute to the science and forecasting of weather — academia, government, and America’s Weather Industry. The National Weather Service describes the interaction between the Weather Enterprise and the public sector as the following: “Each sector plays a critical role in understanding, observing, forecasting, and helping warn communities of danger; and all are working together to build a Weather-Ready Nation. Beyond just weather information, the Weather Enterprise works to meet the needs of water resource managers, businesses, farmers, and water-sensitive communities. Further, longer term decision makers use seasonal, yearly or even longer climate outlooks to help with extensive planning efforts, such as community or agricultural planning. There are very diverse needs and equally diverse users of weather, water, and climate information, which makes it essential for the different sectors of the Weather Enterprise to work together and meet the needs of a constantly changing society.” For more information, visit here.
NOAA Administrative Order (NAO) 216-105B: “Policy on Research and Development Transitions” defines research as basic or applied research.
Basic research is “experimental or theoretical work undertaken primarily to acquire new knowledge of the underlying foundations of phenomena and observable facts, without any particular application or use in view.”
Applied research is “the original investigation undertaken in order to acquire new knowledge. It is, however, directed primarily towards a specific, practical aim or objective.”
The “Policy on Research and Development Transitions” Administrative Order defines development and transition as:
Development is “the systematic work, drawing on knowledge gained from research and practical experience and producing additional knowledge, that is directed to producing new products or processes, or to improve existing products or processes.”
Transition is “the transfer of a research and development output to a capability ready for an operation, application, commercial product or service, or other use.” For more information, please visit here.
High Performance Computing, or HPC, refers to the use of supercomputers to solve complex computational problems that could not be solved on traditional computing resources, such as laptops or desktops. High Performance Computing & Communications (HPCC), established through the High-Performance Computing Act of 1991, improve the accuracy and timeliness of NOAA’s short-term weather warnings, forecasts, hurricane forecast improvements, as well as regional and global climate and ecosystem predictions. High Performance Computing (HPC) Initiatives provide necessary computational and network resources required to advance in environmental modeling capabilities across NOAA. The benefits of HPC Initiatives provide the infrastructure needed to:
* Improvements in short-term warning and weather forecast systems and models,
* Enabling scientists to attack long-lead time problems associated with the physical processes that govern the behavior of the atmosphere and ocean,
* Maintaining NOAA’s leadership position in understanding climate with applications towards critical issues such as hurricanes, drought, sea-level rise, and
* Accelerating modeling and simulation activities and providing relevant decision support information on a timely basis for programs.
The EPIC high-performance computing (HPC) strategy has two core components designed to promote rapid advancement of the Unified Forecast System (UFS) by National Oceanic and Atmospheric Administration (NOAA) scientists and users from outside of NOAA. The first component will enable a scientist to rapidly run models and conduct research using the current version of the operational Numerical Weather Prediction (NWP) system run at NOAA’s National Centers for Environmental Prediction (NCEP) and, potentially, other operational centers. The second component lowers the compute-barrier that often prevents researchers from using NWP systems due to computational constraints caused by the size of model files and the compute power needed to process them. EPIC will provide a system that is able to be used on a variety of platforms, including the cloud, university HPC, and traditional HPC available within several federal organizations. For more information, visit here.
From the Federal Cloud Computing Strategy: The term “cloud” is often used broadly in the Federal Government for any technology solution provided by an outside vendor. The National Institute of Standards and Technology (NIST) defined several cloud deployment models as progressive increases in management by vendors, from * Infrastructure as a Service (IaaS) where vendors provide the infrastructure and hardware, to * Platform as a Service (PaaS) where vendors provide a managed environment for a customer’s application, to * Software as a Service (SaaS) where vendors provide a fully managed application and customers need only supply their data. In practice, many major vendor offerings no longer have such well-defined boundaries. Notwithstanding the term’s common usage, the term “cloud” is most accurately applied to those solutions that exhibit five essential characteristics of cloud computing, as defined by NIST:
1. on-demand service,
2. broad network access,
3. resource pooling,
4. rapid elasticity,
5. and measured service.
According to the Federal Cloud Computing Strategy “These characteristics and the solutions that exhibit them are provider-agnostic – meaning anyone can develop and deploy a cloud solution, whether an outside vendor or a Federal agency. Industry has moved to a more finely differentiated set of capabilities offered at different system layers, making possible nearly any combination of various components managed by either a vendor, a Government agency, or a mix of both. Industries that are leading in technology innovation have also demonstrated that hybrid and multi-cloud environments can be effective and efficient for managing workloads. As a result, the Cloud Smart Strategy encourages agencies to think of cloud as an array of solutions that offer many capabilities and management options to enhance mission and service delivery.”
EPIC will utilize innovative strategies and methods, including cloud-based computing capabilities, to host and manage all or part of the Unified Forecast System (UFS).
For more information please see “From Cloud First to Cloud Smart” Section I: Cloud at a Glance, found here.
Software Infrastructure Software infrastructure is defined as the foundational code and processes upon which applications are built and managed. The UFS is an end-to-end system for earth system prediction that encompasses the assimilation of observations (i.e., efforts such as the Joint Effort for Data assimilation Integration (JEDI)), development of initial conditions, execution of the prediction model, post-processing and analysis of model output, verification of results, and the delivery of products. Software infrastructure must support this entire process. The Memorandum of Agreement between the National Center for Atmospheric Research (NCAR) and NOAA for the co-development of common model infrastructure outlines areas of model development that help define the scope of software infrastructure considered here. These include:
* Open, managed, authoritative repositories
* Workflow management
* Intra- and inter- component coupling
Software Engineering
Modeling systems for Numerical Weather Prediction (NWP), the full-earth system components within the Unified Forecast System (UFS), and the operational applications derived from them, are becoming ever more complex. In order for innovative research to be integrated and tested in the UFS quickly, algorithms must be encapsulated into sections of code that have well-defined interfaces. This way, a scientist can focus on his or her particular area of interest without having to know the details of how other parts of the model function. In the computer science industry, this design principle is known as Separation of Concerns. A key part of EPIC will be to bring modern software engineering practices, such as Separation of Concerns, to bear on the model development process. EPIC will employ professional software engineers to work directly with scientists in an integrated cross-disciplinary environment, to ensure that the UFS code is easily maintainable, extensible and performance optimized. A key part of this will involve ensuring that all code modules include tests and can be reused across components of the system wherever possible. User support services One of the key areas for EPIC success is establishing comprehensive user support services to enable a broad and vibrant community of UFS users and developers. User Support Services (USS) will nurture a collaborative, community-based framework to support, advise and educate a cadre of system and software users and developers.
User support services include, but are not limited to:
* easy access to the latest version of code;
* a code repository maintained under version control software;
* thorough and understandable documentation;
* user-friendly workflows;
* code portability;
* adequate software infrastructure;
* tutorials, workshops, and developer involvement;
* hierarchical testing capabilities;
* clear pathways for incorporating new science;
* easy access to adequate amounts of computing resources;
* and attention to students as next-generation users.
To make changes or updates to an application, programmers must make changes to an applications source code. Source code is written with one of dozens of programming languages (i.e. Linux, Python, or C++) that tell the computer how to execute the commands that make a computer application work. Computer coding can be an error-prone process and a programmer may want to return to their original code, or a more recent version of their code, in their files to make updates or to fix errors. To keep track of changes and updates in files, programmers use Git, a version control system. Think of Git as Google Docs for programming languages. Just like Google Docs, Git allows programmers to see new changes in the text of files, see who made the changes, leave comments in a document, and view old versions of a file. GitHub is a web-based service for Git. Think of GitHub as social media for programmers, GitHub allows programmers to connect with other software developers working on the same files, troubleshoot problems, and share files online. Git and GitHub will play a critical role in EPIC by supplying the community with the source code for the Unified Forecast System (UFS). To encourage the community to develop on the UFS, EPIC will need to have well-managed repositories including policies, documentation, configuration, and testing protocols for all of the individual components of UFS code available on GitHub.
For more information, view the EPIC Vision Paper Section 2.1 “Open, managed, authoritative repositories” available here. For more information on Git and GitHub, visit here.
Proposals were due by November 17th, 2022.
Participants at the Workshop were encouraged to provide their feedback about EPIC, the Workshop, and reflection questions by using a Twitter hashtag and a Google Form called the “Triple S.” Data collected from the “Triple S,” Twitter, and recorder notes was analyzed to inform Session VI: “Strategy, Summary and Recommendations” on Day Three presented by several of the Workshop Planning Committee Members.
Members of the EPIC Community Workshop Planning Committee drafted a potential community-developed EPIC vision and mission statement based on presentations, question and answer sessions, and breakout group discussions at the Workshop.
