High Performance Computing, or HPC, refers to the use of supercomputers to solve complex computational problems that could not be solved on traditional computing resources, such as laptops or desktops.  High Performance Computing & Communications (HPCC), established through the High-Performance Computing Act of 1991, improve the accuracy and timeliness of NOAA’s short-term weather warnings, forecasts, hurricane forecast improvements, as well as regional and global climate and ecosystem predictions. High Performance Computing (HPC) Initiatives provide necessary computational and network resources required to advance in environmental modeling capabilities across NOAA. The benefits of HPC Initiatives provide the infrastructure needed to:

* Improvements in short-term warning and weather forecast systems and models,

* Enabling scientists to attack long-lead time problems associated with the physical processes that govern the behavior of the atmosphere and ocean,

* Maintaining NOAA’s leadership position in understanding climate with applications towards critical issues such as hurricanes, drought, sea-level rise, and

* Accelerating modeling and simulation activities and providing relevant decision support information on a timely basis for programs.

The EPIC high-performance computing (HPC) strategy has two core components designed to promote rapid advancement of the Unified Forecast System (UFS) by National Oceanic and Atmospheric Administration (NOAA) scientists and users from outside of NOAA. The first component will enable a scientist to rapidly run models and conduct research using the current version of the operational Numerical Weather Prediction (NWP) system run at NOAA’s National Centers for Environmental Prediction (NCEP) and, potentially, other operational centers. The second component lowers the compute-barrier that often prevents researchers from using NWP systems due to computational constraints caused by the size of model files and the compute power needed to process them. EPIC will provide a system that is able to be used on a variety of platforms, including the cloud, university HPC, and traditional HPC available within several federal organizations. For more information, visit here.

From the Federal Cloud Computing Strategy:  The term “cloud” is often used broadly in the Federal Government for any technology solution provided by an outside vendor. The National Institute of Standards and Technology (NIST) defined several cloud deployment models as progressive increases in management by vendors, from * Infrastructure as a Service (IaaS) where vendors provide the infrastructure and hardware, to  * Platform as a Service (PaaS) where vendors provide a managed environment for a customer’s application, to  * Software as a Service (SaaS) where vendors provide a fully managed application and customers need only supply their data. In practice, many major vendor offerings no longer have such well-defined boundaries. Notwithstanding the term’s common usage, the term “cloud” is most accurately applied to those solutions that exhibit five essential characteristics of cloud computing, as defined by NIST:

1. on-demand service,

2. broad network access,

3. resource pooling,

4. rapid elasticity,

5. and measured service.

According to the Federal Cloud Computing Strategy “These characteristics and the solutions that exhibit them are provider-agnostic – meaning anyone can develop and deploy a cloud solution, whether an outside vendor or a Federal agency. Industry has moved to a more finely differentiated set of capabilities offered at different system layers, making possible nearly any combination of various components managed by either a vendor, a Government agency, or a mix of both. Industries that are leading in technology innovation have also demonstrated that hybrid and multi-cloud environments can be effective and efficient for managing workloads. As a result, the Cloud Smart Strategy encourages agencies to think of cloud as an array of solutions that offer many capabilities and management options to enhance mission and service delivery.”

EPIC will utilize innovative strategies and methods, including cloud-based computing capabilities, to host and manage all or part of the Unified Forecast System (UFS).

For more information please see “From Cloud First to Cloud Smart” Section I: Cloud at a Glance, found here.

Software Infrastructure  Software infrastructure is defined as the foundational code and processes upon which applications are built and managed. The UFS is an end-to-end system for earth system prediction that encompasses the assimilation of observations (i.e., efforts such as the Joint Effort for Data assimilation Integration (JEDI)), development of initial conditions, execution of the prediction model, post-processing and analysis of model output, verification of results, and the delivery of products. Software infrastructure must support this entire process.  The Memorandum of Agreement between the National Center for Atmospheric Research (NCAR) and NOAA for the co-development of common model infrastructure outlines areas of model development that help define the scope of software infrastructure considered here. These include:

* Open, managed, authoritative repositories

* Workflow management

* Intra- and inter- component coupling

Software Engineering 

Modeling systems for Numerical Weather Prediction (NWP), the full-earth system components within the Unified Forecast System (UFS), and the operational applications derived from them, are becoming ever more complex. In order for innovative research to be integrated and tested in the UFS quickly, algorithms must be encapsulated into sections of code that have well-defined interfaces. This way, a scientist can focus on his or her particular area of interest without having to know the details of how other parts of the model function. In the computer science industry, this design principle is known as Separation of Concerns. A key part of EPIC will be to bring modern software engineering practices, such as Separation of Concerns, to bear on the model development process. EPIC will employ professional software engineers to work directly with scientists in an integrated cross-disciplinary environment, to ensure that the UFS code is easily maintainable, extensible and performance optimized. A key part of this will involve ensuring that all code modules include tests and can be reused across components of the system wherever possible.  User support services  One of the key areas for EPIC success is establishing comprehensive user support services to enable a broad and vibrant community of UFS users and developers. User Support Services (USS) will nurture a collaborative, community-based framework to support, advise and educate a cadre of system and software users and developers.

User support services include, but are not limited to:

* easy access to the latest version of code;

* a code repository maintained under version control software;

* thorough and understandable documentation;

* user-friendly workflows;

* code portability;

* adequate software infrastructure;

* tutorials, workshops, and developer involvement;

* hierarchical testing capabilities;

* clear pathways for incorporating new science;

* easy access to adequate amounts of computing resources;

* and attention to students as next-generation users.

To make changes or updates to an application, programmers must make changes to an applications source code. Source code is written with one of dozens of programming languages (i.e. Linux, Python, or C++) that tell the computer how to execute the commands that make a computer application work. Computer coding can be an error-prone process and a programmer may want to return to their original code, or a more recent version of their code, in their files to make updates or to fix errors. To keep track of changes and updates in files, programmers use Git, a version control system. Think of Git as Google Docs for programming languages. Just like Google Docs, Git allows programmers to see new changes in the text of files, see who made the changes, leave comments in a document, and view old versions of a file. GitHub is a web-based service for Git. Think of GitHub as social media for programmers, GitHub allows programmers to connect with other software developers working on the same files, troubleshoot problems, and share files online. Git and GitHub will play a critical role in EPIC by supplying the community with the source code for the Unified Forecast System (UFS). To encourage the community to develop on the UFS, EPIC will need to have well-managed repositories including policies, documentation, configuration, and testing protocols for all of the individual components of UFS code available on GitHub.

For more information, view the EPIC Vision Paper Section 2.1 “Open, managed, authoritative repositories” available here. For more information on Git and GitHub, visit here.