Evaluating Quality, Relevance, and Performance in our Program Office

Supplemental materials and self-guided content for our inaugural Program Review.

On this page you’ll find materials about the Weather Program Office’s structure, functions and activities that are intended to assist our reviewers in evaluating our quality, relevance and performance as a Program Office.

The National Oceanic and Atmospheric Administration (NOAA), Office of Oceanic and Atmospheric Research (OAR) conducts science reviews every five years to evaluate the quality, relevance, and performance of the research and activities its laboratories and programs conduct and support. These reviews are also intended to ensure that OAR research is linked to NOAA’s mission and priorities, is of high quality as judged by preeminence criteria, and is carried out with a high level of performance.

The purpose of the Weather Program Office (WPO) review, which was held virtually January 24–27, 2023, is to provide an independent, external assessment of the program, including its management structures, and to recommend strategic directions in key areas for WPO and OAR.

The review assesses the research and other science activities WPO funds, how WPO decides what types of research and science to support, and how WPO engages with the public and peer communities to communicate about its work. 

This external review covers the programmatic activities and management of the Weather Program Office over the past five years (2017–2022) and is the first program office review for WPO. The programmatic themes/activity areas and related topics for the WPO review are below.

Activity Area 1.

Organizational Excellence

Activity Area 2.

Weather Research Models, Observations and Forecasting Tools

Activity Area 3.

Advancement and Transition of Weather Research

Activity Area 4.

Effective Communication and Coordination of Weather Research

WPO Response to Panel Review Recommendations

WPO strives to find, fund and foster innovative atmospheric research. As a result of the recommendations from the expert panel during WPO’s Program Review in January 2023, WPO is implementing new strategies to expand and achieve more efficient and effective operations and management of its research portfolio. In the following document, WPO outlines the panel’s recommendations, their alignment with WPO’s Strategic Plan, and the actions WPO is taking to address any areas for improvement. WPO appreciates the opportunity to reflect and gain insight from such an esteemed panel of experts, and is looking forward to integrating their suggestions into daily operations.

The Weather Program Office Response to the Panel Review’s Recommendations can be found here.

Evaluation Report from the Review Panel

In 2023, the Weather Program Office (WPO) completed its first Program Review, covering the years of 2017–22. The Review Panel gave WPO an overall rating of “Exceeds Expectations.’‘ Throughout the assessment, reviewers emphasized the exceptional quality of work by WPO.

The panel concluded all of WPO’s programs were in alignment with NOAA’s mission and the office’s relevance is indisputable. Reflecting on the breadth of WPO’s work, the Review Panel commented: “WPO is a necessary and needed organization, particularly as convergent research comes to the forefront and societal impacts from weather and climate continue to rise.” The office’s work not only saves lives but also substantially enhances the economy as a result of the products produced, technical advancements, and improved communications. By integrating social and physical sciences within a unified framework, WPO produces actionable solutions for society.

You can read the full Evaluation report from the reviewers here.

Charge to Reviewers

Details for Evaluation & Recommendations

NOAA guidance asks reviewers to consider the Quality, Relevance, and Performance of WPO and provide an overall rating for each activity area reviewed.  For each area, each reviewer will provide one of the following overall ratings:

  • Highest Performance: WPO greatly exceeds the satisfactory level and is outstanding in almost all areas.
  • Exceeds Expectations: WPO goes beyond the satisfactory level and is outstanding in many areas.
  • Satisfactory: WPO meets expectations and the criteria for a satisfactory rating. 
  • Needs Improvement: In general, WPO does not reach expectations and does not meet the criteria for a satisfactory rating. The reviewer will identify specific problem areas that need to be addressed.
Download Charge to Reviewers
  • Evaluation Criteria

  • Quality: Evaluate the quality of WPO’s research and development (R&D) portfolio. This refers to the merit of R&D that is funded by WPO and the resulting communication, outputs, and knowledge shared with the scientific community.
  • Relevance:  Evaluate the degree to which WPO funded research and development (R&D) is relevant to NOAA’s mission and of value to the Nation. This primarily refers to the value of R&D to users beyond the scientific community. Relevance includes not only hypothetical value, but actual impact. Examples of ways the impact of R&D can be realized include the application of scientific knowledge to policy decisions, the improvement of operational capabilities at NOAA’s service lines and other collaborating institutions, or licensing of inventions for commercial use.
  • Performance: Assessing performance may include considerations of technical execution, finances, workforce, infrastructure, and leadership necessary to achieve WPO’s goals. This involves understanding the quality of management, including interaction with stakeholders, clear articulation of strategic direction, as well as the balance of WPO’s portfolio across time frames and intended applications.

A Diversity of Expertise

Our Review Panel

Program Review Agenda

Day 1: Organizational Excellence

Day 2: Weather Research Models, Observations and Forecasting Tools

Day 3: Advancement and Transition of Weather Research

Day 4: Effective Communication and Coordination of Weather Research

Download Agenda

Activity Area 1: Organizational Excellence

Download Day 1 Slides

Time (ET) Topic Speaker
10:00–10:15 am Welcome and Introductions Dorothy Koch, WPO Director

Steve Thur, OAR AA

10:15–10:30 am Overview of NOAA Research and Charge to Reviewers Emily Menashes, OAR DAA, Programs and Administration
10:30–10:45 am OAR Portfolios Overview Wayne Higgins, Climate Portfolio Steward
10:45–11:00 am Questions and Open Discussion
11:00–11:15 am Break
11:15–12:00 pm Director’s Overview Dorothy Koch, WPO Director
Activity Area 1: Overview and Introduction to WPO John Ten Hoeve, WPO Deputy Director
12:00–12:30 pm Questions and Open Discussion
12:30–1:30 pm Lunch
1:30–3:00 pm Policy and Budget Overview Melissa Pratt-Zossoungbo, Administrative Officer
Notice of Funding Opportunity (NOFO) Overview Christine Bassett, S2S Program Coordinator
Diversity, Equity, Inclusion, and Accessibility (DEI&A) Segayle Thompson, PAR Acquisitions Project Manager
Intern and Fellow Opportunities Leah Dubots, Management and Program Analyst
3:00–4:00 pm Questions and Open Discussion
4:00–4:15 pm Break
4:15–5:00 pm Closed Session: OAR Representatives Review Panel & OAR Reps
5:00 pm End Day 1

Activity Area 2: Weather Research Models, Observations and Forecasting Tools

Download Day 2 Slides

Time (ET) Topic Speaker
10:00–10:15 am Activity Area 2: Overview and Introduction to Programs Mark Olsen, S2S Deputy Program Manager
10:15–11:15 am Observations Program and Phased Array Radar Mark Vincent and Segayle Thompson, Observations Program
Subseasonal to Seasonal (S2S) Program and Climate Testbed Jessie Carman, S2S Program Manager
Earth Prediction Innovation Center (EPIC) Program Maoyi Huang, EPIC Program Manager
11:15–12:30 pm Questions and Open Discussion
12:30–1:30 pm Lunch
1:30–2:30 pm Closed Session: External Stakeholders Review Panel & Stakeholders
2:30–2:50 pm Break
2:50–4:00 pm Closed Session: Review Panel
4:00 pm End Day 2

Activity Area 3: Advancement and Transition of Weather Research

Download Day 3 Slides

Time (ET) Topic Speaker
10:00–10:15 am Activity Area 3: Overview and Introduction to Programs Aaron Pratt, JTTI Program Coordinator
10:15–11:30 am Air Quality (AQ) Program and Fire Weather Jordan Dale, AQ Program Manager
Joint Technology Transfer Initiative (JTTI) Program Chandra Kondragunta, JTTI Program Manager
Weather Testbeds Program Jordan Dale, Testbeds Program Manager
Research to Applications (R2X) and Transitions Matt Mahalik, Research Transition & Evaluation Lead
11:30–12:30 pm Questions and Open Discussion
12:30–1:30 pm Lunch
1:30–2:30 pm Closed Session: Line Office Representatives Review Panel & LO Reps
2:30–2:50 pm Break
2:50–4:00 pm Closed Session: Review Panel
4:00 pm End Day 3

Activity Area 4: Effective Communication and Coordination of Weather Research

Download Day 4 Slides

Time (ET) Topic Speaker
10:00–10:15 am Activity Area 4: Overview and Introduction to Programs Castle Williamsberg, Social Science R2X Coordinator
10:15–11:15 pm Supplemental Appropriations Program Mark Vincent, Ben Woods, Chris Spells and Renee (Richardson) Keller, Supplemental Program
Social Science Program Gina Eosco, Social Science Program Manager
Policy, Partnerships and Communication Tamara Battle, Policy & Partnerships Lead
11:15–12:30 pm Questions and Open Discussion
12:30–1:30 pm Lunch
1:30–2:15 pm

 

2:15–3:30 pm

Closed Session: Review Panel

Final Discussion and Preliminary Feedback to OAR and WPO

3:30–3:45 pm Wrap Up and Thank You Steve Thur, Dorothy Koch, and John Ten Hoeve
3:45–4:30 pm Closed Session: Review Panel
4:30 pm End Day 4

Resource Library

Everything you need for the review

Click on the image to download the PDF report
Annual Accomplishments Report FY18
Click on the image to download the PDF report
Annual Accomplishments Report FY19
Click on the image to download the PDF report
Annual Accomplishments Report FY20
Annual Accomplishments Report Fiscal Year 2020
Annual Accomplishments Report FY21

Weather Program Office Strategic Plans

Click to download WPO Strategic Plan FY22–26 PDF
OWAQ Strategic Plan FY 2019–2021
WPO Strategic Plan FY 2022–2026
WPO Strategic Plan FY 2022–2026

OAR Strategic Plan & OAR DEIA Strategic Plan

NOAA Strategic Plan & Executive Summary

OAR 2020–2026 Strategic Plan
OAR 2020–2026 Strategic Plan
OAR DEIA Strategic Plan
OAR DEIA Strategic Plan
NOAA FY 2022–2026 Strategic Plan Executive Summary
NOAA FY 2022–2026 Strategic Plan Executive Summary
NOAA FY22–26 Strategic Plan

The Weather Act

Download the Weather Act Reauthorization, 2019
Download the Weather Act Reauthorization (NIDIS Reauthorization Act), 2019
WPO & The Weather Research and Forecasting Innovation Act of 2017
WPO & The Weather Research and Forecasting Innovation Act of 2017
Download Weather Research and Forecasting Innovation Act of 2017
Download Weather Research and Forecasting Innovation Act of 2017

Visit our Weather Act Page

Supplemental Appropriations

Bipartisan Budget Act of 2018
Bipartisan Budget Act of 2018
Additional Supplemental Appropriations for Disaster Relief Act of 2019
Additional Supplemental Appropriations for Disaster Relief Act of 2019
Disaster Relief Supplemental Appropriations Act of 2022
Disaster Relief Supplemental Appropriations Act of 2022

Additional Documents

Weather Program Office Bibliometrics Report
WPO Acronym List
WPO List of Publications
WPO List of Publications
WPO Project List
WPO Project List
WPO FY19-FY23 Annual Operating Plans
WPO FY19-FY23 Annual Operating Plans
An overview of OAR’s Weather Program Office Observations Program, given by Program Manager, Dr. Mark Vincent. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Subseasonal to Seasonal Program, given by Program Manager, Dr. Jessie Carman. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office VORTEX Program, given by Testbeds & Air Quality and Fire Weather Program Coordinator, Felicia Guarriello. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Earth Prediction Innovation Center (EPIC) Program, given by Program Manager, Dr. Maoyi Huang. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Air Quality and Fire Weather Program, given by Program Coordinator, Felicia Guarriello. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office FACETs Program, given by Social Science and FACETs Program Manager, Dr. Gina Eosco. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Testbeds Program, given by Program Coordinator, Felicia Guarriello. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Research Transitions and Evaluation, given by Program Lead, Matt Mahalik. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Social Science Program, given by Social Science and FACETs Program Manager, Dr. Gina Eosco. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Supplemental Appropriations Program, given by Program Coordinator, Ben Woods. This video was recorded for WPO’s 5-year Program Review in January 2023.
An overview of OAR’s Weather Program Office Joint Technology Transfer Initiative (JTTI) Program, given by Program Coordinator, Aaron Pratt. This video was recorded for WPO’s 5-year Program Review in January 2023.

Excellence in Planning & Preparation

Program Review Team

We provide support and coordination across the Weather Program Office for the Program Review.

Contact us at wpo.programreview@noaa.gov for more details about the review process or materials.

Melissa Pratt-Zossoungbo WPO

Strategy and Evaluation Manager

View details
Kristina Kiest WPO

Communications Lead

View details