Evaluating Quality, Relevance and Performance in our Program Office

Supplemental materials and self-guided content for our inaugural Program Review.

On this page you’ll find materials about the Weather Program Office’s structure, functions and activities that are intended to assist our reviewers in evaluating our quality, relevance and performance as a Program Office.

Read the Charge to Reviewers >

Weather Program Office

Review Purpose

The National Oceanic and Atmospheric Administration (NOAA), Office of Oceanic and Atmospheric Research (OAR) conducts science reviews every five years to evaluate the quality, relevance, and performance of the research and activities its laboratories and programs conduct and support. These reviews are also intended to ensure that OAR research is linked to NOAA’s mission and priorities, is of high quality as judged by preeminence criteria, and is carried out with a high level of performance.

The purpose of the Weather Program Office (WPO) review, which will be held virtually January 24–27, 2023, is to provide an independent, external assessment of the program, including its management structures, and to recommend strategic directions in key areas for WPO and OAR.

The review will assess the research and other science activities WPO funds, how WPO decides what types of research and science to support, and how WPO engages with the public and peer communities to communicate about its work.

This external review will cover the programmatic activities and management of the Weather Program Office over the past five years (2017–2022). This will be WPO’s first program office review. The programmatic themes/activity areas and related topics for the WPO review are below.

Activity Area 1.

Organizational Excellence

Activity Area 2.

Weather Research Models, Observations and Forecasting Tools

Activity Area 3.

Advancement and Transition of Weather Research

Activity Area 4.

Effective Communication and Coordination of Weather Research

Charge to Reviewers

Details for Evaluation & Recommendations

NOAA guidance asks reviewers to consider the Quality, Relevance, and Performance of WPO and provide an overall rating for each activity area reviewed. For each area, each reviewer will provide one of the following overall ratings:

Highest Performance: WPO greatly exceeds the satisfactory level and is outstanding in almost all areas.
Exceeds Expectations: WPO goes beyond the satisfactory level and is outstanding in many areas.
Satisfactory: WPO meets expectations and the criteria for a satisfactory rating.
Needs Improvement: In general, WPO does not reach expectations and does not meet the criteria for a satisfactory rating. The reviewer will identify specific problem areas that need to be addressed.

Download Charge to Reviewers

Evaluation Criteria
Quality: Evaluate the quality of WPO’s research and development (R&D) portfolio. This refers to the merit of R&D that is funded by WPO and the resulting communication, outputs, and knowledge shared with the scientific community.
Relevance: Evaluate the degree to which WPO funded research and development (R&D) is relevant to NOAA’s mission and of value to the Nation. This primarily refers to the value of R&D to users beyond the scientific community. Relevance includes not only hypothetical value, but actual impact. Examples of ways the impact of R&D can be realized include the application of scientific knowledge to policy decisions, the improvement of operational capabilities at NOAA’s service lines and other collaborating institutions, or licensing of inventions for commercial use.
Performance: Assessing performance may include considerations of technical execution, finances, workforce, infrastructure, and leadership necessary to achieve WPO’s goals. This involves understanding the quality of management, including interaction with stakeholders, clear articulation of strategic direction, as well as the balance of WPO’s portfolio across time frames and intended applications.

A Diversity of Expertise

Our Review Panel

Program Review Agenda

See the schedule details for our fully virtual 4-day review.

Day 1: Organizational Excellence

Day 2: Weather Research Models, Observations and Forecasting Tools

Day 3: Advancement and Transition of Weather Research

Day 4: Effective Communication and Coordination of Weather Research

Download Agenda

January 24th

Activity Area 1: Organizational Excellence

Click to Join Meeting | Download Day 1 Slides

Time (ET)	Topic	Speaker
10:00–10:15 am	Welcome and Introductions	Dorothy Koch, WPO Director Steve Thur, OAR AA
10:15–10:30 am	Overview of NOAA Research and Charge to Reviewers	Emily Menashes, OAR DAA, Programs and Administration
10:30–10:45 am	OAR Portfolios Overview	Wayne Higgins, Climate Portfolio Steward
10:45–11:00 am	Questions and Open Discussion
11:00–11:15 am	Break
11:15–12:00 pm	Director’s Overview	Dorothy Koch, WPO Director
11:15–12:00 pm	Activity Area 1: Overview and Introduction to WPO	John Ten Hoeve, WPO Deputy Director
12:00–12:30 pm	Questions and Open Discussion
12:30–1:30 pm	Lunch
1:30–3:00 pm	Policy and Budget Overview	Melissa Pratt-Zossoungbo, Administrative Officer
	Notice of Funding Opportunity (NOFO) Overview	Christine Bassett, S2S Program Coordinator
	Diversity, Equity, Inclusion, and Accessibility (DEI&A)	Segayle Thompson, PAR Acquisitions Project Manager
	Intern and Fellow Opportunities	Leah Dubots, Management and Program Analyst
3:00–4:00 pm	Questions and Open Discussion
4:00–4:15 pm	Break
4:15–5:00 pm	Closed Session: OAR Representatives	Review Panel & OAR Reps
5:00 pm	End Day 1

January 25th

Activity Area 2: Weather Research Models, Observations and Forecasting Tools

Click to Join Meeting | Download Day 2 Slides

Time (ET)	Topic	Speaker
10:00–10:15 am	Activity Area 2: Overview and Introduction to Programs	Mark Olsen, S2S Deputy Program Manager
10:15–11:15 am	Observations Program and Phased Array Radar	Mark Vincent and Segayle Thompson, Observations Program
	Subseasonal to Seasonal (S2S) Program and Climate Testbed	Jessie Carman, S2S Program Manager
	Earth Prediction Innovation Center (EPIC) Program	Maoyi Huang, EPIC Program Manager
11:15–12:30 pm	Questions and Open Discussion
12:30–1:30 pm	Lunch
1:30–2:30 pm	Closed Session: External Stakeholders	Review Panel & Stakeholders
2:30–2:50 pm	Break
2:50–4:00 pm	Closed Session: Review Panel
4:00 pm	End Day 2

January 26th

Activity Area 3: Advancement and Transition of Weather Research

Click to Join Meeting | Download Day 3 Slides

Time (ET)	Topic	Speaker
10:00–10:15 am	Activity Area 3: Overview and Introduction to Programs	Aaron Pratt, JTTI Program Coordinator
10:15–11:30 am	Air Quality (AQ) Program and Fire Weather	Jordan Dale, AQ Program Manager
	Joint Technology Transfer Initiative (JTTI) Program	Chandra Kondragunta, JTTI Program Manager
	Weather Testbeds Program	Jordan Dale, Testbeds Program Manager
	Research to Applications (R2X) and Transitions	Matt Mahalik, Research Transition & Evaluation Lead
11:30–12:30 pm	Questions and Open Discussion
12:30–1:30 pm	Lunch
1:30–2:30 pm	Closed Session: Line Office Representatives	Review Panel & LO Reps
2:30–2:50 pm	Break
2:50–4:00 pm	Closed Session: Review Panel
4:00 pm	End Day 3

January 27th

Activity Area 4: Effective Communication and Coordination of Weather Research

Click to Join Meeting | Download Day 4 Slides

Time (ET)	Topic	Speaker
10:00–10:15 am	Activity Area 4: Overview and Introduction to Programs	Castle Williamsberg, Social Science R2X Coordinator
10:15–11:15 pm	Supplemental Appropriations Program	Mark Vincent, Ben Woods, Chris Spells and Renee (Richardson) Keller, Supplemental Program
	Social Science Program	Gina Eosco, Social Science Program Manager
	Policy, Partnerships and Communication	Tamara Battle, Policy & Partnerships Lead
11:15–12:30 pm	Questions and Open Discussion
12:30–1:30 pm	Lunch
1:30–2:15 pm 2:15–3:30 pm	Closed Session: Review Panel Final Discussion and Preliminary Feedback to OAR and WPO
3:30–3:45 pm	Wrap Up and Thank You	Steve Thur, Dorothy Koch, and John Ten Hoeve
3:45–4:30 pm	Closed Session: Review Panel
4:30 pm	End Day 4