Health Profession Opportunity Grants (HPOG) Impact Study’s Six-Year Follow-Up Analysis Plan

In 2010, the Administration for Children and Families awarded the first round of grants from the Health Profession Opportunity Grants (HPOG 1.0) Program to 32 organizations in 23 states, including five Tribal organizations. The researchers used a multipronged research and evaluation strategy to assess the success of the HPOG 1.0 Program. It was evaluated using an experimental design in which program applicants were assigned at random to a “treatment” group that could access the program or to a “control” group that could not. The outcomes of these respective groups were compared in the short term (about 15 to 18 months after randomization), intermediate-term (after three years), and long-term (after six years).

This report presents the plan for evaluating the six-year impacts of the HPOG 1.0 grants on various outcomes. It describes the operationalization of outcome measures, including their source data.

The primary research questions relating to these grants after six years were:

1. What impacts do HPOG programs have on the outcomes of participants and their families and
2. To what extent do impacts vary across selected subpopulations?

The purpose of this report is to describe a plan for answering these questions using six-year outcomes.

(This report is 48 pages long.)

The report identifies three confirmatory outcomes, which are indicators of HPOG 1.0’s progress toward its goals after six years.

These are:

• Receipt of a postsecondary credential requiring one year or more of training.
• Current employment in a healthcare occupation.
• Average quarterly earnings over the 23rd and 24th quarters after random assignment. Because the ultimate goal of any job training program is to increase its participants’ earnings, this six-year analysis will place the greatest weight on earnings as a metric of the program’s long-term success.

The report also identifies secondary and exploratory outcomes. Secondary outcomes are additional important outcomes identified in the HPOG logic model. Exploratory outcomes include additional outcomes of interest and alternative measures of the confirmatory and secondary outcomes.

The report highlights the new data sources that will be used to carry out the six-year impact analysis, including the Six-Year Follow-Up Survey and administrative data from the National Student Clearinghouse. Other data sources, such as baseline data from the HPOG Performance Reporting System, earlier follow-up surveys, and the National Directory of New Hires, have been described in detail in prior project documents.

Finally, the report describes planned tests to measure the influence of COVID-19 on the evaluation and on its findings.

The final report including recommendations is anticipated in early 2022.

• Last Updated:
• Created:
• Resource Publication Date: 2021
• Author(s): D. Litwak, D. Walton, L. Peck
• Funding Source: U.S. Department of Health and Human Services, Administration for Children and Families (ACF)
• Resource Availability: Publicly available
• Posted by: Wesley Peterson
• Posted in: Workforce System Strategies