Career Pathways Intermediate Outcomes Study: Plan for Cost-Benefit Analyses

“Initiated in 2007, Pathways for Advancing Careers and Education (PACE) is a nine-site implementation and impact evaluation of promising programs that are using ‘career pathways’ as their main intervention framework for increasing employment and self-sufficiency among low-income individuals and families….

The impact study portion of the evaluation uses an experimental design to explore the effectiveness of each site’s career pathways program. Eligible applicants at each site (‘study participants’) are assigned at random either to a treatment group that is offered access to that career pathways program (‘PACE program’) or to a control group that is not offered access; either group can use other similar services available in the community. The PACE study is comparing the treatment and control group’s education and training attainment, earnings, and other outcomes at approximately 18 months after random assignment using survey and administrative data to measure the impact of each PACE program. In anticipation of future cost-benefit analyses, the PACE study also collected cost data at each of the nine sites, which was completed in 2015” (p.1).

“Research Questions

The research questions answered in this design document are as follows.

• How will [the authors] determine and monetize the costs and benefits of six selected PACE programs?
• How will [the authors] use the monetized costs and benefits to determine the net present value (NPV) of each PACE program?” (p.i).

The Analysis Plan Covered in the Article

“[The article] lays out a plan for the cost-benefit analyses (CBAs) that [the authors] will conduct for up to six of the nine Pathways for Advancing Careers and Education (PACE) programs” (p.i).

“The elements of [the] analysis plan are as follows.

• Chapter 2 describes the purpose and approach of the CBAs.
• Chapter 3 provides [the authors’] detailed plan for conducting CBAs of PACE programs.
• Chapter 4 describes how [the authors] will select the six PACE programs for CBAs.
• Chapter 5 provides an inventory of data sources [the authors] will use for the CBAs.
• The appendix shows [the] cost data collection template that was used to collect program data as part of the earlier PACE study” (p.3).

(Abstractor: Author and Website staff)

“Key Findings and Highlights • [The authors] will conduct the CBAs from a societal perspective, where as many benefits and costs as possible are included regardless of the party experiencing the benefit or incurring the cost. So, for example, in addition to including the benefits due to increased incomes for treatment group members, [the authors] also will include resulting changes in government tax revenues and spending on public benefits. In addition to determining [Net Present Values] NPVs from this societal perspective, [the authors] will also calculate NPVs that only include the benefits and costs that accrue to key stakeholders such as program participants and the government. • Costs and benefits cannot be identified without context. The PACE programs were implemented with a randomly assigned treatment group (able to enroll in PACE program) and control group (not able to enroll in PACE program). In the CBAs, the outcomes and costs of the control group will provide the necessary context for the outcomes and costs of the treatment group. That is, the benefits of the PACE programs are the value of outcomes for treatment group members less the value of the corresponding “business as usual” outcomes for the control group. Similarly, costs of the PACE programs are calculated as the value of all resources used to provide the program less estimated costs of any use of non-PACE education and training programs that control group participants may have accessed. • The CBAs will use observed study participant and program data wherever possible to measure impacts and outcomes, but complete data on all benefits and costs are not always available. This plan details how [the authors] will impute and assume values for costs or benefits when data are not available. This includes taking benchmarks from external research and developing projections about future earnings based on observed data” (p.i-ii). (Abstractor: Author and Website staff)

• Last Updated:
• Created:
• Resource Publication Date: 2017
• Author(s): Dastrup, Samuel; Burnett, Kimberly; and Buron, Larry.
• Organizational Author(s): Abt Associates Inc.
• Funding Source: U.S. Department of Health and Human Services
• Resource Availability: Publicly available
• Posted by: Wesley Peterson
• Posted in: Workforce System Strategies