When Do Regression-Adjusted Performance Measures Track Longer-Term Program Impacts? A Case…

“The use of performance management systems has increased since the Government Performance and Results Act of 1993. While these systems share the goal of trying to improve service delivery and participant outcomes, they do not necessarily provide information on the causal (value-added) effects of a program, which requires a rigorous impact evaluation. One approach for potentially improving the association between program performance measures and impacts is to adjust performance measures for differences across performance units in participant characteristics and local economic conditions. This article develops a statistical model that describes the conditions under which regression adjustment improves the performance-impact correlation. [The authors] then use the model to examine the performance-impact association using extensive data from a large-scale random assignment evaluation of Job Corps, the nation's largest training program for disadvantaged youths” (p.495). (Abstractor: Author)

Full Publication Title: When Do Regression-Adjusted Performance Measures Track Longer-Term Program Impacts? A Case Study of Job Corps 

The authors found “that while regression adjustment changes the Job Corps center performance measures, the adjusted performance measures are not correlated with the impact estimates. The main reasons are the weak associations between the unadjusted Job Corps performance measures and participants’ longer-term outcomes as measured by the evaluation, as well as the likely presence of unobserved factors across centers that are correlated with outcomes” (p.495). “Though regression adjustment has the potential to improve the performance-impact correlation, it will not be effective if performance measures and outcomes are not well-aligned. Thus, further study is needed to identify inexpensive markers for long-run impacts that are available in the short run and that can be feasibly measured in performance measurement systems” (p. 524). (Abstractor: Author)

• Last Updated:
• Created:
• Resource Publication Date: 2014
• Author(s): Schochet, Peter Z. and Fortson, Jane
• Organizational Author(s): No organizational author identified
• Funding Source: Funding source not identified
• Resource Availability: Publicly available
• Posted by: Wesley Peterson
• Posted in: Workforce System Strategies