The Workforce Innovation Fund (WIF) Evaluations: A Synthesis Report on Findings and Experiences

This synthesis report provides summary information on the WIF interventions and their evaluations, as well as successes and challenges in conducting the evaluations.

In addition to promoting innovation, WIF also sought to build on and expand the use of evaluation research in the workforce system. Using a “tiered-evidence” model, the initiative required that all prospective grantees demonstrate how their proposed innovation was supported by past research and how it would be evaluated to generate future evidence. Grantees were required to contract with an independent third-party evaluator to conduct an evaluation of their project and produce a report covering the implementation of the project and the outcomes (or impacts) achieved.

The primary source for this report was the evaluation reports produced between 2015 and 2019 by the third-party evaluators engaged by the WIF grantees. Other data sources included documents (program manuals, guidance for staff, or local economic data); interviews with and surveys of program administrators, line staff, and program partners; participant focus groups; and project data on enrollment, participation, and completion. Finally, this report highlights lessons learned from the WIF experience and considers how these lessons might be applied in future grantee-funded evaluations.

Overall, WIF was a substantial accomplishment. Across the three rounds of grants, 43 grantees developed and implemented a range of innovative projects and procured external evaluators, who conducted evaluations and produced reports on the projects’ interventions and outcomes. While the evaluation component was one of the more challenging aspects of WIF for some grantees, and the evaluations varied in their strength, all grantees met the evaluation requirement, and many gained significant knowledge from the experience. Further, the grantees and their evaluators produced a broad and useful array of findings on the many and varied interventions, contributing important information to the evidence base.

(This report is 104 pages long.)

(This study took place from 2019–2020.)

The researchers identified the following selected findings:

• Many grantees started with minimal experience supporting program evaluations.
• Grantees had to gain an understanding of the tiered evidence structure.
• To help clarify and enforce evaluation standards, the National Evaluation Coordinator (NEC) provided documentation on expected levels of rigor for the evaluations, the process for submitting evaluation designs, and expectations for evaluator collaboration with the NEC.
• An initial period for planning, implemented as part of later rounds, was valuable in improving the implementation of the interventions and evaluations.
• The WIF structure involved a number of stakeholders in a complex array of relationships that presented challenges for both grantees and evaluators.
• Because grantees funded the evaluations, some issues were encountered in maintaining evaluator independence.
• Communication among the stakeholders involved in WIF emerged as a key component to maintaining effective working relationships and contributing to maintaining evaluator independence.
• Evaluators and grantees experienced challenges in evaluation execution, many of which were common in the evaluation field but some of which were unique to WIF.
• The implementation studies from WIF serve as a rich source of information and learning, in particular for the interventions that were focused on innovative activities, partnerships, or methods.

The researchers identified the following selected recommendations:

• Define roles clearly and increase communication and accountability across stakeholders.
• Develop clear, consistent standards and lines of authority to enforce standards.
• Build capacity within grantee/project staff to ensure evaluation goals and methods are understood.
• Ensure an adequate timeline to measure outcomes relevant to a workforce intervention.
• Ensure comparability of data measures across evaluations.
• Evaluators should note that veterans are excluded from random assignment evaluations and recognize the important implications for the sample size target.
• Require implementation or process studies as a component of an outcome or impact evaluation.
• Include an initial period for comprehensive planning of the intervention, engagement of the evaluator, evaluability assessment, and evaluation design.
• Include detailed evaluation requirements and clear expectations for evaluation deliverables and timelines.

• Last Updated:
• Created:
• Resource Publication Date: 2020
• Author(s): Kean, E.; Cook, R.; Rosa, S.; Williams, J.
• Funding Source: U.S. Department of Labor, Employment and Training Administration
• Resource Availability: Publicly available
• Posted by: Justin Venneri
• Posted in: Workforce System Strategies