Design Options for an Evaluation of TANF Job Search Assistance

“The core of the report describes several broad designs for estimating causal impacts of JSA on individual job seekers, using both individual- and group-randomized experimental designs. It also discusses analytic methods and issues of the generalizability of impact findings over time, geographic areas, and participant characteristics” (p.1).

The report also discusses effects to the labor market when workers and firms that are not JSA recipients are affected by JSA interventions and suggests analyses for estimating these effects. The report concludes with a discussion of the tradeoffs across various evaluation design options when considering research, technical merit, practical merit, and policy relevance. (Abstractor: Author and Website Staff)

“The report concludes with a discussion of the tradeoffs across various evaluation design options when the criteria of research, technical merit, practical merit, and policy relevance are applied. Specifically, the conclusion explains that JSA is a low-intensity intervention, which is likely to have only small effects. The differential impacts of JSA program variants will be smaller yet, with large samples needed to detect them. In response, the report suggests a “maximum variation” comparison, creating a contrast between treatment options that might generate impacts of greater magnitude than a straight comparison of various versions of JSA. Comparing a bare minimum JSA program with a services-rich JSA program would likely generate larger estimated impacts (relative to testing the effectiveness of a single component), reducing sample size needs and therefore evaluation costs and logistical demands. Given that states currently invest resources in JSA programs, despite it being unclear how much these resources help participants, this type of evaluation design has the potential to provide useful information for policy design and decisions. For example, if no difference is detected between bare minimum and services-rich JSA, then states could shift resources from JSA to other efforts. Conversely, if an impact larger than the cost of the program is detected, then states can justify their investments in JSA, knowing that it makes a difference” (p.1). (Abstractor: Author)

• Last Updated:
• Created:
• Resource Publication Date: 2013
• Author(s): Peck, Laura R.; Bell, Stephen H.; Klerman, Jacob Alex; Juras, Randall
• Organizational Author(s): Abt Associates
• Funding Source: U.S. Department of Health and Human Services
• Resource Availability: Publicly available
• Posted by:
• Posted in: Workforce System Strategies