Beyond the Numbers: Data Use for Continuous Improvement of Programs Serving Disconnected Youth

“The American Youth Policy Forum (AYPF) conducted a series of in-depth case studies to examine how three programs which serve a disconnected youth population are utilizing data as a tool for continuous program improvement and ongoing accountability. The resulting publication, Beyond the Numbers: Data Use for Continuous Improvement of Programs Serving Disconnected Youth, described data collection and use at three successful programs, and distilled the key lessons learned and issues to consider both for practitioners and policymakers aiming to improve outcomes for the disconnected youth population” (p. 1). (Abstractor: Author)

“The profiled organizations have made great strides in providing quality interventions to the disconnected youth population. In addition to the well-defined and effectively-executed strategies they have developed, each organization has emphasized a strong data culture among its staff. The results of [the] study have revealed several key lessons learned which [it] hope[s] will aid policymakers, practitioners, and other youth-serving organizations in their efforts to constantly refine and improve the programs and services offered to youth. These include: • The need for a clearly articulated theory of change • Data collection which mirrors program needs • A strong organizational commitment to professional development for all staff related to data collection and data use” (p.34). Organizations utilized data to strengthen their programs in the following ways: • Roca - “Using their customized data system, Roca staff is able to track both benchmarks for individual youth, such as attendance in stage-based programming and placement in transitional employment, as well as substantive outcomes for the entire Roca cohort, such as employment retention and educational attainment” (p.10). Additionally, “Roca is capable of providing funders a complete view of its impact on youth…” (p.10). Partnering with the local Police Department through the “Safe and Successful Youth Initiative (SSYI)”, Police Department “data and information [is used] to help Roca identify potential candidates…for Roca program slots and intervention, while Roca data is shared with Police so they in turn can monitor the progress of Roca clients (p.11). Also, “data collected from the youth worker supervision and quarterly organization-wide reports is used by Roca to inform decisions regarding both program offerings, as well as staff professional development” (p.12). • Our Piece of the Pie (OPP) - “OPP has an organization-wide commitment to using data not only to track and report individual and aggregate outcomes, but also to foster a culture of continuous improvement that hinges on a strong data system” (p.18). “Additionally, some common indicators are tracked across different programs in the city, as Hartford requires OPP to collect demographic measures that are common to all agencies using the city’s ETO enterprise” (p.18). • Diploma Plus National Network (DP) - “Diploma Plus has prioritized and articulated a set of critical data elements to be collected across sites and has provided trainings across the network on data collection and usage at all levels of the organization” (p.26). DP data collection enables “support mechanisms…to be created for those schools [and students] in need,” including continuous coaching and training for schools within the network, tracks individual student progress and robust performance and outcomes data sets are attractive to funders (p.27-31). (Abstractor: Author and Website Staff)

• Last Updated:
• Created:
• Resource Publication Date: 2012
• Author(s): Pate, Austin; Lemer, Jennifer Brown; Browning, Andrea
• Organizational Author(s): American Youth Policy Forum
• Funding Source: Charles Stewart Mott Foundation
• Resource Availability: Publicly available
• Posted by: Laura Aron
• Posted in: Workforce System Strategies