Skip to main content
Research/Evaluation Report

Machine Learning in Workforce Development Research: Lessons and Opportunities

About

This brief presents findings from an exploratory study examining how machine learning can be used to synthesize data about the implementation of career pathways programs. These programs aim to help individuals enter and exit occupational training at different levels—depending on their initial skills and work experience—and advance over time to higher skills, industry-recognized credentials, and better jobs with higher pay. It is one of four linked WSS resources related to career pathways.

This exploratory study was designed to explore how to use machine learning methods, including web scraping, supervised learning, and natural language processing, to collect and analyze a large volume of data on career pathways program implementation.

The study design included two phases: (1) an initial data collection phase to identify an approach and techniques for collecting data, and (2) an analysis phase to synthesize the data. While only the first phase was completed as part of this project, the experience yielded important lessons for future machine learning studies.

This brief summarizes lessons learned from utilizing machine learning to study the implementation of career pathways programs. First, this brief describes the research questions that guided the study and summarizes the machine learning methods designed for the data collection and analysis activities, including study limitations and challenges encountered. It then provides lessons learned on using machine learning methods for social science research. Finally, the brief discusses strategies for using these methods in future workforce development projects and other areas, particularly federally funded efforts.

(This resource is 19 pages in length.)

Major Findings and Recommendations

The researchers identified the following selected findings:

  • Machine Learning can be a powerful tool in the right context. The term “machine learning” covers a wide array of analytic approaches and tools which must be appropriately tailored to the research questions of interest and the data that will be analyzed.
  • Any given application of machine learning methods involves some risk, requiring an understanding of their limitations and expected results. Machine learning methods have been incorporated into the day-to-day work of fields such as information technology and medicine, but they have not been widely applied in workforce development or other social science research. Based on the experience of this project, work is ongoing to determine how these methods can address important questions in workforce development research.
  • In the context of implementation research, machine learning methods may have difficulty replicating the detail and nuance capable of human researchers. While machine learning algorithms’ ability to process large quantities of data—known as big data—is valuable in some contexts, machine learning algorithms will struggle to match the nuance of analyses by human researchers.
  • Some projects should expect to dedicate substantial time and resources to defining key terms, creating and applying a coding framework, and interpreting results. Lack of standardization around key terms in workforce development can present challenges for implementing certain machine learning tasks. Based on this project’s experience, projects applying machine learning methods designed to provide information on concepts without a set of standard definitions and rules may face similar issues.

The researchers identified the following recommendation:

  • The experience of this project suggests that policymakers, government agencies, and other stakeholders interested in using machine learning may first want to execute a “discovery phase” to maximize the efficacy of future machine learning efforts. Specifically, during the discovery phase, data scientists and subject matter experts could collaborate to generate research questions of interest, explore and assess potential data sources, and create design options that could appropriately apply machine learning techniques to answer questions of interest.

Materials

Comments

Content Details

Methodology:

Topics:

Target Populations:

Programs:

Geographic Locations:

Industry Sectors:

  • Last Updated:
  • Created:
  • Resource Publication Date: 2021
  • Author(s): De La Rosa, S. M., Greenstein, N., Schwartz, D., Lloyd, C.
  • Funding Source: U.S. Department of Labor
  • Resource Availability: Publicly available
  • Posted by: Wesley Peterson
  • Posted in: Workforce System Strategies

Error

An error has occurred. The details of this error have been forwarded to our technical team. If you continue to experience issues, please contact our support team at support@workforcegps.org.

Event has ended