Case Study: Selecting Effective Public Benefit Interventions
September 10, 2021 | By David M. Wagner | Download this article as a PDF
It is often easier to reach consensus on broad public benefit goals than to find agreement on how to achieve those goals. How do you design an effective program to achieve those goals and gain support for a chosen set of interventions? This article highlights a few techniques that may be helpful, drawn from a project that sought to recommend solutions to address a known public need.
Key Take-Aways
Use well-defined problem statements to select potential interventions relevant to your mission and rule out those that aren’t.
Survey what others are doing to understand what interventions are working, build on effective program models, and avoid duplicating efforts.
Look to the past for potential options based on what worked well, and what didn’t, to address similar challenges.
Explore a wide variety of options to strengthen your program design, including options that run counter to working plan.
Model the problem space to understand influence points and gain better insights into what interventions may be the most effective.
Project Background
This article draws from a project performed for a government agency charged with closing the cyber workforce gap in the United States. The agency had a preferred solution but wanted to understand how it could be implemented effectively and fit into the ecosystem of existing programs and services. The team charged with recommending an approach was given latitude to identify and analyze options for a program design. [i]
Problem Statements
This project had a succinct problem statement: What government interventions would effectively increase the size of the cyber workforce to close the gap with unmet demand? [ii] This was an effective problem statement for two important reasons.
First, the problem statement set appropriate, high-level bounds for identifying interventions that would be in scope for consideration. This project was about increasing the size of the workforce, which ruled out other ways of closing the workforce gap, such as by reducing employer demand with automated tools. Further, the team was focused on solutions a government agency, rather than a private sector entity, could implement.
Second, the problem statement was outcome-oriented, rather than solution-oriented. This created opportunities to identify a wide array of potential program approaches, including novel ideas that may not have been considered originally. Increasing the size of the cyber workforce is a desired outcome, not a solution. A problem statement that specifies a solution, like “How might a program increase interest in cyber among high school students?,” constrains the available options, perhaps too narrowly.
Careful observers will note that these two factors seem contradictory. A good problem statement both constrains, and broadens, the options to be considered. The trick is to find a balance and limit the options by what’s achievable, but not based on an assumption of how the ultimate goal should be reached.
Identifying Options
The team relied on three methods to identify options for constructing a program to increase the size of the cyber workforce. The first method was to explore the preferred solution that the client agency had identified. Two other methods were effective in expanding the set of options.
First, the team surveyed existing programs and services targeting the same problem. Some approaches that were identified, such as apprenticeship programs, added to options under consideration because they had not been widely adopted in the U.S. Research uncovered existing programs that helped to limit or rule out certain options. For instance, the team found that cyber curriculum guidance was available and supported by influential institutions. This meant that a new program would be most effective by supporting existing curriculum guidance, or by avoiding the topic entirely.
Second, the team looked to past approaches for addressing similar challenges to understand what had and had not worked. In this case, the range of responses to the shortages of nurses, machinists, and other skilled positions during and just after World War II were informative. Many past interventions had parallels in responses to today’s workforce shortages, such as concerted efforts to recruit women into the workforce. Others programs, like government-run job training schools, suggested approaches that would be novel in the cyber realm.
The primary goal of these explorations into current and past interventions was to develop a set of options worth exploring for a new program. A secondary benefit, however, was learning along the way which approaches had been successful, which had not, and what the points of influence were.
Modeling the Problem Space
After identifying program options that could be pursued, it is necessary to evaluate what interventions would most effectively accomplish the public benefit goal. However, public benefit programs can take on wildly different forms. How does one even compare alternatives like normalizing the language used in cyber job postings to an approach aimed at making portrayals of cyber workers more realistic in TV and movies? To tackle this challenge, the team developed two simple models of the problem space.
The first model identified points of influence in the experiences of the intended constituents of the program. For this project, the team modeled a “pipeline” of individuals who could become (or remain) cyber workers. The model included key points in an individual’s professional development: their choice of electives and post-secondary education or training in high school, internship or apprenticeship opportunities, the job hunting and application process, career re-training opportunities, etc. Each of these was an influence point – a choice that helped determine whether an individual would become or remain a cyber worker. The team organized program options according to how they might intervene at each of these influence points. An added benefit was that identifying the influence points also inspired ideas about interventions that had not yet been considered.
The second model was a data-driven model to help quantify the potential benefits of program options. The team relied on public government data and several published studies to model the size of the workforce, demand for cyber workers, and the gap between those values. Some findings were surprising. For instance, the team found that the true cyber workforce gap was only in the low tens of thousands of workers, much lower than had been projected a few years earlier. Most importantly, combining the quantitative and influence points (pipeline) models produced new insights about what interventions might be most effective. For instance, the team found that investing in youth-oriented programs would have an outsized influence on workforce sizes, and that retraining programs needed to focus on adults from outside the information technology field to close the gap.
The key to this project may have been in combining a model of how a program could intervene in the desired outcome with a model that predicted the impact of those interventions. Public policy modeling is an art worthy its own future article. The key is to have a way to explain how new interventions would fit into the ecosystem and what impact they are expected to have.
Summary
Whether you are expanding an existing program, exploring a new one, or trying to guide or influence the direction of public policy broadly, it helps to (1) have a clear articulation of what problem you want to solve, (2) evaluate a wide range of options that could be pursued, and (3) build a model to understand how candidate interventions contribute to your goals. This case study shows how each of these steps proved valuable to a project exploring methods of increasing the size of the cyber workforce.
Please let us know if you find this framework useful for evaluating options for your particular public benefit goals.
[i] The author led this team with a previous employer. Clear Mission Consulting, LLC, was not involved in the project.
[ii] For more on the cyber workforce gap and some examples of existing efforts to address it, see James Lewis’ excellent piece published by the Center for Strategic & International Studies.