Digital Inclusion Outcomes-Based Evaluation

This report describes the challenges facing community-based organizations and other key stakeholders in using outcomes-based evaluation to measure the success of their digital inclusion programs and offers recommendations toward addressing these shared barriers.

In recent years, government agencies, private foundations, and community-based organizations have increasingly sought to understand how programs that promote digital inclusion lead to social and economic outcomes for individuals, programs, and communities. This push to measure outcomes has been driven in part by a larger trend to ensure that dollars are being used efficiently to improve lives rather than simply to deliver services. According to the U.S. Institute of Museum and Library Services (IMLS), outcomes-based evaluation is the measurement of results: achievements or changes in skill, knowledge, attitude, behavior, condition, or life status for program participants. Outcomes-based evaluation

1. identifies observations that can credibly demonstrate change or desirable conditions,
2. systematically collects information about these indicators.
3. uses that information to show the extent to which a program achieved its goals.

In outcomes-based evaluation, a program is a series of services or activities that lead towards observable, intended changes for participants. Generally speaking, outcomes-based evaluation provides a way for programs that promote digital inclusion to understand the impact of the services provided. Outcomes can be understood at various levels, including: individual, client-focused outcomes; program and system-level outcomes; and broader family or community outcomes.

In the field of digital inclusion, an individual, client-focused outcome might be an older adult feeling more connected to her family members or a young adult gaining new job readiness skills as a result of their participation in digital literacy training programs. Whereas, a broader family or community outcome might be understood as increased civic engagement and participation in democratic processes or more collaboration between community-based organizations and other local entities, including local government. Outcomes-based evaluation is used to communicate the “theory of change” underlying a digital inclusion program as well as how a program’s activities can lead to broader outcomes at the individual, program, and community levels. This type of evaluation can be helpful for both funders of digital inclusion programs, policymakers, and for the organizations that offer digital inclusion programs. Outcomes-based evaluation can be useful in the program planning and implementation phases, as well as for measuring program outcomes over time. Therefore, this methodology is often iterative and requires organizations to engage in a continuous process of data gathering and analysis.