Evaluating the effectiveness of your program, intervention, or policy is an important part of understanding its impact and ensuring its success. Evaluation helps to identify what's working well, what could be improved, and whether your initiative is meeting its goals. For many practitioners, understanding the difference between the different types of evaluation —and which to use— can be challenging. In this blog, we’ll explore the different types of evaluation, potential barriers to successful evaluations, and why this matters.
Evaluation Types
There are four main types of evaluation: Formative Evaluation, Summative Evaluation, Process Evaluation, and Outcome evaluation. The sections below detail what those are and how to choose the best evaluation type for your project.
Formative Evaluation
Formative evaluation is a type of evaluation used in the early stages or development of a program or initiative. This type of evaluation assesses the progress and effectiveness of a program while it is still being implemented. Formative evaluations are often used to make early improvements or adjustments to improve the program, evaluate the quality of the program design and implementation strategies, and ensure that the program is aligned with its intended goals. Community needs assessments done ahead of or in the early stages of a program or project is one example of a formative evaluation.
Implementing a formative evaluation can help practitioners ensure programming begins with a solid foundation. By identifying potential issues or areas for improvement early in the process, formative evaluations help practitioners and policymakers make necessary adjustments before an initiative is fully implemented. In this way, formative evaluations help to ensure that initiatives start on the right track and have a greater chance of success.
Summative Evaluation
Summative evaluations are often used to determine whether the program achieved its intended goals and objectives by assessing the overall effectiveness of a program after it has been completed. Summative evaluations collect information from multiple sources over time to demonstrate the evidence of the program’s effectiveness. Summative evaluations are often best used for multi-year programs––where there is sufficient time for practitioners to make adjustments learned from formative evaluations. Summative evaluations can be helpful when deciding whether to continue, end, or expand a program. While summative evaluations are done after a project is completed, the planning for the evaluation needs to start before the project – you need to make sure you have a plan for what you’re going to evaluate and the impact you want to see ahead of time so you know if you’ve achieved your goals.
Process Evaluation
Process evaluation is used to assess how a program is being implemented, including factors such as participation rates, the quality of delivery, and the degree to which the program is being implemented as intended. A process evaluation explores how a program or initiative reaches its short and long-term goals. Unlike summative evaluations, process evaluation focuses on the incremental steps along the way. This type of evaluation focuses on if the program was implemented as planned, if it is reaching its intended audiences, and producing the desired outputs. Process evaluation is mostly quantitative in nature, as it focuses heavily on counts, frequencies, or averages. For example, the number of program participants or the resources used to deliver a program. However, it can be supplemented with qualitative information, such as participant feedback. This type of evaluation should be used at each stage of a program or initiative on an ongoing basis.
Outcome Evaluation
Outcome evaluations focus on the effectiveness of a program in producing change. This type of evaluation is used to assess the impact of a program or the intended outcomes on participants. Outcome evaluations often involve measuring changes in participants' behaviors, attitudes, or conditions as a result of the program. This type of evaluation is used to assess the overall impact of a program or intervention on the individuals, communities, or society as a whole. Outcome (sometimes also called “Impact”) evaluations often involve comparing the outcomes of the program or intervention to a control group or baseline. Outcome evaluations are summative in nature as they look at the program from an overall perspective at the end.
Other Evaluation Types
There are other types of evaluation as well, such as Economic Evaluation (e.g., cost-benefit analysis) or Performance Evaluation (e.g., organizational assessment)! These are less common. If they do apply to the project you’re working on, feel free to contact us and we can help you plan a project using one of those!
How do I know which evaluation types to use?
Each type of evaluation has its own strengths and weaknesses, and the choice of evaluation method is often influenced by the evaluation question, available resources and the evaluation context. Below is a chart that describes the evaluation types, when it is used, potential evaluation questions, and methods to achieve it.
Evaluation Type | Why it is used | When | Evaluation Questions | Methods |
|---|---|---|---|---|
Formative | To make early improvements, evaluate the quality, and to ensure that the program is aligned with its intended goals. | At the beginning |
| Survey Focus group Interview Document review |
Summative | To demonstrate the effectiveness of a program | At the end |
| Survey Focus group Interview Document review Case study |
Process | To explore how a program was implemented | Ongoing |
| Survey Counts tracking Document review |
Outcome | To understand the longitudinal impact across individuals during a certain timeframe | At the end |
| Survey Focus group Interview Document review Case study |
Barriers to successful evaluations
While evaluation can sometimes be easy and completed without any challenges, there are sometimes barriers that affect effective evaluations. Evaluations take a pretty significant allocation of time, financial resources, and technical expertise–– which are often seen as resources that could be better utilized for programmatic goals aimed at serving clients. Often, there is a lack of dedicated funds specifically allocated for conducting evaluations. Program staff have been seen to frequently worry that engaging in evaluation activities will hinder timely access to services or compromise the quality of services provided to clients. In many organizations, there is no designated individual responsible for overseeing the evaluation process, such as a project evaluator. Additionally, depending on your program, concerns may arise regarding data collection or data sharing due to perceived issues related to HIPAA compliance. Finally, evaluation may require forging new partnerships, sometimes between disconnected community organizations or entities, which in turn takes more time to intentionally build relationships to ensure a successful program AND evaluation.
Why should I do an evaluation?
If there are so many barriers, why do evaluation in the first place? To start, evaluations are often required by funding sources to assess the effectiveness of programs and justify the allocation of resources. Through evaluations, organizations can provide evidence for their program expenditures, showcasing that resources are being utilized efficiently and effectively. Evaluations also play a crucial role in securing continued funding by providing support and evidence of the program's impact and value–– which can aid in increasing the likelihood of new or ongoing financial support. By conducting evaluations, organizations can learn to improve managing their limited resources by identifying areas of improvement and approaches to resource allocations.
Evaluations also provide valuable insights and data-driven information that can significantly improve decision-making processes, guiding strategic planning, resource allocation, and programming adjustments. Through evaluations, organizations and their partners can ensure that successful programs can be replicated or expanded in other with other populations or locations; enhancing the potential reach and comprehensiveness of programs. The findings and results of evaluations can also be utilized for marketing and promotional purposes by showcasing the program's effectiveness and impact to external stakeholders, potential funders, and the wider community. For example, sharing evaluation findings from a youth program with prospective parents during recruitment is a strong approach to gaining new participants and support. Sharing evaluation findings can enhance the organization’s visibility and support for its mission.
Conclusion
Evaluations are crucial for assessing program effectiveness and understanding impact. There are four main types of evaluation: formative, summative, process, and outcome evaluations. While barriers such as limited time, resources, and partnerships exist, evaluations provide valuable insights, support resource allocation, and aid in securing funding. They inform decision-making, enable program expansion, and serve marketing purposes. Despite barriers, evaluations are essential for optimizing programs and enhancing impacts for the community.