When is the best time to evaluate your program?

Are we too late? Is this too early? When is the best time to start thinking about evaluation? Here's the short answer: the earlier, the better. However, any time you decide to systematically collect information about your program or effort is a good time. Today, we're going to look at three programs at different phases and how corresponding evaluation activities can support their work.

Scenario 1: We're still developing the strategies for our family support program and are pretty sure we're going to be making changes as we figure things out. Isn’t it too early to think about evaluation?

If you're starting a new program or service and anticipate that you'll need to make some adjustments or adaptations along the way, you may wonder if it's too early to engage an evaluator. Absolutely not! In fact, an evaluation can set you up to have critical data to make important decisions as your programming evolves. This is called formative evaluation and its key purpose is to inform and improve your strategies or activities.

Formative evaluation will answer these questions:

• “Are program activities being implemented as intended?”

• “Are we meeting the needs of our participants and seeing the outcomes we expect?”

• “Are we reaching the intended groups or individuals who can benefit most from our services?”

You'll need this real-time data during a project’s early implementation to make important decisions about service delivery. This iterative process of data collection, rapid feedback, reflection and action planning is commonly referred to as data-to-action.

Taking one step back to the program planning stage, you might have conducted a needs and resource assessment to identify the gaps needing to be addressed in your community and to inform strategy development. Surveys, interviews, focus groups and public records review are ways to collect this critical data that will inform your strategy development.

Scenario 2: We've been operating our youth center for a while, and our teens and other community stakeholders really appreciate what we do. We know we're effective. Why do we need to do an evaluation?

You may be in the enviable position of having a well-established program. Now is also a great time to conduct an evaluation. While you may be hearing anecdotally that your efforts are successful, that isn't reliable enough information to “sell” your program to outside stakeholders and funders. Evaluation results demonstrating positive impact will be necessary if you're looking to scale up a successful program. A summative evaluation can provide the evidence of the impact you're making on your community.

Your evaluator will work with you to design and test data collection instruments and processes, analyze the data, work with you and other stakeholders to interpret results, and prepare visually appealing and clear reports, briefs, infographics and video learnings of results. Both quantitative data (numbers) as well as qualitative findings (descriptive statements and accounts) can help you tell the story of your impact.

Scenario 3: We're at the end of the funding period for our substance misuse prevention program and just learned that we should've been doing an evaluation. Is it too late?

While it isn't optimal to begin thinking of conducting an evaluation at the end of the funding period, this isn't an impossible task. Retrospective methods are available to capture the impact you're making and help you plan for future implementation. For example, key informant interviews and public and program records are a few of the data collection sources that could be used to assess effectiveness and identify important learnings.

In these examples we've been talking about programs. The points made here also apply to efforts around policy change and implementation, social marketing campaigns, community engagement, collaboratives, etc.

What phase is your team at?

Whether your team is at the beginning, middle or final stretch of your program, Strategic Prevention Solutions will tailor an evaluation to your needs. If you'd like to learn more about how we can help, connect with us here.

Also, did you know we have a Planning & Evaluation Workbook? It comes in two formats: hard copy and digital download. This workbook helps you understand how prevention planning and evaluation go together. Concepts are explained clearly, so anyone can understand and walk away with a solid understanding of the basics. The workbook provides exercises for you to practice the concepts and gives the resources you need to bring these concepts to life in your programs.