If you’ve ever worked in the education or health and human services field, you've probably heard the word “evaluation” thrown around. It’s one of those terms that many people might know of, but wouldn’t be able to define it if they were asked to. If you're unsure of what evaluation is or what an evaluator does, that’s OK! We’re here to help.
Fun video makes it easy to understand
In the video Eva the Evaluator by Wendy Tackett (below), Eva is an evaluator who is describing to her confused child what she does for a living. Imagine that! A video to explain to your child what you do for work. Can my family describe what I do? Well, probably not.
So why is it so hard to describe what an evaluator is and what they do? It's because evaluators wear multiple "hats" to get their work done. Personally, I see myself as a researcher, detective, navigator and storyteller all rolled into one. Whew! That’s a lot of roles.
Defining evaluation
Let’s look to a definition of evaluation for some clues about why so many job functions are needed to be an evaluator.
Evaluation is the process of gathering information to provide useful feedback about something.
Now, let's focus on a couple of key phrases in this definition.
1. Gathering information
An evaluator gathers information, but how do we know what information to collect? We ask questions – really, a lot of questions! This is one way to ensure that we're creating an evaluation in partnership with program staff that also is responsive to your community.
Let’s say you want to evaluate a three-hour training workshop with teachers and school administrators (or parents or youth) on how to create a safer and more connected climate on campus. We'd work together to identify key learnings, consider factors related to the training format, understand characteristics and perspectives of participants, and prioritize the best use of everyone’s time and resources. Given all of this, we would identify our burning questions about the workshop. What did participants value about the time they spent training? What will be the impact of the workshop for the school community?
Next, we dive deep into our evaluator’s toolbox to decide on the best ways to collect data to answer your evaluation questions. While people generally think about surveys first, we know that sometimes there are more suitable ways to collect information, such as interviews, focus groups, results of workshop activities and even stories or photographs.
We put all of these together into a written evaluation plan, which also describes the data entry and analysis steps, including considerations of confidentiality of data collection, storage and sharing. Your evaluator will either directly collect the data or coach you on data collection.
2. Provide useful feedback
Nothing will bring an evaluator to tears quicker than learning that the results of all of this effort have been relegated to the figurative or literal bookshelf. After spending immense amounts of time and energy, we all want to ensure that evaluation findings are used to inform a program’s activities and strategies. Your evaluator will work with you to guarantee the evaluation can provide data that you can utilize.
Returning to the example of training school staff, you probably would want to know at the end of the workshop what additional support teachers and administrators require to implement the changes they seek to make. A question identifying those opportunities would be included in the data collection. Provided with those results, your staff can then meet identified needs with additional technical assistance.
Now let's fast forward. We're a year down the road and you have collected some follow-up data about new school policies and practices. Storytelling comes into play to share the ways the workshop and your follow-up support contributed to a safer and more connected school community. The evaluator would work with you to identify optimal approaches to deliver evaluation findings to different community groups and stakeholders (e.g., students, parents, school board or the local newspaper). This may be in the form of a one-page brief, an infographic, a visually engaging report or even a themed video. Take a look at our 'Get The Word Out' handout on our resources page for more ideas.
A typical day for an evaluator
So what might a day in the life of an evaluator look like?
9:00 a.m. – Review wording of online focus group questions
9:30 a.m. – Update evaluation plan with COVID-19 adaptations
11:00 a.m. – Conduct a webinar on how to create and use logic models
1:00 p.m. – Meet with a client to identify potential program outcomes
2:30 p.m. – Design a one-page infographic of pre-post survey results
4:00 p.m. – Draft evaluation findings section of progress report for client to submit to funder
We've got you covered
I hope this blog post clears up for you what an evaluator does and how programs can benefit from their services. Please contact us to discuss how Strategic Prevention Solutions can work with you to gather information, provide feedback to improve your program and tell the story of its impact.