Practical evaluation for greater impact

Evaluation doesn’t always need to be left to “the experts”: anyone can use it to boost the impact of their work, write Kate Randall and Mark Planigale from Lirata Ltd.

Lirata has been evaluating a wide range of projects and services for more than a decade. In this article we share our top tips for evaluation design. We invite you to join our practical training on key Monitoring, Evaluation and Learning (MEL) concepts and tools, and learn how to use them in your context to turn data into powerful insights.

Why MEL?

Have you ever been in any of these situations?

Drowning in data: You are completing a project and suddenly remember you need an evaluation report for your funder. You pull together various data that’s available, then spend days tearing your hair out trying to work out what it means, and which bits are useful for your report. Despite the mountain of information available, it still doesn’t tell the real story about what difference your project has made.

The sound of silence: You have a great program evaluation plan, but when you try to collect data from your participants and stakeholders, hardly anyone responds.

Not quite on the same page: You’ve been funded for an innovative pilot project, and you’re recruiting a new team to deliver it. But once everyone’s on board and the work is underway, you realise some staff have a different understanding of what we’re trying to achieve. Changes are made “on the fly”, and project scope starts creeping.

Great idea, can’t get funding: Your pilot project was a great success. But somehow, your funding applications to expand the program keep getting knocked back, over and over. It’s like the funders don’t understand what you’re trying to achieve.

These are all too common scenarios, for which MEL offers real solutions.

The power of MEL

MEL is about gathering the insights we need to demonstrate outcomes, improve programs and services, and advocate for change.

We believe MEL should be part of the toolkit of every well-run social purpose organisation, just like budgeting, strategic planning and staff development.

Here at Lirata we specialise in MEL, working in partnership with organisations across many sectors and social issues. We’ve seen first-hand the positive difference MEL has made, including:

• Testing new models and pilot projects to deeply understand what works and why, and to identify where to improve or adapt.
• Demonstrating outcomes – highlighting the difference we’re really making.
• Garnering support and influencing change through evidence-informed engagement, fundraising, policy and advocacy.

MEL creates knowledge, and knowledge is power. The power to make smarter decisions, tell a compelling story, and better achieve our purpose.

Making the most of MEL – top tips

We consistently hear two barriers to effective MEL – time and capability. Even when time and resourcing is available, people often lack key skills or confidence – MEL is seen as too specialised, not accessible to people working in program delivery or management. 

While there are certain aspects of MEL that benefit from advanced expertise, we also believe MEL works best when the people delivering programs are involved in evaluating them. That’s why we’re on a mission to build the social sector’s understanding of MEL concepts and tools, and confidence to use and adapt them as needed.

Tip #1: Define it before you measure it

Useful MEL is based on a clear and agreed description of the program that’s being evaluated. Before rushing to evaluate, first make sure the logic behind your program is well documented and understood – how it works, the outcomes it aims to achieve, and what success looks like. This is key to designing effective programs and sensible evaluations, and also incredibly useful for getting stakeholders on the same page.

We suggest two tools: program logic, and/or theory of change. 

One of our recent partners reflected on their experience using these tools to define a pilot project model:

“I felt like we really got down into the heart of what we were trying to achieve. Away from those ‘broad statements’ we often fall back on, and into the details of what sat behind it –
how the initiative would work and where it needed more thinking.” – Manager, homelessness program

Tip #2: Focus on what’s most important

A common dilemma we hear is “How do we know if we’re asking the right questions?” Many organisations aren’t sure where to start with MEL data collection, or there’s so much they want to know it seems overwhelming. We can’t evaluate everything, so prioritising is key – by developing an evaluation framework that’s realistic and scaled appropriately for the program.

Our frameworks centre around a handful of key evaluation questions, focused on the top priority outcomes (or other elements) in the program logic or theory of change. Evaluation questions provide clear boundaries for data collection, and a recipe for designing efficient data collection tools – filtering out any low-value information. This means every piece of data collected is useful and important, and stakeholders are not fatigued (or annoyed) by too many questions.

Tip #3: Nothing about us without us

Super-charge your MEL design and planning by including stakeholders in the process. Who will be affected by, or have a role in, MEL data collection and decision-making – staff, management, program participants, partners, funders, any other key stakeholders?

Facilitating participatory MEL processes enables people to have a voice in MEL design and offer their perspective on your program. This nurtures buy-in, ensures MEL findings are relevant, and helps avoid nasty surprises at reporting time.

Strengthen your MEL skills

Our practical training provides you with the knowledge and skills to design and use MEL effectively and efficiently, to get the information you need.

Our interactive workshops use real projects as working examples, with a strong focus on “learning by doing”. Learn step by step how to design robust program and evaluation plans, develop fit-for-purpose data collection tools, successfully gather data, and synthesise it into useful insights.

Five online modules commence May 2023. Find out more and register today to secure your place. 

This training is also available for in-house delivery: contact us to enquire.

About Lirata

Lirata is an independent not-for-profit organisation based in Naarm (Melbourne).

Lirata is a social justice multiplier. Through consulting and capacity building, we strengthen the ability of organisations to achieve socially just outcomes for communities. 

Lirata partners with social purpose organisations to strengthen program evaluation and review, impact measurement, data systems, strategy and organisational capabilities. 

For further information, please contact us on (+61) 0407 314 716 or email contactus@lirata.com