A Sector Made to Measure

Chris Lee is a humanitarian professional currently posted to Jordan with UNHCR. In this post from the CSI Blog, Lee shares his thoughts on measuring social impact in the Not for Profit sector. 

I write this blog from Jordan, where I am working with UNHCR at a Syrian refugee camp. It’s hard to believe that only a couple of weeks ago I was at a Sofitel hotel in Sydney discussing social impact measurement – and yet, it’s a very relevant topic here when we’re dealing with huge need and limited funding, making efficiency and effectiveness of response really critical.

Measuring for value, not for profit

A sector defined by what we’re ‘not’. Odd. And an absurdity pointed out by Jeremy Nicholls at the Measuring Social Outcomes conference held in Sydney in February.

The disparity in approach when it comes to measurement between the commercial and the social sectors has always been there, and I’ve always known money was at the heart of it – but Jeremy Nicholls really hit the nail on the head when we spoke about ‘value’ – specifically ‘social value’, as opposed to a definition by what money we don’t make. Irrespective of the ‘non-profit’ nature of the social sector, and in fact particularly because of it, measurement is absolutely central to planning, budgeting, reporting and all conversations within such enterprises – and they are very much enterprises, even if they don’t generate profit. If shareholders merit high performance, how much more so those whom we serve.

Principles v boundaries

Part of Jeremy Nicholls’ presentation was talking through Social Return on Investment (SROI) and more specifically talking about the principles underpinning SROI as opposed to talking of a ‘tool’ called ‘SROI’. As with any discussion about frameworks, there were differences in the room about who was married to which framework, and some academic discussions about ‘lots of different approaches’which was less than helpful.

Criticism of SROI as a ‘tool’ can only confuse people trying to introduce social impact measurement in their organisations, and we should be beyond this level of debate by now.

Jeremy Nicholls and Kevin Robbie in particular showed how it wasn’t about competing tools, but a common philosophy about what matters most in driving the greatest impact with limited funding for those in need whom we each try to serve. We have to focus on getting results, not about competitive ‘branding’.

SROI attempts to bring together clear thinking about the challenge of improving performance and increasing funding, and we should now be focused on helpful ideas and techniques, and shared experience to lift our joint capability.

That is why I quietly cheered when Mark Reading used the expression “I am greatly troubled…” in relation to Gianni Zappala’s criticism of SROI and resistance to a standard approach. Why? Because I believe Mark Reading and Jeremy Nicholls are dead right in refusing to accept that we cannot achieve a similar, though clearly not identical, conformity to that achieved in financial accounting. Mark so rightly pointed out that there is plenty of judgement in financial accounting, and analysis and quantification do not displace judgement.

Measurement: a worthy endeavour

It was great to hear Geoff Rowe’s case study of work done at the Endeavour Foundation, where he analysed and allocated costs so as to understand unit costs. This is absolutely essential – in parallel with valuing outcomes – if we are to understand and therefore optimise value for money.

You have to have both a value of outcomes and their cost of delivery, and there is a danger that in our passion for measuring social impact we ignore the necessity for analysing costs. Indeed, demonstrating impact can divert donor and management attention from cost consciousness.

Jennifer Duffecy’s description of her work with Results Based Accountability at Anglicare WA gave another important perspective. This seemed a very practical way of focusing people’s efforts on what matters most and driving performance. Linked to the measurement of outcomes, it provides the improvement mechanism that is needed to complement a focus on value.

Rebecca Scott was inspirational – pragmatic and passionate, with a determination to use measurement directed to areas where STREAT could make most difference to the young people they aimed to serve. For her it was personal. They couldn’t afford not to measure and optimise.

What I wish for the world…

It was a pity that no one spoke in relation to international relief and development. So much of the inequality in the world (as Jeremy Nicholls pointed out) is across international borders. There is already huge funding for international aid, but it is paltry in comparison with the money spent in other areas of the world economy – and so much of it is spent inefficiently and ineffectively.

Even a small improvement in value for money would make a big difference to a lot of people and their families. More than in domestic NGOs, in INGOs there is a strong sense of ‘we know we’re doing good, so we don’t have to worry about doing better’.

This is real here at the Zaatari camp where I’m working, with some 100,000 displaced people, arriving at the rate of some 2,000 per day. People shouldn’t have to be put through this. Efficiency and effectiveness in all our work are not academic issues. They determine how many people don’t get the chance for a fair go.

Perhaps World Vision, who had five participants, and AusAID with two participants, can mobilise much greater participation in the future.

The impact on me

Firstly, I really appreciated receiving a scholarship to attend the event – without which I wouldn’t have been there.

I believe we need three elements for effective outcomes measurement:

1. Understanding value

2. Understanding cost

3. Having a mechanism for making improvement happen

The first two elements are worthless unless translated into action.

Perhaps the main message from the Measuring Social Outcomes conference for me was that those who say it can’t be done, should get out of the way of those who are doing it – and that we should now put all our efforts into learning and improving by doing.

I’ll look forward to the next conference, with 500 participants, and a series of case studies about what people have done and how it’s improved their operations, as well as increased their funding. 
 

About the Author: Chris Lee is on the standby register of RedR Australia for deployment to humanitarian emergencies, as a logistician and water, sanitation and hygiene engineer, and is an associate trainer with RedR. He is currently deployed with UNHCR in Jordan as a supply officer at a large Syrian refugee camp. He previously worked with Oxfam and with the UN's Humanitarian Air Service in Somalia.