Australian Charities Measuring Outcomes Better

Charities are measuring outcomes better than ever before, according to the largest study into outcomes measurement ever undertaken of the Australian community sector.

The Bankwest Foundation in partnership with the Centre for Social Impact at The University of Western Australia (CSI UWA) released the latest findings in its five-year research program Supporting Development and Growth in the Community Sector in Western Australia on Friday.

The report, Australia’s first large-scale study of its kind, aims to increase understanding of the difference community programs make to the lives of Western Australians and what factors enhance the long-term financial viability of the state’s emerging social enterprise sector.

Co-author of the study and director of CSI UWA Professor Paul Flatau told Pro Bono Australia News that measuring impact was important and the research revealed it formed part of the organisations’ mission.

“We found that the majority of community sector organisations have significantly increased their focus and effort on outcomes measurement in the last five years which is one of the most positive signals yet that the sector is undergoing a cultural shift towards measuring impact,” Flatau said.

“The mission of community sector charities is to make a difference and measurement allows community sector charities to know whether they are positively impacting the lives of Western Australians. The report found that the principle driver to increased measurement in the community sector was an organisation’s own drive to achieve a positive social impact for people.

“So if it is to make a difference in terms of mental health, if is it make a difference in terms of unemployment or homelessness or whatever it is, they want to know that what they are doing is achieving their end goal.

“Importantly too is the question of accountability and strategic planning, accountability to funders, planning in terms of what is working and where should you be moving resources, and of course accountability to the people that they serve as well. So I think those elements are fundamental; planning, accountability, are you doing the things that you are setting out to do.”

Anglicare WA CEO Ian Carter said a focus on outcomes or results for their clients was a key part of them moving forward.

“We use results based accountability as a core of this work and we reflect on these results as we look at ensuring that we provide the best services we can for our clients across WA,” Carter said.

One of the key findings of the report was that while most organisations believed they measure their outcomes well, there was also consensus that more needed to be known of the overall impact of the sector.

Flatau said the difference between how organisations rated their own understanding relative to the community sector pointed to a need for a common understanding for the community sector as a whole.

“So if we are an organisation that is in say community mental health or unemployment… what is happening in aggregate across all of our programs around community mental health or around unemployment, are we achieving a difference at a broader population level. That’s one thing that came to mind. And the desire that we should be doing more of that. The more population level social outcomes, social indicators, benchmarks kind of work.

“The other thing that comes to mind is are we actually setting ourselves up for that, in terms of measurement systems.

“People are working in silos, they are working as independent organisations or they are working as funding departments and they are not bringing together the whole system if you like, in terms of measurement. It is all a fragmented world, bits and pieces are doing their own thing. And I think the point that came out there was we need to have a common infrastructure, a common platform, a common way of reporting, a common understanding of exactly where we are going in terms of community outcomes.

“The method of outcomes measurement should be a more shared kind of measurement system and should be producing results at the population level that all different kinds of funding departments and different kinds of sub sectors are working within.”

Flatau said there was also a real opportunity for increased funding.

“Across the board, specific funding for outcomes measurement is still lacking and there’s real opportunity here,” he said.

“Organisations are reporting that outcome measurement in the main is funded from internal, general funds… three-quarters of the outcomes measurement is actually funded by internal funds, general funds, another 7 per cent is funded from a special evaluation measurement fund that they create internally and it is only 17 per cent that has that external funding component.

“So that’s an interesting finding and the findings that follow that, really do suggest that funding for outcomes measurement is nowhere near what it needs to be.”

Other key findings from the report included:

• A majority reported increasing their focus and effort on outcomes measurement in their organisation in the last five years.
• The majority of community sector respondents measured outcomes for all or most of their activities.
• Outcomes measurement practice differs between smaller and larger organisations across several facets.
• When compared to smaller organisations, larger organisations were more likely to measure outcomes, twice as likely to report that high priority was placed on funding agreements in setting the organisation’s outcomes, and were more likely to have increased their effort towards measuring outcomes over the last five years.
• When small organisations do measure outcomes, they measure them as intensively, if not more so, than larger organisations.
• Both smaller and larger organisations report that funding for outcomes measurement is lacking.
• Despite increased expectations for outcomes measurement from funders, very few organisations reported receiving funding specifically for it. Further, even when external funding is received the majority of outcomes measurement is reported as being funded from internal organisational sources.
• There were very few significant differences in outcomes measurement practices between metropolitan and non-metropolitan based organisations. The exception to this was with regards to some of the resources available for outcomes measurement; metropolitan organisations reported significantly greater access to training for staff in data collection, and greater access to technology assisted data collection, specialised data collection staff, and an internal measurement and evaluation unit, though the differences for the latter three were not large.

Flatau said the report was important as it confirmed a number of trends that they thought were happening but previously did not have evidence for.

“So we have had reports of Not for Profits and charities talking about the fact that they are not funded to measure outcomes but where is the evidence of that? Well we now have the evidence of that,” he said.

“There is also questions like to is there a difference between larger and smaller organisations and we certainly see that difference strongly, we also see some interesting findings that we probably wouldn’t know which way it would go, because if you look at the smaller ones… sure less of them do outcomes measurement but when they do do outcomes measurement they are just as strong in terms of their focus as large organisations, that was an interesting finding.

“There are a few interesting finding there… It is putting numbers to things that you have a feeling about but not quite sure.

“This survey as far as we can tell, this is the most significant survey of outcomes measurement in Australia, so it is certainly unique in putting numbers across a very broad range of measurement questions in Australia, and it is pretty extensive relative to US, Canadian, UK surveys… this is a continuing piece of work with the Bankwest Foundation.”

The report marks the fifth publication from the Bankwest Foundation Social Impact Series, and the third report in the research stream of Measuring Outcomes for Impact in the Western Australia Community Sector.

WACOSS CEO Irina Cattalini said the research highlighted the importance of taking a systemic approach to investing in outcome measurement systems.

“Both so that we get good value for money from the investment going into measurement now across individual organisations and so that we can derive meaningful insights from joining up the outcome measures at population levels,” Cattalini said.