Concerns Over Restructure to Social Policy

Labor’s promise of an overhaul to social policy making through an evaluator general has been welcomed by the sector, but some say it could be costly and not effective for programs across the board.

Shadow charities minister Andrew Leigh pledged on Tuesday to create an Office of an Evaluator General within the Treasury, which would thoroughly evaluate government programs to ensure they were working.

Leigh said this would preferably be done through randomised trials, whereby one group is exposed to the new program, and another stays with the existing policy.

He said it would help fund programs making progress on big social challenges in Australia.

“The test-learn-adapt approach is vital to making progress on major challenges such as wage inequality, social mobility and Closing the Gap,” Leigh said.

“At a time when government budgets are under pressure, there’s no excuse for continuing to fund programs that don’t work.”

Gemma Carey, research director at the Centre for Social Impact, told Pro Bono News Labor were making an important investment and said for NFPs working in those policy areas, it would save them time and money.

“Many of those organisations are doing a lot of their own evaluation which costs them. I’d hope that these programs would be evaluated within government when they are contracted to do work, so they don’t have to do it themselves,” Carey said.

“This takes the burden off those charities and NGOs.”

Carey did raise concerns however over the testing method, and said random control trials were very expensive to run, and not always suitable for evaluating all social policy programs.  

“Randomised trials don’t actually always capture everything they need to, and don’t take into consideration that things might work for some groups and not others,” she said.

“If they are run with a medical evidence based policy school of thought… you could actually see things being defunded that might have been working for particular minority groups.”

She said this could then increase work for advocacy organisations who then have to battle their own findings with government.

CEO of Community Council Australia, David Crosbie, told Pro Bono News CCA welcomed any measure that improved the link between agreed policy goals, and the actions of government particularly in terms of what does or does not get funded.  

“Better evaluation and a stronger focus on outcomes and impacts can only strengthen government performance,” Crosbie said.

But he also raised issues with the research method, and said there were limits to randomised trials.

“Real expertise and understanding of impact often lies outside of standard approaches to program evaluation… these limitations need to be carefully considered in developing this welcome new initiative,” he said.

In a speech at the Australian National University, Leigh used various examples of randomised trials which were inexpensive and effective, such as the Chatterbooks program in the UK, which found a government program that hosted after school reading sessions at libraries was not effective.

But Carey said in programs overseas, results of random trials became a battleground between advocacy groups and government over which truth was the most accurate.  

“It was a fight over who had the truth and the best truth. You start with this idea that it really might seem simple and straightforward, but actually it still can be very contested,” she said.

She encouraged researchers and the social sector to keep a close eye on how it was rolled out, and hoped the sector would be consulted before anything was properly implemented.  

Leigh said the initiative would cost $5 million per year, starting in 2019-20.