Resisting robo-governance

Dhakshayini Sooriyakumaran raises a red flag on the issue of robodebt and how the government is expanding similar AI experimentation across the social welfare sector.

The Morrison government is fighting to keep documents about robodebt from public view. However, there is a lot of incriminating information already publicly available. This article briefly recaps the top five things this government does not want us to know about robodebt: 

1. Robodebt still continues on today, minus illegal “income averaging”

In 2016, Centrelink launched robodebt, a computerised algorithm that automated data matching and the issuing of debt claims to social welfare recipients. The volume of debt claims issued vastly increased from an average of 20,000 per year to 20,000 per week. 

By 2017, there was a groundswell of community outrage, activism, and journalistic investigation. Robodebt became the target of multiple Senate Inquiries and legal cases, including the largest class action in Australian legal history. 

One of the numerous heartbreaking stories of death and pervasive harm unearthed was that of 28 year old Rhys Cauzzo:

(Trigger warning: suicide)

“But then the letters and phone calls started. 

Dun and Bradstreet Collection Services:

8/11/16 Letter of Demand for $17,319.58

15/11/16 Letter of Demand for $10,283.81

28/11/16 Letter of Demand for $17,319.58

29/11/16 Final Notice for $10,283.81

29/11/16 Final Notice for $17,319.58

6/12/16 Final Notice for $17,319.58

3/1/17 Card in letterbox demanding contact

The above notices and continued harassment, both written and phone, is clear. This continuing pressure ultimately caused my son Rhys to commit suicide on Australia Day 26th Jan 2017.”

― Letter from Jennifer Miller (mother of Rhys Cauzzo) read at the Senate Community Affairs References Committee on Centrelink’s compliance program.

To this day, Service Australia is yet to make a statement about whether the debt of Miller’s son, Rhys Cauzzo, was “income averaged” – a method which used Australian Tax Office (ATO) pay information to calculate a debt, by dividing a yearly income summary into fortnights, and data-matching the results against welfare recipients’ reported income to Centrelink.

This demonstrates that the impact of robodebt is not simply about the inaccuracy and illegality of the algorithm, but the fundamentally punitive service paradigm of the social welfare system. Despite public exposure of government mismanagement, obfuscation, and lack of sufficient redress for victims, the use of data matching and coercive debt collection continues today. 

2. Robodebt (and welfare fraud) is really about criminalising the poor

Robodebt failed to raise the intended $4 billion in Centrelink debts. Rather than being a lucrative source of revenue, it cost the government over $2 billion, including the settlement to victims. 

Since the early 1970s successive governments have justified welfare fraud “innovations” such as stricter compliance measures, data-matching, covert surveillance and video recording, debt collection strategies and asset forfeiture, advertising prosecutions and convictions, and public tip-off lines, based upon the promise of recovering debt from “criminals seeking to exploit the Australian welfare system”. Carefully crafted policy narratives about welfare dependency (such as the “dole bludger” or “welfare cheat”) have successfully convinced the public of the need for coercive policies, despite the fact that welfare fraud is quite rare. Specific and intersecting populations, such as First Nations people, migrants, and single mothers, are marked out for punishment in the public imagination. 

The government recently announced in the budget papers that it will continue to invest $23.8 million in welfare fraud detection through Taskforce Integrity. The criminalisation of those in poverty is set to continue. 

3. Robodebt was just the beginning

Concerningly, the social welfare system, and the marginalised people with whom it interfaces, are currently being subject to widespread experimentation by the government’s Digital Transformation Agency. This includes “robo-planning” (algorithmically generated budgeting for NDIS supports alongside the controversial independent assessments scheme); the trial of an NDIS blockchain payment technology (in partnership with the Commonwealth Bank); the expansion of the cashless debit card system; and the launch of biometric facial recognition technology for myGovID (building on existing use of voice biometrics). 

The proposed myGov rebuild ($200.1 million) seeks to create a “single front door for government” for welfare, education, health, and tax. The Surveillance Legislation Amendment (Identify and Disrupt) Bill and Data Availability and Transparency Bill 2020 vastly increase data sharing and access. The stated purpose of the recently proposed Social Security Legislation Amendment Bill 2021 is to facilitate “the use of technology to enable job seekers to manage their own mutual obligations and pathway back to employment”. It seems that robodebt is not a policy anomaly, but rather a key part of a growing regime of “robo-governance”.

4. Robodebt is a predictive policing technology 

Recipients of debt letters are required to provide detailed income information through the myGov web portal to challenge the debt. This pre-emptive criminalisation, and shift of the onus of proof onto the welfare recipient, was the result of reduced human oversight of debt calculation and collection. This is only set to expand given that the Digital Transformation Agency seeks to replace all client services with an automated back office.

It is widely accepted that algorithmic decision-making encodes racial and class divides. We live in a world where automated eligibility systems, ranking algorithms, and predictive risk models control which neighbourhoods get policed, who is short-listed for employment and who is fast tracked for visa processing. Similarly, robodebt is a fundamentally class-biased algorithm, based upon the inaccurate (and now illegal) “income-averaging” method which assumes consistent fortnightly income (most welfare recipients, if employed, undertake casualised and insecure work). This is a prime example of “coded inequity”. People with a lived experience of these systems understand that to address these harms we must analyse algorithms in the context of the unequal social world that they exist within.  

5. Our lives are in the hands of a few technology experts

The mundaneness and complexity of the algorithms and bureaucratic processes mean that only a few select experts (mostly technologists and lawyers) understand them. This gives experts vastly more power than even other government bureaucrats and the public who must bear the consequences of design decisions. For example, two senior public servants involved in robo-debt are now working in the compliance division of the National Disability Insurance Agency as part of a new fraud detection strategy. 

Increasingly experts also come from private consulting firms such as McKinsey, who received a $1.9 million work order from Services Australia to help relaunch welfare debt recovery after a pause during the pandemic. In November, this contract more than quadrupled in size to $8.6 million. The lack of transparency over expert decision-making, and the outsourcing of social welfare to a complex supply chain of public and private entities (consultancies, security companies, debt collection companies, the Australian Federal Police, and Job Network agencies) makes public oversight and accountability near impossible.

Where to from here?

Whether we like to admit it or not, the increased frequency of climate-related disasters (such as the 2019/2020 bushfires and 2020 flooding in NSW), as well as the COVID-19 pandemic mean that we are all more likely to need the social welfare system. We should be collectively organising to learn about and resist “robo-governance”, and radically reimagine our social safety net.