Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor

The true danger of automated welfare systems lies in their tendency to conflate poverty with criminality, and thereby invoke police intervention unnecessarily. Most visibly this happens “when poor and working-class people build grassroots movements that directly challenge the status quo through disruptive protest” (177). However, more insidious is the invisible risk of policing that accompanies use of public services—policing that bypasses rights and safeguards through indiscriminate sharing of data, and which is almost impossible for citizens to monitor or keep in check. Big tech automation and coordinated entry has made policing the poor easier than ever, and evidence of data misuse is nearly impossible to verify.

Eubanks acknowledges the potential helpfulness of artificial intelligence and predictive models for addressing inequality. However, she argues that in their current state of implementation, these systems are inaccurate, inadequate, riddled with unintended bias, and harmful. The inability of automated systems to fully grasp and account for numerous competing variables has drastic real-life consequences. This is because our data-based administrative systems are rife with unassigned entities, random variables, and correlation-causation fallacies. Erroneous and ill-designed, these systems can never address the true causes of poverty.

What’s more, incorrect assumptions made at the inception of these algorithmic models instill biases even more deeply into our subconscious attitudes about poverty, bringing a subtle but inescapable shift in habitual acceptance.