Who Is to Blame When AI Makes Mistakes? Technology, Ethics and the Responsibility Gap

Last updated: October 22, 2026

In AI ethics, the question of who is to blame when AI makes a mistake has a name: the “responsibility gap”, a term the philosopher Andreas Matthias introduced in Ethics and Information Technology in 2004. Traditionally the maker or operator answers for what a machine does, but with learning systems, Matthias argued, the manufacturer or operator is “not capable of predicting the future machine behaviour” any more. In practice, someone tends to end up filling the gap. When a self-driving Uber test car killed Elaine Herzberg in Tempe, Arizona, in March 2018, the National Transportation Safety Board found that the car had detected her 5.6 seconds before impact without ever classifying her as a pedestrian. The criminal case ended with the backup driver, who pleaded guilty to endangerment in 2023; the company faced no criminal charges.

The meme at the top of this page turns that pattern into a trolley problem with a driver-assistance system at the wheel. The car is heading for a group of people, and the human in the seat gets two tracks. Grab the wheel and swerve, and you hit only one person, but the system switches itself off and every bit of the damage becomes yours. Keep your hands off, and the car hits the whole group, but you walk away innocent and the manufacturer pays a fine. Only one of the two tracks is bad for the person who could have intervened.

The Meme Is Half Right on Liability and Wrong on the Dilemma

The meme’s legal joke is half right about how driver-assistance systems are classified today, while its trolley set-up misdescribes how a car’s crash behaviour actually gets decided.

SAE J3016 sorts driving automation into six levels. According to SAE’s own chart, at Levels 0 to 2 the human is driving even when the features are engaged: “You must constantly supervise these support features”. Only from Level 3 does the system drive, and even there the human must take over “when the feature requests”. So a car can be marketed with words like “autopilot” while the person in the seat is still, in SAE’s terms, the one driving. The meme’s swerve track, where the system switches off and you inherit the damage, gets this roughly right; its hands-off track, where you walk away innocent, does not, because at these levels you are the driver either way.

The dilemma half is weaker. In a 2016 paper in Ethical Theory and Moral Practice, Sven Nyholm and Jilles Smids listed three ways crash programming differs from the trolley case. A trolley decision is one person’s split-second choice; an accident algorithm is planned in advance by many stakeholders. The trolley case brackets moral and legal responsibility, which is exactly what a car maker cannot do. And the trolley case stipulates certain outcomes, while a real car works with probabilities. Nyholm and Smids keep the ethical question alive, but move it: whoever writes the code decides in advance how risk is shared on the road.

Forty Million Votes on Whom a Car Should Hit

The Moral Machine experiment collected about 40 million decisions from people in 233 countries and territories on whom a self-driving car should spare in an unavoidable crash, and it found strong shared preferences alongside clear cultural differences.

The study, led by Edmond Awad and published in Nature in 2018, let visitors to an online platform choose between two crash outcomes. According to the authors’ manuscript, the three strongest global preferences were sparing humans over animals, sparing more lives over fewer, and sparing the young over the old. The CNRS summary adds the less flattering findings: characters described as poor were more likely to be sacrificed than rich ones, and jaywalkers more likely than pedestrians who obeyed the signals. Counting lives this way is the utilitarian reflex, and our post on utilitarianism and the problem of predicting the future shows where that reflex runs into trouble.

The authors were careful with their data. Their sample was self-selected, and they wrote that “policymakers should not embrace our data as the final word”. They also flagged a direct conflict: Germany’s Ethics Commission on automated driving (2017) prohibits distinctions based on personal features such as age in unavoidable accidents, which, in the paper’s words, “clearly clashes” with the public’s preference for sparing the young. An earlier study by Jean-François Bonnefon, Azim Shariff and Iyad Rahwan, published in Science in 2016, found a second tension: people approved of cars that minimise total deaths, yet “would themselves prefer to ride in AVs that protect their passengers”. The trolley problem that started all this has a longer history, which our Camus trolley problem post traces back to Philippa Foot.

The Responsibility Gap: When Nobody Could Have Known

The responsibility gap is the claim that learning machines can cause harm for which no human can fairly be blamed, because nobody could predict or control what the system would do.

Matthias’ 2004 argument rests on two traditional conditions for blame: you answer for what you could foresee and control. A programmer who writes explicit rules meets both. A system that learns from data and keeps adapting after deployment meets neither in full, since even its makers may be unable to say in advance how it will behave in a new situation. Matthias saw two options, as the abstract puts it: stop using such machines, which he thought unrealistic, or accept a gap that existing ideas of responsibility cannot close. Blaming the machine itself fails too: as the Stanford Encyclopedia of Philosophy notes, critics such as Robert Sparrow argue that computer systems cannot suffer and so cannot be punished.

The gap is contested. Merel Noorman’s SEP entry describes a line of work on “meaningful human control”, developed by Filippo Santoni de Sio and Jeroen van den Hoven, among others, out of the debate on autonomous weapons, which aims to design systems so that humans stay able to track and steer what they do. On that view, the gap is partly a design choice. Deploying a system nobody can predict is itself a foreseeable and controllable decision, and that is one reply to Matthias: the blame travels upstream to whoever approved the deployment. D. W. Tigard goes further in Philosophy & Technology (2021), arguing that responsibility is flexible enough that no special gap exists.

Many Hands, No Fingerprints

When a harm passes through enough desks, each contributor’s share looks too small to blame, and the political philosopher Dennis Thompson called this the problem of many hands.

Thompson named it in 1980 in the American Political Science Review, writing about public officials whose decisions pass through so many hands that blaming any single person seems unfair. His answer was to keep personal responsibility, judged by what each person caused and intended, instead of blaming only the people at the top or the organisation as a whole. Helen Nissenbaum applied the idea to computing, and in a 1996 paper in Science and Engineering Ethics she named four barriers to accountability, summarised in a 2022 paper she co-wrote: many hands, “bugs”, the “computer as scapegoat” and “ownership without liability”. A classic case, according to the SEP, is the Therac-25 radiation machine, which overdosed six patients in the 1980s, contributing to three deaths, through a mix of software errors, inadequate testing and poor interface design.

The scapegoat barrier has an everyday version in online marketing, where a flopped campaign gets pinned on “the algorithm” before anyone rereads the copy. At least one tribunal has refused this move. In Moffatt v. Air Canada (14 February 2024), British Columbia’s Civil Resolution Tribunal heard the airline argue, in effect, that its website chatbot, which had given a customer wrong information about bereavement fares, was a separate legal entity responsible for its own actions. The tribunal called this “a remarkable submission” and held that the chatbot was “still just a part of Air Canada’s website”.

Moral Crumple Zones: The Human Who Absorbs the Impact

A moral crumple zone, a term coined by the researcher Madeleine Clare Elish, is the human operator who takes the blame for a failure of a larger automated system over which they had limited control.

A car’s crumple zone deforms in a crash so that the cabin survives. In Elish’s analogy, published in 2019 in Engaging Science, Technology, and Society, the nearest human plays that part for the technology: the crumple zone “protects the integrity of the technological system”, “at the expense of the nearest human operator”. Her two main case studies are the 1979 Three Mile Island nuclear accident and the 2009 loss of Air France Flight 447, where coverage focused on operators and pilots while confusing displays, design choices and training gaps got less attention. She opens the paper with Tempe: “On March 18, 2018, a self-driving Uber car struck and killed a pedestrian”.

The NTSB named the operator’s failure to monitor the road, while distracted by her phone, as the probable cause. It also found that the system design precluded emergency braking and relied on the operator to intervene. Among the contributing factors it listed were the company’s inadequate safety risk assessment, ineffective oversight of operators and a lack of mechanisms against “automation complacency”, all of which it traced to an inadequate safety culture. It also cited the pedestrian’s crossing outside a crosswalk while impaired and Arizona’s insufficient oversight of automated vehicle testing. A county prosecutor decided in 2019 that the company was not criminally liable; the operator received three years of supervised probation. The operator was distracted, and that matters morally. The crumple zone concept asks a narrower question: why a system built to need a constantly alert human, in a job known to breed complacency, ends with only that human facing criminal charges.

Closing the Gap by Law: Who Pays When Nobody Drove

Recent laws in the UK and the EU, neither applying in full yet, move responsibility away from the person in the seat and towards the company that builds and runs the system.

Under the UK’s Automated Vehicles Act 2024, while an authorised self-driving feature is engaged, the “user-in-charge” gets immunity from offences arising from how the vehicle drives itself, and an “authorised self-driving entity” is, in the explanatory notes’ words, “legally responsible for the way an authorised automated vehicle drives”. Full implementation is targeted for the second half of 2027. The immunity ends when the car hands control back and the takeover period runs out, so the meme’s handover problem survives in a smaller form. In the EU, the revised Product Liability Directive (2024/2853) treats software and AI systems as products, applies strict liability and lets courts presume defectiveness where technical complexity makes proof excessively difficult and a defect is likely; member states must implement it by 9 December 2026. A separate AI Liability Directive was dropped: the Commission announced the withdrawal in February 2025, citing a “lack of agreement on a final text”, and the withdrawal notice appeared in the Official Journal on 6 October 2025.

Liability law answers who pays, which is a different question from who is to blame. A company can compensate a victim without anyone inside it having done something they would call wrong, and that is where Thompson’s insistence on personal responsibility still bites. The philosophical debate has not settled whether the responsibility gap is real or can be closed by better design and a broader idea of responsibility. The UK and EU rules step around that dispute: they attach much of the cost of unpredictability to the organisations that build and run such systems, whether or not anyone inside them is to blame.

If you want to know which philosophy the people building these systems prefer to read, see our post on why Silicon Valley keeps rediscovering Stoicism and getting it wrong.