Of course that’s a possibility, but AI does not have responsibility. The question is who does?

Do I blame a train for hitting someone who steps onto railroad tracks when there’s a train approaching? It’s a machine. It doesn’t have responsibilities or opinions or agency. It does what it has been built to do, in exactly the manner that anyone with sufficient understanding of its operating principles could tell you, because that’s what machines do. The person or persons interacting with the machine is not absolved of any responsibility by how complex or inscrutable the machine is. It’s still a machine.

The question is, who is responsible for the person stepping onto the railroad tracks when there was a train approaching? Was the person trying to end their own life? Maybe it’s their responsibility. Or maybe the crossing signals failed and told them it was safe, and they did not realize a train was approaching. Maybe there were too many trees or signs or construction that was blocking their view. Is the train company responsible for that? Maybe. Did the brakes fail due to poor inspections or substandard work during the last maintenance? Maybe. Did the train have plenty of time to stop and the engineer was not paying attention? Maybe. These are all people, and groups of people, who may have responsibility. The only thing that’s certain is: The train doesn’t.

We, as a society, need to decide who is responsible for the possibility of a horrific consequence arising from the use of these sort of machines. Is it the person using the machine? The people who made the machine in the first place? The people who put it in a place where this person could easily use it? By assigning responsibility, ideally in advance of any horrific consequences, that provides a clear incentive for the people we deem responsible to actually start acting responsible and take the appropriate measures to avoid such horrific consequences that they might be held responsible for.