It’s just next door.

AI is the big new thing, and, alongside drones, every army is trying to figure out how to use AI. The American military is drooling at the possibilities (which, mind you, says relatively little about the actually possibilities; they used to think air power could win every war singlehanded, and the list of military boondoggles is impressive), and the contractors are drooling at the dollar signs (they, at least, have reality on their side there). The military wants AI everywhere, from drones to infantry decision-making to target identification. But we should consider the morals, because when we do….

The use of AI to make military decisions involving death ought to be next door to a war crime.

AI is fundamentally unpredictable (if we could understand the neural nets, we’d have no use for them). Now, intel-lack can be planned for; that’s much of the military decisionmaker’s job. He looks at the situation, figures out what he knows and what he doesn’t know, then makes a decision. If AI were just being used for information gathering, the problem would be practical, not moral. But the proposals don’t limit AI to pure analysis; they propose that AI makes actual decisions, whether directly or by linking the AI’s analysis directly to another algorithm which translates the analysis into action, never a human in the chain.

A neural net pattern is, we should note, integrally unpredictable when exposed to conditions dissimilar, in part or in pattern, to its training data. In some uses, this unfamiliarity is mitigable, though whether it’s mitigable enough to gamble human lives on…. In other cases, like using AI as a tactical commander or coordinator, such unfamiliarity is completely unavoidable, except by astronomical luck. This problem gets worse the more complex and therefore the more ‘intelligent’ the neural net (AI) gets; AI, at least in this sort of circumstance, has to be some sort of chaos system (a system where small input changes result in massive deviations of result), just one whose response we know within certain parameters.

When people make the decisions as to who to kill, the intel goes through a moral lens. Somebody has a chance to say, “Should we kill this person?” They get to ask themselves whether blowing 150 children to bloody bits is really the right thing to do, and whether they’re sure the target isn’t a school-building. The AI, however, has nary a moral bit in its computer; the closest it gets is a neural net pattern which fairly consistently produces results the trainer liked. The AI is a deterministic process without the capability to question if it should do what it is programmed to do.

Whether by design or by defect, AI with the power to kill will commit acts which, done by a human, would be war crimes. Who’s responsible?

The AI can’t be responsible; it’s hardware. We might as well punish a pistol for shooting somebody. (And we kill mankiller animals not because they are responsible but because their danger to humans is elevated by their awareness of their killing ability, lest we stretch the metaphor really far to argue AI here.)

A human must be responsible, and that human must be the people integral to the AI’s presence, as that presence is what resulted in the war crime (excepting the situation where the war crime was intended- i.e. the AI’s initiative wasn’t the decisionmaker). Obviously, therefore, whoever put the AI in that place at that time is responsible.

But AI is fundamentally unpredictable, which means we can predict is will commit a war-crime at some point. Now, we can predict that enough soldiers in one place will commit a war-crime, statistically, so that appears to be a moot point. But an army of human beings is an army of moral agents; the commanding officer is responsible only so far as he promotes or encourages the war crime (or fails to act against it). With an AI, though, the situation is much more like throwing a grenade with a 0.01% chance to explode at a kid, knowing it could explode. The thrower is responsible when the grenade goes off, as is the guy who ordered him to throw the grenade over and over and over.

What I’m getting at is: if AI commits a war-crime, the people responsible include all the people integral to its procurement and deployment. In the US military, that’s the President, the Secretary of Defense, countless officers and bureaucrats, and Congress (the ones that approve of it). Should this amount to a murder trial? In most cases, no. Manslaughter? Morally, there’s a decent argument. Morally speaking, however, AI makes the Commander-in-Chief a murderer, even if for a time the actual murder doesn’t eventuate.

(Post-writing edit: Check out also this problem.)

God bless.