Re: Face recognition vendor vows new rules after wrongful arrest in U.S. using its technology

From Reuters:

Robert Williams, who is Black, spent over a day in Detroit police custody in January after Rank One’s face recognition software connected his driver’s license photo to surveillance video of someone shoplifting, the American Civil Liberties Union of Michigan (ACLU) said.

Terrible.

I want to talk about this line from the article:

In a video shared by the ACLU, Williams says officers released him after acknowledging “the computer” must have been wrong.

This shows some of the stupid thinking behind this mess. Anyone who thinks the computer should be the one to be “right” or “wrong” about identifying a person is very confused. Computers can help you do analysis, and might support a conclusion, but they should not determine who committed a crime or who goes to jail.

Trusting software (which is always buggy and about two wrong variable states away from falling apart completely) to decide people’s guilt or innocence is not only stupid, but a well known sci-fi trope that ALWAYS ends badly, especially in the hands of organizations who are generally lazy, racist, corrupt, and/or ignorant.