After what happened between the American Civil Liberties Union and Amazon, the tech giant is now responding more detailed to the accusations that its facial recognition system Rekognition wrongly identified 28 members of Congress as arrests in an ACLU test that took place during the past week.
Amazon recommends a high setting for accuracy
Matt Wood, Amazon’s general manager for deep learning and artificial intelligence, wrote in a blog post last Friday that while the ACLU used the default setting of an 80% confidence level, the company recommends a far higher setting “for use cases where highly accurate face similarity matches are important.”
“We continue to recommend that customers do not use less than 99 percent confidence levels for law enforcement matches, and then to only use the matches as one input across others that make sense for each agency,” he posted.
He also explained that Amazon tried to duplicate the ACLU’s test involving portraits of the 535 members of the Congress.
Amazon used a larger dataset of face for comparison – 850,000 over the ACLU’s 25,000 mugshots.
“When we set the confidence threshold at 99 percent (as we recommend in our documentation), our misidentification rate dropped to zero despite the fact that we are comparing against a larger corpus of faces (30x larger than the ACLU test),” he posted.
”This illustrates how important it is for those using the technology for public safety issues to pick appropriate confidence levels, so they have few (if any) false positives.”
The Amazon scientist also said that Rekognition is supposed to help law enforcement.
He also said that it’s a really reasonable idea for the government to weigh in and specify what confidence levels they want law enforcement agencies to meet in order to assist them in their public safety work.
Amazon still has not said which law enforcement agencies are evaluating or using the Rekognition service, even though members of Congress have recently demanded this.