FTC takes action on school security tech vendor after investigation
In proposed settlement order, the vendor would be banned from making unsupported claims about detecting weapons by using AI and allow schools to cancel multi-year contracts.
For the last year, I’ve been raising the alarm on the questionable effectiveness of some school security technologies, especially vendors that claim to be AI-powered systems:
CNN Opinion: I study school shootings. Here’s what AI can — and can’t — do to stop them
How do AI security products being sold to schools really work?
Why is image classification of guns so difficult for AI software?
Did OpenAI create the best weapon detection software available with ChatGPT-4o?
Today it looks like the federal government has been listening as the Federal Trade Commission has announced action against the largest company in the school security market.
The FTC’s press release has some important points that are applicable to school administrators including:
Option to cancel multi-year service contracts signed between April 1, 2022, to June 30, 2023.
Banning the vendor from making unsupported claims about AI.
Banning the vendor from making claims about cost or time savings compared to traditional metal detectors (which cost a fraction of the price of the ‘AI Scanners’).
FTC Takes Action Against Evolv Technologies for Deceiving Users About its AI-Powered Security Screening Systems
Proposed settlement would prohibit misrepresentations and allow affected schools to opt out of current contracts for security screening systems
The Federal Trade Commission is taking action against Evolv Technologies over allegations that the company made false claims about the extent to which its AI-powered security screening system can detect weapons and ignore harmless personal items, including in school settings.
In the proposed FTC settlement order, Evolv would be banned from making unsupported claims about its products’ ability to detect weapons by using artificial intelligence and would also have to give certain K-12 school customers the option to cancel their contracts, which generally lock customers into multi-year deals.
The FTC alleged that Evolv misrepresented that its Evolv Express system will detect all weapons; ignore harmless personal items without requiring people to remove them from their pockets or bags; detect weapons more accurately and faster than metal detectors; reduce false alarm rates; and cut labor costs by 70% compared to metal detectors by reducing the need for additional personnel.
In its complaint, the FTC alleged that Evolv’s Express scanners failed in several instances to detect weapons in schools while flagging harmless personal items typically brought to schools, like laptops, binders, and water bottles. For example, Evolv’s Express scanners reportedly failed to detect a seven-inch knife brought into a school in October 2022 that was used to stab a student. Afterwards, school officials increased the system’s sensitivity settings, prompting a 50% false alarm rate.
To reduce false positive rates, Evolv in 2023 introduced a more sensitive setting for Express users with the goal of detecting more knives. Despite this, Evolv said some knives will be missed, more false alarms will occur, and additional staffing may be required to run the machines. It also advised schools to add conveyor belts and other measures to divert harmless items by hand, which makes the system more like traditional lower-cost metal detectors according to the complaint.
Red Flags for School Officials
This decision by the FTC should be a warning that is heard across the school security industry and a reminder to school officials to look closely at the marketing claims made by these vendors. If a company won’t share data about the performance of their equipment or software, school officials should ask why?
As I wrote in my CNN Op Ed:
Schools need to understand the limits of what an AI system can — and cannot — do.
With cameras or hardware, AI isn’t magic. Adding AI software to a magnetometer doesn’t change the physics of a gun and metal water bottle producing the same signal. This is why an AI screening vendor is being investigated by the FCC and SEC for allegedly inaccurate marketing claims made to schools across the country.
Instead of schools choosing to test or acquire the best solutions based on merit, vendors lobby to structure local, state and federal government funding to create a shortlist of specific products that schools are compelled to buy. During a period of rapid AI innovations, schools should be able to select the best product available instead of being forced to contract with one company.
Neither image classification from CCTV nor retrofitted metal detectors address the systemic problem of teens freely carrying a gun at school each day. Solving this challenge requires better sensors with more advanced AI than any product available today.
Schools sit at the brink between a troubled past and a safer future. AI can either inhibit or enable how we get there. The choice is ours.
David Riedman is the creator of the K-12 School Shooting Database, Chief Data Officer at a global risk management firm, and a tenure-track professor. Listen to my weekly podcast—Back to School Shootings—or my recent interviews on Freakonomics Radio, New England Journal of Medicine, and my article on CNN about AI and school security.