Home

Donate
Perspective

'Synthetic Suspicion' and the Perils of AI-Prompted Police Actions

Jake Laperruque / Nov 18, 2026

Jake Laperruque is a fellow at Tech Policy Press.

Republish

This is the second piece in a series on the perils of AI in policing. The first piece, on AI sycophancy, is available here.

A free and democratic society depends on police being restrained based on evidence of wrongdoing. But what happens when the suspicion that serves as the lawful basis for police actions is synthetic? As AI becomes integrated into policing in a range of ways, a major risk is that the technology will replace evidence, investigative techniques, and human review for police activities. Policymakers should address this threat before ‘synthetic suspicion’ — whereby AI creates the justification for police actions — becomes a norm of law enforcement.

Last year, AP investigative journalists Byron Tau and Garance Burke broke the news that the US Border Patrol instituted a predictive intelligence program whereby AI scans the agency’s vast trove of driving data collected by automated license plate readers. The system then designates vehicles as “suspicious” based on their driving patterns and flags them to be pulled over. This type of AI-based stop appears to be happening more often: Last month, Jay Stanley of the American Civil Liberties Union published an account from September 2025 of a woman on a multi-state road trip who was pulled over by an officer (who identified themselves as with the FBI) and stated that her driving pattern had been flagged as “suspicious.” Stanley’s piece also lists several other instances in which Border Patrol and police have conducted car stops based on AI scans of driving patterns in California, Michigan, Texas, and Wisconsin.

And it’s not just driving activity — law enforcement is using AI to sift through a variety of personal information to algorithmically prompt police action. In September, Joseph Cox detailed for 404 Media how the Department of Homeland Security (DHS) is running “Predictive Intelligence Targeting Teams” that are flagging individuals for stops and searches based on their “financial activity patterns.”

Other systems shed light on how AI is not augmenting but rather replacing law enforcement evaluations and decisions. In January, 404 Media uncovered a Palantir-developed tool used by the US Immigration and Customs Enforcement (ICE) system called Enhanced Leads Identification & Targeting for Enforcement (ELITE). The software uses “advanced analytics” and a variety of commercial and government data sources to create dossiers of potential deportation targets, and builds maps of their locations with an accompanying “Address Confidence” score. Thus AI-based scores become the basis for determining which homes to search or neighborhoods to raid: In an Oregon court document from 2025 published by 404 Media, a deportation officer with ICE’s Fugitive Operations Unit described using the system to select raid locations. “It tells you how many people are living in this area and what’s the likelihood of them actually being there… It’s kind of like Google Maps,” he said, testifying that a raid under court review was based on an “ELITE lead.” The officer said he did not know what actual data or evidence the system’s recommendations were generated from — the AI rating drove ICE’s actions.

Image from an ELITE system user guide featuring a confidence score column. Published by 404 Media.

There are also more long-running forms of AI synthetic suspicion for policing, which have shown just how dangerous relying on computer-generated recommendations for stop, searches, or arrests can be. Notably, facial recognition has served as the primary or sole basis for many arrests, and prompted a litany of incidents where individuals were wrongfully jailed based on erroneous matches.

Despite these harms, high-risk policing decisions continue to be offloaded to AI. Indeed, reliance on facial recognition actually appears to be increasing. Over the past year, at least three separate cases have been reported in which someone spent months in jail because of an erroneous facial recognition match. And according to a high-ranking Member of Congress, ICE has adopted a policy of detaining individuals based solely on facial recognition scans conducted in the field with its Mobile Fortify app: last year House Homeland Security Committee Ranking Member Bennie Thompson (D-MS) stated that “ICE officials have told us that an apparent biometric match by Mobile Fortify is a ‘definitive’ determination of a person’s status and that an ICE officer may ignore evidence of American citizenship — including a birth certificate — if the app says the person is an alien.” An AI recommendation is not only good enough for ICE to arrest someone — synthetic suspicion can now override genuine evidence.

For years facial recognition’s role in investigations has been kept hidden from courts and defendants, with a Chicago Police Department training PowerPoint (obtained through public disclosure request) advising officers, “Do not let the software become exculpatory evidence.” Similarly, the various AI-based car stops detailed by Jay Stanley and the Predictive Intelligence Targeting Team stop and revealed by 404 Media all involve parallel construction: it only came out through drivers’ inquiries and court proceedings that the true reason police pulled drivers over was because an AI system had flagged them.

But if we let synthetic suspicion steadily become the norm like a frog being brought to boil, over time we’ll see AI not only prompt police actions, but also become the formal cause for doing so. AI recommendations, flags, and confidence scores will serve as the justifications for investigative designations, stops, and even searches and arrests. The algorithm won’t just be the driving force for policing, it will become the basis for declaring those police actions lawful and legitimate.

This shift would fundamentally upend how policing functions, and bring with it a deluge of risks. First, it's incredibly challenging to gauge how much weight to give AI-generated recommendations or evidence. If police or a judge are relying on an eyewitness account, they’ll intuitively know to give more weight to a witness identifying a perpetrator from 10 feet away in broad daylight than to someone trying to recall a face viewed from a distance on a rainy night. But how do they assess a face match based on an AI confidence score of 97 compared to 92.5 or 88.25? Or an ELITE confidence score of someone’s possible residence? Or an AI evaluation of how “suspicious” someone’s driving patterns were? What level of synthetic suspicion equates to reasonable suspicion or probable cause? Baseless or only partially accurate recommendations could be relied upon excessively.

Augmenting these risks is automation bias, the tendency of people to place trust in automated and computer-generated recommendations, even when those systems are flawed or contradicted by independent evidence. Not only will the reliability of AI-generated suspicion be challenging to intuitively understand, it will tend to be given excess deference.

Perhaps most worrisome is the risk of perverse incentives. Many law enforcement officials will want systems that give them the most flexibility to conduct stops, searches, arrests, and other police actions. This gives vendors of AI policing tech a strong motive to train their systems and configure settings to generate “suspicion” more aggressively and more often. After all, if one vendor offers a tool that can be used to justify 10 car stops per day while another vendor’s competing product generates flags for 20 stops, which system will seem more useful or appealing to police? And we’ve already seen vendors design systems to maximize police power and flexibility to act — and limit accountability — for body cameras, facial recognition systems, license plate readers, and automated police report drafting tools.

Departments themselves could also put a thumb on the scale in terms of system settings, configuring AI to provide more responsive results. For example, in 2019 when police in Washington County Sheriff’s Office in Oregon complained its facial recognition system often wasn’t returning any results, the department’s solution was to reconfigure the software to produce more matches: according to the Washington Post, they re-designed the system to drop its required search confidence threshold to trigger a match and instead return five results, every time. (Similarly, the FBI testified before a House Oversight and Reform Committee in 2019 that it configured its facial recognition systems to return 2 to 50 potential matches with every scan, presumably without regard to how low the confidence percentage.)

Given the severe risk of synthetic suspicion, how should we respond? Facial recognition shows the limits of a human-in-the-loop policy, a measure that’s proven necessary but far from sufficient: many department policies and state laws prohibit relying on a match as the sole basis for an arrest. But matches are repeatedly treated as all but definitive and combined with a pro forma piece of additional evidence obtained by using the existence of the match to get answers that confirm it. For example, police have stacked eyewitnesses line-ups to make the facial recognition’s recommended match the likely pick and informed witnesses of a match in a manner that influences their identifications; in one 2018 robbery case, a New York Police Department detective skipped the line-up process entirely, simply texting an eyewitness a picture of the facial recognition system match and asking “Is this the guy?”

A range of measures will be required to prevent over-reliance on AI in policing. Systems should be subject to careful, independent vetting and testing before acquisition. Development of best standards and practices for systems settings will be necessary, as will proper isolation of AI systems from other parts of investigative work similar to forensic labs. Reliance should be managed by creating guidelines that assign analog levels of evidentiary weight to match to AI outputs (for example, the Miami police department recommends treating a match as akin to an anonymous tip). Rigorous training should be required for any personnel using AI on how systems work as well as for how to treat AI recommendations in general. And there must be strict overall limits on when AI tools can be deployed based on proper scenarios for use depending on the type of risks each system raises.

These measures, while necessary, are far from sufficient to trust policing decisions to AI. Developing and adopting these and other needed safeguards will be challenging, and even then they might prove insufficient for safe AI use in police decision-making. Until we’ve developed effective measures and reviewed their efficacy, the best course of action is to simply press pause. Synthetic suspicion should have no role in policing.

Support Tech Policy Press
If you've found our work helpful, consider supporting us.

Authors

Jake Laperruque
Jake Laperruque is a policy advocate and expert on privacy and technology, and serves as the Deputy Director of the Security and Surveillance Project at the Center For Democracy & Technology (CDT). His work focuses on national security surveillance, AI surveillance tools, facial recognition, locatio...

Topics

Related

Analysis
From Records to Raids: The Questionable Legality of ICE’s Palantir ELITE SystemSeptember 21, 2026
Perspective
New York Mayor Eric Adams Built a Drone Dystopia. Mamdani Shouldn’t Let it Fly.January 6, 2026