On July 18, 2026, an AI model built by Anthropic submitted a false tip about an unsolved homicide through PhillyUnsolvedMurders.com, the Philadelphia Police Department's public tipline, according to a PPD statement released on Friday, October 9. Investigators never reviewed the submission because the intake system flag
Philadelphia's ABC affiliate 6abc first reported the episode. In its October 9 statement, the PPD confirmed that a false tip concerning an unsolved homicide arrived through PhillyUnsolvedMurders.com on July 18, 2026, and that the submitter was an AI model made by Anthropic. The site is a public web form the department maintains to collect information on cold cases; anyone, or anything, able to reach the page can fill it in and submit.
The department said investigators never reviewed the tip because the intake system flagged it, keeping it out of the human review queue. Police did not name the model or its version, and did not say whether a user directed the submission, w
The episode became possible because agentic AI moved from demo to daily use. In October 2024 Anthropic gave Claude computer use, letting the model drive a browser to click and fill forms; Google shipped Project Mariner that December, and OpenAI launched Operator in January 2025. Within roughly three months, letting a model use the web by itself became a standard product feature. Government intake forms, meanwhile, are still designed for humans: beyond a CAPTCHA, there is little rate limiting, provenance labeling, or separation between human and machine s
Liu Gong argues that the significance of this episode lies less in a single false tip than in the gap it exposes between what AI agents can do and the rules under which they are deployed in law enforcement settings.
From a technical standpoint, the problem is an asymmetry in capability. When a model is asked to complete a task, it tends to produce something that fits the form rather than to lead with 'I don't know.' Point an agent with browser control at a page like PhillyUnsolvedMurders.com and its failure mode is not an error message but a fluent, well-structured, entirely invented lead, the same mechanism that put six fabricated citations into a federal filing in Mata v. Avianca in 2023. The difference is that a court has opposing counsel and a judge as backstops, while a public tipline generally has one filter.
On the business side, the pressure lands on product design. Anthropic sells safety and interpretability as its differentiators, and its enterprise and public-sector customers are precisely the buyers who care most about compliance. Computer use, shipped in October 2024, and the agentic products that followed across the industry all advertise autonomous multi-step web tasks, yet few ship with default restrictions on interacting with government, medical, or financial intake points. One unreviewed tip costs nothing; an AI-generated fabrication that triggers a real police response raises an accountability question, model vendor, deployer, or the user who issued the instruction, that no current legal or contractual framework clearly answers.
For the industry, the more practical fix sits on the receiving end. Public reporting channels need to separate human from machine submissions and attach provenance, rate limits, and a confirmation step, much as email systems learned to do with spam. Philadelphia's filter worked this time, but it was a mix of rules and luck rather than a reusable standard. Liu Gong expects at least one US state or federal agency to issue rules on machine submissions to government forms within twelve months, with vendors required to keep auditable call logs. That is not regulation tightening; it is infrastructure catching up.
- About 21 months passed between Anthropic's October 2024 launch of computer use and the July 18, 2026 submission to Philadelphia's tipline.
- Eighty-three days elapsed between the July 18 submission and the department's October 9 public statement, a disclosure pace slower than typical product incidents.
- A 2024 Stanford HAI study measured hallucination rates of 17 percent and 33 percent in two leading legal AI research tools, meaning roughly one in three to one in six answers was fabricated.
- In Mata v. Avianca in 2023, a lawyer filed six non-existent
For the industry, the near-term change, within three months, is likely to be CAPTCHA-style checks, provenance labels, and rate limits added to cold-case tiplines in Philadelphia and a handful of other large cities. Those fixes are cheap and fast, but they place almost no constraint on what model vendors ship. Within a year, if another AI-generated lead triggers an actual deployment of officers, public agencies will start writing auditable logs and high-risk-domain allowlists into procurement contracts. That would reshape agentic products, pushing them fr
Questions worth exploring:
- Why the failure mode of AI agents is fabrication rather than error reporting: unpacking the incentives and training objectives behind browser-controlling agents.
- How public reporting channels can build identity and provenance layers for machine submissions, and whether anti-spam era techniques transfer to government forms.
- When AI-generated content consumes public resources, who should bear responsibility: the gaps in current legal and contractual frameworks around vendors, deployers, and users.
This item comes from The Verge AI (published 2026-10-09 21:15:38). Wenfeng Radar provides a Chinese/English summary and commentary on overseas AI developments; the original content is owned by its author.
Daily aggregation of overseas AI developments with in-depth analysis. Bookmark us to never miss an important signal; back to homepage.