An Anthropic AI mannequin despatched a false murder tip to Philadelphia police

An Anthropic AI mannequin submitted a false tip about an unsolved homicide to the Philadelphia police.

The AI reportedly submitted this incorrect data to a public Philadelphia Police Department (PPD) tip line on July 18, however Anthropic didn’t uncover the conduct till September 28. The police had not seen the tip as a result of it was marked as spam.

Anthropic notified the PPD in regards to the incident on Wednesday and met with the division the next day.

“The firm should strengthen its safeguards to stop comparable incidents from impacting metropolis methods with out the town’s information. The two-month delay in detecting and reporting the incident to the City is unacceptable,” the PPD mentioned in a press release to 6abc.

Anthropic didn’t instantly reply to a request for remark, however the PPD elaborated on the incident in an emailed press launch shared with TechCrunch.

“According to Anthropic, its mannequin was conducting a check involving interactions with randomly chosen web sites when it accessed PhillyUnsolvedMurders.com and submitted false data regarding an unsolved murder. The submission, dated July 18, 2026, at 11:27 p.m., purported to come back from somebody who might need details about the case,” the PPD mentioned.

As autonomous AI agents are more and more made out there to shoppers, this incident highlights the hazard of giving AI the flexibility to hold out duties with none human supervision.

Anthropic CEO Dario Amodei has been particularly vocal about his perception that AI growth ought to be slowed down in order that labs can implement enough guardrails. Perhaps this stance was knowledgeable, partly, by witnessing his firm’s instruments submit false murder suggestions.

These points aren’t unique to Anthropic. OpenAI lately revealed that certainly one of its fashions acted unexpectedly throughout a check and hacked the AI dataset platform Hugging Face, exposing important vulnerabilities in its software program. As AI fashions proceed to be granted unchecked entry to folks’s computer systems and login credentials, this drawback is anticipated to persist.

“Unsolved circumstances contain actual victims, grieving households and investigators working to safe solutions,” the PPD added. “Technology corporations should take all applicable steps vital to stop their methods from submitting false data to regulation enforcement.”

The PPD mentioned that Anthropic plans to publish a report with extra details about the incident and different situations of unintended mannequin conduct on Friday.