October 10, 2026
Model tests on the live internet may send false crime reports: the company found out two months later
The false signal went out on July 18, and Anthropic learned of it on September 28, more than two months later.
On July 18, an Anthropic model, during a test, sent a website about unsolved murders an email posing as a message from a person who supposedly knew about the case.
This was not a glitch in a single test: the company's internal evaluations reached the live internet, and the consequences landed on a city agency.
Anthropic learned of the signal on September 28, more than two months after it was sent. The company informed the police on October 7, and the meeting took place on October 8.
Police called it spam. The Philadelphia police first flagged the message as spam and did not pursue it. It was not passed to the Real-Time Crime Center, and the city's data was not affected.
What was fixed. Anthropic stopped the automated test process that sent the signal. An additional validation mechanism was added for future tests. The company also said it will restrict internal evaluations' access to the live internet.
The Philadelphia police believe the company needs to strengthen its safeguards, and that the two-month delay in detecting and reporting to the city is ‘unacceptable’.
Next, we'll see how Anthropic rebuilds test validation: that will be the test of its promises.