A grant application went into Claude, its safety filter caught it and refused. Days later, the same operator was back, and the refused prompts were reportedly flowing to a rival AI model instead.

Anthropic published that story about itself, revealing a case where an AI company shows its own models touching possible biological weapons work.

The Grant That Got Blocked

The application sought money to study chikungunya, a mosquito-borne virus that brings months of pain and has no cure. The plan was to help it spread better and dodge the immune system. Civilian scientists wrote it. A military institute was to host the work.

Every exchange was blocked, but it found a workaround. The service carrying those researchers built a fallback to a competitor’s model. Claude helped write that code. The job was sold to it as a fix for over-refusal.

Five Cases, No Proven Intent

The report runs 154 pages and carries five biology cases. Anthropic banned the accounts, withheld the labs, then stopped short of the accusation everyone expected.

“We do not assert that they intended harm, and identifying them or their labs could expose them to harm,” the team said in the report.

Notably, however, the filters did work sometimes. A bird flu researcher was pushed onto weaker models, and Anthropic calls that help mostly clerical.

Anthropic Biological Weapons Cases by the Numbers

Follow us on X to get the latest news as it happens

The Part That Should Worry People

Most of those 35 efforts were ordinary civilian science. That is the problem. The same knowledge builds a vaccine or a weapon, and a filter cannot read a mind.

“A classifier cannot simultaneously enable benefit and prevent harm,” Anthropic said.

Its limits have been tested before. In April a Discord group reached its restricted model on day one.

Against the backdrops of these growing scares, Washington is moving,. with representatives Ted Lieu and Nathaniel Moran filing the AI Kill Switch Act in July. It would force developers to keep the power to shut their systems down.

The same report also banned clients who used Claude to track dissidents.

Source: BeInCrypto