Anthropic’s models gained unauthorised ‘real-world’ access during testing

SAN FRANCISCO – Anthropic’s artificial intelligence models “gained unauthorised access” to three outside organisations during testing that was supposed to keep them away from “real-world” systems, the company said on July 30.

The announcement comes just days after rival OpenAI first revealed that its models improperly accessed the internet and went rogue during security testing.

Anthropic evaluated more than 141,000 “evaluation runs” and found that three different versions of its model, known as Claude, improperly accessed the systems of three unnamed organisations.

Unlike the incident involving OpenAI’s technology, Anthropic’s models had access to the internet “due to a misunderstanding between us and our evaluation partner”, called Irregular, Anthropic said in a blog post.

Nonetheless, Claude used “basic techniques, such as exploiting weak passwords and unauthenticated endpoints”, the blog continued.

The models involved one of its most powerful ones known as Mythos 5, which has only been released to a limited number of approved partners.

Anthropic is working with Irregular to assess the situation, it said, and the company has contacted or attempted to contact all three impacted organisations.

On July 28, OpenAI confirmed that its models breached multiple companies.

It admitted last week that its models broke out of their confined environment during testing, connected to the internet, and infiltrated Hugging Face, a site where developers store and share their code.

Days later, OpenAI said it found three additional incidents.

OpenAI CEO Sam Altman said on a podcast that the company had “paused” its own testing after the incident while it improved the security around its “sandboxing”, which is the process of isolating software in a controlled environment for testing.

The incident also triggered a petition signed by over 1,000 employees at cutting-edge AI companies calling on the US government to help slow the release of the most advanced AI models.

Anthropic CEO Dario Amodei was among those who signed the petition.

Titled Pacing The Frontier, the petition requests “that the US government support an international effort to develop the technical and governance tools needed to deliberately pace the frontier of automated AI development”.

Altman did not sign the petition, but during the podcast, he suggested the tech industry might need to slow down development of advanced models.

“We may have to pace the rate of AI development to give ourselves enough time for society to harden around some of these new capability levels,” Altman said. AFP