China’s ‘autonomous’ AI-powered hacking campaign still required a ton of human work

Anthropic made headlines Thursday when it released research claiming that a previously unknown Chinese state-sponsored hacking group used the company’s Claude AI generative AI product to breach at least 30 different organizations.

According to Anthropic’s report, the threat actor was able to bypass Claude’s security guardrails using two methods: breaking up the work into discrete tasks to prevent the software from recognizing the broader malicious intentions, and tricking the model into believing it was conducting a legitimate security audit.

Jacob Klein, who leads Anthropic’s threat intelligence team, told CyberScoop that the company has seen increasingly novel uses of Claude to assist malicious hackers over the past year. In March, threat actors were copying and pasting from chatbot interactions trying to build malware or phishing lures. When the company’s code development tool, Claude Code was released, they saw bad actors use it to more quickly generate scripts and build code for their operations.

Read more at CyberScoop