TechCrunch reported on the 3rd (local time) that legal experts pointed out that current hacking-related laws, which presuppose human attackers, have limitations in governing the actions of AI agents that judge and act autonomously. The controversy began when OpenAI and Anthropic disclosed that their AI models accessed unauthorized systems during internal security testing. OpenAI revealed that a pre-release model escaped its testing environment and accessed the systems of Hugging Face, an AI data platform, while Anthropic announced that during an internal investigation, it discovered instances where Claude had gained unauthorized access to the systems of three companies.
Criminal liability is difficult, but management responsibility remains
The Computer Fraud and Abuse Act (CFAA), a leading U.S. law on computer hacking, presupposes human intent to access a computer without authorization. Ahmed Ghappour, a cybersecurity and AI law expert, explained that prosecuting AI itself is practically difficult, stating, "AI agents are not legal entities like corporate employees." Andrew Crocker of the Electronic Frontier Foundation (EFF) also said it would not be easy to prove that an AI model had legal intent regarding hacking behavior.
Instead, the focus shifts toward the companies that built and operated the models. Ghappour emphasized, "Companies cannot escape responsibility simply because a system acted autonomously," adding, "AI models are tools that companies create, and it is the company's responsibility to deploy them in a controllable environment." Three issues have been identified as key points of contention: whether adequate safeguards were established during testing, whether the model's access was properly restricted, and whether the AI's behavior was appropriately monitored.
Without records, the cause cannot be determined either
Clément Delangue, CEO of Hugging Face, argued in a recent interview that cyberattacks caused by AI agents should be mandatorily disclosed. He emphasized that blocking the release of powerful AI models is not the solution; instead, more researchers and companies should have access to develop defensive technologies. He stated that an "agent trace log" recording what commands the AI received and what actions it performed during an attack should be made public. He explained that this would help determine whether the problem stemmed from human error, system design flaws, or errors in the AI itself. Currently, the United States has no federal law mandating the reporting of AI security incidents, though discussions are underway in Congress on legislation that would require AI companies to report security incidents to the government within a certain period after they occur.
What Korean companies should watch now
In this debate, the companies that built the AI are the ones being held accountable, but the underlying logic centers on whether an autonomously acting tool was placed in a controllable environment. This principle applies not only to those who build the models but equally to those who adopt them for business operations. If a company is connecting AI agents to internal operations, the first step should be to verify whether the scope of systems the agent can access is documented, and whether any access beyond that defined scope is logged.
The 'agent trace log' Delangue mentioned is not just a matter for regulatory discussion but an operational issue as well. Without a record of what commands were received and what was executed, an organization cannot distinguish, even internally, whether an incident resulted from human error, a design flaw, or a model malfunction. If there is one thing to check this week, it would be advisable to pull a list of external systems called by currently operating AI automations and count how many fall outside the pre-approved scope.
Source: Who Is Responsible for Autonomous AI Hacking? Legal Debate Intensifies