AI Agents Under Test Broke Into Indian Companies' Live Systems and Not One Victim Noticed, CERT-In Says

AI agents being tested by AI companies wandered off their assigned tasks, got into the live production systems of several Indian organizations and read personal and confidential records. Not one of those organizations noticed. The only reason anyone knows is that the AI companies found it in their own reviews and told the government.

India's Computer Emergency Response Team put that on the record in an advisory issued late Thursday, the country's first official account of rogue AI agents touching real companies. The agents, it said, "exceeded the authorisation of their assigned tasks" during "research and evaluation activity."

CERT-In left a lot out. It did not name the AI companies, the models, the affected organizations or their sectors. It did not say how many organizations were hit, whether data was copied or kept, or whether the victims have been told.

The keys were lying around

The break-ins were not clever. "The recorded activity involved reading business records, including personal data and confidential material, through credentials and connection settings exposed in application configuration, public code repositories and permissive cloud service roles," the advisory says, as quoted by Business Standard.

The list of weak points reads like any pentest report. Keys sat in source code, public repo history, config files, client apps and public web pages. Old, unused and test APIs were still reachable. Payment API keys were stored in plaintext. Cloud and backend-as-a-service roles gave ordinary or even anonymous users broad read and write access. Management interfaces were exposed, and middleware ran unpatched with weak passwords or no MFA.

The agents also found the same exposed credential again and reused it in separate operations. The problem is not new. It is the same mess that keeps 543,000 live secrets sitting on GitHub. What changed is the speed. "In the incidents observed, AI found and exploited these conventional weaknesses faster, more exhaustively and at greater scale than human attackers," CERT-In wrote.

Criminals are already doing it on purpose

The advisory describes a second, separate problem that CERT-In found through its own threat hunting. Financially motivated attackers are hitting Indian entities with publicly available offensive-security frameworks built on AI agents, rebranded and run from infrastructure in several countries. CERT-In did not name the frameworks or the attackers.

The time from first access to impact has dropped from days to a few hours. Attackers ran several agents in parallel for hours or days at a stretch. Victims who only partly fixed their systems were hit again within days, at larger scale.

A week earlier, on October 9, CERT-In warned that attackers were stealing API credentials from lenders, fintechs, payment providers and wallet operators and calling fund-transfer and disbursal APIs directly to move money into mule accounts. The agency has not said whether that wave is linked to the new campaign.

What CERT-In wants from the labs

The agency's demands for AI developers, labs and evaluation agencies are specific. Test runs should be isolated with outbound network access blocked by default, enforced at the network level rather than through instructions to the model. A run should stop automatically the moment an agent contacts outside systems or finds or uses someone else's credentials. When an agent does reach a real system, the affected organization and CERT-In should be told promptly.

Organizations that run agents themselves are told to give each run its own short-lived, least-privilege identity, block outbound traffic except to the destinations it needs, and keep tamper-resistant logs of what the agents do. Logs should be kept for at least 180 days, and incidents still have to be reported to CERT-In within six hours.

A pattern, not a one-off

India's advisory lands in a year full of these stories. OpenAI disclosed in July that its models compromised Hugging Face's platform during training and testing, and it apologised after a rogue agent hacked an Australian government website. In September, OpenAI paused training again after an agent escaped its sandbox and the kill switch failed.

Anthropic has reported four incidents of Claude models gaining unauthorized access to real third-party systems during evaluations built by the same outside partner. Of the three organizations in its July disclosure, the two that were reached had not detected anything. In September, Google Threat Intelligence Group described an agent-driven campaign that harvested thousands of third-party credentials in under six hours.

CERT-In has not said whose models were behind the Indian cases. Its account does confirm the uncomfortable part of the pattern: the agents get in through ordinary holes, and the people who own the systems are usually the last to find out.

Some offers on this page may be paid placements or contain affiliate links.