Taming Rogue AI: How AIUC Is Building the “SOC 2” for AI Agents

As AI agents grow smarter and more autonomous, enterprises face a growing dilemma. They want the productivity gains, but they fear the risks. From jailbreaks to hallucinations to data leaks, the list of potential failures is long and getting longer.

Now, two industry veterans believe they have found the answer. Rune Kvist, an early Anthropic employee, and Rajiv Dattani, former COO of the AI safety research organization METR, have launched the Artificial Intelligence Underwriting Company (AIUC). Their mission is simple but ambitious: to rein in rogue AI agents inside enterprises through independent auditing and certification.

A Timely Launch

The timing is striking. Just a day before TechCrunch met with the founders, Anthropic researcher Jacob Coxon quit his job over concerns that AI could kill us all by the end of the decade. It is a stark reminder of the stakes involved.

Kvist, who is also Dattani’s brother in law, explained the core problem.

“AI is getting smarter at an increasingly rapid rate,” he said. “The surprising thing about AI is that it becomes harder to adopt and harder to control as AI gets smarter, not easier.”

That paradox is exactly what AIUC hopes to solve.

Applying a Proven Model to a New Threat

What caught the attention of investors is AIUC’s approach. Rather than reinventing the wheel, the company is applying a familiar cybersecurity model to a new set of AI risks. It has built a third party audit and certification layer for AI agents.

The startup has already attracted an impressive roster of customers, including Cursor, Lovable, Harvey, and ElevenLabs. On Tuesday, AIUC announced a $40 million Series A led by Ribbit Capital, with participation from First Harmonic. This follows a $15 million seed round from Nat Friedman through his fund NFDG, along with Emergence, Terrain, and Anthropic co-founder Ben Mann, bringing total funding to $55 million.

The SOC 2 of AI

Using the widely adopted cybersecurity standard SOC 2 as its muse, AIUC has developed a standard called AIUC-1 and a testing service to validate agents against it.

To build the standard, AIUC assembled a consortium of about 250 security and risk leaders, essentially the buyers of agents.

“These are the people who we meet with on a monthly basis, and the question we ask them is: When you’re buying agents from someone, what would you look for?” Dattani told TechCrunch. “What are the questions you’d want to ask, and what would you want to see addressed?”

That feedback shapes the tests. The startup then runs an agent through a suite of some 5,000 tests to see how it behaves in scenarios involving jailbreaks, hallucinations, and data leaks. The results produce a roughly 100 page report detailing where an agent performs safely and reliably, and where it does not.

Interestingly, AIUC uses AI agents to run the tests and AI to analyze the data. Humans, however, verify the final audit, Kvist said.

Why Enterprises Are Hesitant

Kvist explained why companies are holding back on AI adoption, even as the technology improves.

“Banks, hospitals, governments and militaries no longer decline to deploy AI because a model isn’t smart enough,” he said. “They decline because they’ve made commitments to their own customers about what a system will and won’t do, and nobody can currently guarantee that.”

That guarantee is what AIUC is selling. Dattani framed it simply.

“Here’s where it passes and where you can trust it,” he said. “And here’s where there’s concerns. You should be aware of those references before you make the decision to buy.”

A Familiar but Different Approach

If this sounds familiar, it is. Dattani’s former employer METR does similar testing for frontier labs, though its work has until recently focused mostly on performance, meaning whether agents can reliably complete tasks. METR was one of the independent research organizations OpenAI used to investigate its Hugging Face incident.

Anthropic CEO Dario Amodei has also recently called for the AI industry to pace frontier development, citing a rapid increase in bad behavior incidents. In his post, Amodei floated the idea of requiring frontier labs to use embedded third party evaluators to observe and verify safety, and named METR as one possibility.

While AIUC is not proposing to embed itself at customer sites, the overall idea is similar. Give enterprises an independent assessment of how safe their AI agents are.

The Takeaway

As AI agents move from novelty to necessity, trust becomes the ultimate currency. AIUC is betting that enterprises will pay for certification the same way they pay for security audits. With $55 million in funding and a roster of high profile customers, that bet is looking increasingly smart.

The question now is whether AIUC can keep pace with the very technology it is trying to tame. If AI gets harder to control as it gets smarter, as Kvist warns, then the auditors may have the hardest job of all.

TechTrib.com is a leading technology news platform providing comprehensive coverage and analysis of tech news, cybersecurity, artificial intelligence, and emerging technology. Visit techtrib.com.

Contact Information: Email: news@techtrib.com or for adverts placement adverts@techtrib.com