Three years ago, HiddenLayer's $50 million Series A funding raised eyebrows about the viability of its AI threat detection business model. Critics questioned whether the startup was just preparing for a market that wouldn't materialize at scale.

Fast forward to today, and security companies are scrambling to build products that can monitor not just agents but also the tools and add-ons they use.

While there aren't many headlines about agents being exploited, the risk of agents going haywire during production is very real. Gartner estimates that this year alone, companies will spend $2.83 billion on AI tool security products - a 83% increase from last year's estimated $1.53 billion. The market for tools to prevent AI security breaches has exploded.

Read More: Lachy Groom backs Indian startup aiming to keep aircraft in the air with autonomous inspections

Hiddenlayer, which makes tools protect

HiddenLayer, which makes tools to protect AI models, agents, and workflows from adversarial attacks, vulnerabilities, and malicious code injections, is reaping the benefits of this shift. According to CEO Chris Sestito, the startup's annual recurring revenue grew more than 10x over the past year.

"Inference is still inference," Sestito notes. "So whether it's on a traditional machine learning model or Gen AI, our technology still applies." However, he acknowledges that the company had to adapt its products to address prompt injection, agent manipulation, and malicious tool use.

Financial services and large tech companies are currently HiddenLayer's largest verticals, with contracts also in place with the Department of Defense and intelligence community. The startup has also landed a contract with a "leading frontier model provider" that boasts over 700 million weekly users - a company that sounds eerily familiar to OpenAI or Anthropic.

To fuel its momentum, HiddenLayer has raised $100 million in a Series B funding round led The startup plans to use this influx of capital to expand its sales and distribution efforts, as well as further engineering and research initiatives.

Also Read: Adobe Brings Creative Tools Directly into Slack

"We're looking at things like models purporting to be one thing but actually being another - hidden models inside of models," Sestito explains. "Our new products reflect this shift towards runtime security." The startup's focus on runtime security is particularly crucial as AI deployments become more common across businesses.

While HiddenLayer has secured significant funding, it faces stiff competition from established players like Cisco, Palo Alto Networks, and Check Point, which often prefer to buy rather than build AI security products. Sestito acknowledges that some of his company's offerings could eventually be bundled into larger platforms, but he believes governance features will become increasingly important.

"Our job is to scale vertically alongside artificial intelligence," Sestito says. "Eventually, we'll expand horizontally into parts of cybersecurity that increasingly depend on AI." HiddenLayer's ambitious goal is to establish itself as a leader in the rapidly evolving AI security space.