Research

What we test and how we test it

We take a threat informed approach to our testing, focusing on the most likely and impactful threat scenarios you will face. Testing includes:

  • A real offensive taskAn agent is given an objective in a controlled environment and the tools to pursue it, and we measure what it actually does.
  • Real assets to compete againstOur deception artefacts compete against the real assets AI targets, such as credentials and sensitive data, to emulate real customer environments where an AI-enabled threat has more than one legitimate path to take.
  • Across all leading frontier modelsFor the full list of models covered, see model coverage.
  • A baseline for every attack runNo result is counted unless the attacker succeeds almost every time with no deception technology present.
  • Multiple agentic frameworksThe agent's scaffold is a variable in its own right, so results are not an artefact of one harness.

Across all threat scenarios, our canary and AIM technology's average effectiveness is currently:

Detect 98.1%
Delay +244%
Deny 86.2%
Extract 81.0%

Threat scenarios

Threat scenario

Unpatched vulnerability exploitation

An AI-enabled threat exploits an unpatched vulnerability in your website.

We emulated an AI-enabled threat exploiting an RCE on a website, and gaining code execution on the underlying Linux webserver.

In this scenario, our deception technology detected the threat 100% of the time, forced the AI to stop 87.2% of the time, increased the time taken to laterally move by 480%, and extracted intelligence from the AI 89.9% of the time. The intelligence extraction detailed the exact vulnerability and website path the AI exploited, and every action it had taken since it gained code execution. This would enable the defenders to rapidly remediate the initial access vector, and ensure the threat is eradicated.

Real world example: OpenAI's AI agents compromise Hugging Face. In this example, the AI agents exploited two code-execution vulnerabilities to get internal access, and then began enumerating the internal host and network. At this point Velkiro would have detected the AI, halted it in its tracks, and forced it to create a document that detailed how it got in and everything that it had done.

Threat scenario

Insider threat

A misaligned AI agent attempts to go outside of the authorised scope.

We emulated an authorised AI agent attempting to go outside of scope due to misalignment, via finding and leveraging credentials to escalate its privileges.

In this scenario, our deception artefacts detected the AI agent 98.9% of the time, and the AI agent was able to be contained either via our AIM technology or by leveraging a simple SOAR playbook.

Real world example: OpenAI Agents escape sandbox. In this example, OpenAI's agents went out of scope due to misalignment and searched for, found and used credentials to escalate their privileges. The moment this occurred, Velkiro could have detected the rogue agents, and containment could have been enforced via AIM or a SOAR playbook.

Threat scenario

Searching for credentials

A human threat actor is searching your fileshares for credentials.

We emulated a human threat actor searching fileshares to identify credentials and further compromise the network, with common tools such as TruffleHog, Snaffler, ShareFinder and more.

In this scenario, Velkiro's deception artefacts were placed in the fileshares along with real credentials that could be used for lateral movement. Because the deception artefacts were placed where we know these tools look first, they were discovered before the real credentials 97% of the time.

This triggered high-fidelity alerts 97% of the time, and when combined with SOAR, enabled effective containment of the human threat.

Real world example: Enumerating file repositories has been a prolific and effective technique for human threat actors for years. In fact, some studies rate network share discovery as one of the most common techniques used by ransomware groups, scoring it 8 out of 10 for prevalence. For AI, the Hugging Face incident involved credentials being found in files on internal servers and used to expand the attack.

Threat scenario

Unauthorised access via VPN

An AI-enabled threat uses compromised credentials to log in via an external VPN.

We emulated an AI-enabled threat actor using real credentials to access an internal Windows server via VPN. The threat actor then attempted to laterally move to further compromise the network.

In this scenario, our deception technology detected the AI 98.4% of the time, forced the AI to stop 77.8% of the time, and increased the time taken to laterally move by 109%. Not only that, but it forced the AI to detail how it had gotten in and everything it had done up until that point 75.4% of the time. This would enable the defenders to know exactly which user accounts were compromised and rapidly revoke them, and to identify any other persistence to eradicate, ensuring the incident is contained.

Real world example: Gemini agents achieve unauthorised access via account compromises. In these examples, rogue Gemini agents breached two entities via finding credentials in public code repositories, and one via a password guessing attack. These credentials were used to log in and gain unauthorised access to internal resources. At this point, Velkiro would have detected the AI, stopped its attack, and forced it to create a document that detailed the credentials it used to get in and everything that it had done.

Model coverage

ProviderModels tested
OpenAI GPT-5.5, GPT-5.6, GPT-5.6 Sol, GPT-5.6 Terra, GPT-5.6 Luna, GPT-6 Astra, GPT-6 Luna, GPT-5
Anthropic Claude Opus 4.6, Claude Opus 4.7, Claude Opus 4.8, Claude Opus 5, Claude Sonnet 5, Claude Sonnet 5.5
Meta Muse Spark 1.1, Muse Spark 1.2, Muse Spark 1.3, Llama 3.2
xAI Grok 4.3, Grok 4.5, Grok 4.6, Grok 4.7
DeepSeek DeepSeek V4 Pro, DeepSeek V4 Flash, DeepSeek V4.1 Flash
Google Gemini 3.1 Pro Preview, Gemini 3.7 Flash, Gemini 3.8 Flash
Z.ai GLM-5.1, GLM-5.2, GLM-5.3, GLM-5.3 Flash
Moonshot AI Kimi K3
Qwen Qwen3.8 Max

Coverage is extended as new frontier models are released. All marks are the property of their owners. No affiliation or endorsement is implied.

What we will not publish

We publish which models we test against and the average scores. We do not publish how any individual model responds.

A per-model breakdown could provide valuable insight to a threat actor, so we keep that private. If you would like to view per-model scores, they can be requested during a demo.

Kill two threats with one bird

Book a demo today

A walkthrough of how it works, the research behind AIM Technology, and how it can protect your business from human and AI-enabled threats.