Cyvia Research: notes from the lab, published as we learn
Research

We work in the open.

The thinking behind the platform, in public. Where enterprise AI is heading, what we are learning from real deployments, and the research that shapes what we build.

Benchmarks & evals

Numbers with the method attached.

Every benchmark ships with the corpus, the setup, and the parts we did not test, so the result can be checked rather than taken on faith. We refresh these as models and datasets move.

LIVE BENCHMARKS · SELECT ONE TO EXPAND
MODEL ROUTING★ LIVE · 19 SEP
Against an industry-leading general-purpose router

Fixed three-model catalog, comparator configured by an adversarial reviewer, every answer graded blind.

90%
sensitive handled
92.5%
routine sufficient
75%
cost saved
Full method & gapsExpand ↓
DATA SECURITY★ LIVE · 16 AUG
Indian PII detection, format by format

74 files in production shape, vs a leading data-security platform. Coverage diverges where the file type hides the data.

1.20×
more matches
119:1
GSTIN in .txt
5×
Aadhaar .xlsx
Full table & methodExpand ↓
CLASSIFICATION★ LIVE · 30 JUL
Head to head with a leading platform

PII detection & classification across prompt and output surfaces, on a mixed India & ANZ dataset.

+19.1pp
micro recall
95.0%
F1 score
5.6×
faster
Full table & methodExpand ↓
IN PROGRESS
INTENT DETECTIONEVAL IN PROGRESS
Campaign-level intent vs. per-prompt guardrails

Recall on multi-turn jailbreak campaigns, measured against single-prompt classifiers. Publishing with the attack set.

Expected Q4 2026
AGENTIC RISKEVAL IN PROGRESS
Plan-level exfiltration & poisoning catch rate

How often plan-level checks stop a poisoned instruction before it executes, versus content filtering on the output.

Expected Q4 2026
COST & LATENCYEVAL IN PROGRESS
Cascade cost curve at corpus scale

Compute and latency per million documents as the cascade routes cases, benchmarked against a single frontier pass.

Expected Q4 2026
★ LIVE · 19 SEP MODEL ROUTING · VS AN INDUSTRY-LEADING GENERAL-PURPOSE ROUTER

The router wins its own benchmark

On a fixed three-model catalog, with the comparator configured by an independent adversarial reviewer and every answer graded blind, Cyvia won all three lanes enterprises buy on. Preregistered, hash-traceable, and reproducible on your own approved catalog.

90% / 62.5%
Sensitive scenarios handled correctly (36/40 vs 25/40)
92.5% / 70%
Routine work answered sufficiently (37/40 vs 28/40)
75% saved
Weighted token units per 1,000 sufficient answers
200/200
Decision-record audit: every call carries a named rule, policy version & config hash
HOW IT WAS RUN

Fixed three-model catalog, so availability and price are held equal and only decision quality remains. The comparator was configured by an independent adversarial reviewer from public documentation, given no Cyvia material. The judge saw no system names, model identities, costs, or latency.

WHAT WE DON'T CLAIM

Policy fidelity (25/40 vs 19/40) is a consistent but directional win, expected to widen when the semantic matcher ships. Outside-window notices trail at 8/15 vs 10/15, a scoped gap. The cost figure is weighted token units labelled to Round 4b; it is reported as a consumption measure, never as dollars.

Margins of six answers or more over 40 scenarios are decisive; sensitivity and sufficiency clear that bar. Cost basis: 403,608 vs 1,654,818 weighted token units per 1,000 sufficient answers, Round 4b, fixed catalog.

★ LIVE · 30 JUL CLASSIFICATION · MIXED INDIA & ANZ DATASET

PII detection and classification, head to head

Detection and classification across prompt and output surfaces on a mixed India and ANZ dataset, scored against a leading platform. The cascade reads the messy, local entities other tools skip, and does it faster.

+19.1pp
micro recall over the comparator
95.0%
F1 score
5.6×
faster at corpus scale
Full cascade write-up, dataset shape, and the layer-by-layer method. Read the full writeup →
★ LIVE 74 PRODUCTION FILES · VS A LEADING DATA-SECURITY PLATFORM

Indian PII detection on real files, format by format

74 files in production shape, four classifiers, scored on unique matches. Coverage was close on most formats and diverged sharply where the file type hides the data. We take Data Lens output as ground truth here, so read this as coverage overlap, not a scored win, the numbers are what a customer can reproduce on their own estate.

1,072 / 894
Unique matches, Data Lens vs the other platform (1.20×)
119 / 1
GSTIN in .txt files, the biggest blind spot
19 / 4
Aadhaar in .xlsx, nearly 5× the coverage
1.06×
PAN, a near tie, high-volume and well covered by both
UNIQUE MATCHES BY FILE TYPE · DATA LENS / OTHER PLATFORM
EXT
n
AADHAAR
PAN
GSTIN
{{ r.ext }}
{{ r.n }}
{{ r.aadhaar }}
{{ r.pan }}
{{ r.gstin }}

Scored on unique matches, the conservative comparison; occurrence totals widen the gap further. Driving licence omitted as negligible (3 hits in the set). GSTIN and Aadhaar rows in amber mark the format blind spots.

Three benchmarks live, more in flight. We publish each with its dataset and its limits as it lands.

View all benchmarks →
The posts
View all posts →
+19.1pp
CLASSIFICATION ECONOMICS
The economics of classifying with small models instead of frontier LLMs

Why a four-layer cascade beats brute-forcing every document through a frontier model, on both accuracy and cost.

30 Jul 2026Read →
intent lineage
INTENT RESEARCH
Why one-prompt-at-a-time security misses campaigns

A single prompt looks benign. Read the lineage across a session and the campaign gives itself away.

30 Jul 2026Read →
POL-EXFIL-001
AGENTIC RISK
The day an agent tried to email data it should not have

A tainted document, a helpful agent, and the plan-level check that caught the exfiltration before it ran.

30 Jul 2026Read →

Get it as we publish it.

New research, benchmarks, and field notes, in your inbox when they land.

✓ Thanks — you're on the list.