Measuring where LLMs
miss vulnerabilities.
SecLens is MatterSec research for role-specific evaluation of LLMs on real vulnerability detection. It shows why teams cannot rely on a single model score when deciding whether AI can safely participate in software security work.
Real CVEs. Role-specific decisions. Security-first metrics.
SecLens measures where models miss vulnerabilities. The MatterSec 50 catalogs how agents exploit those failures in production work. Together they form the full evidence base.
The best model depends on who is asking.
A model that looks strong for engineering velocity can still be weak for security ownership. MatterSec brings that intelligence into live agent workflows, where model blind spots become code risk.
CISO
Weighs evidence of exploitability and audit trail over raw model capability.
Chief AI Officer
Balances agent productivity against the cost of letting unsafe code ship.
Security Researcher
Cares about per-CWE recall, false-positive rate, and ground-truth provenance.
Head of Engineering
Wants to know which agent + which guardrails get the team to production safely.
AI as Actor
Each agent should know its own weak categories before it touches code at all.