Show HN: SkillFortify, Formal verification for AI agents (auto-discovers)
by varunpratap369 | 2026-03-01 05:51:06+00:00
Hi HN,

I posted SkillFortify here a few days ago as a formal verification tool for 3 agent skill formats. Based on feedback, v0.3 now supports 22 agent frameworks and can scan your entire system with zero configuration.

The problem: In January 2026, the ClawHavoc campaign planted 1,200 malicious skills into agent marketplaces. CVE-2026-25253 was the first RCE in agent software. Researchers catalogued 6,000+ malicious agent tools. The industry responded with heuristic scanners — pattern matching, YARA rules, LLM-as-judge. One popular scanner states in its docs: "No findings does not mean no risk."

SkillFortify eliminates that caveat with formal verification.

What it does:

pip install skillfortify
skillfortify scan

That's it. No arguments, no config files, no paths. It auto-discovers every AI tool on your machine across 23+ IDE profiles:

[*] Auto-discovering AI tools on system...
[+] Found: Claude Code (12 skills)
[+] Found: Cursor (8 skills)
[+] Found: VS Code MCP (5 servers)
[+] Found: Windsurf (3 skills)
[*] Scanning 28 skills across 4 tools...

RESULTS
Critical: 2 skills with capability violations
High: 3 skills with excessive permissions
Clean: 23 skills passed all checks

22 supported frameworks: Claude Code, Cursor, VS Code, Windsurf, Gemini, OpenCode, Cline, Continue, Copilot, n8n, Roo, Trae, Kiro, Kode, Jules, Junie, Codex, SuperVS, Zencoder, CommandCode, Factory, Qoder — plus auto-discovery of unknown tools.

Why formal verification, not heuristics: Heuristic scanners check for known bad patterns. Novel attacks pass through. SkillFortify verifies what a skill CAN do against what it CLAIMS to do. Five mathematical theorems guarantee soundness — if it says safe, it provably cannot exceed declared capabilities.

Results on 540-skill benchmark (270 malicious, 270 benign):
- F1 = 96.95%
- Precision = 100% (zero false positives)
- Recall = 94.07%
- Speed: ~2.5ms per skill

9 CLI commands:
- scan — auto-discover + analyze all AI tools on your system
- verify — formally verify a single skill
- lock — generate skill-lock.json (like package-lock.json for agent skills)
- trust — compute graduated trust score (L0-L3, inspired by SLSA)
- sbom — generate CycloneDX 1.6 Agent Software Bill of Materials
- frameworks — list all 22 supported frameworks + detection status
- dashboard — generate standalone HTML security report (zero dependencies)
- registry-scan — scan MCP/PyPI/npm registries before installing
- verify --recursive — batch verify entire directory trees

1,818 tests. 22 parsers. 97 source modules. MIT licensed. Peer-reviewed paper on Zenodo.

GitHub: https://github.com/varun369/skillfortify
PyPI: https://pypi.org/project/skillfortify/
Paper: https://zenodo.org/records/18787663
Wiki: https://github.com/varun369/skillfortify/wiki
Landing page: https://www.superlocalmemory.com/skillfortify

Built this as part of my research on making AI agents reliable enough for production. The companion project AgentAssert (arXiv:2602.22302) handles behavioral contracts — SkillFortify handles the supply chain.

Happy to answer questions about the formal model, framework support, or auto-discovery.