WITH SKILL$ skillbench challenge ./release-check --runs 31.00
PROVEN+0.60
✓ verification.json existsPASS✓ status contains READYPASS✓ final mentions live evidencePASS
iThe runtime package is copied faithfully; evals/, the rubric, and the other run stay hidden. Use Codex by default or run the same suite with --runner claude.
Treat every skill as untrusted input.
Skillbench reads first and executes nothing. The same deterministic gate protects local installs, registries, and pull requests.
$skillbench audit ./unknown-skill --fail-on high
CRITICALcredential-file-accessBLOCK
CRITICALdownload-pipe-shellBLOCK
HIGHprompt-instruction-overrideBLOCK
HIGHexternal-symlinkBLOCK
static only / package was not executed
One gate, three questions.
01
VALID?Portable metadata, references, and eval contracts.
02
WELL AUTHORED?Optional strict structure without pretending it is a standard.
03
SUSPICIOUS?Secrets, exfiltration, unsafe shell, sandbox bypass, Unicode, and path escape.
Heuristic preflight, not a malware verdict. Hostile packages still belong in a disposable external sandbox.
A real project, not a demo fixture.
We installed Skillbench from npm, generated a project-specific skill for
malikov.tech, and gave the same responsive carousel repair to fresh agents with and without it.
Make carousel status follow the last card a reader can actually see.
The project rule was precise: use the last slidesInView() item, require
inViewThreshold: 0.5, and share that position across the counter, note, progress, and terminal navigation state.
DISCOVERY
8 / 8
positive + near-miss cases
BASELINE
0.50–0.80
varied across two fresh runs
WITH SKILL
1.00 / 1.00
all deterministic checks passed
OBSERVED Δ
+0.20–+0.50
same fixture, hidden rubric
WITHOUT PROJECT MEMORY
One run found visibility events but missed the 50% readability threshold.
→WITH THE SKILL
Both runs reproduced the complete responsive rule and scored 10 / 10.
N=2This is a transparent dogfood result, not a universal benchmark claim. The baseline variance is exactly why Skillbench keeps the fixture, rubric, and repeated runs inspectable.
REDUNDANT
Our own example failed the usefulness challenge.
verify-real-outcome scored 100% with and without the skill over two counterbalanced runs. Fewer tokens did not offset slower execution, so the honest verdict is not “pass”. Inspect evidence JSON.
QUALITY Δ
0.00
TOKENS
−3.1%
LATENCY
+7.1%
CHALLENGE EXIT
1
POLIMAT CHECK: its existing ticket skill now passes portable validation; strict lint separately reports missing Skillbench-specific sections. Fixing that false incompatibility was more valuable than inventing another benchmark win.
Then the mobile bug became a skill.
responsive-release-proof encodes the missing invariant: real desktop/mobile browser evidence, scrollWidth === innerWidth, console health, an interaction, screenshots, and an honest non-ready state when no browser exists.
N=2 AB/BAThe skill improved both orders, but used 26% more tokens and took 45% longer. Challenge belongs at an evidence/release boundary; the deterministic security gate belongs on every PR.
Use it from the TUI or CI.
Start guided, drop to stable JSON when the workflow belongs in automation.
/
01Create
skillbench new
02Inspect + gate
skillbench check ./.agents/skills/release-check --strict --fail-on high
03Test discovery
skillbench eval ./.agents/skills/release-check --runner claude