Vetting agent skills with SkillSpector before you install

Every week a new repo asks you to drop a folder of instructions and scripts next to your agent. Claude Code, Codex and MCP clients then run those skills with your files, credentials and browser session in reach. Nobody gates that step by default, so I tested the tool built to gate it. SkillSpector, NVIDIA’s security scanner for agent skills, sat at 16,912 GitHub stars as of September 11, 2026. I installed it and pointed it at Anthropic’s own official skills repo. The scan returned a risk score of 100 out of 100, CRITICAL, DO NOT INSTALL. That result is the most useful thing I have seen all month, because Anthropic’s repo is not malware. What the scan actually measured was the scanner’s noise floor. Knowing that number changes how you read every report it prints.

What SkillSpector actually checks

The pitch is simple: answer “is this skill safe to install” before it touches your agent. A skill is a folder with a SKILL.md instruction file, often plus executable scripts. The scanner treats that whole bundle as untrusted input. Under the hood it runs two stages. Fast static checks cover every file, and an optional LLM pass looks for issues that need intent comparison. The static layer covers prompt injection, data exfiltration, privilege escalation, supply-chain issues, memory poisoning and MCP tool poisoning. The docs page counts 68 vulnerability patterns across 17 categories in total. The README says 71. That mismatch is small, but I am quoting it anyway. A security tool should count its own detectors consistently, and both numbers were live on September 11, 2026.

The project is moving fast. Releases v2.11.0, v2.11.1 and v2.11.2 landed between August 28 and September 9, 2026. The issue tracker shows same-day fixes. Input is flexible, covering git repos, URLs, zip files, directories and single files, with terminal, JSON, Markdown and SARIF output. It also ships a baseline file, which records findings you reviewed and dismissed so that re-scans only surface what changed. That baseline is the difference between a tool you try once and a gate you can operate.

I ran it on the most official repo I could find

Every scanner demo I have seen scans something obviously evil and reports obvious evil. That proves nothing about noise. So I pointed it at anthropics/skills, the official repo from the company that defined the format. The scan covered 420 components and returned 231 issues: 47 HIGH, 143 MEDIUM and 41 LOW. If a clean, official repo cannot score SAFE, the score is a triage signal, not a verdict.

The category spread tells you where static pattern matching misfires. Data exfiltration led with 63 findings, excessive agency followed with 45, then rogue agent at 38. I opened the HIGH findings expecting something scary and found documentation instead. An ECMA XML schema in the docx skill got flagged HIGH for prompt injection, presumably because schemas are verbose XML. A CLI teaching doc got flagged HIGH for credential handoff, since it shows curl with an Authorization header. A tool-use reference got an anti-refusal hit for text that tells the agent not to skip warnings. None of it is malicious; all of it is text that looks like a pattern. The scan still earns its keep, though, because it also surfaced each skill’s executable scripts and imports. That is exactly the inventory you want before installing someone else’s folder.

The research and the incidents behind the tool

The risk is measured, not theoretical. The README quotes an arXiv study from January 2026. Its team collected 42,447 skills from two marketplaces, then analyzed 31,132. Of those, 26.1 percent carried at least one vulnerability. Another 5.2 percent showed high-severity patterns that suggest malicious intent. The same study found skills bundling executable scripts are 2.12 times more likely to contain vulnerabilities than instruction-only skills. That multiplier is the part a marketing-ops person should internalize. A skill that only advises the model is a prompt risk, while a skill that ships scripts is software. Software needs vetting.

The incidents have been real since January. Security researchers documented malicious skills in the ClawHub registry, which targets OpenClaw users. The first 28 went live on January 27-29, and 386 more followed by February 2, 2026, per the OpenSourceMalware writeup. All of them delivered credential-stealing malware while masquerading as crypto-trading tools.

The newest scare extends past skills entirely. On September 2, 2026, Tom’s Hardware reported that Fortune-500 agents ran arbitrary code after reading poisoned llms.txt guidance files. A trusted instruction file, it turns out, is an attack surface. I covered what the format actually does for AI citations in my llms.txt plumbing post. The lesson stacks: any text an agent ingests as guidance is code, because the agent executes it.

Building the install gate

Agent skill security only works as a habit, and a scanner only helps inside a gate. Here is the one I would run before any third-party skill enters a stack. It borrows the blast-radius logic from my post on human-in-the-loop review gates. The rule there was to gate by what an action can touch rather than by task. Skills are no different. A skill that only formats markdown is a paragraph risk, while a skill with scripts and network calls is software. Software needs the full treatment.

• Scan the repo with skillspector scan <url> --no-llm before reading any of its docs, so the marketing does not prime you.
• Triage HIGH findings by category first: prompt injection, data exfiltration and privilege escalation deserve eyes; anti-refusal hits on docs usually do not.
• Inventory the executable scripts and every URL or domain they contact, and treat unknown domains as disqualifying until explained.
• Record accepted findings in a baseline file, so the next scan only surfaces what changed.
• Re-scan on every skill update, because a repo you vetted in August can ship malware in October.

That last step matters more than the first scan, and the skill supply chain has already shown why. The Nx build-tool compromise in August 2025 ran exactly this playbook on npm. Malware used the Claude Code CLI to explore filesystems while stealing credentials, per Semgrep’s incident writeup. Anyone who has managed npm dependencies will recognize the pattern. Registries get abused, scanners ship, teams add gates, and the next wave finds a fresh door. The same logic applies to MCP servers, as I learned while building my MCP reporting pulls. Anything that joins your agent at tool level gets the scan before the install, not after the incident.

Where scanners fall over

The uncomfortable result of my run is that the score is not calibrated the way a newcomer would assume. NVIDIA’s own tracker agrees. Open issues describe documentation code spans triggering false findings (#515). Others flag inert security-test strings as executable behavior (#523). Another report describes version metadata flipping a skill’s score from SAFE to CAUTION (#524). The maintainers are clearly aware and shipping fixes, with three releases in two weeks. Even so, the false-positive volume stays high enough that a DO NOT INSTALL banner risks training people to ignore it. I also skipped the LLM semantic stage in my run, so I cannot say whether it prunes the noise. The tool’s own warnings said as much during the scan.

Static analysis has a second blind spot: it cannot see intent, and intent is where a clever attack lives. A skill whose SKILL.md says one thing while its scripts do another is the two-stage threat the study documented. No pattern list catches a mismatch it has no baseline for. Treat the scanner as the cheap pass that catches the loud 90 percent. Keep a human on the quiet 10 percent. For me the tool earns its place in the stack anyway, because the alternative is installing blind. My agent token cost experiments taught me the same lesson in another currency: cheap checks upstream beat expensive incidents downstream.

Filed under agents, with every scan and figure dated September 11, 2026.