Skill

red-teaming-llms-with-garak

skill:mukul975/anthropic-cybersecurity-skills#red-teaming-llms-with-garak

Runs NVIDIA garak probe suites (jailbreak, prompt injection, data leakage, toxicity, and more) against an LLM endpoint - Hugging Face models, OpenAI-compatible APIs, or Bedrock - then interprets the resulting hit-rate report for triage. Use when baselining LLM security before/after deployment, validating that a guardrail or fine-tune reduces jailbreak/injection success rates, or producing evidence for an AI risk assessment.

SkillsMP

Verdict

Suspicious

Risky behavior the analysis could not rule out.

Analyzed Oct 27, 2026, 22:29 UTC at 54a7988

findings

medium

highest severity

threat classes

4,532

files scanned

91/91

checks satisfied

package advisories

Why this verdict

suspicious: finding skill:skills/red-teaming-llms-with-garak:skills/red-teaming-llms-with-garak/SKILL.md:55:SUPPLY
suspicious: finding skill:skills/red-teaming-llms-with-garak:skills/red-teaming-llms-with-garak/SKILL.md:58:SUPPLY

Security findings 2

Supply chain

mediumContextual riskmedium confidence

The skill instructs installing garak from PyPI with an unpinned upgrade flag, so the agent pulls whatever the latest published package version is at install time rather than a reviewed pin.

skills/red-teaming-llms-with-garak/SKILL.md:55

python -m pip install -U garak

vector:instruction

Counter-argument considered

Case against: the command sits in the skill's own Prerequisites so it is a normal, openly documented install of the tool the description says it runs. Case for: the carve-out is limited to pinned official-registry installs, and the verbatim command uses -U to pull whatever is latest at install time with no pin, no declared claim covers version/source policy, and no mitigation exists; an agent following the skill is concretely instructed to run it, so the finding stands as a contextual supply risk.

Supply chain

mediumContextual riskhigh confidence

The skill instructs installing garak from the mutable main branch of the NVIDIA/garak GitHub repository via pip, fetching unpinned remote code whose content can change after review.

skills/red-teaming-llms-with-garak/SKILL.md:58-60

python -m pip install -U git+https://github.com/NVIDIA/garak.git@main

vector:instruction

Counter-argument considered

Case against: it targets NVIDIA's official garak repository in a fenced install block, so one might argue a vendor-source carve-out. Case for: the reference is @main with -U, pinned to no commit and no registry artifact, so its content can change after review; neither the pinned-official-registry nor vendor-installer carve-out covers mutable branch installs, no declared claim (D3/D4 cover only API keys and authorization) or mitigation addresses it, and it is a concrete instruction in the skill body, so the finding stands.