Skill

playwright-best-practices

skill:sanity-io/sanity#playwright-best-practices

Use when writing Playwright tests, fixing flaky tests, debugging failures, implementing Page Object Model, configuring CI/CD, optimizing performance, mocking APIs, handling authentication or OAuth, testing accessibility (axe-core), file uploads/downloads, date/time mocking, WebSockets, geolocation, permissions, multi-tab/popup flows, mobile/responsive layouts, touch gestures, GraphQL, error handling, offline mode, multi-user collaboration, third-party services (payments, email verification), console error monitoring, global setup/teardown, test annotations (skip, fixme, slow), test tags (@smoke, @fast, @critical, filtering with --grep), project dependencies, security testing (XSS, CSRF, auth), performance budgets (Web Vitals, Lighthouse), iframes, component testing, canvas/WebGL, service workers/PWA, test coverage, i18n/localization, Electron apps, or browser extension testing. Covers E2E, component, API, visual, accessibility, security, Electron, and extension testing.

SkillsMP

Verdict

Suspicious

Risky behavior the analysis could not rule out.

Analyzed Oct 27, 2026, 23:49 UTC at 64ad107

findings

high

highest severity

threat classes

6,733

files scanned

4,470/4,471

checks satisfied

package advisories

Why this verdict

suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/architecture/when-to-mock.md:187:CRED
suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/core/configuration.md:266:DESTRUCT
suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/infrastructure-ci-cd/github-actions.md:327:SUPPLY
suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/infrastructure-ci-cd/gitlab.md:255:SUPPLY
suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/infrastructure-ci-cd/other-providers.md:252:SUPPLY
suspicious: finding skill:.agents/skills/playwright-best-practices:.agents/skills/playwright-best-practices/infrastructure-ci-cd/reporting.md:370:SUPPLY
suspicious: finding skill:.agents/skills/playwright-best-practices:e2e/README.md:96:HIJACK
suspicious: finding skill:.agents/skills/playwright-best-practices:e2e/tests/releases/utils/methods.ts:192:DESTRUCT
suspicious: unverified finding skill:.agents/skills/playwright-best-practices:e2e/helpers/domPropLeaks/reactDomInternals.ts:162:EXEC

Security findings 9

Credential access

highContextual risklow confidence

The guide instructs users to commit recorded HAR files to version control, and HAR captures of API traffic (which the same section recommends for OAuth flows) contain Authorization headers, cookies, and tokens, exposing captured credentials to everyone with repository access.

.agents/skills/playwright-best-practices/architecture/when-to-mock.md:187-191

Commit `.har` files to version control

vector:instruction

Counter-argument considered

The section recommends HAR recording specifically for OAuth flows and instructs committing the captures; HAR files embed Authorization headers, cookies and tokens, so the instruction persists captured credentials into shared repository history with no sanitization caveat, and no carve-out covers that exposure.

Destructive action

highContextual riskmedium confidence

The documented globalTeardown example runs the shell command `npx prisma db push --force-reset` via child_process.execSync at the end of every test run, force-resetting (wiping all data from) whatever database the project's Prisma/DATABASE_URL configuration points at, which destroys real data if the configuration targets a non-test database.

.agents/skills/playwright-best-practices/core/configuration.md:266

execSync('npx prisma db push --force-reset', {stdio: 'inherit'})

vector:code

Counter-argument considered

The teardown runs 'npx prisma db push --force-reset' unconditionally at the end of every test run against whatever database the project's DATABASE_URL names; a database data-wipe is not among the carve-outs (unlike the CI-gated, compose-scoped volume cleanup), so the destructive scope against a misconfigured target stands.

Code execution

highUnverifiedmedium confidence

Residue R1 adjudicated as unexplained: the e2e DOM-prop-leak helper patches a copy of the installed react-dom development bundle and executes it in-process with vm.runInThisContext to extract react-dom's internal validation tables — a dynamic-code-execution capability the skill's declared purpose does not cover, though it is openly documented and evaluates only local library code already present in node_modules.

e2e/helpers/domPropLeaks/reactDomInternals.ts:162-165

const compiled = vm.runInThisContext(

vector:code

Destructive action

highContextual risklow confidence

The deleteAllReleases test helper queries every release in the environment-configured Sanity dataset and bulk archives and deletes all of them, wiping all release data in whichever dataset SANITY_E2E_DATASET points at.

e2e/tests/releases/utils/methods.ts:192

const query = `*[_type ==

vector:code

Counter-argument considered

The bulk destructive capability is real and remote: every system.release in the env-designated dataset is fetched and archiveAndDeleteRelease is invoked on each; the doc comment and E2E-dataset scoping reduce severity, but remote bulk deletion is not among the accepted carve-outs (which cover only local build/cache rm -rf) and no declared claim covers it.

Supply chain

mediumContextual risklow confidence

The nightly regression workflow template in github-actions.md instructs CI to execute the third-party slackapi/slack-github-action pinned to the mutable @latest ref, so unpinned remote code runs in the user's CI pipeline.

.agents/skills/playwright-best-practices/infrastructure-ci-cd/github-actions.md:327-336

uses: slackapi/slack-github-action@latest

vector:instruction

Counter-argument considered

The nightly template executes the third-party slackapi/slack-github-action at the mutable @latest ref and feeds it SLACK_WEBHOOK_URL; the carve-out covers pinned installs only, so unpinned remote action code executing in the user's CI stands as a supply risk.

Supply chain

mediumContextual risklow confidence

The GitLab CI example configures the test service containers Postgres and Redis with the mutable :latest tag, so the pipeline pulls unpinned remote images at runtime.

.agents/skills/playwright-best-practices/infrastructure-ci-cd/gitlab.md:255

- name: postgres:latest

vector:mixed

Counter-argument considered

The services block instructs the pipeline to pull and run the postgres and redis images at the mutable :latest tag at runtime; unpinned mutable image refs fall outside the pinned-install carve-out, so the supply exposure stands even though it is openly presented.

Supply chain

mediumContextual risklow confidence

The CircleCI example imports the circleci/node orb at the mutable @latest version, executing unpinned orb code in the pipeline.

.agents/skills/playwright-best-practices/infrastructure-ci-cd/other-providers.md:252

node: circleci/node@latest

vector:instructionvector:mixed

Counter-argument considered

Importing the circleci/node orb at @latest injects its commands (e.g. node/install-packages) into the pipeline at whatever the mutable ref resolves to; unpinned orb code is outside the pinned-install carve-out, so the finding stands.

Supply chain

mediumContextual risklow confidence

The troubleshooting example instructs CI to run the third-party GitHub Action dorny/test-reporter at the mutable @latest tag, executing unpinned third-party code in the pipeline.

.agents/skills/playwright-best-practices/infrastructure-ci-cd/reporting.md:370

- uses: dorny/test-reporter@latest

vector:instructionvector:mixed

Counter-argument considered

The troubleshooting example instructs CI to run the third-party dorny/test-reporter action at the mutable @latest tag; unpinned third-party action code executing in the pipeline is outside the pinned-install carve-out, so the finding stands.

Agent hijacking

mediumContextual risklow confidence

The e2e workflow builds a 'Share with an AI agent' PR-comment prompt instructing an AI agent to fetch and act on <report-url>/agent-report.md, whose error messages, stacks, and page snapshots derive from PR-supplied test runs, so instructions injected into a pull request's test output can reach a maintainer's coding agent.

e2e/README.md:96

the PR comment includes a **Share with an AI agent** fenced prompt pointing at that URL

vector:code

Counter-argument considered

The workflow deliberately packages PR-run-derived error messages, stacks and page snapshots into a fenced 'Share with an AI agent' prompt instructing an agent to fetch agent-report.md and fix the tests, so instructions injected into a pull request's test output can reach a maintainer's coding agent; open declaration of the feature does not neutralize the injection channel and no carve-out applies.

Open questions 16

What the analysis could not settle from the code alone.

The Related/Prerequisites sections direct the agent to sibling skill docs (core/configuration.md, core/locators.md, core/test-tags.md, core/assertions-waiting.md, advanced/authentication.md, testing-patterns/api-testing.md, testing-patterns/forms-validation.md, testing-patterns/component-testing.md, architecture/test-architecture.md, debugging/debugging.md, frameworks/react.md, and infrastructure-ci-cd performance.md, parallel-sharding.md, reporting.md, gitlab.md, other-providers.md) that are no⟦… 101 more chars …⟧

The PY-CLAIM-CONTRADICTION leads cite an '**Avoid when**: Tests are read-only...' claim and file_write targets ('[styled-components]', 'Button', '/agent-report.md'); in the shown units these strings appear only as example failure output and as descriptions of a CI-generated test report file, and the cited claim text itself is not present in the shown content, so the contradictions could not be confirmed or refuted from the evidence shown.

Is .github/workflows/e2e.yml consumed by the playwright-best-practices skill or agent at runtime, or is it repository-only CI infrastructure (NOT CONSUMED) rather than part of the declared skill artifact? Its shown content is standard CI (SHA- or tag-pinned official actions, secrets passed only to the project's own deploys), but its inclusion in the skill's repository is unexplained by the declared purpose.

Residue R29 (.env.local file_read, e2e/README.md line 5) is adjudicated necessary: configuring SANITY_E2E_SESSION_TOKEN via shell or .env.local is plainly required for the declared authentication-handling e2e purpose and is declared by claims D131-D133. However, this batch cannot establish whether any agent-facing instruction (e.g. SKILL.md) directs the agent to execute the shipped .github/workflows or dev/preview-iframe app as part of the skill, or whether they ship as inert reference material ⟦… 291 more chars …⟧

Code execution

All 117 units in this batch are inert TypeScript type definitions, type-guard functions, and JSDoc from the Sanity CMS monorepo (packages/@sanity/types), which bear no relation to the target's declared purpose as a 'playwright-best-practices' skill; it cannot be determined from the evidence shown whether this content is intentionally bundled under the skill, is repository context unrelated to the skill, or whether the skill actually ships Playwright content, and no unit shows any executable, net⟦… 93 more chars …⟧

Supply chain

Every unit served in this batch is an inert TypeScript type/interface definition or trivial type-guard from packages/@sanity/types, which bears no relation to the target's declared purpose (a Playwright best-practices skill at .agents/skills/playwright-best-practices); the evidence shown does not establish whether these files are shipped or referenced by the skill itself or are unrelated repository context, so no declared-vs-actual residue could be attributed to the artifact from this batch.

The shown SKILL.md sections are pure navigation tables referencing roughly 50 local markdown guides; only 8 file requests are allowed, so do the remaining referenced guides (e.g. advanced/multi-context.md, core/fixtures-hooks.md, testing-patterns/electron.md, browser-apis/service-workers.md) contain download-and-execute, credential-access, or persistence instructions?

Code execution

Whether the deployed report URL hosting agent-report.md is publicly accessible without authentication, and whether page snapshots or error content in it could expose data beyond the test datasets.

Data exfiltration

Does the CI deploy-report job publish agent-report.md (containing page snapshots, error snippets, and GITHUB_*/SANITY_E2E_* env values) to a URL accessible beyond the team, and is that URL access-controlled?

Data exfiltration

The target is declared as a Playwright best-practices skill, but every owed unit is documentation or pure helper functions from packages/@sanity/types (schema type definitions, asserters, validation types); no Playwright content and no executable behavior chains appear in the shown content, so confirm whether unrelated files were attributed to this target or whether the skill ships additional executable files.

[file request .github/actions/setup/action.yml] Does the composite setup action executed by the e2e and unit-test workflows (uses: ./.github/actions/setup) do anything beyond standard toolchain setup (Node/pnpm install, dependency install), such as fetching remote code or writing host or agent configuration?

Code execution

[file request .github/actions/setup/action.yml] Does the shared local setup action invoked by the e2e workflow only install pinned dependencies, or does it fetch/execute remote code, redirect registries, or read credentials beyond the workflow's declared env?

Supply chain

[file request .github/actions/detect-code-changes/action.yml] Does the local detect-code-changes action referenced by the install job only compute changed paths locally, or does it perform network calls or execute external scripts?

Code execution

[file request e2e/package.json] What do the e2e:setup, test:e2e, and extract-diagnostics scripts executed by the workflow actually run — any network sends, registry redirection, or credential access beyond running Playwright against the deployed preview?

Code execution

[file request dev/studio-e2e-testing/diagnosticsBridge.tsx] Does the studio diagnostics bridge (window.__sanityStudioDiagnostics.gather) referenced by the e2e failure-diagnostics helper only collect in-page diagnostics that get attached locally as test artifacts, or does it send any data off-host?

Data exfiltration

[file request docker-compose.test.yml] Does docker-compose.test.yml (referenced by the webServer command and the CI-gated globalTeardown 'docker compose down -v') declare persistent data volumes or host mounts that the teardown would delete, and is this file shipped in the repository?

Destructive action

Refuted findings 10

Candidates the review stage examined and rejected, kept for transparency.

Credential access

criticalRefutedlow confidence

The Mailinator email-verification fixture in the third-party guide reads the MAILINATOR_API_KEY environment variable and sends it as a Bearer token in live HTTPS requests to api.mailinator.com during test runs.

.agents/skills/playwright-best-practices/advanced/third-party.md:327-332

Authorization: `Bearer ${process.env.MAILINATOR_API_KEY}`,

Why it was refuted

The fixture sends MAILINATOR_API_KEY only to api.mailinator.com, the service that issued it, over HTTPS, as the third-party email-verification integration the declared description explicitly covers ('third-party services (payments, email verification)'); there is no exposure beyond the key's intended use.

Destructive action

criticalRefutedlow confidence

The skill's test-container pattern instructs a CI-gated global teardown that runs a volume-destroying 'docker compose -f docker-compose.test.yml down -v' against a compose file the artifact does not ship, so the scope of what is deleted at runtime is unreviewable.

.agents/skills/playwright-best-practices/architecture/when-to-mock.md:240-247

execSync('docker compose -f docker-compose.test.yml down -v')

Why it was refuted

The teardown is CI-gated and destroys only the volumes of the same docker-compose.test.yml project the config itself starts with 'up --wait', i.e. ephemeral test-container cleanup of the kind the carve-outs accept, and 'global setup/teardown' is an explicitly declared skill capability.

Data exfiltration

criticalRefutedlow confidence

The example PerfReporter collects test titles and performance annotations and POSTs them with CI metadata (GITHUB_SHA, GITHUB_REF) to whatever URL the METRICS_ENDPOINT environment variable names, sending host CI test data off-host whenever that variable is set and pointing at a destination the artifact does not fix.

.agents/skills/playwright-best-practices/testing-patterns/performance-testing.md:397-404

await fetch(process.env.METRICS_ENDPOINT, {

Why it was refuted

The reporter POSTs only test titles, performance annotations and public CI metadata (sha/ref) to an operator-configured METRICS_ENDPOINT, gated on that variable being set, as the in-code '// Send to metrics service' comment states; the destination is chosen by the operator, the payload carries no secrets, and CI performance tracking is the declared subject of this section.

Credential access

criticalRefutedhigh confidence

On unexpected test failure, the fallback diagnostics path reads the Sanity studio session token from localStorage under __studio_auth_token_<projectId> and sends it as a Bearer header on probe requests to the project's api.sanity.work host, so a credential reaches a network sink that is the token's own service and the token value itself is not attached to the report.

e2e/helpers/failureDiagnostics.ts:118-122

const raw = localStorage.getItem(`__studio_auth_token_${projectId}`)

Why it was refuted

The credential only ever reaches its own service: the probe host is hardcoded to the wrapped Sanity service's own domain built from SANITY_E2E_PROJECT_ID, the Bearer header is ordinary authenticated use of that studio's token against that studio's API, and the attached report carries only an `authenticated` boolean, never the token value — wrapped-service credential use, not credential exfiltration.

Credential access

highRefutedlow confidence

The Next.js guide documents committing a .env.test file and a gitignored .env.test.local containing a test NEXTAUTH_SECRET and DATABASE_URL, showing an agent following the skill where credential material lives, though no read-to-network chain or execution sink is present.

.agents/skills/playwright-best-practices/frameworks/nextjs.md:61-69

NEXTAUTH_SECRET=test-secret-local

Why it was refuted

The example follows Next.js convention: the committed .env.test holds only non-secret localhost config while the secret-bearing .env.test.local is gitignored with a placeholder value, so no credential is exposed, and as the claim itself concedes no read-to-network chain or execution sink exists.

Privilege escalation

highRefutedlow confidence

The Electron test fixture documentation instructs launching the application under test with Chromium's --no-sandbox flag, disabling the browser sandbox security boundary during test execution.

.agents/skills/playwright-best-practices/testing-patterns/electron.md:51

args: ['.', '--no-sandbox'],

Why it was refuted

Launching the developer's own app under test with --no-sandbox is the standard, openly documented Electron-test pattern (often required in CI/root containers) inside the skill's explicitly declared 'Electron apps' capability; it weakens the sandbox only for the local test process with no escalation or other sink, so no harmful behavior remains.

Credential access

highRefutedlow confidence

The Electron launch fixtures copy the full ambient process.env into the launched application's environment, exposing any secrets present in the agent's environment variables to the app process under test.

.agents/skills/playwright-best-practices/testing-patterns/electron.md:53

...process.env,

Why it was refuted

Spreading process.env into electron.launch only reproduces default child-process environment inheritance for the locally launched app under test — the same process the developer launches when running the app — with no network, log or other external sink, so no credential is exposed beyond baseline process semantics.

Supply chain

highRefutedlow confidence

The performance-testing documentation instructs injecting into pages under test a script fetched at runtime from the mutable unpkg URL web-vitals@3, loading remotely hosted code that is not fully version-pinned.

.agents/skills/playwright-best-practices/testing-patterns/performance-testing.md:66-69

url: 'https://unpkg.com/web-vitals@3/dist/web-vitals.iife.js',

Why it was refuted

Loading the official web-vitals library from unpkg is the library's documented usage for the skill's explicitly declared 'performance budgets (Web Vitals, Lighthouse)' capability; the @3 major-range pin limits mutability and the code executes only in the tested page's browser context, not in the agent or CI runtime.

Credential access

highRefutedlow confidence

The e2e README directs users and agents to store a Sanity session token in the repository-root .env.local file, which the test tooling reads at runtime, placing a live credential where the instructed agent reads it.

e2e/README.md:5

add it to the `.env.local` file in the repository root

Why it was refuted

Placing the Sanity session token in the repo-root .env.local is the conventional gitignored local-env mechanism for the tested service's own credential and is exactly the env capability the artifact declares for SANITY_E2E_SESSION_TOKEN; the token's only sink is legitimate test authentication, so no exposure chain exists.

Credential access

highRefutedlow confidence

The e2e README instructs the reader to obtain a Sanity session token via `sanity debug --secrets` or a project API token and store it in `.env.local`, so an agent following it places a live credential in a repository-root env file.

e2e/README.md:7

`sanity debug --secrets` will give you the CLI token provided you are logged in `sanity login`

Why it was refuted

Retrieving one's own Sanity CLI token (or a scoped project API token) to run the declared e2e tests is intended use, and this exact text is itself a declared claim of the artifact; the token's only sink is the local .env.local for test authentication, so no harmful chain exists.

Coverage gaps 7

Parts of the artifact the static analysis could not fully examine.

context:unanalyzable_binary binary content is only byte-scanned

dev/test-studio/schema/debug/components/fonts/Rye-Regular.eot

context:unanalyzable_binary pdf content is only byte-scanned

packages/sanity/src/core/form/inputs/PortableText/__tests__/copyPaste/static/dummy.pdf

context:unanalyzable_binary binary content is only byte-scanned

packages/sanity/src/core/form/inputs/PortableText/__tests__/copyPaste/static/dummy.zip!/__MACOSX/._dummy-image-1.jpg

context:manifest_unparseable yaml: YAML input exceeds 1048576 bytes

pnpm-lock.yaml

no_dataflow_for_dynamic_code line 162: invoked code is not structurally extracted

e2e/helpers/domPropLeaks/reactDomInternals.ts

ledger_finding_without_valid_finding run 1: declared-vs-actual.12 SRC:8812e3edce6f515d

invalid_judge_finding declared-vs-actual.12