Vlad Filippov

I open-sourced playwright-ai back in 2023 as an experiment: point a CLI at a URL and have an LLM write Playwright tests for it. It worked, but the models it talked to are long gone. Claude 2 and the text completions API are retired, and the OpenAI side relied on an unofficial package.

So I rewrote it. Version 2.0 is on npm now:

npm install -g playwright-ai

What’s new

• Current models. Claude goes through the Messages API and defaults to Claude Opus 5.5. OpenAI uses the official SDK with GPT-5.5. Aliases like --model sonnet or --model haiku work, and any claude-* or gpt-* id is accepted.
• It renders the page. If Playwright is installed in your project, the URL is loaded in headless Chromium first, so single-page apps produce real markup instead of an empty <div id="root">.
• Smaller prompts. Scripts, styles, SVG paths, inline handlers and base64 blobs are stripped before the HTML reaches the model. Most of a page’s bytes carry no selector information.
• It knows your project. The CLI reads testDir from playwright.config, writes TypeScript when your project is TypeScript, and won’t overwrite an existing spec.
• More control. --fix-attempts sets how many times a failing test is sent back for repair. --focus "login form" steers which component is covered first. --no-run only writes files. --dry-run shows exactly what the model would see. --report writes a JSON summary with per-test status and token usage.

Quick start

1. Have a Playwright project: npm init playwright@latest
2. Install the CLI: npm install -g playwright-ai
3. Set ANTHROPIC_API_KEY or OPENAI_API_KEY
4. From the project directory:

playwright-ai --endpoint https://example.com --tests 3

Each component gets its own spec file. The CLI runs it, feeds any failure back to the model, and lists what passed and what you’ll need to finish by hand.

Upgrading from 1.x

Node 22 or newer is required. The default model changed from gpt4 to claude; the old gpt4 and gpt-3 values still work and map to GPT-5.5 with a notice. Specs now land in your config’s testDir rather than always ./tests. Everything else is additive.

What’s next

Try it on a project with no tests yet and let me know what it generates. Issues and PRs are welcome on GitHub.