Bug bashes

A bug bash runs many e2e explore sessions at the same time, each on one area of the app. A coding agent then writes a test for each finding and reports it as a bug only when the test fails. This page ports upstream’s docs/bug-bash.mdx.

Before you start

• Make sure the app can serve many explorers at the same time. Start the app one time and set reuseExisting: true on the target’s app.command, or set port 0 in the app URL so each run starts its own copy. When CI is set, e2e ignores reuseExisting. Use port 0.
• On mobile, declare one target for each simulator or emulator. Each explorer needs its own device.
• Each explore session uses model calls.
• On a deployed site that other people use, do not change data.

The agent does the rest of the setup, such as seeded accounts and the bug-bash config, from the skill.

Install the skill

The coding agent runs the bug bash from the e2e-dotnet skill, which e2e init installs. To install it without init:

npx skills add hardkoded/e2e-dotnet

Start a bug bash

Give a prompt to your coding agent:

Bug bash the checkout and account settings on this branch.

What the agent does

1. Reads the routes of the app. On a branch, it also reads the diff.
2. Writes 5 to 10 charters. A charter is a one-sentence goal for one area, with one posture: first-time user, numbers and copy, edge input, state after reload, or error paths.
3. Writes e2e.bugbash.config.json, a copy of your config with one agent for each posture (--agent), then runs one e2e explore for each charter, 4 at the same time, each with its own output directory (--output).
4. Merges the findings of all charters.
5. Rejects findings that have a known cause. The cause can be a limit of the explorer (for example, a link that opens a new tab), the local environment, the intended design, or the seed data.
6. Writes a repro test that asserts the expected behavior for each remaining finding. It can start one subagent per area, each with its own e2e mcp session to check locators.
7. Runs each repro test. The finding is a bug only when the test fails with ASSERTION_FAILED.
8. Reports, in this order: confirmed bugs, production risks that it cannot verify locally, rejected findings by reason, and warnings.

Each confirmed bug has its repro test and steps. It also has the explorer’s screenshot and video when the engine records them.

Caution

The video is not masked. If the explorer typed a password, check the video before you share it.

To read the full procedure, run e2e guide bug-bash.

Repro tests

A repro test fails until the bug is fixed. The agent puts each one in a BugBash folder of the test project and tags it bugbash. Keep them out of the run that gates merges:

dotnet test --filter "TestCategory!=bugbash"

After the fix, the test passes. Remove the tag and keep the test as a regression test.

Differences from upstream

The config is e2e.bugbash.config.json, not a TypeScript file that spreads the project’s config. The port has no saved sessions (--session), no --reporter (so no summary.md), and no tests key; --output <dir> writes the record report.json beside <dir>. See COMPATIBILITY.md.