Skip to content

Exploring without a test

e2e explore runs the agent against your app with a goal instead of a test method. The agent plans a step, does it, and records findings until the goal is covered or a budget ends, then writes an assessment. It uses the same model, tools, and action rules as Agent.ActAsync. This page ports upstream’s docs/explore.mdx.

Name the flow and the user goal, and add limits if necessary:

Terminal window
e2e explore 'Explore the checkout flow like a first-time buyer and report anything off'

A limit can be “stop before placing the order.” With no goal, explore uses Explore the app and find bugs. A failed step does not stop the run.

The terminal shows each finding and each finished step as they happen. An issue fails the run with exit code 1; a run that ends blocked exits 1, 2, or 3. A warning does not. The final report puts the most severe issues first. Each finding has the steps to reproduce it and, when possible, a screenshot. Compare each finding with its evidence before you act on it. For the other exit codes, the JSON report, and artifact paths, see the explore reference.

When the target’s engine can resize the viewport (the web engine can), a goal can ask for layouts at other widths:

Terminal window
e2e explore 'Check the pricing page at desktop, tablet, and phone widths for broken layouts'

Add --changed-since <ref> to explore what the branch changed. The planner reads the commit messages and the committed diff since the merge base with the ref. It spends its steps on the screens that the change can affect.

Terminal window
e2e explore --changed-since origin/main

From code, set ExploreOptions.ChangedSince:

var outcome = await Explore.ExploreAsync(new ExploreOptions { ChangedSince = "origin/main", MaxSteps = 4 });

Only a defect the change can cause is an issue. A defect elsewhere in the app is a warning and doesn’t fail the run.

With no goal, the goal is Explore what this change affects and find bugs in it. The planner can then skip a change no user can see, such as docs or tests only. The run ends Skipped and passes. It skips only when it read the whole change. A large diff, a binary file, or an uncommitted file rules a skip out.

The branch’s own commit messages and code can sway the call. To tell a skip from a clean exploration, read outcome.Explore.Ended (Skipped for a skip).

What the model reads: the commit messages, the changed files’ names, and the committed patches since the base, cut to fit its context. A file no commit changed, such as one CI wrote, and a committed file’s later local edits go by name only, never read. Secret values the config gives as strings, and runs of 8 or more of their characters, are redacted. A secret a provider returns reaches the model.

On GitHub Actions, run it in its own job on pull_request. The GitHub reporter posts the findings as their own pull request comment, as upstream does: e2e explore runs the reporters the config lists in reporters. The --reporter flag is not supported, so list the reporter in the config.

.github/workflows/explore.yml
on: pull_request
jobs:
explore:
runs-on: ubuntu-latest
permissions:
contents: read
pull-requests: write
# An issue exits 1. Keep the findings as a review aid, not a merge gate.
continue-on-error: true
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
fetch-depth: 0
# ...install, browsers
- run: e2e explore --changed-since origin/${{ github.base_ref }} --max-steps 4
env:
GITHUB_TOKEN: ${{ github.token }}

fetch-depth: 0 puts the base branch in the checkout. Without it, the run fails with INVALID_CHANGED_SINCE.

Keep the working tree clean after install and build, or the run is never skipped. Ignore .e2e/ and build output.

A run stops after 8 steps or 10 minutes. To reach further into the flow, raise --max-steps <n> (up to 100) or --timeout <ms> (180000 to 7200000, 2 hours).

A run with time left after the main flows returns to them with edge-case inputs, combined filters, and reloads. It finishes early only when nothing new is reachable.

Explore uses the targets, app command, models, and agents from your config. Select them with --target <name> and --agent <name>. Without --target, explore uses the first target. Mobile and Windows targets need ExploreOptions.Engine when you call it from code. The other flags (--headed, --output, --debug, --ai-trace, --trace, --video) are in the explore reference linked above.

The command has no --session. From code, set ExploreOptions.Session to the name of a session you restored into the engine you pass. The planner and the explorer are told the app starts signed in. The port runs no setup test and restores nothing: the engine you pass must already hold the session.

  • Bug bashes: explore a whole app or branch with many explorations, and prove each bug with a test.