Compatibility

Upstream parity target: tester-army/e2e commit ee901b333ed12352418dfb07bda5eafe32e71c8d (docs(agents): show several personas running one suite (#1258)). On 2026-10-19 the 300 upstream commits from 94ddbfe to it were checked: each commit’s title and release note against this file, and against the port’s code where the note named a behavior. A commit that only changes docs, tests of code the port lacks, dependencies, or release notes needs no entry. The sections were not re-read line by line against upstream’s source, so a statement that no commit in that range touched is as it was at 94ddbfe. The packages upstream has are in Upstream packages.

Names are C# versions of the JavaScript API: agent.act is ActAsync, screen.getByRole is GetByRole, expect(locator).toContainText is Expect.That(locator).ToContainTextAsync, and unique() is Values.Unique.

Package map

JavaScript	.NET
e2e and @e2e-dev/web	E2E (WebEngine is Playwright, in the same package)
NUnit [Test]	E2E.NUnit.E2ETest. The fixture commits the replay cache from the NUnit result
xUnit v3 [Fact]	E2E.XUnit.V3.E2ETest. The fixture commits the replay cache from the xUnit result
e2e.config.ts	e2e.config.json, the same keys as JSON. E2ETest finds and applies it; fixture properties override it
Vercel AI SDK model	OpenAiCompatibleModel, OpenAiResponsesModel, AnthropicModel, GoogleModel, and BedrockModel, built from provider in JSON or ModelProviders in code. See Models
e2e/oauth/chatgpt, copilot, grok, opencode-console	E2E.OAuth.Subscriptions
e2e login, e2e logout, e2e models, e2e guide, e2e mcp, e2e explore, e2e-web install	E2E.Cli, a .NET tool whose command is e2e. e2e explore is ported in part: see Explore. e2e guide [topic] prints the bundled skill, as upstream; an unknown topic exits 2. e2e-web install is e2e install: see Browser install command. e2e mcp is ported apart from video recording tools, project tools, and e2e init: see MCP server
@e2e-dev/kernel	E2E.Kernel, KernelProvider: Kernel hosted browsers on the IBrowserProvider seam. See Kernel
@e2e-dev/eas	E2E.Eas, EasSimulators: see EAS Simulators
@e2e-dev/limrun	E2E.Limrun, LimrunSimulators: see Limrun
@e2e-dev/decision, @e2e-dev/smol, @e2e-dev/lightpanda, @e2e-dev/github	Not ported: see Upstream packages
—	E2E.TestMu.Mobile, TestMu AI mobile devices through Appium. Upstream’s @e2e-dev/testmu serves the same devices another way: see TestMu mobile
—	E2E.Mobile.NUnit and E2E.Mobile.XUnit.V3, MobileAppTest: a ready base class for a device test. Not in upstream: see MobileAppTest
—	E2E.Mobile.DevFlow, DevFlowAgents: .NET MAUI apps through the MAUI DevFlow agent. Not in upstream: see MAUI DevFlow
—	E2E.WinApp, WinAppEngine: Windows desktop apps through Windows UI Automation. Not in upstream: see WinApp engine
—	E2E.WinApp.NUnit and E2E.WinApp.XUnit.V3, WinAppTest: a ready base class for a Windows app test. Not in upstream: see WinAppTest
@e2e-dev/testmu (testmuBrowsers(), browsers.ts)	E2E.TestMu, TestMuBrowsers: TestMu AI hosted browsers on the IBrowserProvider seam. Ported in part: see TestMu
—	DocumentEngine, an in-memory page for hosts that do not want a browser

Ported

• NUnit [Test], [SetUp], [TearDown], [OneTimeSetUp], and [OneTimeTearDown]
• Assert.Ignore, [Retry], [Timeout], and [Category]
• Screen queries: role, text, label, test id, placeholder, display value, filter (hasText and has), or, first, last, nth. A locator offers every query kind, scoped to its matches’ descendants. Every query takes a TextMatch: a string or a .NET Regex, tested against the whitespace-normalized text. A string matches the whole text, case-sensitively, unless exact is false, which makes it a case-insensitive substring, as upstream. RoleOptions carries name, checked, disabled, selected, expanded, pressed, and level, and img is read as image. A role query never matches a hidden node; the other kinds keep hidden nodes unless Visible is set, as upstream. getByLabel matches any node whose accessible name matches, as the upstream reference resolver does. Options are C# objects (RoleOptions, TextMatchOptions) instead of object literals, and filter({ hasText, has }) is the two overloads Filter(TextMatch) and Filter(Locator). Regular expressions use .NET syntax, not ECMAScript. GetByTestId also takes Exact, which upstream’s getByTestId does not. locator.or(other) (upstream #1049) is Or(Locator): a union expression that matches either branch once, in document order, and stays strict (STRICT_MODE where upstream says LOCATOR_AMBIGUOUS; add First() when both may match). Both locators must come from the same screen and attempt, else INVALID_LOCATOR. LocatorResolver.Resolve and DeviceLocator.ResolveExpression resolve the union. Differences: the parameter is typed, so upstream’s refusal of a non-locator has no port (null is an ArgumentNullException); and upstream’s UNSUPPORTED_CAPABILITY for a display-value branch has no port, since the port’s engines resolve over their tree. Web branches with different frame-selector chains fail with UNSUPPORTED_CAPABILITY (locator.or requires both branches in the same document) as upstream’s frameSelectors does
• Actions: tap, click (an alias of tap), doubleTap (DblClickAsync is an alias), fill (a string or a Secret), pressSequentially, press, check, uncheck, clear, selectOption, focus, scrollIntoView. They wait up to the action timeout for exactly one enabled, visible match. selectOption takes a string that matches an option value or label. ActionOptions.Timeout overrides the wait per action (PressSequentiallyOptions adds Delay). The web engine’s own Playwright wait still uses the configured action timeout
• Locator reads: textContent, inputValue, getAttribute, isVisible, isHidden, isEnabled, isDisabled, isChecked, boundingBox, count, all, allTextContents. A read of one node (textContent, inputValue, getAttribute, isEnabled, isDisabled, isChecked, boundingBox) waits for its one match until the action timeout, then reads once with no retry for a value; isVisible, isHidden, count, all, and allTextContents answer from the current screen at once. No read verifies an earlier act. textContent and allTextContents collapse each whitespace run to one space and trim, as upstream. A single-node read fails with NOT_FOUND when nothing matches before the action timeout ends
• screen.tree() and screen.debug() (commit d7ec0ec) are Screen.TreeAsync and Screen.DebugAsync. One observation of the whole screen, rendered by FailureEvidence.Listing (the trace’s screen at failure, upstream’s prepareObservation): one node per line, children indented, secrets redacted, cut at the observation byte limit (262144). A node the page does not show is left out. A step of kind screen. DebugAsync writes the text to Console.Out, where a test framework picks it up. Differences: Timeout bounds the read with a linked token and fails with OPERATION_TIMEOUT, where upstream passes it to the engine operation. A Locator has no TreeAsync, so upstream’s refusal on a locator is a compile error. On a FrameScreen they refuse with INVALID_ARGUMENT, as upstream’s do. The same commit’s change to print test output without the list reporter, and to keep stdout for --reporter json, is not ported: the port has no runner. screen-tree.test.ts is ScreenTree/ScreenTreeTests and ScreenDebugTests (not ported: the refusal on a locator, the unknown option, and sdk-types.ts’s checks, which a typed API makes compile errors); screen-tree-secrets.test.ts is ScreenTreeSecrets/ScreenTreeTests (through a session over a fake engine). run-user-output.test.ts’s two new cases are not ported with the runner change.
• screen.scrollUntilVisible and locator.scrollUntilVisible (locator/screen.ts) are Screen.ScrollUntilVisibleAsync and Locator.ScrollUntilVisibleAsync, with ScrollUntilVisibleOptions { Direction, Momentum, Timeout } (down, slow, 30 s by default). Momentum (None, Slow, Fast; upstream’s Momentum) moved from E2E.Mobile to E2E.Engine beside ScrollDirection, and LocatorAction.Swipe carries it. IEngineSession.SwipeAsync(direction, momentum, ct) is the viewport swipe with a stride; its default drops the momentum. WebEngine strides as upstream’s swipeDistance (none half, slow three quarters, fast one and a half of the extent; a swipe naming none keeps three quarters), MobileEngine strides a node swipe by it and keeps the device client’s default for the whole-screen swipe, and DocumentEngine and WinAppEngine have one stride. The port has no operation context, so a swipe’s deadline is the cancellation of the token it runs under, and a swipe the deadline ends fails with an OPERATION_TIMEOUT cause. A locator swipe’s engine error is not wrapped in ACTION_FAILED, since the port has no translateLocatorError. filter({ has }) now refuses a locator of another screen with INVALID_LOCATOR, as upstream’s ownLocator does
• locator.waitFor with attached, detached, visible (default), and hidden, as WaitForAsync(new LocatorWaitForOptions { State, Timeout }). The timeout defaults to the action timeout. LocatorWaitForOptions is named apart from agent.waitFor’s WaitForOptions. Differences: a timeout fails with TIMEOUT (upstream LOCATOR_NOT_FOUND, locator did not become <state>), and a passing waitFor does not verify an earlier act (upstream it does). The timeout carries Details as upstream’s does (the locator, what it asked for, and how long it waited; see Trace page)
• Locator expectations: visible, hidden, attached, enabled, disabled, checked, selected, expanded, focused, text, contained text, value, attribute, accessible name, and count. They poll until the assertion timeout, or the matcher’s own timeout. Not inverts a matcher, which then passes after 1000 ms of continuous truth (or the whole budget when it is shorter). Text matchers take the same TextMatch as queries, ignoreCase, and a list form. The boolean flags are visible, attached, enabled, and isChecked (checked is a C# keyword)
• toHaveScreenshot as Expect.That(Screen).ToHaveScreenshotAsync(name, options) and the same on a locator, with Not and Expect.Soft(locator); see Visual comparison
• expect.soft for every locator expectation, as Expect.Soft(locator), including .Not and the per-matcher timeout. An ASSERTION_FAILED is kept and the body runs on; any other error still throws. E2ETest records each one on the NUnit result, as inside Assert.EnterMultipleScope, so the test fails when the body ends and lists every failure. E2E.XUnit.V3.E2ETest keeps them on the session and fails the test when it is disposed. As upstream, a failed soft assertion outranks a skip: Assert.Ignore or Assert.Inconclusive after one fails the test in NUnit, and the message keeps the skip reason beside the failures. In xUnit v3, Assert.Skip after one fails the test the same way. Other hosts set E2ESessionOptions.OnSoftFailure or call E2ESession.CloseSoftFailures
• expect(browser) as Expect.That(Browser): ToHaveURLAsync (a string or a Regex, with ignoreCase and timeout), ToHaveTitleAsync (a TextMatch, with timeout), and ToHaveClassAsync(locator, TextMatch), .Not, and Expect.Soft(Browser). Like the locator matchers, a passing one verifies the previous act, and .Not passes after 1000 ms of the opposite. A read that outlasts the matcher’s deadline ends it, so a page that stops answering fails at the assertion timeout
• expect.poll as Expect.Poll(read, options), with Timeout, Interval, and Message. A read that throws is retried. The matchers are ToBeAsync, ToSatisfyAsync, and .Not; E2E.NUnit adds ToMatchAsync for any NUnit constraint, such as Is.GreaterThan(3). The default timeout is 5 seconds, not the configured assertion timeout, because the poll does not see the running test, and it does not stop at the test deadline unless a cancellation token is passed
• agent.act, agent.assert, agent.waitFor, agent.extract. ActOptions has Params, Timeout, MaxSteps, MaxModelCalls, and Agent; AssertOptions has Timeout and Agent; WaitForOptions has Timeout, Interval, MaxModelCalls, and Agent; ExtractOptions has Timeout and Agent. See Agents
• Act tools observe, scroll, scroll_to, and back. They are offered when the engine declares EngineCapabilities.Scroll or EngineCapabilities.History. scroll_to with a target scrolls it into view. With text, it pages the viewport, or the target list, until a node reading the text is listed, then scrolls it into view. It stops when the screen stops moving
• Replay cache for a verified act: role, name, test id, and path. Modes are self-finalized, agent-concluded, and missed
• Values.Unique and Secret. Values.Unique rejects an empty or whitespace value, and a value that contains the cache slot marker (U+0001, the port’s form of {{param:). A secret value has at least 6 code points: Secret.Create throws INVALID_ARGUMENT, a config secret INVALID_CONFIG, and Credentials.User INVALID_CONFIG for a short E2E_USER_*_PASSWORD. The values of the secrets an act received in Params are redacted from every later prompt of the attempt, including the assert, waitFor, and extract statement and the agent’s system and context. A secret filled through a locator, or read with Secrets.Get and never passed to act, is not added to that list. Redaction matches any case, JSON escapes, HTML character references, percent encoding, and collapsed inner whitespace. A known marker is never rewritten. The same secrets are redacted from the node names and test ids an act records as targets and end anchors in its replay cache entry, and a target, on replay or from the model, is matched against the screen in that redacted form. The same secrets are redacted from the failed matcher messages of locators and Expect.That(Browser), a toHaveScreenshot failure, a locator.waitFor timeout, the soft-failure aggregate, and the messages of the inner exceptions under them. Upstream redacts error text once, at its report boundary (serializeError), which the port has not, so each of these places redacts its own text. The aggregate redacts when the test ends, so a secret the attempt learned after a failure was kept is covered. expect.poll has no session, so a poll failure redacts the secrets of every live session in the process, as upstream’s setErrorRedactor does for the process. A cause with a secret in its message is rebuilt with the redacted text and keeps its type and code when it is an EngineException, AgentException, or ConfigurationException, otherwise it becomes a TestException with code ERROR, and loses its stack trace. A failed secret fill in WebEngine reports the Playwright message with the value and its fragments of 8 or more characters redacted. A node name cut at 256 characters, and a text or value cut at 512, that ends with the leading part of a secret (8 characters or more, or half of a shorter secret) shows that secret’s marker in its place, as upstream’s redactCut does, in the model’s snapshot, the recorded targets and anchors, and the locate result. StreamRedactor and Redactor.HoldFrom (upstream’s SecretLedger.holdFrom, with maxFormLength as MaxFormLength) hold back the tail of text written in pieces so a value split across two writes is still caught. The port’s Redactor plays upstream’s SecretLedger here (the port’s own SecretLedger only wraps one for the MCP server), so StreamRedactor takes a Redactor, and it takes bytes as a ReadOnlySpan<byte>. Upstream uses them for what user code prints: a test worker’s stdout and stderr, and the e2e mcp process’s output. The port has no worker, and e2e mcp sends Console.Out to stderr as it is, so nothing in the port writes through them yet. The server’s own messages are redacted whole where they are written
• app.open, app.back, app.restart, app.clearState, and app.baseUrl (App.BaseUrl), and the context platform. app.restart and app.clearState reopen the base URL inside their own step, as upstream’s steer, so no app.open step follows them
• The browser fixture subset: goto, reload, back, forward, url, title, waitForURL, evaluate, waitForResponse, route, unroute, unrouteAll, cookies, setCookies, setViewport, emulateMedia, addInitScript, keyboard.press, keyboard.type, keyboard.down, keyboard.up, and mouse.move, wheel, down, up
• Agent budgets: ActOptions.MaxSteps, ActOptions.MaxModelCalls, and WaitForOptions.MaxModelCalls. A per-call budget can only lower the agent’s configured one. A higher or non-positive value throws INVALID_ARGUMENT
• An act past its action budget ends STEP_BUDGET_EXHAUSTED, blocked. The model is told and may still conclude: a passing verdict, or a failure without a code, becomes the budget error. Replayed actions draw on the same budget. navigate, back, scroll, and scroll_to take a slot; observe does not. An act that needs one model call more than its budget also ends STEP_BUDGET_EXHAUSTED, blocked (agent.act exhausted its model-call budget of N)
• As upstream, an act guards against a model that repeats itself. The same call with the same input 3 times in a row, or a sequence of 2 to 4 calls twice, adds a warning. 5 identical calls, or a sequence 3 times, offers the model only done, so it still writes its own verdict. The done call never counts. The failure-streak guard is the port’s own: it counts every failed action and empty turn of the act, not only the trailing run, and only asks for a verdict, without limiting the tools. An executor’s transcript is kept only under E2E_DEBUG=1, as upstream keeps it under --debug.
• ActResult.ModelCalls (0 for a full replay) and ActResult.Actions (replayed and live actions, counting failed attempts)
• AgentException.Blocked and AgentException.Explanation (the same text as Message)
• OpenAI-compatible tool calling

Replay cache

• E2E_CACHE_DIR replaces cache.dir as upstream’s does: it resolves from the config directory, a leading ~ or ~/ is the home directory, a blank value is ignored, and the cache mode stays as configured. Not ported: --cache-dir on e2e run and e2e cache (the port has no such commands), the refusal beside cache.store (the port rejects cache.store), the overlap check against output (the port has no output), and the REPLAY_STALE advice that names the cache directory.
• A passing locator expectation, agent.assert, or agent.waitFor after act verifies it, and a verified act is written. Locator reads, locator.waitFor, and agent.extract verify nothing. agent.assert, waitFor, and extract always run live. As upstream, verification stops when the test fails, so a check in a derived [TearDown] after a failure records nothing.
• Replay runs back and a viewport scroll as recorded, re-finds a scrolled list before each repeat, and pages again for a scroll_to text. Consecutive identical scrolls are recorded as one action with a repeat count. observe is not recorded. Upstream scrolls the viewport when a list that filled the screen cannot be re-found. Replay does not use node geometry, so a lost list stops the replay.
• When a replay ends in end-mismatch and the agent repairs it with more actions, the entry is evicted instead of rewritten; the next clean run records the flow again.
• The key holds the cache schema, the replay policy version, the test, the engine (its platform, never its version, as upstream since e77c17a), the instruction, the params, the agent’s name, a SHA-256 digest of the agent’s context with the known secrets redacted, and the repeat index. Another agent, or a changed context, records again. The repeat index counts per agent and context. Replay policy version 4 re-keys every entry, so the first run after the upgrade records them again: an entry recorded before it may hold a page load as its delta.
• A recording checks the step’s delta, as upstream: up to eight nodes that appeared and up to eight that went away, alerts and status messages first, each with its value (never a secure field’s), secrets redacted from names and values, and its checked, expanded, pressed, and selected states. A node counts as gone only when no node on the end screen still matches its anchor. A replay passes on its own only when the end route matches, every appeared node is present, every gone node is absent, at least one of those changes happened during the replay (counted from the first screen the replay saw on the route it ended on), and no alert shows that was not there before and that the recording never saw. The web engine does not observe Next.js App Router’s route announcer (the next-route-announcer element), so its title echo never counts as an alert. A value the replay typed counts as a change only when the step changed nothing else. An alert’s dates, times, durations, and ids are read as placeholders. A step that changed no node and stayed on its route records nothing, and an entry read for an act that then passed with nothing to record is evicted.
• As upstream, each side lists alerts and status messages first, then leaves, then containers. It skips text that reads differently on every run (ids, durations, dates, clock times, bare numbers, and a count that is the data rather than the step’s result) while a stable anchor remains. An anchor is a visible node with a name, a text, a test id, or a placeholder, and records its role, name, text (when it differs from the name), test id, placeholder, input purpose, value, and states. A node the other fields identify is matched without its test id, so a test id that changed between renders is forgiven; a node with only a test id is matched by it. The screen a step passed on is read once it holds still. Upstream reads a recording’s delta from the step’s settled start screen, and measures a replay’s evidence from the replay’s start. The port reads both from the screen the step’s first action was resolved against: a page that shows a loading state for a few hundred milliseconds after load holds still long enough to pass a held-still look, and a delta read from it is the page load, which a replay finds again before its own effect renders. A step that resumes a replay reads its delta from the screen the replay’s first action was taken on. A node whose name or text holds a Values.Unique value is not an anchor.
• ActResult.Cache carries ReplayedActions, TotalActions, and NotRecorded (param-collision), as upstream’s step cache detail does. It also carries Detail, as upstream’s step.cache.detail does. On a step that read an entry it carries RecordedBy, the model that recorded it, as upstream’s step.cache.recordedBy does (see Trace page). For a replay that stopped at an action it names the action as the hand-off notice does, since the port records no summary (at action 2 of 3, tap button "Apply", then : and the first line of the error when the action threw). After an end-mismatch it names the recorded nodes that did not show (missing alert "Acme Notes"), the ones that did not go away (still showing ...), the alerts the recording never saw (unrecorded alert ...), or that the outcome already showed before the replay acted. A store whose read throws, or that returns an entry of another schema or with no actions, is a miss with invalid-entry.
• Only the first attempt replays. A [Retry] attempt runs live and still records, and its acts report missed with reason retry. Each [Repeat] iteration is a first attempt.
• A recording that opens with navigate replays from any route. Any other recording needs the start route.
• At the end of an attempt that passed, failed, or was skipped, verified acts are written and the unverified acts that recorded or replayed are evicted. An act that missed the cache and then failed leaves its key alone. A model outage or a cancelled test writes and evicts nothing, unless an assertion also failed on the app: then the unverified acts are evicted. An act is reported evicted only once the store deleted its entry (see “Recording outcome” under Trace page).
• An entry that a verified replay finished, or that already holds the same flow, is not rewritten, except to write in the step fields an older entry lacks.
• As upstream, the screen is read by one settle policy per action, for live acts and replays. After a tap, a key press, a select, a check, a navigation, or a back, the next settled look waits up to 2 s for the screen to leave the shape the action was resolved against, then reads until two looks 75 ms apart agree, for at most 1 s. After a fill it waits for the change only; after a scroll, up to 500 ms; after a secret fill, for no change. Node references, focus, and clock-like values do not count as a change, and after an action an empty document does not count as a settled screen. Upstream also counts a screenshot as the shape; the port counts the scroll position instead, so a scroll that moves no node still ends the change wait. An act’s start, the screen after each action, each page of a scroll_to with a text, and the recording’s end screen are held still; a replay settles the look before each action as the previous action’s policy asks, takes a held-still look between the repeats of a scroll, and reads the end state raw.
• A replay waits up to ReplayTimeout (15 s; a .NET-only fixture property, not a config key) for each recorded target and for the recorded end state. Upstream waits 15 s for the end route and sizes the anchor wait from the recording.
• FileStepCache.Write is upstream’s FileCacheStore.write (cache/store.ts): it writes the entry through AtomicWrite.WriteFileAtomic, a sibling temporary file moved over the entry file, so a crashed or concurrent writer leaves no torn entry, and two writers on one key leave the last write, as upstream’s store documents. Upstream’s store awaits writeFileAtomic; IStepCache.Write is synchronous, so the port uses the synchronous twin (upstream’s writeFileAtomicSync, which its CLI and telemetry stores use). Windows-only difference: a POSIX rename replaces the target in one step, but Windows refuses a move over a file that another move is replacing (ERROR_ACCESS_DENIED) or that a reader holds open (ERROR_SHARING_VIOLATION). Upstream’s rename does not retry, so there the losing writer throws and the recording is lost. The port retries such a move up to ten times with a growing pause (about 90 ms in all) before the error is the writer’s, so the write lands and the step reports Saved where it would have reported NotWritten. A denial that lasts, such as a read-only target, reports the same error after the same wait; any other error reports at once; on other systems the retry never runs. .NET-only names: AtomicWrite.MoveAttempts, AccessDenied, SharingViolation, and RetriesMove (the Windows errors and the retry), Utf8 (the encoding both twins write with), and TemporaryPath (upstream’s temporaryPath).
• cache.mode read-only replays but never writes or evicts, including the end-mismatch eviction. off neither replays nor records. Under cache.strict, a stale recording fails the act with REPLAY_STALE and stays in place; a strict run never writes or evicts, and its retries replay. A no-entry miss fails too when the cache directory holds a recording of the same step under another key. Entries record the step as the test, the engine, the instruction, the params digest, the repeat index, and the agent; the port stores the engine where upstream stores the target id, and the instruction text where upstream stores its digest. Only an entry that records the whole step counts, and, outside cache.strict, a verified whole replay in read-write mode writes the missing fields into an older entry. A custom IStepCache is not checked. The message names what changed between the entry’s keyedBy and this run’s key context, as upstream’s keyContextChanges does, and says “the runner, the engine, the app, or the agent’s context” for an entry with no keyedBy.
• The entry’s file name is upstream’s traceCacheKeyHash (cache/identity.ts): the SHA-256 of the RFC 8785 canonical JSON of a trace-1 key (buildTraceCacheKey) of the test, target, engine name and contract version, app identity, instruction digest (instructionDigest, normalized by normalizeInstruction), params digest (paramsDigest), repeat index (createCallIndexer, counted per agent name and redacted context), agent, agent context digest, and replay policy version. The port’s key differs from upstream’s in these parts only: project is the digest of the empty projectId (the port has no projectId); appIdentity is the digest of an empty identity and environment (the port has no app.identity or environment, so the app’s URL never keys an entry); testId is the test title, not <file>::<title path>; targetId is the target’s name and engineName the engine’s platform (the port has no engine manifest); policyVersion is the port’s own 4. Params are projected before the digest as the port’s cache has always projected them: a Values.Unique value is <unique> and a Secret is <secret:name>; a number is written by .NET’s shortest round-trip form, which matches ECMAScript’s for the values a param holds. The entry stores keyedBy (TraceKeyContext) as upstream does; an entry with a malformed keyedBy still reads, without it. Recordings made before this key (the port’s former text-hash key) are not read and are not migrated: they miss as no-entry and must be re-recorded. Under cache.strict they do not fail as REPLAY_STALE either: the check matches the step by its recorded params digest, which the old entries hold in the old form, so the step runs live.

Visual comparison

ToHaveScreenshotAsync compares the viewport, or one locator’s box, against <test file>-snapshots/<name>-<target>-<os>.png, with upstream’s names, OS suffixes (darwin, linux, win32), tolerance rules, retries until the screen matches or holds still, masks, and failure images. The pixel comparison is upstream’s (the pixelmatch threshold and anti-aliasing rules) over PNGs read and written with SkiaSharp. The ported testbed PNGs are upstream’s. Differences:

• A 16-bit PNG is scaled to 8 bits by Skia, which can differ from rounding by one level. Chromium writes 8-bit PNGs.
• DocumentEngine throws UNSUPPORTED_CAPABILITY. MobileEngine over agent-device captures the masked screenshot with normalizeStatusBar on, so the status bar is held fixed; the TestMu (Appium) and MAUI DevFlow clients ignore it and show the device’s real status bar.
• The test file is the file the call is compiled in ([CallerFilePath]), not a runner’s test entry. A call in a helper stores beside the helper. A build that rewrites source paths (DeterministicSourcePaths) breaks it.
• The update flag is E2E_UPDATE_SNAPSHOTS=1, or the fixture property UpdateSnapshots, in place of --update-snapshots (-u). There is no CLI flag or config key.
• Failure images go to .e2e/results/<result>/attempt-<n>/ (the fixture property ResultsDirectory moves it). They are not attached to a step, because the port has no step report. A screenshot written earlier in the run is tracked per process, not per run directory.
• Expect.That(screen) is the only screen matcher. There is no Expect.Soft(screen). Expect.That(frameScreen).ToHaveScreenshotAsync (the scope Browser.FrameLocator returns) fails with INVALID_ARGUMENT, as upstream’s does, and TreeAsync and DebugAsync on it do too (screen.tree reads the whole screen; call it on screen).
• Secure fields are covered by the engine (Playwright masks, in every frame, including a field inside a closed shadow root, which the engine reaches through the e2e-closed= selector engine it registers with Playwright before it opens a browser) and by painting the observed secure nodes. A closed root made by declarative shadow DOM (<template shadowrootmode="closed">) is parsed, not attached, so the hook never sees it and the mask cannot reach it, as upstream. Upstream counts masked regions against secure nodes before it keeps the image; the port counts them (EngineScreenshot.MaskedRegionCount), and a vision judgment compares them (see Agents).
• After a secret fill, ToHaveScreenshotAsync, the assertion screenshots, and the failure screenshot stay on, as upstream since 58a5cec: pixels are withheld only from a model (Exposure.WithholdsModelPixels, upstream’s withholdsModelPixels). They mask secure fields only. ToHaveScreenshotAsync fails with POLICY_DENIED and keeps no screenshot when the engine masked fewer regions than the screen lists secure nodes, or does not say (EngineScreenshot.MaskedRegionCount is null), as upstream’s pixelsProvenMasked does. The failure screenshot applies the same condition, but leaves the image out of the report instead of failing.
• The name is a string? and the options a ScreenshotOptions object, so an unknown option and a non-locator mask cannot be written.
• Locator boxes inside an iframe are in the top-level viewport (past the frame’s border and padding), where they used to be relative to the frame, as upstream’s change for this feature does.

Defaults

Setting	Default	Upstream
Test timeout (timeout)	120 s	Same
launchTimeout	60 s	Same
actionTimeout	30 s	Same
assertionTimeout	5 s	Same
cleanupTimeout	30 s	Same
maxModelCalls	25	Same
maxSteps (actions per act)	25	Same
judgmentTimeout (assert, waitFor, extract)	30 s	Same
waitFor interval	3 s. After the first judgment, the judge runs again only on a changed screen	Same
act timeout	30 s (StepTimeout, .NET-only)	The test timeout
Replay wait	15 s (ReplayTimeout, .NET-only)	15 s for the end route; the anchor wait is sized from the recording
expect.poll timeout	5 s	The assertion timeout

E2EDefaults holds these values, except the expect.poll timeout (PollOptions); E2EConfig, E2ESessionOptions, and E2ETest start from it.

Error codes that differ

	Upstream	.NET
A locator action, read, or expectation matched more than one node	LOCATOR_AMBIGUOUS	STRICT_MODE
A locator action or read matched no node	LOCATOR_NOT_FOUND	NOT_FOUND
locator.waitFor timed out	LOCATOR_NOT_FOUND	TIMEOUT

Not ported

• The e2e command-line tool apart from login, logout, models, guide, and mcp: the custom runner, [E2ETest] discovery, --grep, test.only, and the JSON report
• @e2e-dev/mobile apart from its snapshot model and engine (see Mobile snapshot model and Mobile engine), @e2e-dev/github
• e2e init (see Coding agents), e2e upgrade, the update notice, e2e run --watch, e2e run --max-duration and maxDuration, e2e feedback, and upstream’s Claude and Codex plugins (plugins/e2e, .claude-plugin). The port has no runner, and ships its skill with npx skills add
• The other upstream packages and what they do: see Upstream packages
• Vision on act (the screenshot and point tools). app.screenshot() is not ported either. Video is ported on the web engine and on MobileEngine with agent-device’s own recording: see Video. Vision on judgments is ported: see Agents. ToHaveScreenshotAsync is: see Visual comparison
• Parallel workers and serial suites
• The full locator action set (dragTo, swipe, hover, setInputFiles, secondaryTap, longPress, pointer points). secondaryTap and longPress have no counterpart in the document engine
• The screen-level screen.tapAt and screen.swipe
• Per-action options other than timeout (and delay for pressSequentially): click modifiers on a locator (the agent’s tap and double_tap hold them, above), tap position, and the { label, value, index } form of selectOption
• Diff-only observations. Each model turn receives a full text snapshot
• Telemetry
• STEP_NOT_AWAITED, the error that fails a test body, hook, or fixture teardown that returns before its expect.poll finished, and cancels that poll. The step recorder has AbandonRunning and records a step the test did not await as STEP_NOT_AWAITED, but no fixture calls it yet, and expect.poll is not a step. Nothing cancels a poll the test does not await, so it keeps reading until its timeout or the CancellationToken passed to it. The compiler warns (CS4014) only for a bare, un-awaited Task call in an async method
• Value expectations (expect(value).toBe, toEqual, toMatchObject, toHaveProperty, toMatchSchema, and the rest) and their expect.soft form. Use NUnit Assert.That with constraints, and Assert.EnterMultipleScope (or Assert.Multiple) for soft value checks. expect.poll takes NUnit constraints through ToMatchAsync in their place
• Asymmetric matchers (expect.any, anything, objectContaining, arrayContaining, stringContaining, stringMatching). Use NUnit constraints such as Is.InstanceOf, Is.Not.Null, Has.Property, Is.SupersetOf, Does.Contain, and Does.Match
• The upstream reporter and GitHub pull request comment. The trace page is ported in part: see Trace page

Upstream test-only commits

The tests of upstream commits adcfc2ba, c9445ce6, a13e119c, 1731e0d8, fe2c3450, and 5b6a731e that the port does not run, and why. The tests that follow upstream are in the PRs #517 (adcfc2ba, c9445ce6), #521 (a13e119c), #516 (fe2c3450), #510 and #511 (5b6a731e). 1731e0d8 changed nothing the port has.

Ported in part (the test name carries Partial_ or a PARTIAL suffix):

• keeps the first verdict: a later complete_step cannot turn a failure into a pass and refuses a blocked verdict without a blocking error code and leaves the step open for a real one: the port’s verdict tool keeps no state and sends no result text, so the tests check what the loop does instead (the step outcome, the model-call count, and the text of the rejections) and not the verdict result text upstream reads
• refuses $sink before reading the plaintext: the port has no plaintext resolver to count reads and records no policy decisions
• never sends a secret value to the model: runs on a scripted model, not a real one
• diverges with target-not-found when relocation never matches: no observations == 9 check, because the port retries every 20 ms up to ReplayTimeout
• masks a password inside an iframe in artifact screenshots and observed pixels: the port has one screenshot path
• pins the id of a non-ASCII title path and rejects an empty name, or an empty, repeated, or blank-entry agent list at registration: a .NET test has one title, and [Agent] takes one name
• uses CI defaults for retries, workers, trace, and video: the port has no workers, and its trace default is deliberate (see Defaults)
• refuses both spellings side by side, and an explicit path that does not exist: the port has one config file name and throws INVALID_CONFIG
• refuses another screen's locator or a non-locator before any engine call (locator-union): Or(Locator) is typed, so only the other screen’s locator can be refused

Not ported, because the feature is not in the port:

• No ListReporter, run header, or live window; no JUnit or Markdown report renderer; no serial groups, workers, shards, runner, e2e run, e2e init, or e2e list; no JavaScript config loader; no test.extend; no telemetry; no wordmark; no projectId; no runWithRetries: the report-list, report-junit, report-markdown, cli-run, cli-boot, config-load, parallel-workers, extended-fixtures, telemetry-*, wordmark, target-identity, and retry tests, and the three report-status tests about the last attempt of a serial group
• No value expectations (expect-values truthy/falsy/null/undefined fail on the wrong value, and pass negated), or click-modifier tests; the waitForResponse, route, and unroute predicate tests and the an option ... does not take is refused tests cannot happen, because C# overloads are typed
• No run-wide usage accumulator (the port has ModelUsage, but no overflow-capping total or usage events), withStallGuard, realmSlot, or verifyEndState with waitMs: the model-usage, tool-loop-stall, realm-slot, and trace-replay end-wait tests
• No pixelFallback observe option, tap_at or select_at, upload tool, or schema and vision options on act: the agent-fallback-handoff, agent-act-pixel-tools, agent-act-verbs hidden-file, act-validation, and agent-screen-update tests
• No removed-key hints in agent-config, no artifacts config key (refuses the removed kinds list), no module-error explainer (diagnose), no defineEngine (binds validateApp, prepare, finish, and fixture factories to the spec)
• No stalled-reader pixel fallback in semantic-fallback
• The MCP config takes one target and has no project tools: the three mcp-server and mcp-session-host tests about several targets and project tools. The declared-verbs part is covered by SessionHostTests.Catalogs_only_the_verbs_the_engine_declares
• No --trace flag or retry runner, so the trace-modes test (records retries only under on-all-retries, --trace wins) cannot run. No runner or report, so the report-schema parts of trace-cache-replay cannot run either

Left for other work:

• detectVcs and the error stacks test: covered by #496

Debug

Upstream’s --debug prints the run’s phase timings and a table of agent steps on stderr, and keeps each planned step’s transcript. A test run has no --debug flag, so the .NET-only variable E2E_DEBUG=1 (or true) stands in for it, and a fixture can override the Debug property. e2e explore --debug is the flag itself (ExploreOptions.Debug; the variable turns it on too). E2ESessionOptions.Debug carries it to the session.

• DebugTrace (internal, Internal/DebugTrace.cs) is upstream’s internal/debug.ts: Record, Time (async, rethrows), Summary, and the static Table and FormatMs. It has Off for a caller that has no trace. It has no Drain and no Merge, because the port prints per test and has no worker channel to ship a snapshot over. debug.test.ts is Debug/DebugTraceTests.
• The phases are session.launch (the engine’s start), test.body (from the start of the session until Complete or dispose), session.close (the engine’s disposal), and the agent’s agent.observe, agent.model, and agent.action (Phases, internal; upstream agent/phases.ts, where instrumentPhase also records step events). agent.cache is declared upstream and used nowhere, so the port omits it. Upstream also has config.load, engine.prepare, engine.init, app.start(...), collect, and scheduler: the port has no such phase, because it has no runner, no app process, and no engine prepare step.
• The table is written to stderr when each session is released, once per test, where upstream prints one table for the run. The title is the same, [e2e debug] phase timings (wall ...).
• The agent step table is DebugSteps.AgentStepTable (internal, Report/DebugSteps.cs), upstream’s agentStepTable in report/debug-steps.ts: one row per agent step in execution order, with the step’s time, the time in its model, observe, and action events, its model calls, tokens in and out, the share the provider served from its prompt cache, and its cost. A via column appears when the steps named several models. It is written to stderr after the phase table when each session is released, from that session’s step records (upstream takes the run’s results and serial groups). Differences: StepModelInfo has no provenance (endpoint, adapterVersion, policyVersion, tokenAccounting, peakTokensPerCall), and IAgentModel.Name spells provider and model together, so a step’s label is that name alone unless an executor reports a provider. The model column sums the step’s Model events, one per model call (StepRecorder.AddModelCall). The observe and action columns sum the step’s Observation and Engine events (see Trace page). cached and cost read - unless the provider or an executor reports them in ExecutorModelCall. Every built-in provider returns the cache read tokens (and the write tokens, where it has them) in ModelUsage, and ToolLoop and the agent’s own loop (Agent.cs) pass them on. The AI trace still writes only the input and output tokens. Bedrock and Anthropic report the cached tokens apart from the input tokens, so the port adds them back into InputTokens, as the AI SDK does (convertAmazonBedrockUsage: total: inputTokens + cacheReadTokens + cacheWriteTokens), and upstream’s ModelUsage.record reads that total. Google counts the cached tokens inside promptTokenCount already. The OpenRouter cost is the field upstream’s readCost reads (metadata?.['openrouter']?.['usage'], then cost). A count that is negative or not whole is dropped, as upstream’s tokenCount does. Only an OpenAI-compatible endpoint reports a cost, and only when it sends usage.cost, as OpenRouter does. Upstream’s readCost also reads the AI Gateway metadata; the port has no gateway. debug-steps.test.ts is DebugSteps/AgentStepTableTests (both it; the first is partial: the port has no serial groups, so its third step is a second attempt’s). .NET-only: the agent step table case of SessionDebugTests, ModelCallUsageTests and Provider_usage_carries_the_cache_split_and_the_cost_when_reported.
• An executor’s AttachTranscript text is kept as transcript-<n>.txt only under debug. .NET-only: SessionDebugTests, and the debug-on and debug-off cases of AttachedTranscriptTests.

AI trace

Upstream’s --ai-trace writes every model call of the run to <output>/ai-trace.json, in the AI SDK devtools shape ({ runs[], steps[] }) that npx unbox-ai reads. One run is one agent step, and one step is one model round trip. A test run has no --ai-trace flag, so the .NET-only variable E2E_AI_TRACE=1 (or true) stands in for it, a fixture can override the AiTrace property, e2e explore --ai-trace is the flag itself (ExploreOptions.AiTrace; the variable turns it on too), and E2ESessionOptions.AiTrace carries it to the session.

• AiTraceRecorder, AiTraceCollector, AiTraceScope, AiTraceRun, AiTraceStep, AiTraceDocument, withAiTraceScope and withAiTraceStep (as AiTraceScopes.WithAiTraceScope and WithAiTraceStep), and stringify (as AiTrace.Stringify) are upstream’s internal/ai-trace.ts (all internal, Internal/AiTrace.cs). The recorder keeps upstream’s rules: a run per step, a nested run per tool, steps numbered without collisions, the prompt as sent with the system prompt in front, and inline media replaced by [binary N bytes omitted]. Its drain is Drain(all).
• Upstream’s recorder is an AI SDK telemetry integration, so registerAiTraceRecorder and dispose’s removal from the SDK’s list have no port. The recorder’s onStart, onStepStart, onLanguageModelCallStart, onLanguageModelCallEnd, onStepEnd, onEnd, onError, and executeTool are methods (OnStart, …), and AiTraceModel.CompleteAsync (.NET-only, Agent/AiTraceModel.cs) fires them around each model call. Both places that ask a model for a turn, the tool loop and the judgment path, go through it, so a custom executor with a model of its own is traced too. A custom executor that calls a provider directly is not seen. ExecuteTool is not called by any tool. Upstream’s AI SDK keeps a generation open while its tools run, so a generation a tool starts is a child of the step that called it. The port closes a generation (OnEnd) as soon as the model answers, before the tool loop runs the tools, and no tool of the port starts a model call, so there is no open generation for ExecuteTool to attribute a nested one to.
• The scope comes from StepRecorder, as upstream’s steps.ts does: each step enters the attempt’s scope and narrows it to its api and label. The attempt’s scope is built by the session (the test title is also its TestId). The port has no worker, so there is no drain over a channel: the session drains its recorder when it is released, the process adds it to one AiTraceCollector per file, and it rewrites the file at most every 2 seconds and when it exits.
• The file is ai-trace-<process id>.json next to the results directory (.e2e/ by default), not ai-trace.json. A test run can use several processes (one per test project), and each rewrites only its own file. Open it with npx unbox-ai <file>.
• What the model sees is redacted by the same redactor as the request, and the trace redacts it again on the way in: text, tool arguments, the response, and a failure message. Screenshots are byte counts. The port’s ModelUsage has the cache read and write tokens but no inputTokenDetails object, so usage has no inputTokenDetails, and output has no providerMetadata. The response has no id. provider is the model class without Model, such as Anthropic.
• Tests: ai-trace.test.ts is AiTrace/ (13 of 14; “registers through the SDK loader and tolerates a missing SDK” is skipped, since there is no SDK; “records the status and the body of a provider error the SDK could not parse” goes through OpenAiCompatibleModel and expects the port’s The model provider failed: HTTP 400: ... message). ai-trace-media.test.ts is AiTraceMedia/ (3 of 3). agent-ai-trace.test.ts is AgentAiTrace/ (2 of 3, partial: the model is the port’s ScriptedModel, so the model id is scripted and the provider Scripted, and there is no run outcome, so the test reads the file; “–ai-trace on child-process workers” is skipped, since the port has no workers). .NET-only: AiTraceModelTests (redaction, screenshots, concurrent tests).

Upstream packages

The packages in tester-army/e2e at the target commit, and what the port has:

Upstream	Status	Notes
e2e (packages/e2e)	Ported in part	Everything under Ported and the sections after it. The runner, reporters, e2e run, e2e init, e2e upgrade, telemetry, and the hosted session (e2e/runner) are not ported: see Not ported
@e2e-dev/web	Ported	WebEngine in E2E: see Web engine
@e2e-dev/mobile	Ported in part	E2E.Mobile: see Mobile snapshot model and Mobile engine
@e2e-dev/eas	Ported	E2E.Eas: see EAS Simulators
@e2e-dev/limrun	Ported	E2E.Limrun: see Limrun
@e2e-dev/kernel	Ported	E2E.Kernel: see Kernel
@e2e-dev/testmu	Not ported	A mobile device provider for TestMu AI, through agent-device. E2E.TestMu.Mobile serves the same devices with Appium: see TestMu mobile, and what still differs
@e2e-dev/decision	Not ported	decisionFilter (see Test filters, where the filter seam it plugs into is ported) and a decision-model executor: a model that picks one action from a grammar (taps, points, vision). The port has the executor seam (IStepExecutor, see Agents) but no such executor
@e2e-dev/smol	Not ported	A browser provider that runs each browser and the app in a branchable smol machines microVM. The port has the provider seam (IBrowserProvider, see Web engine) but no such provider
@e2e-dev/lightpanda	Not ported	A browser provider for the Lightpanda headless browser. Upstream added it on 2026-10-15 (b41afa2), after the earlier target. The engine drives Chromium only
@e2e-dev/github	Not ported	A reporter that posts the run as a pull request comment. The port has no reporters

A config that selects a decision model or smol cannot be written in e2e.config.json, and it fails at load with INVALID_CONFIG instead of being ignored (E2EConfig):

• Decision models are an executor. Upstream takes executor: decision(...) in agents.<name>. The port rejects the key with config key "executor" is not supported by the .NET port: an executor is code, so set AgentOptions.Executor. A custom executor in code works, but the port ships no decision executor. provider accepts only the names in Models, so a provider of decision is rejected with the list of valid names.
• smol is a browser provider on the target (web({ browser: smol() })). The target accepts name, platform, app, trace, and video. A browser key is rejected as an unknown target key, and an engine key as not supported by the .NET port. To use a provider, set WebEngineOptions.Browser in the fixture’s CreateEngine. The port ships KernelProvider and TestMuBrowsers there, and none for smol.

Skill text: start near the screen, fix from the trace page

skills/e2e-dotnet ports upstream’s 74c2b73 (#1311, start each test near the screen it covers) and c5e3248 (#1309, “Workflow: fixing a failed test”). Differences:

• The port has no saved session, test.setup, or test.extend fixture, so “Start near the screen under test” signs in and seeds data in the framework’s per-test setup (NUnit [SetUp], xUnit v3 InitializeAsync) with an HttpClient, and Browser.SetCookiesAsync hands the cookies to the browser. On a device it is Device.OpenLinkAsync.
• The test runner prints the exception, not upstream’s failure block, so the port has no ❯ at, ❯ on screen, or ❯ trace lines. The skill points to the exception’s Code, message, and Details, and to the trace page’s ## Screen at failure (its URL and Closest to what the locator asked for). The trace page, ## Screen at failure, and screen-at-failure.txt exist as upstream.
• Expect.Soft, Values.Unique, App.OpenAsync, and Device.OpenLinkAsync stand in for expect.soft, unique, app.open, and device.openLink. In CI the cache is read-only unless the config sets a mode (E2EConfig), where upstream’s default records nothing there. Upstream’s core-concepts.mdx and coding-agents.mdx have no counterpart.

Skill text: off-site iframes (not ported)

Upstream’s 4982d8d (#1333) documents that the browser engine’s semantic tree leaves out the content of a frame from another site, so agent steps there need browser.frameLocator and vision, and an agent.act whose only change is in that frame records no replay. The port does not take the text, because the behavior does not exist here: the web engine reads every iframe’s document through Playwright, same-origin or cross-origin, and puts it under the iframe node (see Web engine), so the agent sees and acts on that content. Browser.FrameLocator exists, and it resolves over that same tree. The changes to skills/e2e-dotnet/references/{agent,debugging,writing-tests}.md and docs/ are skipped because the behavior they warn about does not exist here.

Verification skill

skills/create-verification-skill/ ports upstream’s skill of the same name (commit d85702a2, feat(skills): create-verification-skill generates a verify-<app> skill on e2e (#981)). Like upstream, it is not bundled in E2E.Cli and e2e guide does not read it; install it with npx skills add hardkoded/e2e-dotnet --skill create-verification-skill. Differences:

• The steps use what the port has. The launch is the target’s app.command or the app started by hand, the doctor is a smoke test run with dotnet test --filter, and the evidence is the dotnet test result and the test’s own checks, since the port writes no report or trace page, and a video only when video is on.
• Not ported: the generated bug bash fan-out. It depends on e2e.bugbash.config.ts, which the port lacks, and runs many e2e explore commands at once, so a feature’s charters are goals for a live MCP walk, and a finding is a bug once a repro test fails with ASSERTION_FAILED.
• Not ported: npx e2e init as the setup step, e2e list as a config check, --output for proofs, and the .gitignore lines for .e2e/bugbash/ and .e2e/proof/. The repo already ignores .e2e/.
• The example under references/example/ is verify-todomvc for samples/TodoMvc, in place of verify-playground for upstream’s apps/testbed, which the port does not have. It has three features (adding, completing, filtering), not the playground’s three (todos, forms, login).
• Upstream added no tests for this skill, so none are ported.

skills/migrate-to-e2e/ ports upstream’s third skill (e6885dc, feat(skills): add migrate-to-e2e (#1109), and 6609428). Install it with npx skills add hardkoded/e2e-dotnet --skill migrate-to-e2e; like upstream, it is not bundled in E2E.Cli. Differences:

• The source runners are Playwright .NET, Selenium .NET, and Appium, with upstream’s Node ones (Playwright, Cypress, Detox, Maestro) kept. The e2e side is an NUnit or xUnit v3 test project run with dotnet test, in place of e2e.config.ts, *.e2e.ts files, npx e2e init, e2e run --repeat-each 3, and e2e list.
• Not ported: upstream’s migration guides (docs/migrate/*.mdx) and the edits to them in e6885dc. The port has no such guides, so the skill reads the e2e-dotnet skill’s topics, and the hand-over offers an issue at hardkoded/e2e-dotnet in place of e2e feedback.
• Not ported: the plugin copy (plugins/e2e/skills/migrate-to-e2e), since the port has no plugin; the AGENTS.md and CONTRIBUTING.md lines about it map to CONTRIBUTING.md here.
• Upstream added no tests for this skill, so none are ported.
• docs-site/src/content/docs/coding-agents.mdx ports upstream’s docs/coding-agents.mdx for the commands the port has.

Coding agents

Not ported: e2e init, which scaffolds the config, installs the skill into .agents/skills and .claude/skills, registers the MCP server, and adds a test:e2e script. The port installs the skill with npx skills add hardkoded/e2e-dotnet and registers the MCP server with claude mcp add e2e -- e2e mcp. The bug-bash config (e2e.bugbash.config.ts) and its fan-out are not ported either: see Verification skill.

MobileAppTest (not in upstream)

E2E.Mobile.NUnit.MobileAppTest and E2E.Mobile.XUnit.V3.MobileAppTest derive from E2ETest and make a device test one using and one config file. Upstream declares a device in e2e.config.ts and its runner owns the engine and the hosted device. The port’s adapters do, so these classes do it. Every symbol here is .NET-only. Differences:

• e2e.config.json accepts "platform": "ios" and "android". A device target takes app.bundleId and app.appPath, as upstream’s app. It takes app.command and app.readyUrl too (see Config). It does not take the other upstream app keys (identity, environment, launchArguments, permissions). The port still runs one target.
• E2E_MOBILE_PLATFORM and E2E_MOBILE_APP_PATH win over the config, so one project can run on iOS and Android. E2E_MOBILE_DEVICE, E2E_MOBILE_PROVIDER (local, limrun, devflow), E2E_DEVFLOW_PORT, and E2E_DEVFLOW_APP are .NET-only.
• A hosted run leases one device on the first fixture and releases it when the process ends, because the adapters have no hook after the last test. A killed process is covered by the provider’s hard timeout (15 minutes for Limrun).
• MobileAppHost is one source file, compiled into both packages. HostedDeviceProviders (the provider each E2E_MOBILE_PROVIDER name makes) is a second, compiled into both packages and into E2E.Cli. CreateMobileOptions, AppId, AppPath, Platform, and AttachesToRunningApp are the members a test project overrides or reads.

Mobile snapshot model

E2E.Mobile ports the pure logic of @e2e-dev/mobile: the projection of an agent-device accessibility snapshot onto SemanticNode (SnapshotProjection), locator expressions and agent-device selectors over it (DeviceLocator, AgentDeviceSelector), error translation (DeviceErrors), links, screenshot masking, and the shared helpers (DeviceSupport). The client, pool, provider, and engine are in Mobile engine. Differences:

• agent-device is a Node package. AgentDeviceError stands in for its AppError, and AgentDeviceSelector reads the selector grammar of parseSelectorChain and applies the term rules of listSelectorChainMatches to the raw nodes, for the target’s platform (the keys id, role, text, label, value, appname, windowtitle, and the flags visible, hidden, editable, selected, focused, enabled, hittable).
• An AMBIGUOUS_MATCH failure (several devices match and no selector narrows it) keeps agent-device’s message, code, details, and hint, and DeviceErrors adds one sentence to the text: Set MobileOptions.Device to a device id, or boot exactly one device. Upstream adds nothing. AmbiguousMatchTests covers it and has no upstream counterpart.
• SemanticNode.States holds plain booleans, so it cannot tell an unchecked switch from a node with no checked state. ProjectedNode.States keeps the states the platform reported, and ProjectedNode.Selector keeps the id= hint upstream puts on the node.
• A filter with hasText honors the exactness of its text pattern, as upstream’s matchesText does. Locator.Filter(TextMatch) is a substring match (upstream’s toTextPattern(hasText, { exact: false })), so a locator test sees no difference. TextMatch holds no exactness, so the expression carries it: LocatorExpression.HasTextExact (.NET-only; default false) makes HasText equal the text, as upstream’s exact('About') does in locate.test.ts. A Regex ignores it. The core resolver (Screen) reads it too.
• A role query has no hidden state. Role queries skip hidden nodes, as upstream’s do.
• Link parsing reads a URL with System.Uri, after the whitespace rules a browser applies.
• MaskPng refuses a PNG with no end chunk, as upstream’s parser does.

Mobile engine

MobileEngine ports engine.ts, options.ts, bindings.ts, pool.ts, provider.ts, surface.ts, actions.ts, device.ts, and tools.ts. It implements IEngine and declares every capability, video included. DeviceSession is the surface, DeviceActions (the IDeviceActions the engine uses by default) turns a node and an action into commands, Device is the device fixture, and MobileTools is the agent tool pack. The eas and limrun providers are ported (see EAS Simulators and Limrun); a provider’s own recording (DeviceProvider.record) is IDeviceProvider.RecordAsync (see Provider recording). Differences:

• agent-device is a Node program. IDeviceClient stands in for upstream’s createAgentDeviceClient: a call is namespace.method with an options object, as upstream’s client names them. AgentDeviceRpcClient implements it over the daemon’s HTTP JSON-RPC 2.0 (POST /rpc, method agent_device.command, GET /health). The wire format was read from agent-device 0.21.22’s code and checked against a running 0.21.22 daemon for devices, boot, close, session_list, prepare, and snapshot. A running 0.21.25 daemon was checked for GET /health (protocol 2) and session_list. A running 0.21.29 daemon answered GET /health (protocol 2, version 0.21.29), session_list, and devices in the same envelope, and its daemon.json still carries httpPort, token, and version (it adds capabilities and plugins, which the client ignores). The 0.21.26 to 0.21.29 release notes list no change to the RPC envelope or to the commands the engine sends; the changes that touch the engine are in its commands (a coordinate fill that names no text input is refused on iOS, a find focus match behind the visible keyboard is refused, an alert dismiss picks a listed decline, and a delayed launch confirmation is surfaced), and none of them was exercised on a device. It is not a documented protocol, so the version is pinned (AgentDeviceVersion.Pinned) and the client checks rpcProtocolVersion and the daemon version. DeviceCommands turns each call into a command, positionals, and flags. The calls the engine makes today are covered. A call it does not know is refused.
• A local daemon needs Node 22.12 or newer. The first call starts it the way agent-device’s own CLI does (npx --yes agent-device@0.21.29 session list --json, with AGENT_DEVICE_DAEMON_TRANSPORT=http), in the state directory ~/.agent-device/e2e-dotnet, and it keeps running after the command. E2E_AGENT_DEVICE_BIN names an agent-device.mjs to run in place of npx. A missing Node, a Node that is too old, an agent-device that does not start, and a daemon of another version fail with INVALID_CONFIG and the recovery. Starting a daemon on Windows is not verified.
• MobileOptions carries the app (App: bundle id, build path, launch arguments, permissions), the target name, and the device choice as three properties (Device, Devices, Provider; at most one), because the port has no target config for a device yet. Permission names and states are enums, so an unknown one cannot be written. Settle and SettleMilliseconds stand in for settle: number | false.
• The core engine contract has no prepare, finish, init, or worker slot. MobileEngine.PrepareAsync and FinishAsync take their own info records, and StartAsync(MobileStartInfo) starts the worker of one slot. StartAsync(EngineStartOptions) starts slot 0 of the target. Bindings travel from prepare to a worker through MobilePrepareResult.Env, as upstream’s environment variable does. In one process the pool also keeps them.
• app.open() is OpenAsync: it launches the pinned app fresh. A deep link or any absolute URL is refused. EngineStartOptions.BaseUrl is refused as upstream refuses app.url.
• DeviceSession.BackAsync always settles (upstream’s agent origin), since a test step has no origin here.
• The core session has no attempt hooks, pixel observations, or artifact directory. So DeviceSession is one attempt (it registers on the engine until it closes), and ScreenshotAsync returns the masked, status-bar-normalized PNG for toHaveScreenshot instead of writing numbered files. A secure field that cannot be masked still withholds the image with ENGINE_FAILURE.
• DeviceSession.LocateAsync(LocatorExpression, TimeSpan timeout, CancellationToken) is upstream’s locate (surface.ts), so a locator or Expect on a device resolves in the engine. It snapshots with interactiveOnly: false, runs DeviceLocator.ResolveExpression, and binds the matches in a separate map capped at 2048 (the oldest goes first; an older ref is NODE_STALE). It hands the capture it matched against to EngineStartOptions.Screen, and tells EngineStartOptions.Environment the platform and device once, when the session starts. Differences: timeout is the budget left (upstream’s operation.timeoutMs), since the core session has no operation context; the expression is one flat record (LocatorExpression) where upstream’s is a union of query, filter, index, union, selector, and frame; and the port’s refs carry no revision, so upstream’s rejectSupersededLocate has no port (a device mints a new id for every projection, so a later locate never reuses a ref). Core retries a retryable EngineException of a locate until the wait’s deadline, as resolveOnce does.
• Upstream’s Android locate (fix(mobile): keep locator polling through unreadable android snapshots, 4fe6baf) is ported. A snapshot agent-device cannot read yet (details.androidSnapshotHelperFailureReason of empty-helper-output, system-window-only, or content-poor-app-window, the message Android snapshot helper returned insufficient foreground app content) matches nothing in a locate, so a locator or Expect keeps polling until its wait runs out, and a locate whose capture leaves no budget fails with ENGINE_FAILURE and that message. DeviceErrors.IsUnreadableCapture reads the reason off the AgentDeviceError that the translated EngineException keeps as its InnerException. An observation, such as the agent’s next turn, still fails on it at once, as upstream’s.
• Upstream’s iOS locate (2b2b845, tester-army/e2e#1281) is ported. Just after OpenAppAsync, RestartAsync or OpenLinkAsync, a snapshot that iOS refuses with details.runnerErrorCode of APP_NOT_RUNNING (DeviceErrors.IsAppNotRunning) matches nothing in a locate (locator waits and Expect only; an observation still fails at once), like an unreadable Android capture, until a snapshot succeeds (DeviceSession._launching, upstream’s launching). A wait that runs out while the app is still down fails with ENGINE_FAILURE and that reason. An app that exits after its first readable screen still fails at once.
• Upstream’s Android test id match without the <package>:id/ prefix (feat(mobile): match Android test ids without their package prefix, 2791786) is in DeviceLocator, which DeviceSession.LocateAsync now calls, so Screen.GetByTestId("greet") matches <package>:id/greet on Android when nothing matches the full id.
• The core does not say who calls PerformAsync, so DeviceSession.Origin does. ActionOrigin.Agent (the default) settles after each action. ActionOrigin.Test never settles and waits out a transition (TransitionMilliseconds) for a control that arrived with the last action. The core LocatorAction has no long press, hover, drag, or swipe momentum, so LongPressAsync, HoverAsync, DragToAsync, SwipeAsync(node, direction, momentum), and PerformAtAsync(point, PointerAction) are methods of DeviceSession. ScrollIntoView, Select, and PressSequentially are UNSUPPORTED_CAPABILITY.
• Device is a plain class (session.Device) with ...Async methods that take a cancellation token per call. The harness does not record device.<method> steps or labels, and Device has no locator. appearance, network, alert, and biometric results are enums. A permission name or state cannot be unknown, so the INVALID_ARGUMENT check for one is not needed.
• The sample (samples/MobileApp.E2ETests) derives from MobileAppTest (see MobileAppTest). It launches the app with Device.OpenAppAsync, because the core App.OpenAsync needs a URL, and reads the Device fixture from MobileEngine.ActiveSessions, because the core session does not expose the engine’s session. It was run against an iPhone 17 simulator. The Android path is not run.
• MobileTools.Create returns AgentTools by name (the port’s defineTool), with Platforms as the strings the engines report (ios, android). Pass MobileTools.Create(engine) as the agent’s tools (AgentTools on E2ETest), and the agent offers open_app, swipe, and alert on the platforms of the engines it was built from. ai-imports has no counterpart: there is no ai package.
• agent-device takes no abort signal, so a cancelled command keeps running. Upstream’s surface tracks those commands and makes the next attempt wait for them. The port does not: each attempt has its own session and client.
• The key grammar (DeviceKeys) is ported here, since the core has none.
• Video is DeviceSession.StartVideoAsync and StopVideoAsync (upstream’s startVideo and stopVideo of AgentDeviceSurface), over recording.record (command record, with quality: medium, recordingScope: device, and hideTouches when MobileOptions.VideoTouches is false). The file is video/video.mp4 in MobileStartInfo.ArtifactsDir (upstream’s artifactsDir; EngineStartOptions.ArtifactsDir fills it). A recording is marked before the device is asked, so a start that outlived its budget is stopped when the session is disposed, as is one whose stop failed (upstream’s endAttempt and dispose). A recording the device finalized elsewhere is moved into place. A session is one attempt, so upstream’s startAttempt cleanup (sessions.close of a leftover recording) has no port, and the in-flight commands outlive the session on the engine (see In-flight commands). Closing a session does not wait for a recording start that outlived its budget before it stops the recording (upstream’s endAttempt settles the in-flight commands first), so the daemon may see the stop while the start still runs. The device provider’s recording (LeaseRecording) is in Provider recording. The agent-device parameters were read from upstream and the 0.21.25 daemon’s command table; on an iOS 27 simulator with Xcode 27.1 the daemon accepted the start, and its stop timed out in xcrun after 30 s, so a recorded file is not yet seen end to end.
• VideoTests (describe('video') of engine.test.ts) are ported in full on a scripted client; “records and stops on the pinned $platform device after closeApp ends the previous session” runs the second attempt as a new session. ManifestTests declares the video capability.
• Upstream’s keyboard drag start (fix(mobile): start the iOS keyboard-dismissing drag clear of controls, 787a5d5, tester-army/e2e#1305) is ported. Keyboard.KeyboardDragStart in Keyboard.cs is upstream’s keyboardDragStart in keyboard.ts (the class name is the module’s, since C# has no free functions). DeviceSession.KeyboardTop is upstream’s keyboardTop, and RawSnapshot.KeyboardFrame carries the keyboard fact’s frame it reads. SnapshotProjection.ScreenKinds is exported as upstream exports SCREEN_KINDS. DismissKeyboardAsync drags from that point, and a screen with none fails with UNSUPPORTED_CAPABILITY before any drag. Tests: five its of describe('device fixture') in Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests. Not ported: ‘dismissing the keyboard over choice controls leaves them as they were’ (apps/mobile-benchmark/tests/device.e2e.ts), which needs the mobile benchmark app, a React Native app the port lacks.
• Upstream’s known-viewport fix (fix(mobile): keep the known viewport when the app reads smaller than the screen, 7674a2c, tester-army/e2e#1291) is ported. DeviceSession.Project learns a snapshot’s viewport only when it is plainly the screen’s: the first one the session reads, the one it knows, or that one rotated (SameScreen, upstream’s sameScreen; two sizes within a point are one screen, since rects from the tree carry fractions). ViewportForAsync (upstream’s viewportFor) checks a tree of any other size against the device’s screenshot, in either orientation, before it replaces the known one: a screenshot reports the portrait size even in landscape, so it confirms a size but cannot pick the orientation. The size it checks against is DeviceSupport.LogicalScreenSize (upstream’s logicalScreenSize): the screenshot’s logical dimensions, else its pixel dimensions over its density. A simulator’s screenshot reports those; a real device’s reports neither, so there the tree’s size is taken as it comes. A screenshot that fails during that check fails the observation, as upstream’s does. TraceScreen and MaskedScreenshotAsync size the trace screen and the secure-field masks by the known viewport, not the tree’s. On iOS, a swipe from a sheet’s grabber can pull down Notification Center, and the app’s tree behind it reported a third of the screen (130x281 on a 390x844 device), which misplaced every rect and tap after it. Upstream records this only in its changeset; its skill references and docs say nothing about it, so SKILL.md and docs/ have nothing to mirror. Tests: in Engine/ObservationTests, ‘keeps the known viewport for an app drawn smaller than the screen, and follows a rotation’ (new) and ‘measures the screen in the viewport the snapshot publishes, and in its tree when it publishes none’ (split into two harnesses, as upstream’s); in Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests, ‘drags at the centre of a screen rotated since the last observation’ (new), since the keyboard drag measures its start through ViewportForAsync too.
• Upstream’s input budget (fix(mobile): bound each input by its budget, 14b77d3, tester-army/e2e#1073) is ported. An input (a tap or other node action, a point action, typing, a key, back, a keyboard dismissal, and the node LongPressAsync, HoverAsync, DragToAsync, and SwipeAsync the port has in place of upstream’s longPress, hover, dragTo, and swipe actions) runs its device commands under InputBudget (DeviceSupport.cs, upstream’s support.ts): with no budget left the input is not sent and fails with OPERATION_TIMEOUT, an input that does not answer in time fails with ACTION_MAY_HAVE_COMMITTED, one sent with almost no budget left still gets 250 ms to answer, a hold (a long press or timed swipe) at least as long as the budget left is refused with NOT_ACTIONABLE, and each command of a multi-command input (a key on an unfocused field) is sent on its own. Reads, launches, artifacts, and the Device fixture calls (including device.dismissKeyboard, which passes no budget, as upstream’s dismissKeyboard(signal) does) are unbounded. Differences: upstream reads the budget per call, from operation.timeoutMs, which is the call’s timeout or actionTimeout. The port’s session calls carry no timeout, so every input gets DeviceSession.ActionTimeout, which MobileStartInfo.ActionTimeout sets (EngineStartOptions.ActionTimeout fills it), as the web engine’s operations get the action timeout. DeviceSession.InputBudget is upstream’s inputBudget and DeviceSession.SendInputAsync its sendInput. InputBudget’s track is an Action<Task> that registers the task (upstream’s returns the promise it is given); see In-flight commands. The OperationContext.signal doc change in docs/reference/engine.mdx and contract.ts has no counterpart: the port has no OperationContext. Tests: in Engine/ObservationTests, ‘ends %s, hung on %s, at its budget as possibly committed’ (a theory of the same seven entries; the screen swipe is a perform on the root), ‘sends nothing once a settling target spent the budget, and no keystroke once a focus tap used it up’, and ‘refuses a long press that holds longer than its timeout leaves, before sending it’ (also on the node LongPressAsync); in Support/SupportHelpersTests, ‘sends an input with what is left of its budget, and reports one cut off as possibly committed’. These run on real timers of a few hundred milliseconds, where upstream’s fake timers do not wait. ‘lets a locate outlive its budget’ was already ported.
• Upstream’s WebDriver element-type fix (fix(mobile): read XCUIElementType class names as their XCTest types, 3f9a7d8, tester-army/e2e#1133) is ported. SnapshotProjection.NormalizeKind is upstream’s normalizeKind in nodes.ts: it drops ^XCUIElementType(?=[A-Z]) (XcuiElementTypePrefix, a .NET [GeneratedRegex] for upstream’s inline regex) before it maps the type, so the XCUIElementTypeButton that agent-device’s WebDriver runtime reports on hosted Appium clouds reads as Button. DeviceSession.KeyboardShowing is upstream’s keyboardShowing in surface.ts: it routes node types through NormalizeKind, so an XCUIElementTypeKeyboard counts as a shown keyboard. Upstream records this only in its changeset; its skill references and docs say nothing about it, so SKILL.md and docs/ have nothing to mirror. Tests: in Nodes/SnapshotProjectionTests, ‘reads the XCUIElementType class names a WebDriver session reports as the native XCTest types’; in Engine/DeviceFixtureDismissingAKeyboardThatHasNoDismissKeyTests, ‘reads an XCUIElementTypeKeyboard from a WebDriver session as a shown keyboard’.
• Not ported: the tool-pixels test (needs the full runner and a scripted model), and the device.openApp check of an unknown permission name.

EAS Simulators

EasSimulators (the E2E.Eas package, namespace E2E.Eas) ports @e2e-dev/eas: hosted iOS simulators and Android emulators from Expo, leased through Expo’s GraphQL API over HttpClient, with no SDK. The provider is its own package, E2E.Eas, which depends on E2E.Mobile, as upstream’s @e2e-dev/eas is a package of its own. It keeps a private copy of the shared HttpClient, so E2E.Mobile exposes nothing extra for it. Pass it as MobileOptions.Provider. The sessions, the queue and startup limits, the idle backstop, the EXPO_TOKEN or eas-cli login, and the project id from the app config follow upstream. Differences:

• Options are the EasSimulatorsOptions record, so an unknown option key is a compile error and upstream’s rejectUnknownKeys has no counterpart. A bad combination (BuildId with ApplicationArchiveUrl, an idle limit not below the duration) is INVALID_CONFIG from the constructor.
• A dynamic app config (app.config.ts and its siblings) is evaluated by running the project’s own expo config with node from the path of the process, so it needs Node. A static app.json needs nothing.
• The request’s cancellation is a CancellationToken. A cancelled read of the app config is an OperationCanceledException; upstream wraps it as “could not read the Expo app config”.
• A device request always carries the project root, so upstream’s check for an engine that passes none has no counterpart.

Limrun

LimrunSimulators (the E2E.Limrun package, namespace E2E.Limrun) ports @e2e-dev/limrun: one hosted iOS simulator or Android emulator per worker slot, created, polled to ready, driven through an agent-device daemon the provider starts for it, and deleted on release. Pass it as MobileOptions.Provider. The key and base URL (LIMRUN_API_KEY, LIMRUN_BASE_URL), the instance spec, the daemon environment, the retained lease, the 20-second heartbeat, and the cleanup follow upstream. The client has the lease commands the provider needs: leases.allocate, leases.heartbeat, and leases.release (DeviceMethods), sent as agent-device’s agent_device.lease.* JSON-RPC methods. Differences:

• The provider is its own package, E2E.Limrun, which depends on E2E.Mobile. Upstream’s @e2e-dev/limrun is a package of its own too. The package keeps private copies of two small E2E.Mobile internals (the argument builder and the shared HttpClient), so E2E.Mobile exposes nothing extra for it.
• Limrun’s REST API (/v1/ios_instances and /v1/android_instances: create, get, delete) is called over HttpClient, with no @limrun/api SDK. Like upstream’s configuration of the SDK, it retries nothing and gives each request 60 seconds.
• Upstream forks a Node process per device, so the daemon and its lease client get the instance’s credentials without changing the runner’s environment. The port starts the daemon itself, with exactly the run’s environment plus the instance’s (AgentDeviceDaemon.LocalAsync with an environment), and runs the lease and install calls in the runner. AgentDeviceDaemon.StopAsync runs agent-device daemon stop --clean.
• Upstream pins agent-device as a dependency of @e2e-dev/limrun, at the engine’s version (0.21.29), and checks that the installed one matches. The port always starts the pinned release (AgentDeviceVersion.Pinned), so nothing is installed by hand. It only checks that the engine asked for that release, and the daemon’s own version check runs when it starts.
• The lease request leaves out providerCredentialFingerprint. The daemon only compares it with the credentials of a daemon that was already running, and the port starts a fresh daemon with the right credentials for every device.
• Options are the LimrunSimulatorsOptions record, with enums for the device models, so an unknown option is a compile error and upstream’s rejectUnknownKeys has no counterpart.
• Upstream’s Limrun page (docs/integrations/limrun.mdx) is ported as docs-site/src/content/docs/integrations/limrun.mdx, with the port’s C# names. Its examples/with-limrun app-flow example is not ported: samples/MobileApp is the port’s mobile example.
• Not verified against a live service: the lease wire format was read from agent-device’s code (the agent_device.lease.* methods and their params in the daemon, and the client that writes them), and Limrun’s REST calls from the @limrun/api 0.59.0 source. No test reaches Limrun or a daemon.

In-flight commands

agent-device takes no abort signal, so a command a cancel or a timeout abandoned keeps running on the device. Upstream’s surface keeps them in inflight (track, settleInflight) and its startAttempt waits for them before it opens anything, so a ghost tap never lands in a retry. The port has the same behavior:

• DeviceSession.Track is upstream’s track: CommandAsync, ActAsync (upstream’s perform registration), and every InputBudget register the device command until it settles. DeviceSession.SettleInflightAsync is settleInflight, and DeviceSession.StartAsync calls it first, with the caller’s token, as startAttempt does: a command that never settles fails the start as CANCELLED once the token ends, and the boot never goes out.
• Differences: upstream’s surface is the worker, so it holds the set. The port’s session is one attempt, so MobileEngine.InflightOf(target, slot) holds the set of a worker slot (InflightCommands, .NET-only) across the attempts that drive it. A command that fails after its caller left is read once and never reported. The set is keyed by target, slot, and the lease or device the binding names, so an engine reused for another run does not wait on a released device’s commands. The wait is also bounded by E2EDefaults.LaunchTimeout when the caller’s token never ends (ENGINE_FAILURE). Upstream’s client takes no abort signal, so an abandoned command is still running when it is tracked; the port passes the caller’s CancellationToken to IDeviceClient.CallAsync, and the HTTP client aborts its request on a cancel, so a command the caller cancelled (not one an input budget timed out) is tracked only until that call returns, while the daemon may still be running it. Closing a session (upstream’s endAttempt or dispose) does not settle the in-flight commands.
• Tests: Engine/ObservationTests, ‘keeps a timed-out input in flight: the next attempt waits for the device command to settle’, and Engine/SessionHooksViewportSwipeLocationArtifactsTests, ‘waits for an abandoned command to settle before the next attempt starts and its launch goes out’. Both wait on a command the test releases, not on a timer, apart from the 200 ms input budget of the first.

Provider recording

IDeviceProvider.RecordAsync(lease, context, ct) is upstream’s DeviceProvider.record. When an attempt records video and the slot rides a provider’s lease, DeviceSession.StartVideoAsync calls it in place of agent-device’s recording, with the lease as it traveled to the worker (DeviceProviders.TravelledLease, upstream’s travelledLease: the id and the declared fields, never what a derived lease class kept) and a ProviderRecordContext of RunId, TargetName, AttemptId, and Env. DeviceProviders.RecordLeaseAsync is recordLease: a failure is ENGINE_FAILURE naming the provider and the lease, a failure met once the token was cancelled is CANCELLED, and a token already cancelled never reaches the provider. StopVideoAsync ends the recording with ProviderRecordings.StopProviderRecordingAsync (upstream’s stopProviderRecording) into a VideoFile or a VideoLink; the recording is marked at once and cleared when the provider confirmed the stop, before what it wrote is checked; a stop shared by concurrent callers runs once. Closing the session stops a recording still marked, best effort. DevicePool.Recorder is pool.recorder, and LeasedDevices.Recorder is recorder. Differences:

• Upstream’s record is optional, and a provider without one records with agent-device. A C# interface member has no absence to test, so the default RecordAsync returns null and a provider that returns null records with agent-device, as IBrowserProvider.RecordAsync does. LeasedDevices.Recorder is always the provider. Upstream’s asDeviceProvider check that record is a function and isProviderRecording’s check for a startedAt and a stop have no port: the types say it. The context’s signal is the CancellationToken parameter.
• MobileStartInfo.AttemptId (upstream’s EngineAttemptContext.attemptId) names the recording’s attempt. EngineStartOptions.AttemptId fills it, and a new id is made when it is absent. A run through the test host sets no RunId yet, so ProviderRecordContext.RunId is empty there.
• Upstream’s worker outlives its attempts and keeps what an attempt could not stop: a provider recording whose stop failed is retried at the next startAttempt (which then blocks), a native recording that could not be confirmed stopped resets the leased session before the next attempt, and a worker disposed while the lease is reused keeps its session. The port’s session is one attempt that always ends its agent-device session: a recording whose stop failed is retried once, when the session closes, and then abandoned, and nothing is retained for the next attempt. Providers must clean up their recordings when they release a lease.
• Tests: Provider/DeviceProviderRecordingTests ports 10 of the 22 it of describe('device provider recording'). Eight follow upstream: ‘records through the provider instead of agent-device, from the worker, with the lease id and the declared lease fields’, ‘reports a recording the service keeps as a link’, ‘keeps the agent-device recording for a provider without record’, ‘keeps the agent-device recording for a worker without a lease, when no prepare ran’, ‘names the provider and the lease when record fails, and has nothing to stop at attempt end’, ‘cancels a start whose budget ran out, and stops the recording that came up anyway at attempt end’, ‘never asks the provider to record for an operation already cancelled’, and ‘keeps a recording whose stop failed, so endAttempt stops it once more’. Two are partial (Partial_ prefix): ‘stops a provider recording without closing its leased session when the worker is disposed mid-attempt’ (the port closes the session, so it checks the stop only) and ‘clears a confirmed provider stop before rejecting its invalid artifact’ (it checks the stops, not the reuse of the session). Skipped: ‘refuses a provider whose record is not a function’ (the type says it), and 11 that need a worker that survives attempts and retains a recording or its session: ‘keeps a reusable leased session when the provider recording still cannot stop during disposal’, ‘finishes a retained provider recording before the next attempt records in the same worker’, ‘blocks the next attempt while its retained provider recording cannot stop and preserves its owner’, ‘does not begin another attempt or record when a retained provider stop wait is cancelled’, ‘resets a leased session whose native recording still cannot stop after endAttempt’, ‘resets an unconfirmed native recording before another attempt in the same worker’, ‘fails the next attempt when its native recording reset cannot close the session’, ‘keeps an unconfirmed native recording after its stop wait was cancelled’, ‘preserves a session after native stop succeeded even when moving its artifact fails’, ‘retries an unconfirmed provider stop after endAttempt during disposal’, and ‘shares a pending provider stop and closes on reuse revocation before the stop resolves’. .NET-only: ‘keeps the agent-device recording for a provider that records nothing of this device’ (a provider whose RecordAsync returns null).

device.locator

Device.Locator(string selector) is upstream’s device.locator: a Locator over a selector expression, which DeviceLocator answers with the agent-device selector grammar (see Mobile snapshot model). Upstream builds it with EngineFixtureContext.locator. The port has EngineFixtureContext (public, Locator(LocatorExpression) only) and IEngineFixtureHost (.NET-only): E2ESession hands the context to an engine session that implements the interface, once the attempt’s Screen exists, and DeviceSession keeps it for its Device. Device.Locator on a session no test harness started (an engine used on its own) is INVALID_STATE. The other members of upstream’s EngineFixtureContext (fixture, app, timeouts, signal, operation, attachArtifact, attachViewport, screen, expectable) have no port: the port’s fixtures take their own tokens and record no steps. Upstream’s engine test has no it for locator; Engine/DeviceFixtureTests has a .NET-only test, ‘mints a locator from an agent-device selector on the attempt’s screen’.

Kernel

KernelProvider (the E2E.Kernel package, namespace E2E.Kernel) ports @e2e-dev/kernel at the port’s parity commit: kernel() from provider.ts, the Kernel client from client.ts (KernelBrowsers, kernelBrowsers, KernelBrowserParams, KernelReplayParams), and envValue from env.ts. It is a WebEngineOptions.Browser provider: Name is kernel, AcquireAsync creates a browser, ReleaseAsync deletes it, SweepAsync deletes the run’s active browsers, Downloads reads a download back through the browser filesystem, and RecordAsync records a Kernel replay. Lease tags, the 600 s idle timeout, the cancelled-request delete, the sweep message, and the replay.mp4 replay with its retried download follow upstream. Differences:

• Upstream’s function kernel(options) is the class KernelProvider (a type named Kernel would clash with the namespace). KernelOptions is a record that extends KernelBrowserParams, as upstream’s interface does.
• There is no Kernel .NET SDK (no package from Kernel on NuGet; Kernel publishes Node, Python, and Go SDKs). The client calls Kernel’s REST API with HttpClient, the API key as a bearer token, and replaces @onkernel/sdk: POST /browsers, DELETE /browsers/{id} (404 counts as deleted), GET /browsers?status=active&tags[k]=v (paged by X-Has-More and X-Next-Offset), POST /browsers/{id}/replays, POST .../replays/{replayId}/stop, GET .../replays/{replayId} (Accept: video/mp4), and GET /browsers/{id}/fs/read_file?path=. The base URL is https://api.onkernel.com/, or KERNEL_BASE_URL from the run’s environment (the SDK reads it from the process environment). The SDK retries a failed request twice; the client retries nothing, since a retried create could allocate a second browser. The SDK keeps one client per API key; the client here is stateless, so the provider builds one for each request. An error answer becomes a KernelApiException with the SDK’s message, trimmed (401 Invalid or disabled API key).
• Upstream types the create-browser body as the SDK’s BrowserCreateParams. KernelBrowserParams has properties for stealth, headless, gpu, timeout_seconds, profile, proxy_id, viewport, and tags (Stealth, Headless, Gpu, TimeoutSeconds, Profile, ProxyId, Viewport, Tags), and Extra, a JSON object for any other field by its wire name (proxy, extensions, region, …), which the properties win over, so the run’s tags and timeout stay. Upstream’s replay is a boolean or the start-replay body; the port has Replay (default true) and ReplayParams. Scope is null for the default, BrowserProviderScope.Worker.
• The web engine calls RecordAsync for an attempt that records video (see Video): the replay is the attempt’s video/replay.mp4, and a headless browser, or Replay = false, gets the engine’s screencast. It does not call Downloads yet: the port has no waitForDownload. The provider tests both against a fake Kernel; the recording is not checked against a live Kernel.
• Not verified against a live Kernel, because no KERNEL_API_KEY was available. The endpoints, field names, and paging headers were read from the @onkernel/sdk 0.113.0 source (browsers.ts, replays.ts, fs.ts, core/pagination.ts, client.ts), the version upstream pins. The wire format may differ from what is written here; a first run against a real account is the check.
• tests/E2E.Kernel.Tests ports packages/kernel/tests/unit/kernel.test.ts: Kernel/KernelTests.cs for describe('kernel()') and Kernel/KernelRecordTests.cs for describe('record'), 15 of the 16 it. The 16th, “keeps the idle timeout backstop for an undefined option from a JavaScript config”, has no C# counterpart: an unset TimeoutSeconds is null, and “creates a browser with the run tags and the idle timeout backstop” checks it. Two have the suffix PARTIAL_ in the name: “fails without an API key in the run environment and reuses one SDK client per key” (no SDK client to reuse; the test checks that each request carries the key of its run) and “records a Kernel replay of the browser …” (a call has no signal of its own to compare with the step’s). The mocked-SDK harness is FakeKernel, a fake HttpMessageHandler. RestApi/KernelRestApiTests.cs has six .NET-only tests of the REST requests (base URL, create body fields, a JSON error body, paging), which no upstream it covers.

TestMu mobile

E2E.TestMu.Mobile drives TestMu AI real and virtual mobile devices through Appium. Upstream has @e2e-dev/testmu (commit 88ed717), a testmu() device provider that leases a device for each worker slot and drives it through an agent-device daemon (daemon.ts, driver.ts, heartbeat.ts, sessions.ts). The port does not port it. It talks to TestMu’s hub with Appium instead, so every type here is .NET-only. The package depends on E2E.Mobile and holds an IDeviceClient over a W3C WebDriver session on TestMu’s hub (https://mobile-hub.lambdatest.com/wd/hub), with the flat LT:Options capabilities TestMu documents. It is a preview: no provider leases a device yet, and the page source is not converted to a snapshot yet, so capture.snapshot fails with NOT_IMPLEMENTED (UNSUPPORTED_CAPABILITY in a test). What it does:

• E2E.Mobile gets one new public interface, IDeviceClientSource, which has no upstream counterpart. A MobileOptions.Provider that implements it makes the IDeviceClient for each session, in place of the agent-device client. A worker in another process builds its engine from the same options, so its provider object makes the client there. What the client needs to find the leased device travels in DeviceLease.Client, which reaches the worker as DeviceConnection.Client. Any other provider is unchanged.
• Supported: launch and terminate an app, deep links, tap, double tap, long press, fill, type, swipe, pan, drag, scroll, screenshot, home, back, native alerts, the keyboard (Enter and dismiss), orientation (portrait and landscape), and the clipboard. A ref finds the node’s rect in the newest snapshot, and a tap lands on its centre. A ref of an older snapshot is stale (NODE_STALE).
• Refused as UNSUPPORTED_OPERATION, which a test sees as UNSUPPORTED_CAPABILITY: permissions, appearance, biometrics, location, wifi and airplane mode, clearing app state, the keychain, hover, folding, the foreground app, installing a build from a test, and launch arguments except on an iOS relaunch. The provider will install the app when it leases the device.
• sessions.close does nothing. The engine closes at the end of every attempt, and the next attempt drives the same device, so only the provider’s release will end the session.
• fill taps the field and writes to the focused element, because a snapshot ref is not an Appium element id. back on iOS taps the first button in the navigation bar, as agent-device does, and fails when the screen has none; Android uses the system back.
• settle options are ignored: the engine settles on observe. normalizeStatusBar is sent on for the masked screenshot an observation takes, but the Appium client ignores it (Appium cannot normalize the status bar), so a screenshot shows the device’s real status bar.
• The package is in CI’s pack step but not in the release workflow until the provider lands. Not verified against a live account: every wire detail was read from TestMu’s and Appium’s documentation, and the mobile: script names and arguments were not sent to a real hub. The unit tests use a fake hub. Among them: whether the hub accepts the LT:Options-only capabilities without a top-level platformName, the session id field, and which mobile: commands it forwards.

What still differs from upstream’s @e2e-dev/testmu

E2E.TestMu.Mobile serves the same TestMu AI devices another way, so it is a client, not a port. Compared with upstream’s device provider (provider.ts, daemon.ts, driver.ts, heartbeat.ts, sessions.ts):

• No IDeviceProvider leases a device: upstream’s testmu(options) (device, osVersion, app, deviceType, project, build, sessionName, orientation, geoLocation, timezone, language, locale, appiumVersion) has no counterpart. Nothing uploads the build from app, names the dashboard session (e2e-<run id>-<target>-<slot>, with the project and the build), or releases the session at the end of a run. The package opens its session from a TestMuSessionRequest a caller builds.
• No RecordAsync: upstream’s record finds the lease’s session on the dashboard (findSession) and returns its video URL (sessionVideoUrl) as a video/mp4 link, with the session’s start time. The seam is ported (see Provider recording), and nothing on TestMu’s side feeds it. The client refuses recording.record, so a TestMu device has no video.
• No agent-device daemon or plugin: upstream starts one daemon per slot with @agent-device/testmu, checks its version (assertDriverVersion), keeps the lease alive with a heartbeat (LEASE_TTL_MS), and allocates with retainOnClose. The port has no heartbeat.
• The page source is not converted to a snapshot, so a test cannot observe the screen (capture.snapshot is NOT_IMPLEMENTED).
• Credentials: upstream reads LT_USERNAME and LT_ACCESS_KEY from the run’s environment (testmuCredentials); this package takes the hub credentials from its request.

MAUI DevFlow (not in upstream)

E2E.Mobile.DevFlow drives a .NET MAUI app through the MAUI DevFlow agent, the HTTP server Microsoft.Maui.DevFlow.Agent runs inside the app in Debug builds (builder.AddMauiDevFlowAgent()). Upstream has no counterpart, so every type here is .NET-only. The package depends on E2E.Mobile and holds an IDeviceClient over the agent’s /api/v1 API plus the provider, DevFlowAgents, an IDeviceProvider and IDeviceClientSource to pass as MobileOptions.Provider. It is a preview: it attaches to an app that is already running, and the device-side verbs are not there yet. What it does:

• DevFlowAgents.AcquireAsync finds the agent (DevFlowAgentsOptions.Port, else the one agent of the platform and AppName a running DevFlow broker lists at port 19223, else port 9223), waits ConnectTimeout (60 s) for its status, checks the platform (status.device.platform, where the agent writes it) and the app’s package id against the target, and claims the agent’s mutation lease as holder e2e, so the DevFlow Inspector and the MCP server see who drives the app. The agent answers a claim with 200 and says who holds the lease; one another host holds fails the run naming the holder. The lease travels to the worker in DeviceLease.Client (host, port, leaseId, label). Release gives the lease back and leaves the app running. A worker without a binding finds the agent on its first call, with the platform devices.boot names, and claims a lease of its own there. The broker is never started from here. One agent is one running app, so a target with more than one worker slot is INVALID_CONFIG.
• The agent lets a lease lapse ten seconds after the holder was last heard from, and refuses a mutation under a lapsed lease with 409 reason: lease. So, as Microsoft’s own AgentClient does, every request that changes the app claims the lease again first (a claim the holder has is a heartbeat). A 409 lease on an action names the holder from the agent’s details, or says the lease lapsed when there is none.
• capture.snapshot reads GET /ui/tree?envelope=true and flattens it into the snapshot the engine projects. The agent answers 409 capture-changed-during-read when the screen moves during the read, as it does while an animation runs, so the tree read and the hit-test are tried up to four times with a short wait before they fail. MAUI types become agent-device kinds (Entry is text-field, Label is static-text, Switch is switch); a type without a mapping keeps its name in kebab case (vertical-stack-layout), and the agent’s own link, heading, and listitem roles win over the type. The AutomationId is the identifier GetByTestId reads. A field’s text is its value; a password field reports password: true and no value, because the agent writes [REDACTED] in place of the text and never sends the plaintext. A switch, check box, or radio button reports checked. displayed and opacity give visibleToUser. The viewport is the window’s bounds. interactiveOnly keeps the interactive, labelled, and identified elements and re-parents their children.
• Actions carry the element id and the captureEpoch of the tree the ref came from: tap (a tap at a bare point hit-tests the point first, GET /ui/hit-test, since the agent’s tap takes an element), double tap (gesture doubletap), long press (gesture longpress, 1 s by default), fill, clear (a fill with empty text), type (the agent’s key with text on the field that has the focus, read off a fresh tree, and key: "text", a name that is no key, so a typed delete or enter is appended rather than pressed), Enter (key on the focused field), a screen scroll (gesture swipe on the page the other way, half the viewport by default), a node swipe and scroll, a pan, back (POST /ui/actions/back, which pops the navigation, a modal, or a Shell route), the app’s theme (PUT /device/app/theme for device.setAppearance), the foreground app (the agent’s status), and screenshots (GET /ui/screenshot?scale=native, with the logical size from the window’s bounds).
• apps.open checks that the agent runs the app the target names and answers its identity. A relaunch, launch arguments, deep links, install, close, home, alerts, orientation, folding, the clipboard, the keyboard dismiss, hover, a drag between nodes, a long press or pan at a bare point, and every device setting but the theme are refused as UNSUPPORTED_OPERATION, which a test sees as UNSUPPORTED_CAPABILITY. sessions.close does nothing. settle options are ignored: the engine settles on observe. normalizeStatusBar is ignored.
• Errors: a 409 (a stale capture, a changed tree), an element the agent does not find, or a ref the newest tree does not have is COMMAND_FAILED with reason: ref_not_found, a retryable NODE_STALE; a 501, a 404 that names no element, and the not_supported and unsupported-capability codes are UNSUPPORTED_OPERATION; a mutation lease another host holds names the holder; an agent that does not answer is SESSION_NOT_FOUND (INVALID_STATE), with the hint that the app may not be running. The agent answers { success: false, error, reason } rather than the RFC 7807 body its spec shows, so both shapes are read.
• E2E_DEVFLOW_TRACE=1 prints each call to the agent (method, URL, status) to stderr, since loopback calls skip a system proxy such as Proxyman. Bodies are not printed: they hold typed text.
• The sample (samples/MobileApp) registers the agent under #if DEBUG and runs the same tests with E2E_MOBILE_PROVIDER=devflow against the app you launched. The two tests that send the app to the background or relaunch it are inconclusive in this mode.
• The package is in CI’s pack step but not in the release workflow until the device-side verbs land. Not verified against a running app: every wire detail was read from the agent’s source and spec in dotnet/maui-labs at main (the 0.1.0-preview.13 packages). The unit tests use a fake agent. Among the things to check on a device: the window bounds as the viewport on iOS and Android, whether role is set on MAUI elements, the key action with text, the lease body, and the broker list.

WinApp engine (not in upstream)

E2E.WinApp drives a Windows desktop app through Windows UI Automation, with the library behind the winapp ui command of the Windows App CLI (Microsoft.Windows.SDK.BuildTools.WinApp.UIAutomation), in the test’s own process. Upstream has no desktop engine, so every type here is .NET-only. The package follows the shape of E2E.Mobile: WinAppEngine is the IEngine (Platform windows, capabilities observation, actions, keyboard, scroll, screenshot, and pinned app; no location, history, video, or tap modifiers), WindowSession the IEngineSession, UiaProjection the projection into the shared node model, WindowActions the action mapping, WindowKeys the key grammar, WinAppErrors the error table, Window the fixture, WinAppTools the agent tools, and WinAppTargetEngine the one that reads a TargetConfig. The library is reached through one seam, IWindowDriver, whose only implementation, UiAutomationDriver, is the only type marked [SupportedOSPlatform("windows")]; the rest of the package builds and is unit-tested on any platform (tests/E2E.WinApp.Tests, with a scripted driver). What it does:

• A windows target takes app.appPath (the executable to launch, relative to the config folder; E2E_WINAPP_APP_PATH overrides it) and app.bundleId (the process name). With a path the engine launches the executable itself (WindowProcess: no shell, ArgumentList, stdout and stderr pumped into the attempt’s app log, a visible top-level window awaited within LaunchTimeout, the process tree killed on timeout, cancel, or close). With a name alone it attaches to the running app and never ends it. app.open() has no URL to open, so it relaunches the app; restart and clearState relaunch too, and nothing is deleted. A URL passed to open is UNSUPPORTED_CAPABILITY.
• An observation inspects the main window and every other visible window of the process (a WinUI flyout, menu, or combo box list opens in a popup window of its own), nests the flat element list the library returns by depth, and projects UI Automation control types onto the browser roles (Button is button, a ToggleSwitch class is switch, Edit is textbox, a PasswordBox class is a secure textbox with no value, List/ListItem, ComboBox, Tab/TabItem, Window is window for the app window and dialog for a ContentDialog, popup, or flyout; an unmapped type is its name in lower case). The test id is the AutomationId. Boxes are logical pixels, relative to the main window, read through the window’s DPI. Refs are the library’s selector slugs, bound for the session in a bounded list, since UI Automation identity is stable across inspections: an element the library no longer finds is NODE_STALE.
• Actions prefer the control’s own pattern: tap invokes, selects, toggles, or expands an invokable control and clicks the middle of any other; fill and clear use the value pattern and fall back to focus, Ctrl+A, and typed text for a control that refuses it (a RichEditBox); press and type focus the control, bring the window forward, and send keys through SendInput, which reaches the foreground window only (WinUI controls are windowless and drop posted messages); a window that cannot come forward is NOT_ACTIONABLE; select expands a combo box, observes again, and selects the option by name, value, or automation id; check and uncheck use the toggle pattern (a radio button is selected and cannot be unchecked; nothing is sent when the state already matches); a swipe turns the mouse wheel over the node for up and down, and uses its scroll pattern for left and right or when the desktop cannot take input. On the screen it turns the mouse wheel over the largest scrollable control (the pattern steps by a share of the whole extent, too coarse to land a row of a long virtualized list in view; six notches move a screen or so), scrolls that control by pattern when the desktop cannot take input, and turns the wheel over the window when there is none (up and down only: a sideways swipe with nothing to scroll is NOT_ACTIONABLE). A sensitive fill never puts the value in a message. The session also offers hover, long press, drag, a right click, and typed text.
• A screenshot is the main window, encoded as PNG by the package’s own PngEncoder (the CLI tool strips the Skia natives), with every secure field painted black; a secure field without a box withholds the image.
• Window is the fixture: OpenAppAsync (the app under test, with Relaunch, or another running app by name), CloseAppAsync, ForegroundAppAsync, ListWindowsAsync, ResizeAsync, MaximizeAsync, RestoreAsync, MinimizeAsync, and AlertAsync, which answers the dialog the newest observation saw: accept presses its PrimaryButton (else a button named OK, Yes, Confirm, Accept, Save, or Continue), dismiss its CloseButton or SecondaryButton (else sends Escape). WinAppTools.Create(engine) gives the agent open_app and alert, scoped to windows.
• Errors: a launch that fails or shows no window is ENVIRONMENT_UNAVAILABLE; a vanished element is a retryable NODE_STALE; an offscreen control, a refused value, and a lost foreground are NOT_ACTIONABLE; an app the engine cannot find is INVALID_STATE (with the hint to call app.open()); anything else the library reports is ENGINE_FAILURE. A caller’s cancel is CANCELLED. The engine never produces OPERATION_TIMEOUT.
• e2e explore and e2e mcp take a windows target (TargetEngines.Create gets a windows factory the way it gets a device one; WindowsTargets in the CLI builds the engine on Windows and is INVALID_CONFIG on another host). Pattern actions work with the window anywhere; keys, clicks, drags, and the wheel need an unlocked, interactive desktop with the window in the foreground.
• Not there: App.BackAsync (UNSUPPORTED_CAPABILITY: a desktop app has no history), video, tap modifiers, popup windows in screenshots, a deep-link tool, and the clean-up of the app’s data on clearState. A packaged (MSIX) app starts through a broker and exits at once, so it is attached by app.bundleId, not launched.

WinAppBenchmark

apps/WinAppBenchmark is upstream’s apps/mobile-benchmark rebuilt as a WinUI 3 app with Microsoft.UI.Reactor: 19 of its 25 scenarios, with the same names and the same AutomationId where upstream has a testID. Not ported, because the surface is mobile-only: Permission Prompt, Photo Picker, Stripe PaymentSheet, Apple Pay, Apple Pay Billing Address, and WebView Accessibility. The README of the app lists each change made to fit a desktop window.

apps/WinAppBenchmark.E2ETests ports upstream’s apps/mobile-benchmark suites (smoke, login-form, flows, flows-lists, controls, device, device-fixture, and tests-agent/scenarios), one directory per upstream file and one class per describe. They are real-model tests of the Windows category, so CI builds them and does not run them. Not ported: the tests of the six scenarios the app lacks, the fixtures a desktop window lacks (appearance, clipboard, home, network, location, biometrics, keyboard), describe('refusals') (the Windows engine declares the verbs it checks), the placeholder query, and the pointer-point test (tapAt, a tap position, screen.swipe). The agent scenarios Broken Accessibility, Vision Only, Flattened Registration Form, Flattened Login, and Gestures are declared and skipped with a reason, as upstream declares a gap: agent.act gives the model the tree only, never a screenshot, and it has no long press. The test project README lists each test that differs from its upstream it.

WinAppTest (not in upstream)

E2E.WinApp.NUnit and E2E.WinApp.XUnit.V3 add WinAppTest, a base class over E2ETest for a windows target, the way MobileAppTest serves a device target. It reads AppId (app.bundleId) and AppPath (E2E_WINAPP_APP_PATH, else app.appPath) from the target, builds a WinAppEngine from CreateWinAppOptions(), hands the agent WinAppTools, sets the step timeout to 90 seconds, and relaunches the app before each test (WinAppHost.OpenAsync, shared by both adapters from src/Shared). Window is the session’s fixture. The base class adds no [Category].

TestMu

TestMuBrowsers (the E2E.TestMu package, namespace E2E.TestMu) is the port of upstream’s testmuBrowsers() (packages/testmu/src/browsers.ts, with dashboard.ts and web.ts, commit 92f8cb0), ported in part. It is a WebEngineOptions.Browser provider for the TestMu AI (formerly LambdaTest) web grid. Upstream’s mobile device provider is a separate part of the same package (see TestMu mobile). Taken from upstream: the Edge --user-data-dir=remote-debug-profile-edge entry in browserOptions (appended to the given ones, kept if a --user-data-dir= is already there), the idleTimeout default of 600 seconds, the project default of e2e, the session name e2e-<run id>-<target>-<slot> (a per-attempt lease is e2e-<run id>-<target>-attempt-<attempt id>), the Windows 11 default for Platform, the credentials read trimmed with a blank value counted as unset (the error names each missing variable), and the validation of Capabilities at construction (INVALID_CONFIG for a nested LT:Options, browserName, user, accessKey, name, a browserOptions that is not an array of strings, and a number JSON cannot encode). Not taken: the /puppeteer CDP URL (the port keeps the Playwright-protocol endpoint, below), the browserName check (the port also lets pw-chromium through), and the unknown-key check (a record has no unknown keys). The port keeps the seam’s names (IBrowserProvider, BrowserLease) and adds only the provider, TestMuBrowsers, and its options record, TestMuBrowsersOptions. Only the web grid is covered; TestMu’s real devices are not. Differences from a provider like upstream’s kernel:

• It calls no TestMu API. AcquireAsync builds wss://cdp.lambdatest.com/playwright?capabilities=<JSON> from browserName, browserVersion, and LT:Options (platform, build, name, user, accessKey, and the options that were set). The grid starts the session when the engine connects. The lease’s Protocol is Playwright: the grid answers Playwright’s own protocol on this URL, not CDP, so the engine dials with Chromium.ConnectAsync and opens its own browser context, because the browser has none. ReleaseAsync calls nothing, because the grid ends a session when its connection closes (checked against a real account). SweepAsync is the interface default.
• The TestMu dashboard shows these sessions without a Passed or Failed mark. TestMu marks a session through its Automation API or a lambdatest_action call, which needs the test result, and e2e passes no test result to a provider (upstream’s BrowserReleaseContext has none either). The provider also does not look up the session id.
• The credentials are LT_USERNAME and LT_ACCESS_KEY, or the aliases TESTMU_USERNAME and TESTMU_ACCESS_KEY, read from the run’s environment. The access key is part of the endpoint, so the lease id is the session name (e2e-<run id>-<target>-<slot> or e2e-<run id>-<target>-attempt-<attempt id>), which is what progress lines print. The engine puts no endpoint in an error or a log line, and Playwright drops the query string from the URL it prints when a ws:// or wss:// connection fails, so a refused connection does not show the key (a test checks it). Each lease endpoint is about 650 bytes, and a worker’s leases travel in one environment variable capped at 16384 bytes, so a target with more than about 20 worker slots fails at prepare.
• Checked against a real account: ConnectOverCDP on this URL fails with Protocol error (Browser.getVersion), Chromium.ConnectAsync works, and closing the connection ends the session. TestMuLiveTests repeats that through the engine and reads the session’s end from TestMu’s Automation API. It is skipped without LT_USERNAME and LT_ACCESS_KEY. Mobile devices are not covered.
• Behavior change: a Capabilities entry named name, user, or accessKey used to replace the provider’s value. It is now refused with INVALID_CONFIG, as upstream does.
• TestMuBrowsersFunctionTests (Browsers/) ports the its of upstream’s browsers.test.ts for the Edge flag and the validation. Two are partial (Partial_ prefix): the Edge it that also checks upstream’s build (run-1; the port’s is e2e-<run id>) and browserVersion inside capabilities, and the rejection it.each, of which the rows for user/accessKey, name, nested LT:Options, browserName, browserOptions (a string, [1], null) and a value JSON cannot encode are covered. The project and idleTimeout defaults are checked in the .NET-only TestMuBrowsersTests, because the upstream it that checks them also checks the /puppeteer URL and the Windows 11 default together with build: run-1 (the port’s build is e2e-<run id>). The its for the session name and for a missing or blank credential are ported. Not ported: the /puppeteer URL (the port’s endpoint differs), the lease id without spaces, undefined values keeping the defaults (C# has no undefined), the unknown-key, browserName, non-object capabilities, Date, Map, function, BigInt and cycle rows (a typed record and JsonNode cannot hold them), the log line (the port logs nothing), the credentials taken from process.env (a test would change the process environment, which the live test reads). The other TestMuBrowsersTests and TestMuLiveTests are .NET-only, in tests/E2E.TestMu.Tests.

Browser install command

Upstream’s e2e-web command (packages/web/src/cli.ts) has one subcommand, install. It runs the pinned Playwright CLI. The port has it as e2e install [chromium|firefox|webkit ...] [--with-deps], in the tool that already ships the Playwright driver. It installs Chromium when no browser is named, forwards the options before the browsers, exits with the Playwright code, prints usage for -h or --help anywhere, and exits 2 for an unknown option or browser. As upstream, the install runs with PLAYWRIGHT_SKIP_BROWSER_GC=1 unless the user set it. The engine itself drives Chromium only.

Upstream’s command lives in the web package (e2e-web, from @e2e-dev/web), apart from the e2e command. The port is one tool, and WebEngine is in the E2E package, so the command is e2e install in E2E.Cli, the one place the Playwright driver ships. The code follows upstream’s split: E2eWebCli.MainAsync is main and CliIo is its CliIo (E2eWebCli is named for the command; a method called Main would be taken for an entry point), WebEngine.RunPlaywrightCli is runPlaywrightCli, and BrowserConnection.BrowserNames and IsBrowserName are BROWSER_NAMES and isBrowserName.

Not ported:

• A command named e2e-web, and npx, bunx, and pnpm exec usage lines. dotnet e2e install is the local tool form.
• Upstream’s exit code 2 for a missing or unknown command. e2e exits 1 for both, as it does for every command.
• Forwarding SIGINT and SIGTERM to the installer and the 128-plus-signal exit code. Microsoft.Playwright.Program.Main runs in this process. Ctrl+C in a terminal reaches its driver in the same process group; a SIGTERM sent to e2e alone does not.
• install.test.ts (ensureBrowsersInstalled) is the first-launch install of the engine, not the command. The port covers it in E2E.Tests/Install, and upstream’s own checks of the installed-browser cache and of the run’s environment have no counterpart: the port leaves that to Playwright’s install (see Web engine).

MCP server

Ported from upstream packages/e2e/src/mcp at commit ee901b333ed12352418dfb07bda5eafe32e71c8d (docs(agents): show several personas running one suite (#1258)), apart from what is listed below. e2e mcp serves over stdio with the server name e2e, the tools, resources, and logging capabilities, and upstream’s four fixed tools with their names, descriptions, and closed argument schemas. Stdout carries only the protocol; every other write goes to stderr, and the MCP SDK logs nothing. The flags are --config, --target, --headed, a hidden --headless, and --max-sessions (1 through 16, default 4). A session is headless by default; open_session takes headed to show the browser or hide it, and --headed makes headed the default for a session that does not say (upstream commit dc2afca). A bad command line exits with 2, a disconnect or a signal with 0, and an unexpected error with 1. The resources e2e://guide and e2e://guide/<topic> serve the bundled skill; an unknown topic fails with UNKNOWN_TOPIC.

A session is an attempt on the config’s target with an engine of its own: a Chromium WebEngine for a web target, or a MobileEngine for an ios or android target (see Device sessions below), or a WinAppEngine for a windows target (see WinApp engine), so parallel agents each drive their own browser, device, or app (a windows target with only app.bundleId attaches to one running app, which they share). open_session loads e2e.config.json fresh, claims the config (CONFIG_IN_USE for another config while a session is open), and fails UNKNOWN_TARGET for a target the config does not declare, CONFIG_NOT_FOUND for no config, SESSION_OPEN when no slot is free. call and tools name a session, or fail NO_SESSION or SESSION_REQUIRED. A session ends on close_session, after 30 minutes without a call, or after 4 hours. It never reads or writes the replay cache. Every configured secret is a secret of the session: any output, error, log line, and the close summary shows its value by name (<secret:name>). type_secret fills a secret into an editable field (a password secret needs a password field), and after a fill screenshot answers PIXEL_TAINTED and attaches nothing.

The catalog is what the testing agent’s tools do, so a coding agent acts as the model of an act does: observe, tap, double_tap, type, press, select, check, uncheck, clear, scroll_to, scroll, navigate, back, screenshot, type_secret, and locate. scroll_to and scroll need EngineCapabilities.Scroll, back needs History, screenshot needs Screenshot, and type_secret needs a secrets entry. navigate allows http and https only (POLICY_DENIED for any other scheme).

Differences:

• Arguments name a node by its ref (target: "e12", from observe), as the testing agent’s tools do, where upstream uses node ids (n42) and a revision. Actions answer with the screen after the action, not with a diff of what changed since the last one. A failed action is an error result: tap e9 failed: NOT_FOUND: node e9 is not on the current screen; re-observe for the newest refs.
• The grammar is the port’s: type and type_secret take text and secret; uncheck and clear are tools of their own. Upstream’s long_press, right_click, hover, drag, upload, and the point tools (tap_at, hover_at, type_at, press_at, select_at) have no counterpart in the port’s actions.
• locate writes the Screen call a test uses (Screen.GetByRole("button", "Save")) and the port’s codes: NOT_FOUND and STRICT_MODE, where upstream names LOCATOR_NOT_FOUND and LOCATOR_AMBIGUOUS.
• As upstream, a locate with a status, alert, or listitem name that matches nothing points at Screen.GetByRole(role).Filter(text) and Screen.GetByText (spelled as GetByRole(role).filter({ hasText }) upstream), and an exact: false query is not told to loosen the match again (upstream #1147, 5e9597c): the advice says “an” before a role that starts with a vowel (an alert, a status), and for an exact: false query it keeps the option in the text call, Screen.GetByText("Saved.", exact: false) (getByText(q, { exact: false }) upstream). The web engine names those three roles from their content, so there the name also matches; the advice still holds, since the text matches either way.
• Test lines (upstream #1317, 9ceb071): every action answers with the Screen or App call that repeats it, after the first line of its result, and the read-only test_lines tool returns them as a draft test body. The code is C# (await Screen.GetByRole("button", "Save").TapAsync();, Secrets.Get(name), Filter(text), Nth(i), new RoleOptions { Visible = true }, App.OpenAsync(path)), and SessionTestLines (Mcp/TestLines.cs) is upstream’s SessionTestLines (mcp/test-lines.ts). Differences: the port’s tree carries a node’s test id, so GetByTestId is the last query tried for a node, where upstream reads the id off the live node; the port resolves each locator through the engine’s LocateAsync (2 s each, binding only the matches, so a device keeps the agent’s node ids valid) when the engine has one, else on one fresh look at the screen per node with LocatorResolver, where upstream spends one engine resolve on each (the cap of 10 tries stays); the live look comes in the port’s resolver order, so the twin of a shadow root is told apart by its place in that order and never by the box it kept; there are no credentials, so a fill names Secrets.Get(name) only; a type that fell back to the keyboard does not exist, because the port’s type always fills; a locator tap takes no modifiers, so a tap or double_tap with modifiers prints No test line:; a press has no times. The web engine names a list item from its content, so a row is found by that name (GetByRole("listitem", "Offer B Reserve")) before it is found by Filter(text). The draft ends with a line that tells the agent to add an Expect after each step, where upstream also names the credentials and secrets imports. The session catalog reads the newest observation from the agent’s observation feed, where upstream wraps the step context. Not ported with it: the reserved tool name (the port has no project tools), the telemetry event of an MCP session. The drafted lines run as a test class of a project built in a temporary directory with dotnet test (DraftedTestRun), where upstream runs them with its runProject.
• The fixed tools check their arguments with a strict check of the JSON Schema subset tool schemas use, since the .NET MCP SDK does not validate them. The messages are zod’s, so an unknown argument reads Invalid arguments for tool <name>: Unrecognized key: "<key>", as upstream.
• The server instructions name e2e.config.json, E2ETest classes, and dotnet test in place of e2e.config.ts, tests/*.e2e.ts, and npx e2e run.
• The guide has upstream’s eight topics, written for .NET, plus writing-tests-nunit and writing-tests-xunit, one per test framework. explore describes Explore.ExploreAsync, and explore and bug-bash say that the bug bash fan-out is not ported.
• The 2026-07-28 MCP revision deprecates logging. The server still declares it and sends its log lines, as upstream does.
• The config declares one target, so TARGET_REQUIRED cannot happen. A session’s engine is its own, so sessions never share an engine instance, and ENGINE_IN_USE happens only for a device (see Device sessions below). The credential registry upstream keeps process-wide has no counterpart (the port has no credentials).
• The e2e tool ships the Playwright driver of every platform (about 215 MB) and none of the SkiaSharp native libraries, since it never compares screenshots. Chromium installs on the first session, as for a test.
• Device sessions: open_session on an ios or android target builds a MobileEngine on a local simulator or emulator (see Explore). The catalog leaves out navigate for an engine that declares PinnedApp, as the engine has no URL to open; before the first observation the session launches the pinned app fresh (TargetEngines.LaunchesPinnedApp, then App.OpenAsync), as upstream’s launchesPinnedApp does (upstream #1252, 3bd32ef). The summary adds App: <bundleId or appPath>, launched fresh., and the open_session description names the pinned app. It installs no build, so a local device target needs app.bundleId. The launch is bounded by the session’s time limit, as upstream bounds it by the attempt’s timeout, and a cancel of the request during the launch surfaces as CANCELLED. A device runs one app, so a second session on a device target fails with ENGINE_IN_USE (upstream’s code for two sessions on one engine instance; the port builds an engine per session, so SessionRegistry.ClaimEngine claims the device, .NET-only) until the first closes. Hosted devices are not ported yet. A windows target builds a WinAppEngine the same way (WindowsTargets.Create, E2E.Cli), on Windows only (INVALID_CONFIG elsewhere), and launches the executable app.appPath names or attaches to the process app.bundleId names before the first observation.

Not ported:

• start_recording and stop_recording (Kernel video), the project’s own tools (defineTool, toModelOutput), custom engines, mobile tools, and per-session telemetry.
• Redaction of what user code prints: a config is JSON, so no user code runs in the server.
• e2e init registering the server. The port has no e2e init (see below); register it with claude mcp add e2e -- e2e mcp or a .mcp.json entry.

Agents

E2ESessionOptions describes the default agent with Model, Judge, AgentExecutor, AgentSystem, AgentTools, AgentContext, MaxSteps, MaxModelCalls, JudgmentTimeout, and ProviderOptions, and names the others in Agents (an AgentOptions per name). E2ETest fills both from agents.<name> in e2e.config.json. A call picks an agent with its Agent option; an empty or unknown name throws INVALID_ARGUMENT (unknown agent "x"; configured: default, ...). Each agent starts from the defaults and inherits nothing from another. Settings are checked when the session starts and fail with INVALID_CONFIG.

• A test or a class picks the agent for its calls with [Agent("name")] (upstream’s agent option of test and describe). The closest pin wins: a call’s Agent, the method, the class, a nested class’s enclosing class, then E2E_AGENT (the port’s --agent, one name; a pin wins over it, as upstream). An unknown name fails the test with INVALID_ARGUMENT when the session starts, where upstream fails at collection. [AgentContext("...")] is upstream’s agentContext: the layers join outer to inner with newlines and follow the agent’s own context, for every agent a call picks, and the cache key sees the joined text. .NET-only symbols: AgentAttribute, AgentContextAttribute, E2EFixture.RunAgent, E2ESessionOptions.Agent and TestAgentContext, and TestAgentOptions (internal). Not ported: a list of agents (agent: ['buyer', 'admin'], one result per agent), because NUnit and xUnit have no collection step to fan a test out; the serial-group and setup-test rules that go with it; and --agent listing several names.

• judge judges assert, waitFor, and extract, and defaults to the model. act always uses the model

• system is appended to the built-in act rules. Judges never see it

• context (at most 16384 UTF-8 bytes) is told to every model call: after the act rules as Project context:, and to judges inside <project-context>

• maxSteps and maxModelCalls are 1 through 100, 25 by default

• judgmentTimeout (30 s) bounds assert, waitFor, and extract unless the call sets Timeout. A per-call timeout must be positive; a waitFor interval is 100 ms through 60 s

• providerOptions is a dictionary of JSON objects by provider. It rides every ModelRequest, and OpenAiCompatibleModel adds the fields under its Provider key (openai by default) to the chat-completions body as given, so write the wire names (reasoning_effort, not reasoningEffort). It cannot replace model, messages, or tools. Every other client does the same with its own key (anthropic, google, bedrock, azure, xai, openrouter, gateway) and refuses its own core fields. As upstream, requests to OpenAI (and Azure over Responses) carry store: false and a prompt cache key per system prompt, and Anthropic requests carry cache breakpoints on the system prompt and the newest message; provider options win over both

• assert and extract make one model call and one repair round. waitFor judges at once, then again only after Interval and on a changed screen, and every call, repairs included, counts against MaxModelCalls. It ends STEP_TIMEOUT (waitFor timed out; last judgment: ...) or STEP_BUDGET_EXHAUSTED (waitFor exhausted its model-call budget; last judgment: ...)

• As upstream, the judge prompt says a value shown in more than one place holds only when every place agrees, unless the statement names the place or is about some item among several. The wording follows upstream, with “instruction” as “statement” and the "fails" verdict as “false”, because the port’s judge answers through done

• ExtractAsync<T>: the type argument is the schema, in place of a Standard Schema. The judge gets the JSON schema of T (JsonSchemaExporter), and the answer must deserialize into T with required members and nullable annotations respected. A failure gets one repair round with the validation error, then MODEL_OUTPUT_INVALID (extracted data failed schema validation: ...). A judge that says the data is not shown ends ASSERTION_INCONCLUSIVE (nothing to extract: ...)

• tools (AgentOptions.Tools, AgentTools on the default agent) are project tools the model is offered beside the agent’s own. As upstream’s defineTool, an AgentTool declares mutates and platforms: a tool is offered only on the platforms it names, and a mutating tool takes an action slot. Its output is redacted and cut at 16384 characters. A name the agent already uses (observe, done, navigate, back, scroll, scroll_to, or a verb) is refused with INVALID_CONFIG. Differences: the tool is an AgentTool built with its schema as a JsonObject and a delegate, since the port has no defineTool, AI SDK tool, or toModelOutput; a tool that wants upstream’s getToolContext takes the second constructor, whose delegate gets an AgentToolContext (ObserveAsync, AttachScreenshotAsync), and the first constructor keeps working; and upstream’s reserved names for tools it adds elsewhere (complete_step, locate, the recording tools, report_finding) have no counterpart. Upstream records a mutating tool as a gap in the trace, and so does the port: a recording holds an action of kind tool with the tool name, a replay runs the actions before it and then hands the step to the model, and the step is never replayed whole. Older versions of this port cannot read that action and run the step live. As upstream, a recording that stops at such a gap (reason gap) is not stale: a strict cache run replays what it can and runs the rest of the step live. The recording keeps the tool’s name only, never its arguments

• tap and double_tap take an optional modifiers list (Shift, Control, Alt, Meta, ControlOrMeta) on an engine that declares EngineCapabilities.TapModifiers, as upstream’s tap, double_tap, and right_click do on an engine that declares tapModifiers. The web engine declares it. The device engines and the document engine do not, and refuse the field with UNSUPPORTED_CAPABILITY. The held keys are recorded as modifiers of the tap or doubleTap action and replayed from the cache. An entry whose modifiers is empty, repeats a key, or sits on another action is a miss (invalid-entry). The tools do not close their schemas, so a tool called with a field its engine does not declare is refused by the action, not by the schema. There is no right_click, because no engine here has a secondary tap.

• act also offers a double_tap tool, which upstream does not have. Without it, the agent cannot open a control that reacts only to a double-click, such as a TodoMVC label that opens its editor. It runs the locator doubleTap, and the replay cache records it as doubleTap. Upstream and older versions of this port cannot replay that entry, so they run the step live

• ActOptions.OnAction is upstream’s onAction: a hook that runs after each action once the screen settles, for replayed actions too. The hook is an ActionHook delegate returning a Task, and the event’s source is the ActionSource enum (Model, Replay). Upstream’s check that onAction is a function has no counterpart, since the delegate type says it. A hook that throws fails the step (ACTION_FAILED, agent.act onAction failed: ...) and leaves the step’s recording alone. Upstream takes a screenshot from the hook with app.screenshot, which is not ported, so the port’s tests record the events and the screen the hook reads. Tests: agent-act-on-action.test.ts is AgentActOnAction/ (6 of 6, plus one .NET-only test for a replayed step whose hook never completes; the first is renamed and partial: it does not check that each hook step holds one screenshot artifact. The others record the hook’s events in a list where upstream takes an app.screenshot step, and check the same behavior). The act-validation it “takes an onAction function and refuses any other value” is skipped: a delegate cannot be anything but a function. .NET-only: AgentException.FromActionHook (internal), where upstream has an ActionHookError subclass, because AgentException is sealed

• act params are checked as upstream: JSON-safe values, at most 32 levels deep, no cycle, and at most 64 KiB once a Secret is projected to its name and purpose and a Values.Unique to its value. The instruction is at most 8192 UTF-8 bytes. Each limit throws INVALID_ARGUMENT with the upstream message before any model call. A value reached twice through different paths is not a cycle

• Vision on AssertAsync, WaitForAsync, and ExtractAsync is a VisionMode (Off, On, Only) in place of upstream’s false, true, and 'only', and an undefined value throws INVALID_ARGUMENT. On sends a masked screenshot beside the tree, and Only sends the screenshot alone, with a note that no tree is attached on purpose. The screenshot is sized for the model (long side at most 768 pixels) and its note names the regions masked for security. A secret filled in the attempt (by a locator or by the agent), an engine that captures no pixels, or masking the engine cannot prove (fewer masked regions than secure nodes in the tree, or no count at all) takes the image away and the judgment goes on with the tree; Only has no tree to go on with, so it fails with POLICY_DENIED that names PIXEL_TAINTED, MASKING_UNPROVEN, or UNSUPPORTED_CAPABILITY, before any model call. The judge’s system text says an attached screenshot is data, as upstream’s policy does. WaitForAsync takes a screenshot with each look and counts a changed picture as a changed screen. An inconclusive judgment made from the tree alone ends with the hint to pass Vision = VisionMode.On, firm once a judgment of the attempt has received pixels and conditional before (“if the engine captures pixels”), and none when pixels cannot help: a secret was filled, the engine declares no Screenshot capability, or an earlier request for pixels was degraded. The hint’s words name the C# option

• AssertOptions.Screenshot (on by default) keeps the redacted viewport after the verdict, a passing one too, under .e2e/results/<result>/attempt-<n>/screenshots/<n>-assert.png, and AgentException.Screenshot names it on a failed judgment as a path relative to the project root, where upstream names it relative to the artifacts directory. An engine that captures no pixels keeps none. A secret fill does not stop it: the screenshot is the project’s own artifact. Taking it never fails the judgment. The port’s step record has no visionInput, visionDegraded, visionOnly, or pixelBytes on a step, no vision.pixels or assert.screenshot policy events, and no observation revision in a pixels-only request

• executor (AgentOptions.Executor, AgentExecutor on E2ESessionOptions and the fixtures) replaces the built-in agent for act and assert. The socket is IStepExecutor: Name, and Version, Cache, Model, and Judge with defaults, so a hand-rolled executor writes Name and RunStepAsync(context, cancellationToken). The token is upstream’s signal; the context’s Attempt.CancellationToken is upstream’s attempt.signal. StepExecutorContext is a class a test can build and override: Step, Attempt, Target, ReplayedPrefix, Model, ProviderOptions, Ledger, AgentContext, Budgets, Actions, PixelsTainted, ObserveAsync, AttachTranscript, and AttachScreenshotAsync. An executor is code, so agents.<name>.executor in e2e.config.json stays INVALID_CONFIG and names AgentOptions.Executor. As upstream, System and Tools are refused beside an executor, and the Model and Judge an executor brought are the agent’s, for the judgments too. An entry that names another is INVALID_CONFIG (agents.default.model (gpt-5.4) and the executor's own model (gpt-5.4-mini) differ; configure the model in one place). Two models are the same when they are one instance or share a Name, where upstream compares the provider and the id

• The Ledger text an act reads is upstream’s serializeLedger over the attempt’s step records: 1. app.open passed :: / per completed step, then observed: <handoff> when it has one, up to 8192 bytes, newest kept, with [n earlier step(s) omitted] first. A failed step is in it, with its status. Each label and handoff is stripped of ANSI sequences and control characters and cut to 256 and 700 bytes. A step is recorded for agent.act, agent.assert, agent.waitFor, agent.extract, a locator action and locator.waitFor, an expect matcher on a locator, the browser, or the screen, and app.open, app.back, app.restart, and app.clearState. The Browser fixture’s own calls and expect.poll are not steps yet, except that Browser.WaitForURLAsync runs ToHaveURLAsync, so it is recorded as expect.toHaveURL. The records are internal; no report reads them yet. A replayed act hands later steps the text the replay returns (Replayed recorded actions.), where upstream hands them the verdict the recording run wrote, because the port’s cache entry keeps no verdict.

• ToolLoop.CreateToolLoopExecutor and DefaultAgent.CreateBuiltInAgent are public. Differences: ToolLoopExecutorOptions.Tools returns a list of ToolLoopTool (a ModelTool and a delegate that answers a ToolLoopResult), where upstream’s returns an AI SDK ToolSet; BuildPrompt answers a string, where upstream’s can answer a whole message history; the chassis adds done, where upstream adds complete_step; there is no maxTurns to remove. ToolLoopHelpers has GuardAsync, Concluding, and ReportHardStop, and the loop guards every tool call itself. The built-in agent drives the context the runtime builds, so it runs as a step’s executor or inside a custom executor that passes its context on, and not on a context built by hand

• The verbs are the ones the port’s engines have: TapAsync and DoubleTapAsync (both take ClickOptions, whose Modifiers need Target.TapModifiers, as upstream’s tap(target, { modifiers }) needs target.tapModifiers), TypeAsync, TypeSecretAsync, ClearAsync (upstream clears with type), PressAsync, SelectAsync, CheckAsync, ScrollToAsync, ScrollUntilAsync, ScrollAsync, NavigateAsync, and BackAsync. Not ported, because no engine here has them: long press, secondary tap, hover, drag, upload, the point verbs (tapAt, hoverAt, hitTest), typeText, pressKey, and dismissKeyboard. Target.Verbs follows the engine’s capabilities: the scroll verbs need Scroll, Back needs History. A node is named by its ref (new ExecutorTarget("e12"), the [ref=e12] of a line), where upstream names n42. Actions and looks run in call order, mutating project tools included, and a mutating RunToolAsync body that calls an action or a look waits behind itself, as upstream’s does. A verb checks its argument before it takes a slot (INVALID_ARGUMENT), but the port has no key grammar, so PressAsync takes any non-blank key

• ObserveAsync answers Text, Truncated, KeyboardVisible, and Path, and with ExecutorObserveOptions Tree (a list of root ExecutorNodes, redacted, with no value or selection on a secure field, a link target absolute and cut as a line cuts it) and Pixels (the masked screenshot sized for the model, or PixelsWithheld, a VisionDegradation). It has no Revision, no Viewport, and no framePath, since the port has none. ReplayedPrefix carries the actions as the text the hand-off notice lists, ReplayHandOffReason adds the port’s ActionBudget and InvalidEntry, and has no viewport-changed. A replay that meets ACTION_MAY_HAVE_COMMITTED stops as action-uncertain and names the action, which the built-in agent’s notice repeats. KeyboardVisible is upstream’s keyboardVisible (commit 5fce7b2): Observation.KeyboardVisible carries the engine’s measured on-screen keyboard state to ExecutorObservation.KeyboardVisible, and the mobile DeviceSession sets it from the capture’s keyboard fact (visible is true, absent is false, and an unmeasured capture leaves it null). The web engine measures none, so it is always null there. Not ported: the part of that commit in @e2e-dev/decision (dismiss_keyboard and the page fingerprint), and the sdk-types.ts checks, which the C# types make. Tests: FixtureRuntime/ProjectToolDispatchTests and E2E.Mobile.Tests’ Engine/ObservationTests.

• The runtime owns the budgets and the clock. RecordModelCall throws STEP_BUDGET_EXHAUSTED past MaxModelCalls, and a verdict never outranks a hard stop: a passed one over a spent budget, a timeout, or a code-less failure after one ends with the runtime’s code. An executor that ignores its token gets one second to unwind after the step is stopped, and then the step settles without it. The closed verdict grammar is upstream’s: an empty summary, an unknown code, a passed verdict with a code, or a blocked one without a blockable code is MODEL_OUTPUT_INVALID, and so is a runtime code the runtime did not record. A thrown error that is no agent error becomes ACTION_FAILED, as upstream’s toAgentError does, and a cancel by the caller stays an OperationCanceledException. The message of a failed or blocked verdict is upstream’s agent.act failed: <summary> or agent.act blocked: <summary> (agent.assert for an assert). The step keeps the bare summary as its explanation, as upstream does. A hard stop (Step timed out, a spent budget) that ends the step with no verdict, or over a passed or code-less one, has a message with no prefix (the reason, then the verdict’s summary if there is one); a coded failed or blocked verdict gets the prefix, also after a hard stop

• AssertAsync on an agent with an executor goes to it as a step of kind Assert, with the agent’s budgets and JudgmentTimeout. An assert is never cached, and a passing one verifies the acts before it. Vision is refused with UNSUPPORTED_CAPABILITY, since the executor decides what its model sees, and Screenshot keeps its evidence after the verdict, bounded by the action timeout: a capture that outlives it is dropped and the verdict stands. WaitForAsync and ExtractAsync keep the judge, which is the executor’s Judge or Model. ExecutorCache.Off never replays or records, and takes no look before the step. A step with the cache on that cannot read its first screen (no app opened) still runs, and records nothing

• TypeSecretAsync fills a secret the step’s params declared, and refuses with POLICY_DENIED any other name, a disabled or non-editable field, and a field whose purpose does not fit a password secret. A secret the attempt does not know is AUTH_CREDENTIAL_UNAVAILABLE. The built-in agent’s fill_secret goes through the same check, so its refusals read as the dispatcher’s messages and not as Secret fill refused: .... ExecutorStep.Params is JSON: a Secret is its name and, when set, its purpose, and every string and key that holds a secret value the attempt knows is redacted. ExecutorAttempt has the session’s TestTitle as TestId, a fresh id per session, and Index as the attempt number minus one; ExecutorStep.Index counts the agent steps of the attempt, where upstream’s counts every step. AttachTranscript and AttachScreenshotAsync write under the attempt’s results directory (transcript-<n>.txt, screenshots/<n>-<label>.png), and, as upstream, only under E2E_DEBUG=1. AttachTurns keeps the step’s last 12 turns on the step record, redacted, as upstream’s attachTurns does, and the tool loop reports every turn it takes with the same clips. Its loop notes cover the events the port’s loop has (a reply without a tool call, a shrunk history); upstream’s others (a forced tool choice, the wind-down notice, a resent stalled request) have no counterpart. Deliberate difference: upstream clips a turn’s text and then redacts it, so a cut can split a secret and leave a part the redactor no longer matches. The port redacts first and clips after, both in the loop and again in AttachTurns, which also enforces the bounds itself (the last 12 turns, 8 calls of 200 characters, an outcome of 2048) for any executor. A clip never splits a surrogate pair. The loop does not yet attach its debug transcript. The port has no step events

Not ported:

• maxObservationBytes and maxInputTokens. The port sends each observation whole, so it has nothing to cut; the keys are rejected in e2e.config.json

Explore

Explore.ExploreAsync ports upstream’s explore (packages/e2e/src/explore), the library under e2e explore, and e2e explore ports the command (see below). An exploration is the planner (ExtractAsync), the explorer (the built-in agent with the exploration rules and report_finding), and the loop of charters under the exploration’s budgets, with upstream’s outcome policy: a failed step is not a finding, a step at its limit is neither, three unproductive steps in a row end the run, reported issues fail it, and a run that explored nothing, or whose charters were all blocked, stays blocked. The planner’s prompt, the repairs of its answers (padded digit runs, a lone title or charter), and the explorer’s rules are upstream’s text, with done where upstream says complete_step and fill_secret where it says type_secret.

• The port has no runner, so ExploreAsync starts one E2ESession and returns an ExploreOutcome: Status (RunStatus: Passed, Failed, Blocked), ExitCode (0, 1, 2, or 3, from the error that ended the run by upstream’s table of agent codes), Error (the E2EException that ended the run), and Explore (upstream’s run.explore block as ReportExplore). A config, agent, target, goal, or budget error throws before anything starts, as upstream does. A cancelled token throws OperationCanceledException, where upstream’s interrupt is exit code 130.
• A failure while closing the exploration (the engine’s finish hook, or the session’s own release) does not throw and does not replace the outcome: as upstream’s runner records it as a run error of phase cleanup and folds its category into the exit code (precedence 130, 4, 3, 2, 1, 0), ExploreOutcome.ExitCode is the verdict’s combined with the cleanup failure’s (an engine failure is 3), Error is the verdict’s error or else the first cleanup failure, and a failure that is not an e2e error is an ENGINE_FAILURE, as upstream’s translateProvisioningError makes it. The findings stay in Explore.
• Times are TimeSpans: ExploreOptions.Timeout, ExploreBudgets.Timeout, and ReportExploreStep.Duration where upstream has timeoutMs and durationMs. The step bounds are Explore.StepBounds (1 to 12, 8) and the time bounds Explore.TimeoutBounds (3 to 15 minutes, 10).
• Upstream’s ExploreOptions.rawConfig is RawConfig, an E2EConfig. Because e2e.config.json holds a provider and a model id only, Agents gives AgentOptions by name over the config’s, for a model of your own or a project tool, and Engine gives the engine, where upstream’s config file holds both. OnProgress receives the ExploreProgress moments that upstream sends as explore run events through onEvent; FindingReported is upstream’s finding phase.
• The config declares one target, so there is no choice among several and no notice for it; Target only has to name that one (UNKNOWN_TARGET otherwise). The port rejects credentials in the config, so Credentials gives the accounts in code, by name, as UserCredentials (Credentials.User reads one from the environment). Each travels into every step as the username and the secret <name>.password (purpose password), and a secret anywhere in the params of an act, at any depth, is declared and remembered, as upstream’s walk of the params finds it. The size bound of the account inventory, a quarter of the params limit, is upstream’s.
• A target’s engine is chosen from its platform, a .NET-only bridge: upstream’s config holds a live engine handle, and e2e.config.json still rejects an engine key. web is a WebEngine. ios and android are a MobileEngine (MobileTargetEngine.Create, E2E.Mobile) built from app.bundleId or app.appPath and E2E_MOBILE_DEVICE, as for MobileAppTest. windows is a WinAppEngine (WinAppTargetEngine.Create, E2E.WinApp; the CLI passes WindowsTargets.Create as the windows factory of TargetEngines.Create, which is INVALID_CONFIG off Windows) built from app.appPath or app.bundleId, as for WinAppTest. ExploreOptions.Engine still wins. The device engine needs Node 22.12 or newer and a booted simulator or emulator. E2E_MOBILE_PROVIDER=limrun or devflow makes the engine lease a hosted device, as the config of upstream’s CLI does with mobile({ provider: limrun() }) (the port names the provider by variable, like the adapters). Any other value fails with INVALID_CONFIG; E2E.Eas and E2E.TestMu.Mobile are not referenced by E2E.Cli, because upstream’s CLI loads providers from the config and the port’s tool ships only Limrun and DevFlow. Upstream’s openStandaloneAttempt (run/standalone.ts) prepares the engine for one slot of a run of its own and finishes that run when the attempt closes, on every path (engines.prepare and engines.finish, run/provision.ts). The port has the same: Standalone.OpenAttemptAsync is openStandaloneAttempt, over Standalone.OpenWorkerAsync (openWorker) and its StandaloneHost (openAttempt, close), and PreparedEngines is provision.ts’s class of that name. Only upstream’s one-shot kind exists: the hosted kind publishes each session for another test runner, and the port’s tests run in their own. An engine takes part by implementing IPreparableEngine (.NET-only: upstream’s engine handle carries optional prepare and finish members). MobileEngine does; WebEngine does not, because it leases for the attempt it starts (below). e2e explore and e2e mcp open their session through the host, so the lease happens when the session opens, never when the engine is built or when ENGINE_IN_USE refuses a second session, and MobileEngine.StartAsync leases nothing itself. What prepare leaves (the device binding) reaches the worker in EngineStartOptions.Env, under the run’s own variables. The finish runs once, within the cleanup timeout, and a hook that outlives it is abandoned and reported as CLEANUP_TIMEOUT whether or not it honors its token, as upstream does. A finish that fails is an error (close_session shows it as a Cleanup: line, and e2e explore ends with it), because a hosted device bills until it is released. A start that fails or is cancelled after the lease was granted finishes the engine, with a failed finish added to the start’s message; a caller cancel surfaces as CANCELLED and not as a provider failure; a lease the provider grants after the caller stopped waiting is released where it lands; and the end of the process finishes an engine still held (AppDomain.ProcessExit, .NET-only; a killed process is covered by the provider’s own time limit, 15 minutes for Limrun). Progress lines of the provider (session URLs, lease ids) have nowhere to go in a session and are dropped, as in WebEngine’s standalone run; LIMRUN_API_KEY is registered as a secret, so an error that quotes it is redacted. DevFlow’s agent runs inside the app, so IDeviceProvider.AttachesToRunningApp (.NET-only; a default interface member, false) makes the first app.open() attach with no relaunch. Live runs are not possible without Limrun credits or a DevFlow agent: the tests use a fake provider that counts leases and releases. Ported, in part, through the host: OpenHostLifecycleTests.Partial_Host_close_cancels_an_opening_session_and_waits_for_disposal_and_finish_exactly_once (upstream’s open-host-lifecycle it, without the count of worker disposals, because the port has no worker dispose hook) and EngineTargetsTests.Partial_Ends_the_run_before_any_worker_starts_when_prepare_fails and Partial_Abandons_a_finish_hook_that_outlives_the_cleanup_budget_as_a_cleanup_error_instead_of_hanging_the_run (upstream’s engine-target it, without the run’s status, results, and error events, because the port has no run). .NET-only: StandaloneHostTests, PreparableEngineTests, HostedDeviceSessionTests, ExploreHostedDeviceTests, and DeviceTargetsTests. The .NET-only symbols of this change are: IPreparableEngine, StandaloneAttempt (upstream’s type of that name, over the port’s E2ESession), the process-exit finish of StandaloneHost, PrepareScope and FinishScope (upstream’s, in C# casing), E2ESession.LaunchAsync (the start with the worker’s environment), DeviceTargets and HostedDeviceProviders (they stand in for the provider upstream’s config holds as code), IDeviceProvider.AttachesToRunningApp, the MobileTargetEngine.Create overloads that take a provider, SessionHostOptions.Credentials, and FakeCloud (test helper). The E2E_MOBILE_PROVIDER mapping to Limrun and DevFlow is .NET-only, because upstream names a provider in e2e.config.ts and the port’s config file cannot hold code. DevFlow is .NET-only too: upstream has no DevFlow package, so the attach behavior has no upstream counterpart. EngineCapabilities.PinnedApp is the .NET-only stand-in for upstream’s test of the engine handle (a session with restart and no open): MobileEngine declares it. TargetEngines.LaunchesPinnedApp(target, engine) is upstream’s launchesPinnedApp (config/targets.ts, upstream #1252): the target has no app.url and the engine pins an app. The port’s config holds a platform and no engine handle, so it takes the engine too. Explore opens the app first when the target has a URL or LaunchesPinnedApp holds (openApp), so a device target’s pinned app launches before the planner’s first look. To make that call work, App.OpenAsync on a target with no URL on a pinned-app engine launches the app through RestartAsync, as upstream’s launchPinnedApp does in run/fixtures.ts, and refuses a path with APP_URL_REQUIRED; upstream’s UNSUPPORTED_CAPABILITY for an engine without restart has no port, because PinnedApp implies it. Tests: TargetEngineChoiceTests and MobileTargetEngineTests (.NET-only); E2eExploreTests.Launches_a_device_targets_pinned_app_before_the_planners_first_look and SessionHostTests.Launches_a_device_targets_pinned_app_before_the_first_observation port upstream’s two it of #1252. AppSteeringHooksTests ports three it of upstream’s app steering hooks for a device (app.open() launches through restart, app.open(path) is APP_URL_REQUIRED, and the engine’s own reason is kept); the fourth, an engine with no relaunch hook, has no port, because PinnedApp stands for that hook. .NET-only: SessionHostTests.Surfaces_a_cancel_during_the_launch_of_a_pinned_app_as_a_cancel_and_disposes_the_engine. The explore test reads the attempt’s steps from report.json, as upstream reads the report.
• The goal is redacted for the secrets of the config and the credentials’ passwords, which the session also redacts wherever they appear.
• The tool is named report_finding. A project tool of that name is INVALID_CONFIG for every agent, as upstream’s HARNESS_TOOL_NAMES has it (done is reserved with the grammar’s names). The finding is recorded with ArtifactId, the path of the evidence screenshot relative to the project root (.e2e/results/<result>/attempt-1/screenshots/<n>-finding-<k>.png), where upstream names an artifact of its report. The exploration is a test of the virtual file explore (Explore.ExploreFile, upstream’s EXPLORE_FILE): the file joins the goal in the result id and leads <result> (explore-<goal words>-<id>), and every prompt opens with Test: <goal>, as upstream’s does, where an e2e mcp session, which runs no test (no TestFile), gets no such line. The explore executor keeps the model’s screenshot, so none is kept once a secret was filled (the model gets no pixels then) or when the engine captures none. A finding has no observationRevision, since the port has no observation revision.
• e2e explore [goal] (E2E.Cli, ExploreCommand) takes --config, --target, --agent, --max-steps, --timeout (milliseconds), --headed, --output, --debug, --ai-trace, --trace, and --video, with upstream’s bounds, and exits with the outcome’s code: 0, 1, 2, 3, and 130 for an interrupt. A bad flag exits 2 with commander’s message (option '--max-steps <n>' argument '99' is invalid. must be an integer from 1 through 12). Each E2E_USER_<NAME>_USERNAME and _PASSWORD pair in the environment is passed as Credentials (named <name> in lower case), where upstream reads the config’s credentials. A username with no password, or a short one, exits 2.
• The output is upstream’s list reporter without its live window: explore-text.ts is ExploreText (the severity words, why-it-ended text, step counts, and the order of findings), and list-explore.ts is ExploreView in E2E.Cli. The header, each finding and each step stream as they happen, then the Findings section, the Assessment, and the Findings and Steps summary rows. Differences: no RUN header, no live window, a step line counts no actions (the port has no engine events), the screenshot is named by its path under the project root, and the failure of a run that is not the findings’ verdict prints as CODE: message below the record.
• --debug, --ai-trace, --trace [mode], and --video [mode] are upstream’s, with its parser: a bare --trace or --video is on, the optional value is greedy (--trace "Explore checkout" reads the goal as the mode and is refused with write --trace=<mode>, or put test files before --trace), and the old spellings all and retries are refused with the mode they meant. ExploreOptions.Trace, Video, Debug, and AiTrace carry them; a Trace or Video is a Run source over the target’s and the config’s, else the target’s mode. E2E_TRACE, E2E_VIDEO, E2E_DEBUG, and E2E_AI_TRACE set them without the flag. An exploration is one attempt, so a retry mode keeps nothing, and ExploreOptions.Notice says to pass --trace on or --video on (upstream’s recordingNotices for explore, as Explore.RecordingNotices, internal), as it says that a --video the engine cannot record is skipped. An exploration’s verdict (ASSERTION_FAILED with an issue finding) gets no trace page (isExploreVerdict, as TracePages.IsExploreVerdict), and ReportRun.Explore carries the record for that. e2e mcp takes none of the four, as upstream.
• Not ported: --session and --reporter (the command refuses them with exit 2, since running without a session or a report would change what the run means), --reporter json (the port has no reporters; .e2e/report.json has the record under run.explore, see Report document), a start from a restored session, and the --artifacts refusal (the port never had the flag).
• Tests: E2eExploreTests ports the it of explore.test.ts that says to pass --video on when a retry mode would record nothing (partial: the notice comes through ExploreOptions.Notice, and a scripted engine that declares the video capability stands in for the web engine). The --trace and --video parser tests of cli.test.ts are about e2e run, which the port lacks; ExploreFlagsTests (.NET-only) covers the same parser on e2e explore, and ExploreRecordingTests (.NET-only) covers the files.

Config

E2ETest loads the nearest e2e.config.json above the test assembly directory, then above the working directory, once per run. Supported keys: targets (one entry with name, platform, and app with url, command, and readyUrl), timeout, launchTimeout, actionTimeout, assertionTimeout, cleanupTimeout, retries, agents, cache (mode, dir, strict), and secrets. Defaults, the CI cache demotion to read-only, E2E_SECRET_<NAME> overrides, the 6 code point secret minimum, and INVALID_CONFIG for unknown keys follow upstream.

app.url is normalized as upstream does, with System.Uri instead of WHATWG: a missing scheme becomes https://, or http:// for a loopback host. Userinfo, a query, a fragment, a scheme other than http(s), and plain HTTP on a non-loopback host are INVALID_APP_URL, with upstream’s messages. A base URL set in code (E2EFixture.BaseUrl, E2ESessionOptions.BaseUrl) follows the same rule. Port 0 on 127.0.0.1 or [::1] asks for a free port and needs an app.command to start the app on it (INVALID_CONFIG without one, or with reuseExisting); any other host with port 0 is INVALID_APP_URL. A base URL set in code with port 0 is INVALID_CONFIG, since only app.command starts an app on a free port. Keys of the old .NET shape (app, engine, agent, timeouts) fail with a hint to the upstream key.

app.command and app.readyUrl port upstream’s config/command.ts and config/app.ts (CommandConfig, Command.Normalize, TargetApps.ResolveTargetApp), and ManagedProcess ports run/managed-process.ts (readiness polling, the preflight probe and reuseExisting, the startup-failure tail of command.log, inheritEnv). executable is started with an argument list, never a shell. {port} expands in args, in env values, and in readyUrl, and a url on port 0 takes a free loopback port (AppPorts.AllocateTargetPorts, upstream’s allocateTargetPorts). The first test, e2e explore, or e2e mcp session that needs the command starts it (E2EFixture.StartSessionAsync, Explore, and the MCP session call RunApps.StartAsync) and it runs until the process ends, where ManagedProcess.KillManagedProcessGroups takes it down on the way out; on Windows the process runs in its own job object (ProcessJob, .NET-only), which ends it and what it started when it is stopped, and when this process is killed outright, as a test host is once its run ends. On Unix a descendant is read with pgrep -P when the app is stopped, so one that outlives a leader that exited earlier is not found. The app command differs from upstream’s in these ways:

• There is no runner, so a command starts once per test process (a .NET-only holder, RunApps), not once per run. Two processes that declare one command, such as dotnet test and e2e mcp, each start their own, so give a shared app a fixed port and reuseExisting. Upstream’s SharedAppProcesses pool, declaredCommands, and the engine prepare hooks that startDeclaredProcesses shares a provision with are not ported (one target, no pool).
• A started process is stopped with SIGTERM and then Process.Kill(entireProcessTree: true) after shutdownTimeout. On Windows, which has no signal to ask a process to stop, the first step is the kill. There are no process groups, so a descendant that outlives its leader is not signalled; the two managed-process integration tests about that (stops a surviving child after the command leader exits on SIGTERM, lets a child finish its shutdown after the leader exits) are not ported. stdout and stderr are read together into log (or dropped when there is none), line by line as the process writes them.
• A cancelled token ends ManagedProcess.StartAsync by throwing OperationCanceledException after taking the process down, where upstream’s start returns and the caller reads the signal. startupTimeout and shutdownTimeout are TimeSpans (CommandConfig.StartupTimeout, ShutdownTimeout), and a value over 2147483647 milliseconds, which a .NET timer cannot wait, is INVALID_CONFIG. A readiness probe still waiting when the process exits ends there.
• JSON has no secrets.get() handle, so the message that names one for a string value of args, env, or url is not reachable, and an env entry of null stands for upstream’s undefined. command.log containment resolves symlinks as relativeToProjectRoot does (Paths.RelativeToProjectRoot). An app key outside the port’s set is named in the port’s own words (unknown targets[0].app key "readyURL"), where upstream says target "web" app has unknown key; the command’s own keys use upstream’s message.
• The resolved app has Base, PortRequest, Command, and ReadyUrl; site, environment, and identity are not ported, and neither is the config digest (digestCommand, digestTargetApp), so the config.test.ts tests that digest an app, and the identity and environment tests, are not ported.
• Tests: ManagedProcess/ ports the it of tests/unit/managed-process.test.ts and ManagedProcessIntegration/ those of tests/integration/managed-process.test.ts, with the program they start (E2E.Tests.App) where upstream runs node -e; the notice tests wait out a real budget where upstream moves a fake clock. AppPorts/ ports app-ports.test.ts (partial: one target, and no identity or digest). Config/ ports the command and readyUrl tests of config.test.ts. RunApps/ is .NET-only. Not ported: target-processes.test.ts and shared-app-processes.test.ts, which drive the runner.

Differences:

• JSON has no engine handles or model instances. The fixture’s CreateEngine chooses the engine, platform must be web, and targets holds one entry. agents.<name>.model and judge are model ids for the .NET-only provider (see Models), with the .NET-only baseUrl and apiKeyEnv beside them. A subscription provider refuses baseUrl and apiKeyEnv. E2ETest passes agents.default as the session’s own agent settings and every other entry through CreateAgents
• A secret set to null reads only E2E_SECRET_<NAME>. Provider functions do not exist in JSON. A test reads secrets with Secrets.Get(name)
• retries is validated and resolved (1 in CI, 0 elsewhere) but NUnit retries still come from [Retry]
• cache.strict treats any replay that finds a recording but does not finish it as stale. Retries replay too, and a strict run never writes or deletes a recording, whatever cache.mode says
• A launch or cleanup timeout fails with ENVIRONMENT_UNAVAILABLE
• trace and video are accepted at the config root and on the target, as upstream. The modes are off, on, retain-on-failure, on-first-retry, and on-all-retries (RecordingMode, RecordingSource, ResolvedRecording). The mode is resolved as upstream’s resolveConfig does it: the flag, else the target’s key, else the config root’s, else the default (retain-on-failure for trace, off for video), and a mistake in a target’s key is reported even when the flag wins. The old trace: 'all' and trace: 'retries' spellings, bare or as { record }, are refused with the mode they meant. The port records video on the web engine (see Video) and writes the trace page (see Trace page): the settings are read and carried on TargetConfig.Trace and Video, the fixture’s Trace and Video properties, and E2ESessionOptions. The port has no --trace and --video flags, so the .NET-only variables E2E_TRACE and E2E_VIDEO stand in for them, with source Run. RecordingSource.Test is the source of a setting made for one test; the port has no per-test option yet. The per-attempt rules (attemptKeep, attemptVideo, keeps, recordsOnSomeAttempt) are in RecordingModes, internal. The port has no config digest, so upstream’s “keeps the mode, at the top and on a target, out of the digest” has no port; the replay cache key does not read these settings. config.test.ts tests are in Config/: TraceTests and VideoTests (the three of the four it that do not need a digest; the flag is the variable, so “lets a target override the config, and the flag override both” and “refuses a mode it does not know” are partial), ArtifactsConfigTests (only the old trace spellings; the port has no artifacts key), and “uses CI defaults for retries and workers…” in ResolveConfigTests (partial, no workers). recording-modes.test.ts is RecordingModes/
• Not supported, and rejected with INVALID_CONFIG: projectId, tests, failOnSkippedFailure, workers, artifacts, output, reporters, credentials, cache.store, the non-URL app keys, and the agent keys tools (set AgentOptions.Tools in code), executor (set AgentOptions.Executor in code), maxObservationBytes, and maxInputTokens. Credentials still come from E2E_USER_<NAME>_USERNAME and _PASSWORD

Web engine

WebEngine launches Chromium through Playwright and builds a semantic tree in the page: explicit roles, the upstream implicit role table (landmarks, lists, tables with rows and cells, dialogs, options, img alt="" as presentation unless it has a title, a tabindex, or a global ARIA attribute, and role="presentation" or role="none" ignored on a focusable element or one with a global ARIA attribute, select multiple as listbox, and the rest), accessible name, text, test id, heading level, and the disabled, checked, expanded, selected, pressed, focused, and hidden states. As upstream, the name reads aria-labelledby before aria-label and associated labels. Disabled covers :disabled (including a disabled fieldset), aria-disabled="true" on the element, and aria-disabled inherited from an ancestor for the roles it applies to. Checked is the native state for checkbox and radio inputs and aria-checked="true" elsewhere; mixed reads as not checked. Hidden is what renders, as upstream: display: none, a visibility other than visible, content under content-visibility: hidden, and content a closed details folds away. A display: contents element is shown when a child element is, or when it is visibility: visible and its own text has a box. One exception, described below: a box with no size does not hide a node. aria-hidden is a separate state, not part of hidden. A child that sets visibility: visible under a hidden parent is visible; nothing under display: none is. An explicit role is the first recognized token of the role attribute; with none recognized, the node keeps its native role. role="img" is reported as image, and an open details is expanded. A closed select lists up to 60 options under it: the ones it offers, so an option that is hidden, display: none, aria-hidden, or inert, or sits under such an optgroup, is left out before the limit counts, as upstream’s isOfferedOption does (#900). A visibility: hidden option stays listed, since a closed select paints none of its options anyway, and a disabled one keeps its state. Each observed element keeps a stable ref for the life of the document, and the engine runs an action on the element that ref names, with no fallback to a text or role lookup in the page. The agent adds the fallback upstream’s agent has: a tap, double_tap, fill, fill_secret, press, select, check, uncheck, or clear whose node the engine reports NODE_STALE finds the node again by its role, name, and test id in a fresh look and retries, up to twice, scroll and swipe do not retry, and a ref the newest look no longer lists (not one it lists as hidden) is found the same way from the last node it named. A description that matches no node or several fails the verb with LOCATOR_NOT_FOUND. Upstream also matches a node’s placeholder, input purpose, and container, and forgives a test id that changed; the port matches the three fields it records, and never finds again a node with no name and no test id. Because the port records no container for a node, upstream’s containerKey fix for windowed lists (841ea3fa, a row is not keyed by its list’s first rendered row) does not apply. Upstream wraps the retry in commitTargeted and the step’s accounting; the port has no such wrapper, so each verb runs the loop (TargetedAsync) directly, and ObservationFeed.Relocate takes the redactor as an argument. Firefox and WebKit launch options are not exposed yet; the package reference can drive them later. .NET-only: the first launch in a process runs Playwright’s Chromium install, and E2E_SKIP_BROWSER_INSTALL=1 turns it off. .NET-only: dotnet test has no flag like upstream’s --headed, so when WebEngineOptions.Headless is unset, E2E_HEADLESS=0 or false shows the window. As upstream, a run is headless by default. As upstream, a headless run installs with --only-shell and a headed run installs the full build, and the install runs with PLAYWRIGHT_SKIP_BROWSER_GC=1 unless the user set it. Upstream checks the cache before it spawns the install; the port leaves that to Playwright’s install, which does nothing when the build is there. Playwright’s .NET Program.Main starts the driver with the process environment, so the port sets PLAYWRIGHT_SKIP_BROWSER_GC on the process, not on the install alone.

The walk goes through open shadow roots and closed ones that page script attached (an init script records them, as upstream does). Each iframe is a boundary node whatever its role, so an <iframe role="none"> stays listed with the role none, as upstream’s walk emits it before any role check, and the engine reads that frame’s document through Playwright and puts it under the node, so same-origin and cross-origin frames are both listed and acted on. One observation lists at most 3000 nodes across all frames. When a document has more, nodes that intersect the viewport come first and the rest of the budget goes to the others in document order; the observation reports Truncated, and the snapshot tells the model to scroll. A locator resolves against that tree, so matches come in tree order: a match inside a closed root sits where its host is, where upstream lists the light-DOM matches first and the closed root’s after them. OpenAsync waits for the load event.

WebEngineOptions carries the upstream web() options: Viewport (default 1280 by 720, null for no emulation), TestIdAttribute, Headers, BasicAuth, UserAgent, Locale, TimezoneId, ColorScheme, ReducedMotion, InitScripts, Connect (a CDP endpoint resolver, with an optional ReconnectEndpoint), and Browser (an IBrowserProvider, upstream’s browser: BrowserProvider). Differences:

• Headers ride requests to the base URL’s host. Upstream scopes them to the app’s site, which can include sibling subdomains
• Browser ports upstream’s BrowserProvider seam (provider.ts): IBrowserProvider (Name, Scope, AcquireAsync, ReleaseAsync, SweepAsync), BrowserProviderScope (Worker, Attempt), BrowserRequest, BrowserLease, and BrowserReleaseContext, and the engine side, LeasedBrowsers. BrowserLease.Protocol (BrowserProtocol.Cdp, the default, or Playwright) is .NET-only: a Playwright lease is dialed with Chromium.ConnectAsync, the engine opens its own context with the options it gives a launched one, and it needs Worker scope and no ReconnectEndpoint because that connection cannot be reattached (a dropped one is replaced by a new lease). A provider is an interface, so upstream’s asBrowserProvider checks that a method exists have no port; the engine still refuses a blank name, a scope outside the enum, and Connect beside Browser (INVALID_CONFIG). Scope and SweepAsync are default interface members: a provider that leaves them out leases per worker and sweeps nothing. Neither a request nor a context carries upstream’s signal: AcquireAsync, ReleaseAsync, and SweepAsync take a CancellationToken. Downloads (BrowserProviderDownloads, with BrowserDownloadContext) and RecordAsync (ProviderRecordContext, ProviderRecording, ProviderRecordingStopContext, ProviderRecordingResult) are default interface members too: a provider with no downloads returns null, and one that cannot record a browser returns null from RecordAsync, where upstream leaves the method out. LeasedBrowsers.Downloads() and Recorder() are upstream’s downloads() and recorder(), and ProviderRecordings.StopProviderRecordingAsync is stopProviderRecording (recording.ts, with VideoSegment as VideoFile or VideoLink). ProviderRecording.StartedAt is a DateTimeOffset, so upstream’s isProviderRecording has no port. The engine calls RecordAsync through Recorder when an attempt records video (see Video). Browser.WaitForDownloadAsync calls Downloads (upstream’s waitForDownload, with saveDownloadsTo, saveFromBrowser, and saveLocally of downloads.ts as Downloads.SaveDownloadsToAsync, SaveFromBrowserAsync, and SaveLocallyAsync). A provider’s directory is checked at config load too
• A runner calls WebEngine.PrepareAsync, Init, FinishAsync, and DisposeAsync as upstream’s prepare, init, finish, and dispose, with EnginePrepareInfo, EnginePrepareResult, EngineInitInfo, and EngineFinishInfo; an attempt is StartAsync and the dispose of its session (startAttempt and endAttempt). The hand-off to a worker travels in the same E2E_WEB_BROWSERS_<TARGET>_<DIGEST> environment variable, capped at 16384 bytes. The port has no runner, so an engine whose Init never ran leases for the one attempt it starts, as upstream’s standalone run does: it prepares one slot of a run of its own, reading EngineStartOptions.Env (the process environment by default), and gives the lease back, and sweeps, when the session is disposed, with the default cleanup timeout for each step (the engine does not see the session’s). EngineStartOptions.AttemptId names the attempt to a per-attempt lease
• Differences from upstream’s lease flow, all from the port’s engine. It keeps no browser connection between attempts, so a worker dials the endpoint of the lease it holds at every attempt, and leases a replacement only after an attempt attached to a browser that was no longer connected when the attempt ended (upstream: when the next attempt finds the connection dropped). A start that fails gives its lease back itself, since there is no endAttempt to call after it. A lease that arrives too late, after the attempt was cancelled or the worker disposed, is given back under a cleanup budget of its own (upstream passes the signal that already aborted). Upstream’s state capability, which an attempt-scoped provider drops, does not exist in the port, so only app.clearState() is refused (UNSUPPORTED_CAPABILITY, naming the scope)
• The lease tests are BrowserProvider/ in E2E.Tests, from browser-provider.test.ts. Not ported: “rejects a sweep that is not a function” and “rejects a record that is not a function” (both are methods of the interface), “releases the replacement on dispose even when ending the attempt failed” (the port’s DisposeAsync does not end an attempt), and “refuses, at the start and named, a recording without a start time and a stop” (a recording is typed). The other tests of the recording group are ported in full, on a real Chromium the provider leases (upstream drives a scripted page), through StartVideoAsync and StopVideoAsync of the engine session. Also not ported: “knows a recording by a parseable start time and a stop” of provider-recording.test.ts. The rest of that file is ProviderRecording/, and provider-downloads.test.ts is ProviderDownloads/, on a real Chrome over CDP, with the Browser fixture over the engine session
• Browser.OnDialogAsync and Browser.WaitForDownloadAsync are upstream’s onDialog and waitForDownload. DialogHandler is upstream’s DialogHandler (DialogHandler.Accept and Dismiss are its 'accept' and 'dismiss', and a Func<Dialog, Task> converts to it), Dialog.Type is a DialogType (Alert, Confirm, Prompt, BeforeUnload) where upstream’s is a string union, and OnDialogAsync returns Func<Task> for upstream’s unsubscribe. The engine side is DialogRouter (dialogs.ts) and ErrorLatch and WebSupport.SanitizeFilename (support.ts), and WebSession.ArtifactPath is the surface’s artifactPath. Differences: the engine session has OnDialogAsync and WaitForDownloadAsync (default members of IBrowserSession), and the Browser fixture attaches the file to the step (attachArtifact), since the port’s fixture holds no context; BrowserDownload is the result upstream types inline. A dialog handler’s failure is latched and fails the next step, as a route handler’s does, but the port has no settleAttempt, so one that no step follows is dropped (upstream fails the attempt with it). A test framework’s assertion is not an E2EException, so one thrown in a handler fails the next step as ENGINE_FAILURE, dialog handler failed: ..., where upstream’s expect failure keeps its ASSERTION_FAILED. A download is a download artifact of the browser.waitForDownload step, with the media type of its extension. The port has no artifact store, so upstream’s rewrite of a text-like download through the secret ledger has no port: the record says incomplete, as upstream’s does for a session without secrecy. Not ported: “a dialog handler assertion that no step follows” and “fails a %s handler assertion at the attempt end when no step follows it” of web-platform.test.ts (no settleAttempt), and nothing else. The frame tests of dialogs-frames.e2e.ts are DialogsFrames/FramesTests, and the download tests trigger the link with browser.locator('a[download]') as upstream does
• With ReconnectEndpoint, as upstream, each attempt rides the default context of a fresh browser from CdpEndpoint, refuses a browser an earlier attempt of the same WebEngine rode, and reconnects once to the same browser and page after a transport drop. Headers, BasicAuth, UserAgent, Locale, and TimezoneId are INVALID_CONFIG, and app.clearState() is UNSUPPORTED_CAPABILITY. After a reconnect, PerformAsync, the agent’s key press, and the viewport swipe fail with NODE_STALE until the next observation; the browser fixture’s keyboard and mouse are direct input and keep working. As upstream, the reconnect is spent from the budget of the operation that found the transport dropped, both for the CDP dial and for the operation itself
• BasicAuth takes a Secret password. As upstream, the attempt redacts the password and the Authorization credential the browser sends with it, padded and not, as text only: it does not withhold screenshot pixels, and the agent cannot fill it by name. The port has no engine secrets to declare, so the engine resolves the password through the attempt when it starts. A plain string password is not redacted
• An InitScripts entry is a WebInitScript: source (a string converts to one), WebInitScript.FromPath, or WebInitScript.FromFunction. A C# lambda cannot run in the page, so a function is its JavaScript source. A relative path resolves against the project root (EngineStartOptions.ProjectRoot, which E2ESession passes from E2ESessionOptions.ProjectRoot). Each attempt reads the files when its session starts; upstream reads them once per process
• ColorScheme and ReducedMotion are the enums WebColorScheme and WebReducedMotion, and Browser.EmulateMediaAsync takes a WebMedia; upstream’s are string unions. A value outside an enum (a cast number) is INVALID_CONFIG in the options and INVALID_ARGUMENT in EmulateMediaAsync; a null media throws ArgumentNullException. A feature or value outside the type cannot be written, so upstream’s unknown-key and wrong-type refusals have no port. Playwright sets the media per page, so a tab the app opens itself gets a test’s change as it appears, on a persistent context too, so a script it runs at load may still read the configured values, as upstream. After a reconnect the new transport sets the media again on the app’s own tabs

The tree is a subset of upstream’s: names follow upstream’s accname rules, and nodes are listed by role, test id, or direct text (not by name alone, or as an empty painted box). So getByText answers with the innermost element that reads the text, such as a <strong> inside a paragraph, as Playwright does. Text inside a <label> of a control, or in an element an aria-labelledby names, is not listed: the labelled element carries it as its name, so getByText answers with that element (Playwright answers with the label and retargets a fill to the control). A node listed for its text alone does not count against the 3000-node budget: such nodes have an allowance of their own, so a page full of text cannot push controls out. In the agent snapshot, a node that lists its children shows its own text, read as a person reads its line: the text of an inline link, emphasis, code, or span in the sentence stays in place, so <p>Read our <a>privacy policy</a> for details.</p> reads whole while the link is also listed as a control. Inline text-only elements are not listed again, unless the line is longer than the 512-character text bound, or the element has a test id, a role, or can be acted on (tabindex, onclick, or an <a> with no href). getByText still answers with such an inline element: the page script sends it as a bare text node under the line, which the agent does not read. This differs from upstream, whose text engine finds the element in the page. A leaf such as a button or a heading shows its whole text. Locator reads (TextContentAsync, text matchers) still read the whole rendered text, as upstream’s node read does. A contenteditable editing host (the root of a rich-text editor, never a canvas, a frame, or a native control) is a textbox named by aria-label, aria-placeholder, or the data-placeholder its editor paints, with its rendered text as the value, as upstream. Its children stay in the tree as content, so getByText finds them. A <p> inside an editor has no role. A document in designMode is not an editing host, though its <html> is editable; upstream has no such check. Playwright’s role selector knows a bare host only by an explicit role="textbox", while the port’s GetByRole reads the tree and finds it. Upstream’s getByText runs Playwright’s text engine; the port matches the tree, so text split across child elements with no text of its own between them (<div><b>a</b><i>b</i></div>) is not one match.

The snapshot leaves out a checkbox’s or radio’s value, which inputValue still reads (on by default).

The page script reads the text selected inside the focused input, textarea, or contenteditable host into SemanticNode.Selection, as upstream does. A caret, an unfocused field, and a secure field have none. The agent snapshot lists it as selection="..." after the field’s value, verbatim and escaped (a lone space counts, an empty selection does not), as upstream does. The runner drops it from a field whose value or text holds a registered secret, and from a field cut at its length limit when the selection is not in the part shown. Differences: the port’s press has no times, so one call presses the key once; and the snapshot shows a value as it is, a newline included, where upstream escapes it.

Upstream leaves an aria-hidden subtree out of its tree walk, and its text and visibility reads go through Playwright. Here every locator read comes from the tree, so the port lists the subtree and marks each node AriaHidden (.NET-only), and role queries and the agent skip those nodes. They are not hidden: a painted aria-hidden node is visible to isVisible, toBeVisible, and toBeHidden, as upstream, and a visible query keeps it. An action on one is not refused as hidden. A box with no size is not hidden here, since role queries skip hidden nodes and an empty landmark has no size. A hidden node reads its DOM text (textContent) instead of innerText, so a text query still finds a visibility: hidden copy.

Password fields and autocomplete=current-password fields are marked secure. Their values, and their value attribute, are omitted from the snapshot. A text, value, or attribute expectation on a secure field fails with POLICY_DENIED, as upstream. So do the textContent, inputValue, allTextContents, and getAttribute reads (for any attribute).

Selected is aria-selected="true" or a selected <option>, expanded is aria-expanded="true" or an open details, and focused is the active element, followed into shadow roots. Attributes (for getAttribute and toHaveAttribute) and the client rect (for boundingBox, null for a node with no layout box: display: none on it or an ancestor, or any display: contents element) are read for every kept node; they are not sent to the model. The tree reduces a link’s href attribute as upstream’s tree does: origin and whole path, without userinfo, with ?… or #… for a dropped query or fragment, mailto: and tel: kept, blob: reduced to its inner origin and path, and data:… or javascript:… for an inline payload. Upstream’s node read keeps the whole value; the port reads every locator attribute from the tree, so it keeps the whole href on the side and getAttribute and toHaveAttribute read that. The agent snapshot lists a node’s test id, its href, and its placeholder when it has no name, as upstream’s line does: a link target on the app’s origin shows as its path, any other as the whole URL, and a target over 256 characters is redacted, then cut there and ends with …. The executor tree carries treeHref’s result: a link target on the app’s origin keeps the origin and loses no more of its path than the line does. The snapshot is not part of the cache key or of a recording, so showing them does not change the replay policy version. The snapshot joins its lines with \n on every OS, as upstream does (the port used Environment.NewLine, so on Windows the lines ended \r\n); the cache key and recordings do not embed the snapshot text, so no recording is invalidated. A rect in an iframe is in the top-level viewport, shifted past the frame’s border and padding and clipped to its content box.

A display value query never matches a checkbox or radio, which carries a value it never shows. Text, label, placeholder, test id, and display value queries keep hidden matches, so toBeVisible fails on a hidden match and toBeHidden passes on no match or one hidden match. Role queries skip hidden nodes, and toBeAttached and waitFor attached or detached also match hidden nodes for role queries. A single-node matcher that still sees more than one match at its deadline fails with STRICT_MODE.

Playwright failures never escape as PlaywrightException. WebEngine throws EngineException with an upstream engine code from EngineErrorCodes: NOT_ACTIONABLE when an action timed out before its input was dispatched or the target does not take that input, ACTION_MAY_HAVE_COMMITTED when it timed out after dispatch, NODE_STALE when the element or its document is gone, OPERATION_TIMEOUT for a navigation or key press timeout, and ENGINE_FAILURE otherwise. EngineException.Retryable is true only for NODE_STALE and FRAME_NOT_FOUND; any other retryable claim becomes ENGINE_FAILURE, as upstream. As upstream (commit c23944c), a Playwright timeout stays a timeout: StaleOr and NavigationStaleOr never turn a TimeoutException into NODE_STALE, even when its call log says an element detached or the page navigated, so a read or a navigation that times out is OPERATION_TIMEOUT and is not retried. The action path is unchanged by that commit. Tests: Errors/StaleOrTests and Errors/NavigationStaleOrTests. A sensitive fill’s value is redacted from the message. As upstream, check and uncheck click once and read the state back on the same element until just before the action timeout, so a control that commits its state late passes. A control gone by the first read (replaced, or navigated away) took the click, so the action passes. Each of these is NOT_ACTIONABLE: a control that goes later in its old state, a click that leaves the state unchanged at the end of the wait, and an uncheck on a checked radio. The port ends the poll min(250 ms, timeout/4) before the action deadline, which upstream does not: upstream polls to the full timeout, but here the deadline that abandons the call would win the race and report ACTION_MAY_HAVE_COMMITTED. The document engine still uses NOT_FOUND for a ref missing from its last observation.

A page stuck in a script fails an operation at the action timeout, not the test timeout. As upstream, each page operation (a navigation, back, forward, reload, observe, an action, a scroll, browser.evaluate, browser.title, browser.setViewport, browser.emulateMedia, a key press, typing, and browser.mouse) has one deadline, the action timeout, that every page call in it shares. Playwright’s own timeout is set 250 ms short of what is left, or half of it when less than 500 ms is left, so Playwright’s reason (what blocked an action, or that its input was dispatched) arrives first. A call Playwright never answers, such as an evaluate or input on a page stuck in a script, is abandoned at the deadline. A read cut off that way fails with OPERATION_TIMEOUT. An action or input cut off that way is ACTION_MAY_HAVE_COMMITTED, because it may have reached the page.

A scroll moves three quarters of the viewport or the scrolled element with scrollBy. Upstream sends a wheel gesture. back is the browser history.

Video

video records an attempt on the web engine. The modes, their precedence, and E2E_VIDEO are under Config. This is the rest of upstream’s video.ts, provider-video.ts, attempt-session.ts (startVideo, collectVideo), and the video half of execute.ts.

• IEngineSession.LocateAsync is upstream’s locate (EngineSession.locate), with the flat LocatorExpression record and the budget left as a timeout argument. Core resolves a locator through it when the session says CanLocate (.NET-only, transitional). WebEngine and DocumentEngine implement it: each reads its semantic tree and resolves the expression over it with the core resolver, so a match is a node of that observation with the ref the page script minted. Upstream’s web locate (surface.ts) projects the expression onto Playwright selectors (locators.ts), which the port’s page script replaces; the port resolves over the tree it already reads, so only the selector and frame expression kinds ask Playwright (see the Browser.Locator bullet below). The web engine reads under the timeout it is given (what is left of the wait, at least 500 ms), or the action timeout when that is shorter; the document engine answers at once and does not use it. LocatorExpression.IncludeHidden is .NET-only: toBeAttached and waitFor set it so an engine that resolves over its tree matches hidden role nodes, which they matched before the engine located; upstream’s Playwright has no such switch. A locate hands the trace the observation it read (TraceFeed.ShowScreen, and EngineStartOptions.Screen on the document engine), as core’s Agent.ObserveAsync did for the resolve it replaces; upstream’s web locate shows none. A retryable EngineException of the read (NODE_STALE during a navigation) is now retried until the wait’s deadline, as resolveOnce does. Any session that says no CanLocate (the WinApp engine, a scripted engine) has its locator resolved over ObserveAsync in core, as before. Screen.GetByTestId on such an engine, and the LocatorResolver readers of screenshot masks and e2e mcp, still resolve over the observation.
• Browser.Locator(selector) and Browser.FrameLocator(selector) are upstream’s browser.locator and browser.frameLocator (browser.ts). FrameScreen is upstream’s FrameScreen: a Screen (so Screen is no longer sealed, though only the port can derive from it) whose every query goes through a frame expression, with Locator(selector) and FrameLocator(selector) for a CSS or XPath selector and a nested frame. A locator of a frame screen can Or, Filter, and scope as any locator, since Screen.OwnLocator compares the attempt’s root screen (upstream compares the context). LocatorExpression has the selector kind (the device’s, now the web engine’s too) and the frame kind (Selector is the iframe’s selector, Source the scoped expression). WebEngine.LocateAsync resolves them in WebSession.ProjectAsync, over the tree: Playwright says which elements a selector, or an iframe, names (WebLocators is upstream’s locators.ts, with FrameSelectors), the elements’ refs go to the resolver as a refs expression (LocatorExpression.Refs, .NET-only, internal), and a frame scopes its queries to the iframe node’s descendants, where the walk hangs the frame’s document. Differences: an element the tree leaves out (no role, test id, or text of its own) matches nothing, where upstream’s Playwright locator matches any element; an iframe the page does not show has no node, so it is FRAME_NOT_FOUND (retryable) and its content is not reachable; a read that does not wait (Count, AllAsync, AllTextContentsAsync, IsVisible, IsHidden, toBeAttached, toBeVisible, toBeHidden, toHaveCount, a text list, and WaitFor for Detached or Hidden) answers no match for a missing frame (MissingFrame.Empty, upstream’s missingFrame: 'empty'), and any other resolve polls for it until its timeout and fails with NOT_FOUND (FRAME_NOT_FOUND) or STRICT_MODE (FRAME_AMBIGUOUS), as upstream’s translateLocatorError maps them to LOCATOR_NOT_FOUND and LOCATOR_AMBIGUOUS. A Filter(has) resolves inside each match, so a frame its locator shares with the source is the source’s; a has from another frame fails with UNSUPPORTED_CAPABILITY. A frame selector must match an iframe or frame element, else the frame is missing. A selector that captures with * before its last part fails with INVALID_LOCATOR when the locator resolves; upstream finds out when its read selector returns nothing for the captured element, the port reads the spelling (WebLocators.RefuseCapture, which splits on >> outside quotes), and its message says Filter(Locator) where upstream says filter({ has }).
• BrowserExpect.ToHaveClassAsync(locator, expected) is upstream’s expect(browser).toHaveClass, on Expect.That(browser) and Expect.Soft(browser). A string is the whole class list, whitespace-normalized, and a Regex is tested against it. A locator with no match yet keeps it polling (observed: no node), a node with no class attribute fails it (observed: no class attribute; class="" is an empty list), and several matches fail it at once with STRICT_MODE (upstream’s LOCATOR_AMBIGUOUS). The step’s label is page, as the other expect(browser) matchers’ here, where upstream’s is the expected pattern.
• IEngineSession.StartVideoAsync and StopVideoAsync are upstream’s artifacts.startVideo and artifacts.stopVideo, and EngineCapabilities.Video is its video artifact capability. EngineStartOptions.ArtifactsDir is EngineAttemptContext.artifactsDir: the attempt’s .e2e/results/<result>/attempt-<n>. WebEngine starts the video once the attempt’s page is open, and WebSession (upstream’s AttemptSession) tells it when pages open and close.
• VideoRecorder and ProviderVideo implement IAttemptVideo (internal), upstream’s AttemptVideo. VideoRecorder writes video/video.webm and, for later pages, video/video-part<n>.webm, each with its StartedAt, from Playwright’s IPage.Screencast.StartAsync. ProviderVideo returns the one segment the provider made: a file it wrote under video/ (Kernel’s replay.mp4) or a VideoLink. The first-frame wait is the same FirstFrameWait (1 s). WebScreencastOptions (WebEngineOptions.Screencast) is upstream’s WebScreencastOptions; Size is a WebViewport and a size under 1 pixel or a Quality outside 0 to 100 is INVALID_CONFIG. The old web({ video }) key has no port, since the options are typed.
• A browser provider that records is asked through LeasedBrowsers.Recorder. Upstream leaves record out of a provider that cannot record, and the engine then records the screencast. A .NET provider is an interface, so it returns null from RecordAsync and ProviderVideo.IsRecording is false: the session then arms a VideoRecorder instead. A page the app closes under a ProviderVideo needs nothing, as upstream.
• ExecuteVideo (internal) is upstream’s execute.ts video code: Plan is validateEngine for the one attempt a session is (UNSUPPORTED_ARTIFACT when a mode a target or a test set cannot be recorded by the engine; a config-root or E2E_VIDEO mode is skipped), StartAsync is the video part of launchSession (a default mode fails quietly, a set one fails the start), and StopAsync is stopVideo, closeSession, and registerVideos (a video the mode does not keep for the verdict is deleted, a link that is not http(s) fails the cleanup after the other recordings are kept). E2ESession starts the video after the engine, finalizes it before the engine is released when the session is disposed, and reads the verdict from the error Complete is given: no error is a pass, a SkipException is a skip. A required video that cannot be finalized makes DisposeAsync throw. The kept segments are E2ESession.Videos (internal); the port has no report to register them in.
• Differences: a test run has no --video flag (E2E_VIDEO, see Config; e2e explore --video is the flag itself), no run-wide check, and no notice, so a run-wide mode that an engine cannot record is skipped without one, and a retry mode with no retries says nothing. e2e explore has both notices (see Explore). A session knows only its attempt number, not the test’s retries, so UNSUPPORTED_ARTIFACT is raised only when the attempt records, where upstream raises it before any test when some attempt would. xUnit v3 has no retry, so on-first-retry and on-all-retries record nothing there. The test option video and test.describe options have no port (RecordingSource.Test is set by code that builds the session). A mode set on a test is not checked against a serial group, which the port does not have. MobileEngine records the device screen with agent-device (see Mobile engine); a device provider’s own recording (DeviceProvider.record) is not ported.
• Tests: Video/ (video.test.ts, VideoRecorderTests, the nine it), ProviderVideo/ (provider-video.test.ts, both), BrowserProvider/RecordingTests (the recording group of browser-provider.test.ts, see Web engine), ProtectedAppOptions/WebScreencastTests (the first it of describe('web({ screencast })'), partial: typed options have no fractional size or unknown key; “refuses the old video key, naming screencast” has no port), EngineContract/VideoArtifactsTests (describe('video artifacts'), all seven partial, session level, no report; describe('video capability and plan notices') is not ported, it needs the runner’s notices) and “surfaces UNSUPPORTED_ARTIFACT before any attempt…” in RunnerEngineContractTests (partial), and VideoModes/ (video-modes.test.ts, partial: no runner, report, or serial group, so the wizard group is not run, and the modes are already resolved). E2ESessionCleanupTests now fails the start with a required video, since a bad results directory is refused before the engine starts.

Trace feed

What the trace page shows about the app is collected as upstream’s trace-feed.ts collects it, and kept on the attempt’s step records. The page that reads it is described in Trace page.

• EngineStartOptions.Screen and EngineStartOptions.Environment are upstream’s EngineAttemptContext.screen and environment. TraceFeed (Engine/TraceFeed.cs, internal) is upstream’s class: after an action, ScreenAfterActionAsync captures the screen like an observation without publishing it, so the refs a test or the model holds stay valid, within one second; TellEnvironment tells the browser (chromium 141.0) and the page’s user agent once an attempt, without anyone waiting for it. The web engine builds the feed when the attempt starts. It hands the screen after OpenAsync, BackAsync, PerformAsync, PressAsync, and SwipeAsync (upstream’s open, back, perform, pressKey, and the viewport swipe), and tells the environment after OpenAsync and BackAsync (upstream’s navigating). Upstream’s typeText and the point actions have no engine member here.
• An attempt that keeps a trace (the same keep rule as the video: attemptKeep of the resolved trace mode) hands the engine a Screen sink. An attempt that keeps none hands none, and takes no screen from what the test observes. The default mode, retain-on-failure, keeps one for every attempt, since the verdict is not known before the end.
• SessionScreens (Run/SessionScreens.cs) is upstream’s screens of TargetSession and EnvironmentFacts is its createEnvironmentFacts: a name told again overwrites, blank values are dropped, at most 8 facts, each redacted and then cut to 40 and 200 characters. The port has no createEngineSession, so they are classes of their own. AttemptScope.ObserveAsync shows each observation the test, the screen fixture, and the agent take to the same sink.
• StepRecorder.RecordScreen and StepScreens.CompareScreens and TraceScreen (Run/StepScreens.cs) are upstream’s recordScreen and step-screens.ts. A step keeps how the last screen it saw differs from the step before’s (StepRecord.Screen): the node lines added, removed, or changed, without node ids, 12 at most. Differences: the engine hands the port a typed Observation, so SessionScreens has no check for a malformed snapshot or treeUnavailable, and no revision to mint. The screen’s location is the route (/todos), not the whole URL, and a node that is hidden or aria-hidden is left out, as the lines the model reads leave it out. The byte budget is upstream’s default of agents.<name>.maxObservationBytes, 262144, since the port has no such key.
• E2ESession.Steps, E2ESession.Environment, and E2ESession.AppLog (internal) hold what the page reads.
• The app log is collected apart from the feed, as upstream does it. EngineStartOptions.AppLog and EngineStartOptions.Navigation are upstream’s EngineAttemptContext.appLog and navigation, and AppLogEntry is its AppLogEntry; AppLogSource and AppLogLevel, the records’ enums, are public for it. AppLog.Install and NavigationLog (Engine/AppLog.cs, internal) are installAppLog and NavigationLog of web/src/app-log.ts. On every context the web engine configures, a persistent one too, it reports console output (error and assert as errors, warning, log and info as info; debug and the rest are left out, and so is Chrome’s Failed to load resource: echo, which the network lines tell), uncaught errors, failed requests (one the browser cancelled is left out), and responses with a status of 400 or more (5xx an error, 4xx a warning, a missing /favicon.ico left out). It tells where the page went: navigated to /checkout, the new tab navigated to /help, frame "pay" loaded /embed, and the app opened a new tab at /help; the test stays on its page. A navigation the engine makes itself is not told, since its step says it: NavigatingAsync (upstream’s navigating) wraps OpenAsync and BackAsync, and also ForwardAsync, ReloadAsync, and GotoAsync, as upstream wraps browser.goto, reload, back, and forward. It also tells the environment, as upstream’s navigating does. Differences: the console location is Playwright’s URL:line:column string, read to (at URL:line), and an uncaught error is the text Playwright gives (the message, then the first at frame), where upstream reads the error’s name, message, and stack.
• As upstream, a tab’s first document is told by its opening, a page that is not the test’s current page counts as a tab (so the test page’s navigation while RestartAsync replaces it reads as a tab’s), and a request a test aborts itself (net::ERR_FAILED, ERR_BLOCKED_BY_CLIENT) is logged: only the browser’s own cancellations are left out.
• .NET-only names: StepHooks, E2ESession.Phase and E2ESession.AppLog (internal), StepEvents (the step’s events, safe to read while the engine appends), and AppLog, NavigationLog, and AppLogRoute as the homes of upstream’s installAppLog, NavigationLog, and AppLogRoute. .NET-only tests: StepHooks/StepHooksTests in E2E.Tests and E2E.NUnit.Tests. EngineStartOptions.AppLog and Navigation are for engine authors, as Screen and Environment are; no user guide describes the engine contract, so the guides tell only what the trace page shows.
• AppLogRoute (Run/AppLogRoute.cs) is upstream’s AppLogRoute of TargetSession, built by createAppLogRoute (engine/session.ts), and AppEvent and AppLogLimits are its AppEvent and APP_LOG_LIMITS. The engine reports from the moment it starts, before any step records, so what arrives before the recorder takes it waits: errors and warnings up to 200, info up to 100, navigations up to 50, so chatter never crowds out an error. An entry with a source or a level outside the enums, or a blank text, is dropped. The route is a class of its own, since the port has no createEngineSession. StepRecorder.RecordAppEvent and AppLog() are upstream’s recordAppEvent and appLog(): a line goes to the attempt’s app log with the step running in this async context, else the step that started last, redacted and then cut to 300 characters; where the app went is a navigation event of that step, and one before the first step is told under it.
• Step events, #443. A locator action records an engine event, upstream’s recordAction of the locator engine: the action, the node it landed on as the resolve read it (role and label, secrets redacted), and appeared after Nms when the node took more than one round to appear. An action that failed records its code. The port retries no stale node, so an event never says went stale; PressSequentiallyAsync is one pressSequentially event where upstream focuses first and records focus; the action selectOption is named select, as the engine contract names it. RecordedTarget.TargetLabel is upstream’s targetLabel (cache/trace.ts). An assertion on a locator and WaitForAsync record a poll event with the count of reads and, when the reading changed or the poll failed, the readings in order: SampleHistory (Expect/SampleHistory.cs) is upstream’s class (expect/samples.ts), RecordAsync its record. An agent step records the events upstream’s instrumentPhase records (Phases.InstrumentPhaseAsync, agent/phases.ts; PhaseHost, PhaseSpec, and PhaseDetail for its host, spec, and detail; PhaseDetail is .NET-only, since upstream’s is a Partial<StepEvent>; the port’s PhaseSpec has no api, which upstream never reads). Every look the step takes is an observation event with the nodes it read and its size in bytes, as a model reads it, and a look taken inside an action is named: relocate for the fresh look that re-finds a stale node, page for the look between the pages of a scroll. Every grammar action is an engine event named for the verb (tap, doubleTap, type, typeSecret, press, selectOption, check, uncheck, scrollIntoView, scrollUntil, scroll, navigate, back; and clear, which upstream has no verb for), with the action it recorded as its detail, and a project tool is one named tool:<name> (ActionNames.ProjectToolEventPrefix, agent/action-names.ts). A judgment records one observation event for its look. Differences: the port’s action body commits and notifies inside the phase (the action is recorded and the test’s OnAction hook has run when the event ends), where upstream’s phase ends before it, so an engine event also holds the settle look and the hook; the model events are recorded where each model call is counted (AddModelCall), so the model phase announces no activity; and there is no agent.cache phase or policy event.
• Agent step events, #648. Observation.Revision (public, set by the harness; upstream’s Observation.revision) is b<n> for the attempt’s n-th capture through AttemptScope.ObserveAsync, so an observation an engine hands a locator directly has none. A step records the revision of the last screen it read as StepRecord.ObservationRevision (StepAgentDetails.ObservationRevision), and the report writes it as observationRevision on any agent step that read one. StepAgentDetails.ObservationBytes keeps the largest observation in StepMetrics.ObservationBytes. Tests: AgentPhases/InstrumentPhaseTests has the one it of upstream’s describe('instrumentPhase'); the retryingObserve and checkStepClock blocks of agent-phases.test.ts are not ported, because the port has not extracted those two functions. The events and the revision are read in the ported tests of agent.test.ts (the waitFor and repair cases), agent-act.test.ts (the executor step and the stale node), agent-vision.test.ts (observationBytes) and the secret-fill describe of agent.test.ts (typeSecret). Not ported: the policy events (recordPolicyEvent) and the observation event of a denied pixel fallback (agent-fallback-handoff.test.ts’s POLICY_DENIED observation event), because the port has no pixel fallback.
• The hook a step ran in, #443. StepRecorder.Phase is upstream’s phase, and E2ESession.Phase (internal) its enter. Upstream’s runner enters beforeEach and afterEach itself. The port has no runner: NUnit or xUnit calls the hooks. So when Phase is null, StepHooks (Run/StepHooks.cs, .NET-only) reads the step’s call stack. The first method outside the library that carries a mark decides: an NUnit [SetUp] is beforeEach, a [TearDown] is afterEach, a [Test], [TestCase], or [Fact] is the body, and a test class’s InitializeAsync or DisposeAsync is the xUnit pair. A step called after an await inside a helper the hook called belongs to the body, since the stack no longer shows the hook. [Theory] and [TestCaseSource] are test methods too, and InitializeAsync and DisposeAsync count on a class that implements IAsyncLifetime. OneTimeSetUp and OneTimeTearDown are not hooks of a test (upstream’s beforeAll and afterAll), and their steps carry none.
• Tests: StepScreens/ (step-screens.test.ts, all 11 it; “hands every engine screen to the routed sink …” is partial, with no malformed or treeUnavailable snapshot to drop, and is named for what it checks), Traces/TracesTests (see Trace page), AppLog/ (unit/app-log.test.ts, all 7 it: StepRecorderRecordAppEventTests and TheSessionAppLogRouteTests; “holds lines and navigations until a sink is routed …” is partial and named for what it checks, since a typed entry cannot have a source that is no source), WebAppLog/ (web/tests/integration/app-log.test.ts, both it, in Chromium; AppLog/ is taken by the unit test of the same name; the server of upstream’s first it destroys the socket of /api/reset, which an HttpListener cannot do, so a route aborts that request and the line ends net::ERR_FAILED), SampleHistory/ (unit/sample-history.test.ts, all 6 it), and the .NET-only TraceFeed/WebTraceFeedTests, which drive the web engine in Chromium, where no upstream it does, and E2E.NUnit.Tests’ StepHooks/StepHooksTests, which runs a fixture with a [SetUp] and a [TearDown] through NUnit and reads the trace page.

Trace page

Upstream’s --trace writes <output>/results/<result>/trace.md, a markdown page that tells the whole attempt: what failed and where to look, every step with what it did, the cache’s decisions, what the app logged, the agent’s last turns, the screen at failure, and what the attempt ran on. You open the file; there is no viewer. A test run has no --trace flag, so trace and the .NET-only variable E2E_TRACE choose which attempts keep a page (see Config); the default is retain-on-failure. e2e explore --trace [mode] is the flag itself.

• A session writes the page when it is released, after the video is stopped, the engine is closed, and the debug table and the AI trace are written, and before a required video’s failure is rethrown. It writes when an attempt of the test kept a trace, by attemptKeep and keeps (the rule the video follows), and the page then tells every attempt, see below. A page that cannot be written prints a line on stderr and never hides the test’s result. The page goes in the test’s directory under the results directory, .e2e/results/<result>/trace.md, beside the attempt-<n> directories, as upstream’s does. A session is one attempt, and each adds itself to RunReport (.NET-only, Report/RunReport.cs), which collects the attempts of a test: a first attempt starts the test over, a later one extends it. The page tells every attempt the test has had so far, as upstream’s does from the run report, and is written when any of them kept a trace (upstream’s tracedResultIds). So a flaky test reads Passed after 1 failed attempt with the last failure, and a test that failed every attempt reads Failed the same way on both attempts, even when the passing or last attempt kept none itself. The run keeps the attempts of a test only while it has not passed and the mode is not off: an attempt that passed or was skipped ends the test and drops them. The port cannot know the last attempt of a test that keeps failing, since the adapters do not tell the session the retry count, so those entries stay until the process exits. Upstream’s runner holds every result of the run in memory too (const results: ResultRecord[] = [] in run/runner.ts), for its report.
• The MCP server’s sessions set trace to off, so they write no page and take no screens: an MCP session is for looking, and e2e mcp has no --trace, as upstream. An exploration takes the target’s mode, or --trace; its own verdict gets no page.
• TracePages.WriteTracePagesAsync and TracePages.TracePage are upstream’s writeTracePages and TRACE_PAGE (report/traces.ts). FailureText.RenderTracePage, ToldAttempt, FailedStepOf, AttemptsLine, FailureSource, EvidenceOf, DetailLines, and StepLabel are the matching functions of report/failure-text.ts, and MarkdownText and ReportFormat are report/markdown-text.ts and the parts of report/format.ts the page uses. ArtifactPaths.ResultSegment is resultSegment (run/artifacts.ts; it also strips a .cs extension). All are internal.
• Report1Document, ReportRun, ReportResult, ReportAttempt, ReportError, ReportArtifact, FailureEvidence, and Outcome (Report/ReportRecords.cs) are the part of upstream’s report the page reads, and AttemptReport.Build (Run/AttemptReport.cs, .NET-only) fills them from the step recorder, the environment facts, and the video segments of one attempt. ReportAttempt is upstream’s ReportAttempt and AttemptView in one, since the port has no serial groups. The file is the first step’s file (unknown without a step), the run id is one per process, and the title is the test’s title as one part. RunReport.Add joins an attempt to its test’s result and derives the result’s status from the attempts (Flaky when the final attempt passed after one that did not), where upstream’s runner keeps the result record and buildReport reads it.
• What the page tells: the title, the file, the status and duration, what the attempt ran on, the error (a fence in it cannot close the block), Look at:, whether the attempts failed alike, every step (its label, duration, model calls, the video offset, the hook, the explanation of a failed one, the cache’s decision, the steps’ screenshots, and how the screen changed), the app log, the agent’s last turns, the screen at failure, other errors, and the evidence. Every piece of text goes through MarkdownText (a mention or a URL becomes code, markup is escaped, a loopback origin is dropped), and the whole page goes through the attempt’s redactor again before it is written.
• Not filled by the port yet, each with its issue. The renderer shows them when a record carries them, and the tests build such records by hand. A serial group’s members: the port has no serial groups. Filled since: the agent’s turns of a step (#481), the step events, the hook, and the app log (see Trace feed), and what became of a step’s recording with the model that recorded it (see Trace page).
• Recording outcome. E2ESession.Flush is upstream’s settleStagedTraces: it returns a CacheWrite per step index, and the session sets StepRecord.CacheWrite on each step that carries cache detail. A step decides two itself when it concludes: Evicted (a passed step an end-mismatch hand-off had to repair) and NoChange (the step changed nothing a replay could check and read no entry; one that read an entry is evicted at the settlement instead). The settlement reports the rest: Saved, Kept (the replayed entry, or one holding the same flow, which ignores the recording model), Unconfirmed, and Evicted (an entry the step read that the settlement deleted: a whole replay no check confirmed, a replay that ended in a failure, or an entry the step read and replaced with nothing). A flow the step recorded anew is Unconfirmed when no check confirmed it, whether or not an older entry was deleted, as upstream’s settleStagedTraces reports it (only an entry replayed whole is evicted). Both report Evicted only after IStepCache.Delete returned, as upstream’s evict and settleStagedTraces do since 689620d (#1072). IStepCache.Delete returns when the entry is absent and throws when deletion fails, as upstream’s CacheStore.delete does, and FileStepCache.Delete ignores only a missing file or directory, as upstream’s FileCacheStore ignores only ENOENT. A Delete that throws leaves the entry in place: E2ESession.Flush reports Unconfirmed for a whole replay or a new recording and nothing for an act that recorded nothing (upstream’s evict stages nothing then), and StepTraceSession.ConcludeAsync (upstream’s evict for a repaired end-mismatch) stages no decision. Upstream also evicts a failed step’s consumed replay in conclude; the port does both at the settlement, so a delete ConcludeAsync could not make is tried once more there. Tests: StepCache/StepTraceSessionTests “does not stage an eviction outcome when deleting a consumed entry rejects” and the three it of trace-cache-config.test.ts in TraceCacheConfig/FlushStagedTracesTests, the default-store one a no-op as root and on Windows, as upstream. NotWritten is a store whose Write throws; the port’s FileStepCache has no size limit on a write. Unlike upstream, a cancelled attempt reports none, since it never settles. The page tells the model after the cache decision: recorded with `fake-loop/model-A` . CacheEntry.RecordedBy and CacheInfo.RecordedBy are upstream’s recordedBy, but one string, IAgentModel.Name, which spells provider and model id together (upstream’s is a provider and a model). The recorder redacts it and drops one that is blank or past 300 characters, and a read does the same, so a malformed one never makes an entry a miss. It is the agent’s configured model, which an executor’s Model becomes when the agent sets none; an agent with none records none. It never keys the entry. The port has no list reporter, so the recorded with note on a list line has no port; cache.recordedBy and cache.write are in report.json (see Report document). Tests: CacheModelProvenance/ holds both it of cache-model-provenance.test.ts, each partial: they read the step record, the entry file, and report.json (checked against the schema), not the list line or trace: 'on', and the malformed cases are one string (the nine cases of the first are seven). The second is also partial where an executor reports no model: upstream’s report names the executor, the port’s says unknown. StepCache/CacheWriteTests (.NET-only) covers each outcome. TraceRecorderTests has the two oversized cases as one theory, the blank cases as one theory, and the redaction in “records a secret fill by name only”. The two it of trace-cache.test.ts that read a bounded or blank recordedBy are covered by the oversized-model, blank-model, and invalid-model cases of the first test.
• Error details. ErrorDetails (E2EException.Details, public) is upstream’s ErrorDetails of internal/errors.ts, with the same fields in C# casing: Locator, Expected, Observed, Matches, Role, Name, TestId, and WaitedMs. A locator that matched nothing (NOT_FOUND) carries the locator as written, what it asked for, and how long it waited. One that matched too much (STRICT_MODE) carries the same without the wait and with Matches. A waitFor that timed out carries the wait. A locator matcher that failed carries Locator, Expected (with not when negated), Observed, and Matches, and ToHaveScreenshot carries the stored screenshot’s path as Expected and what it observed. LocatorResolver.LocatorDetails and LocatorExpression.ExpressionHints are upstream’s locatorDetails and expressionHints. ErrorSerializer.Serialize keeps the details as upstream’s serializeError does: text redacted, stripped of control characters, cut at 2 KiB, empty text dropped. The port has no report boundary, so ErrorRedaction.Failure also redacts the details text when it builds the error. Screen.ScrollUntilVisibleAsync sets them too, on its LOCATOR_NOT_FOUND. Upstream sets details in one more place the port has no feature for: a tap at a point. Not ported: the details of an error classified from another module copy (there is one copy of an assembly).
• Failure evidence. FailureEvidence.CaptureAsync is upstream’s captureFailureEvidence (run/failure-evidence.ts), with its options, its 5 s budget for the observation and the screenshot together, its limits, and its scoring of the nodes closest to what a locator asked for. A failed attempt that keeps a trace page looks at the app once more and keeps the screen as attempt-<n>/screen-at-failure.txt, one #id role "name" text="…" [states] line for each node (indented by depth, hidden nodes left out as the lines the model reads leave them out, a node with no role read as text), a masked screenshot as attempt-<n>/screenshots/failure.png, the location, and for a locator that matched nothing or too much, up to five candidates. The page shows them under ## Screen at failure. The look is taken once per attempt, the moment the test body fails, ahead of the test’s own teardown (upstream’s captureEvidence, called after the body, the late soft failures, and the settled callbacks, and again after the teardown for a failure that first landed there). E2ESession.CaptureFailureEvidenceAsync is that look, and an adapter calls it: the NUnit adapter wraps the test method (IWrapTestMethod, since ITestAction runs after [TearDown]) and looks when the body throws or an Expect.Soft failed; the xUnit adapter looks from a BeforeAfterTestAttribute after the test method, when the result is Failed. A failure that first lands in the teardown is looked at when the session is released, as upstream does. Differences: an xUnit Expect.Soft failure is kept on the session until the test is disposed, so it is looked at then, after the test’s own DisposeAsync (closing the soft failures early would move them out of the test’s result). An attempt whose trace page is not kept looks at nothing. The screenshot is dropped when the attempt knows a secret (as every pixel the port keeps; upstream keeps its masked failure screenshot), when the look at the screen failed, and when the engine does not report that it covered a region for each secure node. The viewport is the screenshot’s size in viewport pixels, since the port’s observation has none, so it is absent without a screenshot. When the look fails (a crashed page, a closed app) there is no evidence, not the last screen a step saw. The candidates are listed for the codes NOT_FOUND, STRICT_MODE, and TIMEOUT (upstream’s LOCATOR_NOT_FOUND and LOCATOR_AMBIGUOUS too, which an agent action raises).
• Differences: the page has no sourceUrl or artifactsUrl link and no Standalone mode, and its footer does not point at report.json, which the port writes beside the results (see Report document). An attempt that did not skip but carries a skip (a test that called Skip and then failed on a soft failure) names it on its page after Look at:, as Also skipped: <reason>. lastTurnLines and screenLines belong to the run page and are not ported. Not ported: writeHostTrace (run/host-trace.ts) and the hosted session’s page, which need the hosted session (#403).
• .NET-only names: E2ESession.Evidence (what the session captured; upstream reads it from the report), E2ESession.CaptureFailureEvidenceAsync and E2EFixture.CaptureFailureEvidenceAsync (upstream’s local captureEvidence closure; the fixture’s is for the adapters), LooksBeforeTeardownAttribute (NUnit and xUnit; internal), NUnit E2ETest.CaptureEvidenceOf (internal; reaches the protected fixture method), xUnit E2ETest.LookAtFailure and its Running map (the attribute is handed the test, not the instance), FailureEvidence.ScreenFile, ScreenshotFile, ScreenArtifactId, and ScreenshotArtifactId (upstream registers the files with its artifact sink), FailureEvidenceOptions.Observe and Capture (upstream hands the open session), AttemptReport (and its ResultId and ResultSegmentOf, which give the result id and directory name; a .NET test has no one source file, so the test class stands for upstream’s file: the fixture passes its full name as E2ESessionOptions.TestFile, it joins the title in the id hash as upstream’s testId does, and its simple name leads the segment; a session started with no TestFile, such as explore or a direct E2ESession.StartAsync, hashes the title alone), AttemptReport.TestId and EncodeTitle (internal; upstream’s testId and encodeTitle in internal/ids.ts, with a title path of one title; the title is percent-encoded so a :: in it cannot pass for the separator; the hash input carries a NUL mark so a title-only session cannot share an id with a test id), E2ESessionOptions.TestFile (public, optional; the one new member of the public API, set by E2EFixture, so the NUnit and xUnit adapters change nothing), TraceOutput, WriteTracePagesOptions, ReportResult.Directory (an attempt writes under ArtifactPaths.ResultSegment of the result’s id and title, so two titles that slug alike keep apart; the segment leads with the test class’s name when the fixture started the session, and the page goes in that directory), StepRecorder.AddModelCall (upstream’s usage.record in StepAccounting), FileStepCache.CacheDirectory (internal), and CacheInfo.Entry (upstream’s entry of StepCacheInfo), OptionalTextConverter (upstream’s readRecordedModel tolerates a malformed recordedBy; the port’s is a string, which a converter reads leniently), PendingAct.Decided (upstream’s decided staged entry), and CacheInfo.WithRecordedBy (upstream spreads recordedBy into the info).
• Tests: Traces/ holds unit/traces.test.ts (all 11 it: WriteTracePagesTests, TracePageNamesTests, TheStepsOnATracePageTests, TheAppLogOnATracePageTests, over hand-built records) and integration/traces.test.ts (TracesTests, all 6 it, in a session over a fake engine, with no runner). In TracesTests, “keeps each step’s events, files what the app logged …” is named for the part it checks, Keeps_each_steps_events_and_files_what_the_app_logged_under_the_step_it_happened_in_on_the_page (partial: it reads no run event, which the port has no run to emit); the two “names the step … on the page and in the summary” tests are partial (no summary reporter); “keeps the traces the mode asks for” runs sessions with the attempt number in place of a runner’s retries. ReportFailureText/ holds report-failure-text.test.ts: detailLines, attemptsLine, failedStepOf, failureSource, evidenceOf, and renderTracePage (15 it; the footer of “tells the whole story” has no report.json pointer). Not ported from that file: “lastTurnLines and screenLines” (the run page), and “a serial member on its page” (no serial groups). .NET-only: TracePageRedactionTests. FailureEvidence/ holds failure-evidence.test.ts (FailureEvidenceTests, 6 of its 7 it in E2E.Tests, in sessions over a fake engine; the it.each is one theory). “looks at the screen before teardown runs …” is E2E.NUnit.Tests/FailureEvidence/FailureEvidenceTests (through NUnit, with a [TearDown] in place of afterEach), so all 7 it of the file are ported, with a .NET-only soft-failure variant beside it; the xUnit adapter’s look has no test project. “keeps the evidence of two tests whose ids sanitize to the same directory name apart, each report digest matching its file” is Keeps_the_evidence_of_two_tests_whose_titles_slug_alike_apart (partial: the port’s report artifacts carry no sha256 to compare). Keeps_the_evidence_of_two_tests_with_one_title_in_two_classes_apart and Gives_a_title_with_a_separator_in_it_another_id_than_the_same_text_split_into_file_and_title are .NET-only (upstream’s runner rejects a duplicate test id; the port keys on the class instead of the file). Errors/ErrorDetailsTests holds the error details tests of unit/errors.test.ts: two of three, the redaction one partial (the port keeps no stack text), and not “keeps the details of an error classified from another module copy”. The details of the failure are asserted in ScriptedMatchers/, ScreenshotAssertions/, and AgentExecutorContext/ where upstream asserts them, and the failure capture beside an assertion’s own screenshot in AssertEvidenceUnderACustomExecutorTests. open-host-trace.test.ts is not ported with the hosted session (#403).

Test filters

Upstream #1206 (da13624, feat(runner): run only the tests a pull request can affect) adds filters to the config, --changed-since <ref> on e2e run and e2e list, and the “left out” part of the report. #1345 (47127f7) adds context.configFile. The port has no runner, no collect or select, no e2e run or e2e list, and no reporters other than the JSON report, so it takes the filter seam, the change detection, and the report fields, and leaves the wiring to a host. A test framework selects tests itself (dotnet test --filter), and the port keeps no list of every test before the first one runs, which a filter needs, so nothing in E2ETest calls a filter.

Ported:

• ITestFilter (upstream’s TestFilter: Name, and FilterAsync for filter), TestFilterCandidate, TestFilterChanges, TestFilterContext (with ConfigFile, #1345), and TestFilterResult (Exclude, Note) are public (Run/TestFilter.cs, upstream’s types.ts). A host calls the internal TestFilters.ApplyTestFilters (Run/TestFilters.cs, upstream’s run/test-filters.ts) with its candidates: filters run in order on the tests the ones before kept, a filter that throws, returns an invalid result, or runs past 5 minutes (FilterTimeout) leaves nothing out with a warning, and reasons, notes, and log lines are redacted and cut to one line of 1024 code points. context.Redact also masks every run of 8 or more characters of a secret. TestFilters.RepoConfigFile is upstream’s repoConfigFile, TestFilters.FilterRecords and FilterSummary are filterRecords and filterSummary, and LeftOut.LeftOutTests is leftOutTests (Report/LeftOut.cs, report/left-out.ts).
• Changes.FindChangesAsync (Run/Changes.cs, upstream’s findChanges): every file that differs from the merge base of the ref and HEAD, staged, unstaged, deleted, and untracked files included, a rename as both paths, sorted by code point, with a 30 second budget. A ref git cannot measure from is INVALID_CHANGED_SINCE (a ConfigurationException). Git.RunGitAsync, GitResult, and GitFailure are upstream’s runGit, GitResult, and GitFailure; Paths.OnDiskPath is onDiskPath; Time.BudgetText is budgetText; ReportFormat.FilterDecision is filterDecision.
• The report: run.changes (ReportChanges) and run.filters (ReportFilter, FilterOutcome) on E2E.Reports.ReportRun, SkipInfo.Filter (a filtered skip names the filter), ErrorPhase.Filter, and the filter phase in the wire form. schema/report-v1.schema.json and both fixtures are upstream’s at #1345.

Differences:

• An interrupt surfaces as an OperationCanceledException, where upstream rejects with INTERRUPTED. The context’s Signal follows the interrupt only while the call runs. The work a timed-out or interrupted call leaves behind is not recorded (upstream’s leaveWorkBehind keeps the CLI from waiting for it; the port has no CLI process to keep).
• A result is typed, so the refusals of a result that is not { exclude, note? } (an array, a string, an unknown key, a map, a numeric note, a value that cannot be shown as text, and a reason that is not a string) cannot happen. A null result, a null Exclude, an empty reason, and an id no collected test has are refused, with the port’s wording for the first two.
• The redactor reads a newline and a space alike in a run it masks, so it takes the space before a key’s text into the marker in a diff line, where upstream keeps it. Nothing leaks.

Not ported:

• filters in e2e.config.json (a filter is code, and the JSON config holds none), --changed-since, --no-filter, INVALID_CHANGED_SINCE as a CLI exit 2, the NO_FILTERS warning, filterSelection, filterCandidates, select with excluded (applyExclusions), the list and run output of a left-out pair, the left-out lines of the list, markdown, and JUnit reporters, telemetry, the filter setup step of a run, pull_request_target guidance, and the CI recipe. They need the runner, collect, select, and the reporters.
• @e2e-dev/decision’s decisionFilter and its tests, including #1345’s leaves nothing out, making no call, when the config the run loaded changed, whatever its name, and goes by name for any other: the package is not ported (see Upstream packages).
• Tests: Changes/FindChangesWithChangedSinceTests has all 10 it of changes.test.ts (the INTERRUPTED one is Partial_rejects_once_the_run_is_interrupted; the --no-relative one runs apart from Windows, as upstream’s does). Time/BudgetTextTests is budgetText. SchemaFixtures/ReportV1SchemaTests has the filter it of schema-fixtures.test.ts. TestFilters/ApplyTestFiltersTests, AFilterPastTheDeadlineTests, and FilterRecordsAndFilterSummaryTests port test-filters.test.ts over candidates built by hand, since the port has no collect or select; the ones that differ are Partial_. Not ported from it: the two filterSelection it, the three filterCandidates it, the refuses a result cases above, and leaves nothing out when a filter throws a value that cannot be shown as text. TestFilters/WhatAFilterIsToldTests is #1345’s names the config file the run loaded over RepoConfigFile, partial. Not ported: the rest of integration/test-filters.test.ts, select.test.ts’s cases for excluded, config.test.ts’s filters cases, cli*.test.ts, report-list, report-markdown, report-junit, telemetry-events, and packages/decision’s tests.

Report document

Upstream writes report.json, a report-1 document of the whole run (report/build.ts), checked by schema/report-v1.schema.json. The port writes it too (#496), with the differences below.

• When and where. Upstream builds the document once, when its runner ends the run, and writes it to <output>/report.json. A test framework runs the port’s attempts as sessions with no run around them, so each session adds its attempt to the process’s run (ReportFile.WriteAsync, .NET-only) and the whole document is written again, atomically, to report.json beside the results directory (.e2e/report.json by default, as ai-trace.json is). The file is complete after every attempt. run.finishedAt and run.status are those of the attempts so far, not final, and run.exitCode is the worst exit code of the results, as upstream’s verdictExitCodes derives it: a failed or timed-out result exits 1, or 2, 3, or 4 when the category of an attempt error is configuration, infrastructure, or internal (an agent code’s category comes from AgentError.CategoryByCode, upstream’s CATEGORY_BY_CODE, so a blocked AUTH_CREDENTIAL_UNAVAILABLE exits 2), and an interruption exits 130, which outweighs the rest. run.status follows the code (passed, failed, interrupted, else error), and blocked is derived from the results afterwards. A result keeps its wire form until it gains an attempt, so a large suite does not serialize every result again after each attempt. A result id is the test’s file and title, so a test that runs twice in one process (another target, a repeat) is one result that its last run replaces, as RunReport already treats it. Test processes that share an output directory overwrite one another’s file, and a file from an earlier run is replaced by the first attempt. It is written whatever the trace mode, for passing tests too. A write that fails (disk, permission) prints one line to stderr and does not fail the test.
• A closed session’s report (feat(runner): expose the closed hosted session report, b750125). E2ESession.Report() is upstream’s session.report(): once the session is disposed and its trace page, when the mode keeps one, is written, it returns the attempt’s report-1 document as a single-result run (ReportFile.SingleAsync, .NET-only; upstream’s writeHostTrace returns the Report it builds). It returns null before that and when the page cannot be written. The return type is the public E2E.Reports.Report, upstream’s Report (an alias of Report1Document) with its nested types in C# casing and the names of report/build.ts and run/steps.ts (ReportRun, ReportTarget, ReportResult, ReportAttempt, ReportStep, ReportError, ArtifactRecord, StepEvent, StepMetrics, StepModelInfo, StepCacheRecord, FailureEvidence, SkipInfo, VcsInfo, and the rest). They live in the E2E.Reports namespace because the port’s own internal records for the same parts keep upstream’s names in E2E (ReportRunStatus and ReportExplore are shared and stay in E2E). The typed report is read from the document that report.json holds, so the two tell one attempt alike; ReportSerializer is .NET-only and the only place that reads it; the library never writes the typed form. Differences: the typed report has no serialGroups and no carried (the port has none), no extensions and no stack; enum members stand for upstream’s string unions (ResultStatus.TimedOut for timed-out); durations stay DurationMs as upstream, but ReportExplore keeps its TimeSpan members; a session that keeps no trace (the mode keeps none for the attempt’s verdict) still returns its report, where upstream’s hosted session always writes a trace; a trace page that cannot be written is reported on stderr and the session releases without a failure, so upstream’s close() failure for it has no port. E2ESession.AttemptId (internal) is upstream’s attemptId. Tests: OpenHostReport/OpenHostReportTests has both it of open-host-report.test.ts; the first is partial and renamed Returns_independent_closed_reports_that_remain_stable_across_a_second_session (no gadget.dropFile() fixture, and the soft failure is CloseSoftFailures’ result rather than an entry of close()’s failures), the second is partial and renamed Returns_null_when_the_trace_write_fails_and_the_session_still_releases (no close() failure check). .NET-only: Tells_the_same_result_as_report_json_for_the_same_session checks that Report() and report.json tell one attempt alike, which upstream’s hosted session (no report.json) has no counterpart for. TypedReport/TypedReportTests (.NET-only) reads the valid report-1 fixture into a Report and writes it back, and checks that the document is the same bar the serial groups, carried, extensions, and stack. The document’s run.id is the session’s own, so it is not compared. ReportFile.DetectVcs (.NET-only) asks version control once per process and project root for both report.json and Report(); a failed lookup is asked again, and Asks_for_the_commit_once_per_project_root_however_many_sessions_ask_and_asks_again_after_a_failure covers both.
• schema/report-v1.schema.json is upstream’s schema, unchanged. schema/fixtures/report-v1.valid.json and report-v1.invalid.json are upstream’s fixtures. The E2E package packs schema/*.schema.json under schema/ (not the fixtures), as upstream’s files does. schema/session-v1.schema.json and its two fixtures are upstream’s, unchanged. Upstream’s agent-judgment-v1 and agent-judgment-v2 schemas are not ported: the port has no agent-judgment-2 response grammar (JUDGMENT_SCHEMA, validateJudgmentResponse), so its judge ends with the done tool.
• Session store (run/sessions.ts, SessionStore). SessionStore (internal) is upstream’s: one AES-256-GCM envelope file per session, authenticated over the whole envelope (identity, expiry, and secrecy in the clear), the learned secret values only inside the ciphertext, a memoized decrypt that still checks expiry on every load, and file names that are percent-escaped, cut, and hashed. SessionIdentity, SavedSession, EngineState, and SavedSecrecy are the records of upstream’s interfaces. Methods are SaveAsync and LoadAsync (save, load); Create and ForWorker take an optional TimeProvider (.NET-only, a test’s clock). ForWorker takes the key as byte[]. A key of the wrong length is E2EException with SESSION_KEY_INVALID, since the port has no infrastructure error class. A file is written 0600 and the directory 0700 on Unix (AtomicWrite.WriteFileAtomicAsync takes the mode; Windows has none). A payload or envelope field of the wrong type is SESSION_INVALID, where upstream’s TypeError escapes. Not ported yet: targetIdentity, SessionStaging, openSession, writeHostTrace, and openHostTrace, which need setup sessions and the hosted session’s trace (#705). The agent-judgment schemas are #706.
• Tests: SessionEnvelope/ is unit/session-envelope.test.ts (all 10 it: SessionEnvelopeTests, SessionLoadTests, SessionFileNamesTests), and SchemaFixtures/SessionV1SchemaTests is the two session it of describe.each(schemas).
• WriteReport (Report/WriteReport.cs, internal) is upstream’s report/write.ts: WriteTextReportAsync, WriteJsonReportAsync (two-space indent, a trailing newline), and ToPosixPath. AtomicWrite.WriteFileAtomicAsync is upstream’s writeFileAtomic (internal/atomic-write.ts): a sibling temporary file with random bytes in its name, moved over the target; AtomicWrite.WriteFileAtomic is the synchronous twin (writeFileAtomicSync), which FileStepCache.Write uses because IStepCache.Write is synchronous. On Windows both retry a move the file system refused while another move was replacing the target or a reader held it open (see Replay cache). The trace page and the failure screen file are written through WriteTextReportAsync, as upstream’s traces.ts does (it replaces the port’s own TracePages.WriteTextAsync). The writer serializes with the port’s JSON defaults (camel case, null omitted), so a caller redacts the text and shapes the document. Difference: a string with a lone UTF-16 surrogate makes WriteJsonReportAsync throw, where JSON.stringify writes an escape; the builder that cuts text to the schema’s limits must not split a surrogate pair. Not ported: the file mode option, and fileReporter, since the port has no reporters.
• The schema is checked in tests only. The tests use JsonSchema.Net 8.0.5 (MIT; version 9 and later use another license), in the test helper project tests/E2E.Tests.Schema. The shipped package validates nothing and has no new dependency. ReportSchema.AssertValidReport is upstream’s assertValidReport, with format asserted as Ajv does.
• Run-level blocks (Report/BuildReport.cs, upstream’s report/build.ts): DescribeTarget, ProjectSource, ComputeSummary, DeriveRunStatus, ComputeUsage, SerializeTarget, and Build, which assembles Report1Document with ReportRun’s new members (Status, ExitCode, StartedAt, FinishedAt, Project, Environment, Vcs, Errors, Summary, Limits, Usage). ReportRun.Targets are ReportTargets now, not names. ReportResult gained DeclarationIndex, Tags, and Selected; ReportArtifact gained Size; StepMetrics gained ObservationBytes (the largest observation the step read, in bytes of the text a model reads; a judgment that sends pixels alone counts none, as upstream’s does not; so usage.maxObservationBytes is the largest of them). AttemptReport.Build fills each kept file’s Size, so usage.artifactBytes adds them up. The engine version of a per-attempt target is IEngine.Version, which the session carries in AttemptScope.EngineVersion; a blank version reads unversioned. Git.RunAsync removes the variables that point git at a repository (GIT_DIR, GIT_WORK_TREE, and the like) before it starts git, so a hook or wrapper cannot move the repository it reads; upstream passes the parent’s environment through. A target’s baseOrigin is the scheme, host, and non-default port of the app URL, without credentials, and only for http and https, as new URL(...).origin is for a web URL. Differences: the port has no serial groups, no --last-failed rerun (carried), and no failOnSkippedFailure, so DeriveRunStatus reads results only. DescribeTarget takes an IEngine, which has no name, so the engine name is its platform; its flag capabilities are spelled in camel case (tapModifiers); stateCapability is false, since the port has no state hooks. A target’s environment is always test (no app.environment key), the project is unknown with a zero digest (no project id or config digest, as upstream reports without a config), limits are the defaults (the config rejects maxObservationBytes and maxInputTokens), and environment.runtime reads .NET <version> where upstream’s reads node <version>.
• .NET-only names in this part: ReportEngine, ReportProject, ReportEnvironment (upstream writes these as inline object shapes), ReportRunStatus (upstream’s status string union), BuildReportOptions.TargetProvenance is upstream’s own, Git.RunAsync’s budget parameter (for tests), and DescribeTarget(platform, version, capabilities) (the session knows its engine by those three, not as an IEngine).
• Vcs.DetectAsync and Git.RunAsync (Internal/Vcs.cs, Internal/Git.cs) are upstream’s detectVcs and git (internal/vcs.ts, internal/git.ts): git is asked with a one-second budget, and a checkout without git falls back to GITHUB_SHA and GITHUB_HEAD_REF or GITHUB_REF_NAME. Difference: a cancelled token surfaces as a cancellation, where upstream has no token.
• Wire form (Report/ReportWire.cs, internal; upstream’s serialize* functions and the report-1 shapes of report/build.ts). Upstream types the shapes and lets JSON.stringify drop what is absent; the port builds JsonObjects, which omit what is absent the same way. Text is cut to the schema’s bounds without splitting a UTF-16 surrogate pair, and a lone surrogate reads as U+FFFD (JSON.stringify escapes it; the port’s writer refuses it). A result’s text goes through the redactors of that test’s attempts, as its trace page does. ReportError gained Category and Retryable; the stack never enters (SerializedError has none). ReportAttempt gained Id (a UUID version 7, AttemptScope.AttemptId, now minted with the dashes), StartedAt, Cleanup (always complete: the port does not tell a failed or forced cleanup from the attempt), and Skip; ReportResult gained Platform; ReportSkip and AttemptCleanup are .NET-only. The run id is a UUID version 7, one per process. A result id is the full 64-hex SHA-256 that its directory name is the head of (the port’s digest, not upstream’s). testId is file::title as AttemptReport.TestId spells it.
• Fields the port cannot fill, and what the wire says: a judgment step that failed before it read a screen has observationRevision unrecorded (upstream requires one; the revision, b1, b2, and so on, is minted for each capture of the attempt by AttemptScope.ObserveAsync and kept as Observation.Revision) and its explanation reads empty when the judge gave none; model.endpoint is provider-default, adapterVersion the package version, policyVersion unversioned, tokenAccounting provider, peakTokensPerCall 0, and provider and model are split at the first slash of IAgentModel.Name, unknown when the name has none; cache.recordedBy is dropped when the name has no slash; cache.entry is dropped unless it is a SHA-256; a skip is always explicit (a Skip call); visionInput, visionDegraded, visionOnly, and the tool-proposal event kind have no port; a finding’s artifactId is left out, since the port names its screenshot by path; an artifact is complete only for a log whose file was read, and every video and screenshot is incomplete (pixels cannot be proven free of a secret); the size and sha256 come from the file under the results directory when the report is written; serialGroups is [] and carried is absent; a target’s index is the order its first attempt reached the report. Results sort by file, declaration index, target index, then, since the port declares no index, by source line, title, and id, so the order does not depend on which attempt finished first.
• Tests: ReportStatus/ holds report-status.test.ts: ResultTagsTests, ResultSourceTests, RunStatusDerivationTests (the three non-serial cases), and RunLimitsTests (the default-limits case), each asserting the document is valid report-1 as upstream’s build does, and ErrorStacksTests (partial: no serial member; the port’s error keeps no stack, so the test throws a real exception and checks that neither its stack nor its path reaches the document). Not ported: the three serial-group cases (“is blocked when the failing member of the last attempt carries a blockable code”, “stays failed when the failing member of the last attempt failed on the product”, “reads the last attempt, not an earlier blocked one”; the port has no serial groups), and “fills the block from the largest per-agent values” (the config rejects those keys). Vcs/DetectVcsTests has all three it of vcs.test.ts. .NET-only: ReportParts/ReportPartsTests (upstream reaches these blocks only through whole runs), ReportJson/ReportFileTests (the file as sessions write it, the artifact digest, redaction, the surrogate cut), and the helper GitRepo (upstream’s tests/helpers/git-repo.ts).
• Tests: SchemaFixtures/ReportV1SchemaTests has every it of upstream’s describe.each(schemas) for report-v1; SchemaFixtures/SchemaFixturesTests has the first top-level it (a schema for every fixture pair); the second (it publishes every schema) is partial: it reads the project file’s packed items and the glob, not package.json, and dotnet pack was checked by hand to hold the schema and no fixture. ReportWrite/WriteJsonReportTests has both it of report-write.test.ts. .NET-only: the two-space/newline test and the ToPosixPath test.
• observationTruncated (upstream 82039ad, #1301) is ReportStep.ObservationTruncated, written by ReportWire.Step when true and absent otherwise. ObservationFeed marks it for any look of the step that a capture or byte limit cut, including one a settled look took before its last, and ObservationFeed.ReportDetails() is upstream’s reportDetails. A judgment records it in Agent.LookAsync, where upstream does it in Invocation.observe. Tests: AgentObservationReport/AgentObservationReportFieldsTests ports agent-observation-report.test.ts, both partial (Partial_ prefix): the port has no fixture-engine runProject, so each test runs its own session over an engine wrapper that marks every observation truncated and validates its own report.json, where upstream runs one project of both tests and validates its report. SchemaFixtures/ReportV1SchemaTests has the boolean/string/absent it. Not ported: the sdk-types.ts check (the C# type makes it).
• Tests that read the report: a test whose upstream it reads outcome.report gives its session a results directory (ReportDirectory, .NET-only, in ReportJson/) and reads report.json after the session is disposed, checked against the schema. They are AgentAct/ (the hand-rolled executor, the default tool loop, and the blocked verdict), EngineTargets/, CacheModelProvenance/, ScriptedActions/, TruncatedSecrets/, TransformedSecrets/, and AgentPolicy/ASecretTheObservedScreenShowsTests. What the report still lacks keeps a test partial: the run’s exitCode is 0, 1, or 130 (a blocked run reads 1, where upstream reads 2); an engine has no name besides its platform (targets[].engine.name is the platform); an action an executor takes records no engine event; app.restart and app.clearState record the reopen as a step of its own; click and dblclick are the tap and doubleTap steps; a step whose executor reports no model reads unknown where upstream names the executor. Traces/TracesTests stays partial: the lead it reads is in upstream’s summary.md, and the port has no summary reporter.

App and browser fixtures

App.RestartAsync closes the tab and opens a new one in the same browser context, so cookies and storage survive. App.ClearStateAsync replaces the context with a clean one. Both then reopen the app at its base URL, as upstream, and keep a viewport set with Browser.SetViewportAsync. Without a base URL they leave a blank page.

Differences from upstream:

• app.open, back, forward, and reload wait for load within the action timeout. Upstream waits within the test timeout.
• Browser.GotoAsync(url, GotoOptions?) resolves the URL against the base URL like App.OpenAsync, with the same scheme rule. GotoOptions.WaitUntil is Load (the default), DomContentLoaded, or NetworkIdle, and Timeout defaults to the action timeout. Upstream’s default is the test timeout.
• Browser.WaitForURLAsync is ToHaveURLAsync as a wait, as upstream, so it verifies the previous act and fails with expect.toHaveURL failed. It takes a string, resolved against the base URL, or a Regex, and polls until the assertion timeout or the given timeout. ToHaveURLAsync takes a string or a Regex instead of upstream’s string | RegExp. A string is compared after both sides go through System.Uri (not WHATWG), which agrees with Chromium on the default port, the trailing slash, dot segments, and percent-encoding. A Regex is tested against the URL as Chromium serializes it.
• Browser.WaitForResponseAsync takes upstream’s glob string or a .NET Regex (upstream an ECMAScript RegExp), and the timeout as a TimeSpan?. As upstream, the timeout bounds the match only; WebResponse.TextAsync and JsonAsync<T> wait for the body up to the action timeout, which is fixed for the session (upstream reads the budget when the body is read). A failed match is OPERATION_TIMEOUT with the port’s message form (waitForResponse: <Playwright message>).
• Browser.EvaluateAsync<T> takes the script as a string (an expression, or a function source that is called with the optional argument) and deserializes the JSON result to T. A throwing script fails with EVALUATE_FAILED and the message it threw, a result that is not JSON (such as Infinity) with INVALID_ARGUMENT, and a browser that closes under it with ENGINE_FAILURE.
• Browser.RouteAsync takes the same glob string or Regex as WaitForResponseAsync, and a handler that returns a Task. WebRoute has Request, FulfillAsync (RouteFulfillResponse: Status, Headers, ContentType, Json, Body, Path), ContinueAsync (RouteContinueOverrides: Url, Method, Headers, PostData), FallbackAsync, and AbortAsync, with upstream’s checks, codes, and messages. UnrouteAsync removes the routes added with an equal glob, or a Regex with the same source and options. UnrouteAllAsync removes every route added through the browser and keeps the configured site headers. Path resolves against the project root (E2ESessionOptions.ProjectRoot, which E2ETest sets to the config’s directory, else the working directory). Differences: options are typed, so an unknown key, a non-string header value, and an abort error code do not compile instead of failing with INVALID_ARGUMENT; Json is serialized with System.Text.Json web defaults, and null means no JSON body; a handler error raised after the test’s last browser call is dropped (upstream waits for running handlers and fails the attempt with it).
• Browser.AddInitScriptAsync takes source or a WebInitScript, and a JSON argument only with WebInitScript.FromFunction, the function’s JavaScript source.
• Fixture calls are not recorded as harness steps.
• Navigation URLs resolve with System.Uri, not WHATWG. As upstream, only http:, https:, and the exact about:blank are admitted. But a tab or newline inside a URL is escaped, not stripped, so view-\tsource:file: opens as an http path under the base URL (or fails with APP_URL_REQUIRED without one) instead of POLICY_DENIED. A percent-encoded unreserved character in the path is decoded (%66ile: becomes file:). Neither opens a scheme other than http(s).

DocumentEngine supports app.back (it rebuilds the previous route), app.restart, and app.clearState (both drop the history and reopen the base URL). It has no viewport: DocumentPage.OnScroll and DocumentElement.OnScroll let a page load more rows when it is scrolled. It has no browser, so every Browser member fails with UNSUPPORTED_CAPABILITY. A custom engine opts in by implementing IBrowserSession, and by overriding the default BackAsync, RestartAsync, and ClearStateAsync on IEngineSession, which otherwise fail with UNSUPPORTED_CAPABILITY.

Models

Upstream takes any AI SDK model instance. JSON cannot hold one, so agents.<name>.provider names the client, and code builds one with ModelProviders, Subscriptions, or a client’s options. Each client speaks its vendor’s wire protocol directly:

Upstream	provider	.NET
@ai-sdk/openai chat, @ai-sdk/openai-compatible, Ollama	openai, openai-compatible	OpenAiCompatibleModel
@ai-sdk/openai Responses, @ai-sdk/azure	openai-responses, azure	OpenAiResponsesModel (ModelProviders.AzureOpenAi)
@ai-sdk/anthropic	anthropic	AnthropicModel
@ai-sdk/google	google	GoogleModel
@ai-sdk/amazon-bedrock	bedrock	BedrockModel, signed with SigV4 or a Bedrock API key
@ai-sdk/xai	xai	OpenAiCompatibleModel at api.x.ai
@openrouter/ai-sdk-provider	openrouter	OpenAiCompatibleModel at openrouter.ai
gateway() from ai	gateway	OpenAiCompatibleModel at ai-gateway.vercel.sh
chatgpt()	chatgpt	Subscriptions.ChatGpt
copilot()	copilot	Subscriptions.Copilot (CopilotModel)
grok()	grok	Subscriptions.Grok
opencodeConsole()	opencode-console	Subscriptions.OpenCodeConsole (OpenCodeConsoleModel)

Keys come from the environment variable the AI SDK reads (OPENAI_API_KEY, ANTHROPIC_API_KEY, GOOGLE_GENERATIVE_AI_API_KEY, AZURE_API_KEY, XAI_API_KEY, OPENROUTER_API_KEY, AI_GATEWAY_API_KEY, AWS_*), or the one apiKeyEnv names.

Subscriptions follow upstream: the same four logins (openai PKCE on port 1455 or --device, github-copilot from gh or an OAuth App’s device flow with enterprise hosts, opencode-console and spacexai device flows), the same credentials file ($XDG_CONFIG_HOME/e2e/oauth.json, mode 0600, locked and replaced atomically, unknown entries kept), E2E_OAUTH_CREDENTIALS, refresh two minutes ahead of expiry with one refresh shared per file (a cancelled call stops waiting for it, and the refresh finishes for the others), a retry after a 401, Copilot’s chat-or-Responses choice from the plan’s listing, OpenCode Console’s per-model protocol from the workspace config, and OPENCODE_API_KEY, which replaces the login for the model and for e2e models opencode-console; bare e2e models also lists OpenCode Console when only the key is set, and a missing login’s message names the key. The OpenCode Console model always asks the Anthropic route for 8192 output tokens, as upstream does when a call sets no cap. A missing or rejected login fails the step with MODEL_PROVIDER_FAILED, blocked, naming the OAuthException code (NOT_LOGGED_IN, LOGIN_REQUIRED). Tests: oauth/errors.test.ts is OauthErrors/ (DescribeResponseTests: both it of describeResponse; ReadJsonBodyTests: the it of readJsonBody, which is TokenEndpoint.ReadJsonBodyAsync) and oauth/token-endpoint.test.ts is OauthTokenEndpoint/RequestTokensTests (all 3 it).

Differences:

• Requests are not streamed. The Codex backend only streams, so its event stream is folded back into the final response, as upstream folds it
• The Vercel AI Gateway and OpenRouter are reached over their OpenAI-compatible chat API, so a gateway model gets no Anthropic cache breakpoints or OpenAI cache key. The gateway reads VERCEL_OIDC_TOKEN from the environment but does not run vercel env pull itself
• Gemini tool schemas go as parametersJsonSchema, unconverted. Gemini 3 thought signatures are kept per model instance and sent back with their function calls
• A Responses model (OpenAI, Azure, and the subscription backends) replays its reasoning on the next turn as upstream does: nothing is stored, so the request asks for reasoning.encrypted_content and sends each encrypted reasoning item back inline, ahead of the turn’s text and tool calls (all reasoning items first, not interleaved as returned). The request asks for it from every model except those named gpt-3*, gpt-4*, chatgpt-4o*, and gpt-5-chat*, as the AI SDK does. Anthropic replays its thinking blocks, signature and redacted data included, ahead of the turn’s text and tool calls, and chat completions replay reasoning_content (read from reasoning_content or reasoning) on the assistant message, as the AI SDK does. Google replays its thought parts (thought: true, with their thoughtSignature) and Bedrock its reasoningContent blocks (reasoningText with its signature, or the redacted content) the same way, ahead of the turn’s text and tool calls. A Gemini text part with a thoughtSignature and no thought: true keeps its text but loses the signature (#684). A Bedrock request is signed again for every attempt (ModelHttp.SendAsync takes a sign callback), as the AI SDK signs inside each fetch, so a retry after a long wait carries a fresh x-amz-date
• On Anthropic, the tool loop stops replaying the thinking from a message it rewrote since the last request (an executor’s PrepareMessages changed it, or the history was shrunk after a context overflow) on, as upstream does, since Anthropic binds each thinking block to the prefix before it. The loop compares each message by value with what the last request sent (MessageShape), drops the reasoning with ProviderHints.AfterEdit (upstream’s afterEdit, taking the model, since the port has no providerHints(model) factory; its SpeaksAnthropic check reads the model object where upstream’s reads a provider and model id), and notes it on the turn. The drop is permanent for the step, and reasoning produced after the rewrite stays. Only AnthropicModel and such a BedrockModel do this: a Bedrock model whose id names Anthropic behind an optional region prefix (eu.anthropic.claude-…) does it too, as upstream’s speaksAnthropic recognizes it (the id is read off the BedrockModel; a Bedrock application inference profile ARN is not recognized, as upstream). The port has no Vertex or OpenRouter client that replays Anthropic thinking, and it has no per-turn elision of superseded screens, so those two rewrites are the only ones. The part of the change that keeps the system prompt fixed across a switch to auto has nothing to do here, since the port sends tool_choice: auto from the first turn
• azure uses the Responses API at /openai/v1 with api-version=v1, as the AI SDK’s azure() does. Bedrock signs with AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY (and AWS_SESSION_TOKEN) when both are set and otherwise resolves the AWS credential chain (shared profile via AWS_PROFILE or the default profile, SSO, container and instance roles) with the AWS SDK for .NET (AWSSDK.Core), as the AI SDK’s amazon-bedrock provider does with the AWS SDK for JavaScript. Upstream has no symbol for it, since the AI SDK owns the chain, so AwsCredentialChain and BedrockModelOptions.CredentialChain (internal, a test seam) are .NET-only. A chain that finds nothing ends the step with MODEL_PROVIDER_FAILED; the chain’s own message is redacted
• A model request the provider refuses as too large ends with CONTEXT_OVERFLOW, blocked, and names the same status and body. The port uses upstream’s patterns for the refusal (every provider’s wording, any HTTP 413, a throttling text vetoed) on the message and the body. As upstream does, agent.act then shrinks the step’s history once and sends the request again: every superseded screen is elided, and any text past 16,384 characters is cut to its head with a notice. The refused request spends no model call, and the retry runs through the turn’s preparation again, so an executor’s PrepareMessages sees the shrunk history, as upstream’s prepareMessages does. A second refusal, or a first one with nothing to shrink, ends the step and says which. The port has no screenshots and no earlier budget for stale screens, so a screen is elided only at a refusal.
• A model that refuses a request (finish reason content-filter) ends the step with MODEL_REFUSED instead of MODEL_PROVIDER_FAILED, as upstream does, and the step cache reads it as a failed flow like MODEL_OUTPUT_INVALID. The port has no AI SDK to map each provider’s stop word, so its providers do: refusal (Anthropic), content_filter (chat completions, and Responses incomplete_details), SAFETY, RECITATION, BLOCKLIST, PROHIBITED_CONTENT, SPII, and IMAGE_SAFETY (Gemini, and a prompt it blocks with no candidate, by promptFeedback.blockReason), guardrail_intervened and content_filtered (Bedrock). A refused turn that still called a tool is kept, as upstream keeps a usable answer. The message names the model without a provider, because an IAgentModel has only a name, and quotes the provider’s word redacted and cut at 128 characters
• A model GitHub Copilot does not serve to the account (model_not_supported, a 4xx answer) ends the step with MODEL_UNAVAILABLE, blocked, naming the model and the causes (the plan, the Copilot settings, an unknown id), as upstream’s copilot() does. Upstream also marks the failure so its runner reports it once and stops the run; the port has no run-level error list and no workers, so every step that calls the model fails the same way. A model on the wrong endpoint (unsupported_api_for_model) stays a MODEL_PROVIDER_FAILED
• A model request answered with an error status ends with MODEL_PROVIDER_FAILED and names the HTTP status and the response body, secrets redacted, cut at 1 KB. The port never parses a provider’s error body, so it quotes every one. Before it fails, the request is sent again up to 5 times (upstream’s TRANSPORT_RETRIES, ModelHttp.TransportRetries) after a network failure or a status the AI SDK treats as retryable (408, 409, 429, and every 5xx), waiting the provider’s retry-after-ms or retry-after when it is under a minute (or under the backoff) and otherwise 2 seconds doubled each time; the failure is the last attempt’s. A request that got no response in 120 seconds is not sent again, and the step’s deadline or a cancel ends the wait. The port has no forced tool_choice to retry with auto: every request already sends auto
• Every model request sends upstream’s identity headers, replacing the same headers set on the client, but names this port so its traffic is not counted as upstream’s: User-Agent: e2e-dotnet/<version> (<platform>; <arch>) (the NuGet version; the platform and arch by Node’s names), HTTP-Referer: https://github.com/hardkoded/e2e-dotnet, and X-Title: e2e-dotnet. Model requests append runtime/dotnet/<version> after it, where upstream has the AI SDK’s user agent. Login requests send the same user agent, as upstream’s do, and still name the e2e originator and referrer that upstream registered
• e2e login with no provider shows a numbered list instead of upstream’s picker. e2e models prints the same columns. e2e login prints the sign-in URL on a line of its own, as upstream does, with plain lines where upstream uses clack log steps
• A ModelMessage of the user role can carry Images (ModelImage: the bytes, a media type, and the size), and every provider sends them after the turn’s text: an image_url part with a data URL for chat completions, input_image for Responses, a base64 image block for Anthropic, inlineData for Google, and an image block with its format for Bedrock. The AI SDK builds these parts upstream, so the port writes each provider’s shape itself. A model that cannot read images answers with the provider’s error, which ends the step with MODEL_PROVIDER_FAILED. A judgment sends one when it asks for vision (see Agents); vision on act is not ported. Pixels.SizeForModel (internal) caps a screenshot’s long side at 768 pixels with upstream’s box filter and keeps the scale (image pixels per CSS pixel) in step
• EngineScreenshot.MaskedRegionCount counts the regions the engine covered, which upstream carries as maskedRegionCount on observed pixels: the web engine counts the secure fields of every frame, and the mobile engine the secure nodes it painted. It is null for an engine that cannot say. A secret fill, by a locator or by the agent, raises the attempt’s exposure, which withholds pixels from a model for the rest of the attempt; the port has upstream’s filled level only, because it has no engine-held secrets
• Not ported: e2e init writing the model config