10 · Continuous integration

Dimension

10 · Continuous integration

Platform

Vitest + Playwright · via exo-verify

Vitest

Open-source test runner for typed unit and integration checks. Ecosystem standard · authority: Exo

Playwright

Open-source browser checks that exercise every Staging release before promotion. Core standard · authority: EXO

Status

exo-verify partly built on staging (2026-10-01): contract, CLI, build gates, 15-min live checks, pre-push hook; AI review waits on the AI gateway.

What it isCopy Anchor Link

Continuous integration means code checks and AI evals. The goal: every change is type-checked, tested, and clicked through in a real browser on Staging before anyone approves it (the stack description on exo.now). Today the type checks, tests, and live route checks run; the browser click-through is planned. A release is healthy, reviewable, and approved, not merely green in a provider dashboard. This dimension is about verifying code, not identity; sign-in lives under Identity. The service design is on the exo-verify page.

PlatformCopy Anchor Link

• Vitest (open source) runs typed unit and integration checks. In this repository, pnpm test:int runs Vitest with vitest.config.mts.
• Playwright (open source) drives browser checks against Staging. In this repository, pnpm test:e2e runs Playwright with playwright.config.ts.

The rest of exo-verify is first-party: a verify contract, Railway sandbox workers, live checks, and an AI review pass whose model calls go through exo-router.

Boundary and replaceabilityCopy Anchor Link

• The contract is the command list each repository declares (typecheck, lint, test, smoke). Exo runs what is declared, and a missing command is a visible gap, not a silent pass.
• The AI review layer uses an open-source engine or Exo's own prompts. Third-party review (Greptile Enterprise) stays a later option if Exo's review quality falls short.
• No GitHub Actions or GitHub-app bots: the owner is leaving GitHub.
• The AI eval format (what an eval declares, how it is scored, and where results are stored): TBD. ADR 0005 records AI review and agent-written targeted tests, not a separate eval contract.

Cost notesCopy Anchor Link

From ADR 0005 (2026-09-27), at ~300 agent commits per month:

Cost per verified commit is roughly $0.02–0.10, recorded per App so it appears in that App's cost to serve. Vitest and Playwright are open source.

StatusCopy Anchor Link

• Today in this repo: pnpm check (legacy-backend freeze, tsc, ESLint plus the UI-system gates, and Vitest integration tests), pnpm verify (adds the production build), and Playwright e2e via pnpm test:e2e.
• Built on staging (2026-10-01): the exo-verify contract, CLI, and results store; Railway build gates (Exo and Observatory enforce, Construct and Rampart report); 15-minute live smoke, drift, and domain checks; the pre-push hook; and the PR-Agent review pass through exo-router. Review is off until the Convex AI gateway is enabled. Measured: the Exo gate adds 42 s to each deploy. See exo-verify.
• Open source parts: PR-Agent (MIT), Vitest (MIT), Playwright (Apache-2.0; browser journeys not wired yet).
• Open question: does Entire expose webhooks or a checks API for triggering runs?