Paidwen

paidwen.yml tells Paidwen how to start your app and which journeys to replay.

paidwen.yml sits at the root of your repository, on the default branch. Paidwen reads it on every pull request, always from the base branch, so a pull request cannot change its own verification.

npx paidwen init writes a first version. npx paidwen record <id> <url> adds a journey by clicking through your app. Your coding agent can also record a journey from one sentence, with the Paidwen MCP tools of npx paidwen mcp.

This example starts an app and replays four journeys.

yaml
version: 1

app:
  compose: docker-compose.yml
  service: web
  port: 3000
  path: /
  ready:
    path: /api/health
    timeoutSeconds: 240
  env:
    NEXT_PUBLIC_FEATURE_FLAGS: ""
  secrets: [PAIDWEN_STRIPE_TEST_KEY]
  setup:
    - node dist/seed.js
  setupService: api

services:
  email: mailpit
  stripe: mock
  stripeWebhookUrl: http://api:4000/webhooks/stripe
  storage: mock
  ai: mock

journeys:
  - id: signup
    name: Sign up and reach the dashboard
    paths: ["apps/web/app/signup/**", "apps/api/src/auth/**"]
    steps:
      - goto: /signup
      - fill: { field: Company name, value: Paidwen test }
      - fill: { field: Email, value: "{{email}}" }
      - fill: { field: Password, value: "{{password}}" }
      - click: Create account
      - email: { to: "{{email}}", click: Confirm my email }
      - expect: { url: /dashboard, text: Welcome }

  - id: subscribe
    name: Subscribe to Pro with a test card
    account: true
    steps:
      - goto: /login
      - fill: { field: Email, value: "{{account.email}}" }
      - fill: { field: Password, value: "{{account.password}}" }
      - click: Log in
      - goto: /billing
      - click: Subscribe to Pro
      - click: Subscribe
      - expect: { url: /billing, text: Pro }

  - id: export
    name: Export invoices as CSV
    account: true
    steps:
      - goto: /login
      - fill: { field: Email, value: "{{account.email}}" }
      - fill: { field: Password, value: "{{account.password}}" }
      - click: Log in
      - goto: /invoices
      - download: { click: Export CSV, name: "*.csv", contains: INV }

  - id: sort
    name: Sort the invoices by amount
    account: true
    goal: Log in, open the invoices and sort them from the largest amount.
    expect: { text: "$5,936.40" }

model:
  name: claude-opus-5-5
  maxCalls: 40

settings:
  failClosed: false
  previews: auto
  agentsOnly: false
  reviewMinutes: 15
  budgetMinutes: 20

version is always 1.

Paidwen refuses any other value.

app tells Paidwen how to start your application.

The application starts inside the disposable environment. Every key of app is optional.

  • compose: the compose file to start. When absent, Paidwen uses dockerfile if it is set, then looks for docker-compose.yml, docker-compose.yaml, compose.yml and compose.yaml at the root, then for a Dockerfile.
  • dockerfile: the Dockerfile to build as a single service named app. It wins over a compose file at the root. With build, you choose its stage, its context and its arguments. When the app declares a Postgres URL, Paidwen adds a disposable Postgres database: see services.database.
  • service: the compose service the browser opens. When absent, Paidwen takes a service that publishes a port, skips databases, caches and mail catchers, prefers a service built from the repository, then the names web, app, frontend, site or client.
  • port: the container port of that service. When absent, Paidwen reads it from the service's ports or expose, or from the EXPOSE lines of the stage it builds.
  • path: the first page. Default /.
  • ready.path: the path Paidwen polls until it answers, before the journeys start. Default: path. Any HTTP answer counts, even a 404: the server is up. Paidwen stops waiting as soon as the app container exits.
  • ready.timeoutSeconds: how long to wait for the application. Default 240. Paidwen does not wait for the healthcheck of the app container: during this time, a failing healthcheck cannot stop it.
  • env: literal environment variables given to every service of your compose file.
  • unset: variables removed from every service, whatever their source: your compose file, an env_file, env or Paidwen. For example unset: [PUBLIC_URL].
  • secrets: the names of the secrets Paidwen forwards to your application, among the fixed names below. Nothing else is ever forwarded.
  • services: the compose services to start, for example [web, worker]. Paidwen starts them with every service they depend on, and builds nothing else. Default: every service without a profile.
  • profiles: the compose profiles to turn on, as docker compose --profile does. A service with a profile starts only when one of its profiles is listed, or when services names it.
  • build: builds a service from your repository instead of pulling its image, with service, context, dockerfile, target and args. Paths start at the root of the repository. Every service that uses the same image is built the same way. Without service, it applies to the app service. Give a list to build several services.
  • images: the image a service runs instead of its build or its image in the compose file, by service. It can be an image your own CI builds for each commit, for example images: { web: "ghcr.io/acme/web:{{sha}}" }, or the new name of an image that left its registry. {{sha}} is the commit Paidwen verifies and {{shortSha}} its first seven characters. Paidwen waits for the image as long as settings.buildTimeoutMinutes allows. The runner must be able to pull it.
  • setup: commands run inside a container after the application answers and before the journeys, for example a seed script. Each command has five minutes. A command can use {{account.email}}, {{account.password}} and {{secrets.NAME}} with the fixed names below. Paidwen quotes each value for the shell, so the command writes the variable without quotes, for example node seed.mjs --email {{account.email}}. Paidwen hands the command to the shell through its standard input, never through the arguments or the log. A command whose secret is not set is skipped with a warning.
  • setupService: the service that runs the setup commands. Default: service.
  • setupTarget: a stage of the Dockerfile of that service, for example the stage that keeps the development dependencies. Paidwen builds it next to the app and runs each setup command in a new container of it.

Secrets have fixed names.

Secrets live in the GitHub environment named paidwen of your repository. Paidwen reads only these names:

  • PAIDWEN_TEST_LOGIN and PAIDWEN_TEST_PASSWORD: the test account that journeys with account: true use as {{account.email}} and {{account.password}}.
  • PAIDWEN_STRIPE_TEST_KEY: your Stripe test secret key, used only when services.stripe is test.
  • PAIDWEN_ENV: a dotenv block (one KEY=value per line) injected into every service. Use it for the test values your application needs.
  • PAIDWEN_MODEL_KEY: your own Anthropic API key, used only by the optional model layer: see the section model. Only the engine reads it, never your application.

Pull requests from forks run without any secret.

Paidwen sets only the variables your app declares.

Inside the disposable environment, your application must talk to the mock services and must build links that the browser can open. Paidwen therefore sets the variables below, but only those your app declares: in the environment of your compose file, in an env_file, in a .env.example file of the repository, or in an ENV or ARG line of a Dockerfile it builds. They win over the values of your compose file. app.env and PAIDWEN_ENV win over them, and app.unset removes any of them.

  • The application URL as seen from the browser: APP_URL, PUBLIC_URL, BASE_URL, SITE_URL, NEXT_PUBLIC_APP_URL, NEXT_PUBLIC_SITE_URL, NEXTAUTH_URL, AUTH_URL, VITE_APP_URL, WEBAPP_URL, NEXT_PUBLIC_WEBAPP_URL, BETTER_AUTH_URL, WEB_URL, APP_BASE_URL, FRONTEND_URL, PUBLIC_APP_URL.
  • Email through Mailpit: SMTP_HOST, MAIL_HOST, EMAIL_HOST, EMAIL_SERVER_HOST, EMAIL_SMTP_HOST, NEXT_PRIVATE_SMTP_HOST and SMTP_SERVER receive paidwen-mailpit; SMTP_PORT, MAIL_PORT, EMAIL_PORT, EMAIL_SERVER_PORT, EMAIL_SMTP_PORT and NEXT_PRIVATE_SMTP_PORT receive 1025; SMTP_SECURE, NEXT_PRIVATE_SMTP_SECURE, SMTP_SECURE_ENABLED, EMAIL_USE_TLS, EMAIL_USE_SSL and EMAIL_SMTP_NO_TLS turn TLS off.
  • Stripe mock, as soon as your app declares one STRIPE_ variable: STRIPE_API_BASE, STRIPE_API_HOST, STRIPE_API_PORT, STRIPE_API_PROTOCOL, STRIPE_SECRET_KEY, STRIPE_PUBLISHABLE_KEY, STRIPE_WEBHOOK_SECRET.
  • AI mocks, as soon as your app declares one OPENAI_ or ANTHROPIC_ variable: OPENAI_BASE_URL, OPENAI_API_BASE, OPENAI_API_KEY, ANTHROPIC_BASE_URL, ANTHROPIC_API_KEY.
  • Storage mock, when your app declares an S3 endpoint and your compose file runs no storage server: the declared endpoints among AWS_S3_ENDPOINT, AWS_S3_ENDPOINT_URL, S3_ENDPOINT, S3_ENDPOINT_URL, STORAGE_S3_ENDPOINT and NEXT_PRIVATE_UPLOAD_ENDPOINT, plus AWS_ENDPOINT_URL_S3 that the AWS SDKs read, and the declared access keys and path style settings.
  • Disposable database: DATABASE_URL and every declared variable whose example holds a Postgres URL.

Your application reads the Stripe host from these variables. With the official Node SDK: new Stripe(key, { host: process.env.STRIPE_API_HOST, port: Number(process.env.STRIPE_API_PORT), protocol: process.env.STRIPE_API_PROTOCOL }), only when STRIPE_API_HOST is set.

Paidwen reads your compose file as docker compose does, and refuses what opens the machine.

The environment has no internet and no access to the machine that runs it. Paidwen copies your compose file into a safe version and refuses, with a message on the pull request: privileged, cap_add, devices, security_opt, pid, ipc, network_mode, userns_mode, extends, include, secrets, configs, bind mounts of paths outside the repository, build contexts outside the repository, and ${VARIABLE:?message} values that are still unresolved.

  • Named volumes, healthchecks, depends_on, profiles, env_file inside the repository and ${VARIABLE:-default} work as usual.
  • A bind mount of a path of the repository works, always read-only. When the path is missing from the repository, Paidwen skips the mount with a warning.
  • When an env_file is missing, for example a .env that git ignores, Paidwen reads the .env.example file next to it and says so in the log. When the .env file next to the compose file is missing, Paidwen reads the .env.example file there to resolve ${VARIABLE} values.
  • Published ports are ignored: the browser reaches your service by its name on the internal network.
  • Each service runs with no-new-privileges, 2 GB of memory and 2 processors at most. It keeps only the capabilities an image needs to start and drop its own privileges: CHOWN, SETUID, SETGID, SETPCAP, DAC_OVERRIDE, FOWNER, KILL and NET_BIND_SERVICE.

services lists the third parties Paidwen replaces.

  • email: mailpit (default) or none. With Mailpit, journeys can open the emails your application sends.
  • stripe: mock (default), test or none. The mock answers the Stripe API, accepts any price identifier, serves a Checkout page whose button reads Subscribe for a subscription and Pay for a one-time payment, as on Stripe, and sends signed webhooks to stripeWebhookUrl. With test, your application uses your Stripe test key from PAIDWEN_STRIPE_TEST_KEY.
  • stripeWebhookUrl: where the mock posts its webhooks, as seen from inside the environment, for example http://api:4000/webhooks/stripe.
  • storage: mock (default) or none. The mock is a small S3 server that keeps the files of the run in memory. It serves presigned links, browser uploads and multipart uploads, and creates a bucket on its first file. It replaces only an S3 endpoint your app declares, when your compose file runs no storage server of its own.
  • ai: mock (default) or none. The mock answers the OpenAI and Anthropic APIs with a fixed sentence.
  • database: auto (default), postgres or none. With auto, Paidwen adds a disposable Postgres 16 database when it starts a Dockerfile and your app declares a Postgres URL. With postgres, it always adds one. The database is postgresql://paidwen:paidwen@paidwen-postgres:5432/paidwen.
  • allow: extra hosts your application may reach. Empty by default.

journeys lists the user journeys Paidwen replays.

Each journey needs an id, a name, and either steps, or a goal with expect.

  • id: lowercase letters, digits and hyphens.
  • name: what the lead reads in the verdict. Write it as a sentence: Subscribe to Pro with a test card.
  • paths: file globs. A pull request that changes one of these files makes the journey a priority.
  • account: the journey uses the test account of the paidwen environment.
  • timeoutSeconds: the time the whole journey may take, without the time your model takes to answer. Default 120.
  • goal: the journey in one sentence. A journey with a goal and no steps needs expect. Your agent records its steps with npx paidwen mcp, as the next section explains, or the model layer records them with your own key: see the section model. Without steps and without a key, such a journey is not replayed.
  • expect: the final checks of the journey, with the keys of the expect step below. Paidwen runs them after the last step. A journey passes only when they hold, whoever wrote the steps.

Your agent records a journey from one sentence.

npx paidwen mcp gives your coding agent a browser: paidwen_record_start, paidwen_page, paidwen_click, paidwen_fill, paidwen_select, paidwen_check, paidwen_press, paidwen_goto, paidwen_expect, paidwen_record_save, paidwen_journey_replay and paidwen_browser_close. Each tool returns the page reduced to its active elements, one line per element with its role and its exact name.

The agent opens your running app, acts like a user, proves the end with paidwen_expect, and saves the journey in paidwen.yml with the steps below. paidwen_journey_replay then replays the saved steps from the start without any model, as Paidwen does on a pull request. The agent types the test account as {{account.email}} and {{account.password}}: start npx paidwen mcp with PAIDWEN_TEST_LOGIN and PAIDWEN_TEST_PASSWORD in its environment. The Paidwen plugin for Claude Code does all of this with /paidwen:record.

Each step is one action on one line.

  • goto: /path: open a page of your application.
  • click: Create account: click the element with that name. Paidwen looks for a button, a link, then any element with that text.
  • fill: { field: Email, value: "{{email}}" }: type in the field with that label, placeholder or name.
  • type: { field: Search, value: Acme, delayMs: 40 }: press the keys one by one, like a person, for a field that reacts to each key, such as suggestions or a code in several boxes. delayMs sets the pause between two keys, 40 by default.
  • select: { field: Client, value: Acme }: choose an option by its label, then by its value.
  • upload: { field: Contract, file: fixtures/contract.pdf }: attach a file of your repository to the file field with that label, or to the button that opens the file dialog. sample: pdf attaches a small sample file instead, and png, csv and txt work too. The file keeps its name and must stay inside the repository.
  • check: I agree: check a checkbox or a radio by its label.
  • press: Enter: press a key.
  • email: { to: "{{email}}", subject: Confirm, click: Confirm my email }: wait for the email sent to that address, then open the link whose text or address contains click.
  • download: { click: Export CSV, name: "*.csv", contains: INV }: click, wait for the download, then check the file name and its content.
  • wait: { text: Saved }, wait: { url: /dashboard }, wait: { seconds: 2 }.
  • expect: { url: /dashboard, text: Welcome, notText: Error, title: Dashboard }: check the page. Every key given must hold.
  • expect: { visible: { role: button, name: Sign }, hidden: Loading }: wait until an element shows, or until an element disappears.

Each target can also be a precise locator, the form npx paidwen record writes: { role: button, name: Pay }, { label: Email }, { placeholder: Search }, { text: Welcome }, { testId: submit }, { css: "#pay" }.

Templates insert values into the steps.

  • {{email}}: a fresh address for this run, delivered to Mailpit.
  • {{password}}: a strong password for this run.
  • {{account.email}} and {{account.password}}: the test account.
  • {{secrets.PAIDWEN_ENV}} and the other fixed secret names.
  • {{now}}: the current date and time.

A pull request that renames a control updates the target of its step.

Paidwen reads paidwen.yml from the base branch, so a pull request never changes the checks that judge it. A pull request that renames or moves a button, a link, a field or a page on purpose updates, in the same pull request, the target of each step that points to it: the role and the name of the control, or the path of a goto.

Paidwen replays the steps of the base branch first. When a step no longer finds its element or its page, Paidwen replays the steps that the pull request writes, under four rules:

  • the journey keeps the same number of steps, of the same kinds, in the same order;
  • each step keeps its typed value, its option and its file;
  • the expect steps and the expect of the journey stay identical;
  • the first changed step comes at or before the step that broke.

The journey passes as changed only when all its checks hold, and the verdict names each change, for example "Largest first" is now "Highest amount first". When the steps of the base branch fail on a check, Paidwen never tries the steps of the pull request: the journey breaks and blocks the merge. This needs no model and no key.

model sets the layer that records and repairs your journeys with your own model.

The model layer is optional. It is off until the GitHub environment paidwen holds PAIDWEN_MODEL_KEY, an Anthropic API key. Without it, every journey replays its steps exactly, the pull request updates the target of a renamed control itself, as the section above explains, and the verdict explains a break in one simple sentence, such as After clicking "Largest first", the page /invoices?sort=amount does not show "$5,936.40".

With the key, your model does four things during a verification:

  • It records a journey written as a goal. Paidwen starts your default branch, your model drives the browser until the goal looks reached, then Paidwen replays the recorded steps without the model and checks expect. The pull request then replays these same steps. The verdict gives them, ready to copy into paidwen.yml, so that the next verifications need no model for this journey.
  • It repairs a step whose element is gone. When a click, a field, a choice, a box or a download no longer finds its element on the pull request, your model reads the page and proposes the action that does the same thing, such as the same button under its new name. It never opens an address and never changes a check. The journey passes as changed only when every check after it holds; otherwise it breaks at the original step and blocks the merge. The verdict names the change and gives the new steps, to copy into paidwen.yml in the same pull request.
  • It explains a break in one sentence, from the page of the pull request and the page of the base branch at the same step.
  • It reads the title, the description and the commits of the pull request, and lists as not observed the visible changes they promise that no journey goes through. These never block, and never remove a journey.

Your model never sees your code: only the page reduced to its active elements, and the text of the pull request. It never decides that a journey passes: the checks do. The page and the pull request are data for it, never instructions.

The engine calls the model from the machine of your runner, and only the engine reads PAIDWEN_MODEL_KEY. Chromium and your application run in a network without internet: Chromium hands each question to the engine through a folder that only they share, and reads the answer there. No container of your application receives the key, and the network of your application stays closed. A pull request from a fork never receives the key.

  • name: the model, claude-opus-5-5 by default. Another recent Claude model works too, such as claude-sonnet-5-5 or claude-haiku-4-5. When the model declines a request, the API can hand it to another Claude model, billed at the price of that model.
  • maxCalls: the most model calls of one verification. Default 40.
  • maxTokens: the most tokens one verification sends and receives, cache reads included. Default 800000.

Once a ceiling is reached, the verification goes on without the model. The log of the run ends with the calls, the tokens and the cost of the verification on your key, and the dashboard shows them next to the verdict.

settings changes how Paidwen verifies.

  • failClosed: when Paidwen itself cannot verify, block the merge (true) instead of a neutral check (false, default).
  • previews: the animated preview in the pull request comment: auto (public repositories only, default), on, off.
  • agentsOnly: verify only the pull requests written by an agent. Default false.
  • reviewMinutes: the review minutes Paidwen counts as saved for each verified pull request, in the monthly report. Default 15.
  • engine: pin a version of the Paidwen engine.
  • budgetMinutes: the time all journeys may take together. Default 20. When the budget runs out, the journeys touched by the pull request have run first.
  • buildTimeoutMinutes: the time the build of your images may take. Default 15, at most 45. It is also the time Paidwen waits for the images of app.images.
  • parallelBuilds: build the images of several services at the same time. Default false: Paidwen builds them one after the other, so a runner needs the memory of one build only. Set it to true on a large runner.