Verity
Start free

Verity docs

Verity is CI-native (Path B): your tests run in your own pipeline and POST results to the ingestion API. We never host your browsers for paid usage.

1. Create a project & token

In the dashboard, create a project and copy its ingest token into your CI secrets as VERITY_INGEST_TOKEN.

2. GitHub Action

# .github/workflows/verity.yml
name: Verity
on: [pull_request, push]
jobs:
  verity:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      # Quick check (diff-aware subset) on PR commits — a fast sanity check.
      - uses: qashift/verity-action@v1
        if: github.event_name == 'pull_request'
        with:
          tier: fast
          token: ${{ secrets.VERITY_INGEST_TOKEN }}
      # Full regression on merge to main — the safety net. Never shortcut.
      - uses: qashift/verity-action@v1
        if: github.ref == 'refs/heads/main'
        with:
          tier: full
          token: ${{ secrets.VERITY_INGEST_TOKEN }}

3. CLI

npm i -D @qashifthq/verity

# Fast tier — only runs tests mapped to your changed files.
npx verity run --tier fast --token $VERITY_INGEST_TOKEN

# Full tier — the entire suite.
npx verity run --tier full --token $VERITY_INGEST_TOKEN

4. Ingestion API

The Action and CLI both call this. You can call it directly too:

POST /api/ingest
Authorization: Bearer $VERITY_INGEST_TOKEN
Content-Type: application/json

{
  "tier": "fast",                  // "fast" = Quick check, "full" = Full regression
  "trigger": "pr",
  "commit_sha": "abc123",
  "branch": "feature/checkout",
  "pr_number": 42,
  "changed_files": ["src/checkout.ts"],
  "results": [
    { "spec_path": "tests/checkout.spec.ts", "attempts": [false, true, true], "duration_ms": 4200 }
  ]
}

attempts is the per-rerun pass/fail array. Mixed results are classified flaky, all-false is a real failure, all-true passes.

5. Predictions

  • Regression-prone module flagging (Live) — files with a high historical regression rate are flagged in PR comments.
  • Flake vs. real-failure disambiguation (Live) — each failure shows its historical flake rate.
  • Cross-dimension risk correlation (Calibrating) — switches on after ~20 deploys of history.
  • Pre-merge deploy confidence (Calibrating) — a decomposed score, hidden until calibrated.

6. Self-healing

When a locator breaks, the runner sends the old selector + surrounding DOM to POST /api/heal. Verity proposes a corrected locator and logs it as a suggestion. It is never written into your source automatically — you review and apply the diff in the dashboard.

7. How tests are stored & run

Stored:each generated spec lives in Verity (Supabase), keyed to your project and org and protected by row-level security. You can view, version, and export them any time — they're yours.

  • Cloud (trial): generation and a capped number of runs execute on our infra so you can get the feel in minutes — no setup. Intentionally limited.
  • CI (the real thing): the CLI / GitHub Action runs the specs in your pipeline against your own environments, reruns failures to detect flakes, and posts results back. We never host your browsers or receive your secrets.

8. Export & the FDE workflow

AI gives you a first draft; an engineer makes it an asset. From a project, hit Export tests (or call the API) to download every spec plus a Page Object Model scaffold and Playwright config:

GET /api/projects/:projectId/tests/export
# → verity-<project>-tests.json
#   { files: [{ path: "tests/checkout.spec.ts", content: "..." },
#             { path: "pages/BasePage.ts", content: "..." },
#             { path: "playwright.config.ts", content: "..." }] }

On a paid plan, your forward-deployed engineer then:

  1. Restructures the flat AI specs into a maintainable Page Object Model under pages/.
  2. Commits them to your repo and wires the Verity GitHub Action.
  3. Maintains the suite as your product changes — while Verity self-heals locators, triages flakes, and predicts risk.

9. Open source

The core test selection, flake, and scoring engine (@qashift/verity-core) is MIT-licensed. The hosted dashboard, predictions, and team management are proprietary.