CI on GitHub Actions

Fail pull requests automatically on assertion regressions.

CI: GitHub Actions Gate

Use this workflow to fail builds when error severity assertions fail in FakeMCP runs.

Why this exists

Most teams want a simple CI gate, not another custom harness. This pattern provides deterministic run creation, polling, and JUnit output in one workflow.

When to use this vs DIY script

Use FakeMCP CI gate when you need stable assertions and downloadable reports across pull requests. Use DIY scripts for single-project experiments where no shared reporting is needed.

Example workflow

name: fakemcp-reliability-gate

on:
  pull_request:
  push:
    branches: [main]

jobs:
  reliability:
    runs-on: ubuntu-latest
    env:
      BASE_URL: https://fakemcp.com
      SUITE_ID: suite_replace_me
      PROJECT_TOKEN: ${{ secrets.FAKEMCP_PROJECT_TOKEN }}
    steps:
      - uses: actions/checkout@v4

      - name: Create reliability run
        id: run_create
        run: |
          set -euo pipefail
          response="$(curl -sS -X POST "$BASE_URL/v1/runs" \
            -H "Content-Type: application/json" \
            -H "X-Project-Token: $PROJECT_TOKEN" \
            -d "{\"source\":\"ci\",\"suiteId\":\"$SUITE_ID\",\"seed\":\"${GITHUB_SHA}\"}")"
          echo "$response" > run.json
          run_id="$(node -e 'const b=JSON.parse(require("fs").readFileSync("run.json","utf8")); if(!b.id) process.exit(1); process.stdout.write(b.id)')"
          echo "run_id=$run_id" >> "$GITHUB_OUTPUT"

      - name: Poll run status
        run: |
          set -euo pipefail
          run_id="${{ steps.run_create.outputs.run_id }}"
          for i in {1..20}; do
            body="$(curl -sS "$BASE_URL/v1/runs/$run_id" -H "X-Project-Token: $PROJECT_TOKEN")"
            echo "$body" > run-status.json
            status="$(node -e 'const b=JSON.parse(require("fs").readFileSync("run-status.json","utf8")); process.stdout.write(String(b.status ?? ""))')"
            if [[ "$status" == "completed" || "$status" == "failed" ]]; then
              break
            fi
            sleep 1
          done

      - name: Download reports
        run: |
          set -euo pipefail
          run_id="${{ steps.run_create.outputs.run_id }}"
          curl -sS "$BASE_URL/v1/runs/$run_id/report?format=json" -H "X-Project-Token: $PROJECT_TOKEN" > fakemcp-report.json
          curl -sS "$BASE_URL/v1/runs/$run_id/report?format=junit" -H "X-Project-Token: $PROJECT_TOKEN" > fakemcp-junit.xml

      - name: Enforce gate
        run: |
          set -euo pipefail
          node -e 'const r=require("./fakemcp-report.json"); if ((r.summary?.failedCases ?? 0) > 0) { console.error("FakeMCP gate failed"); process.exit(1); }'

      - name: Upload JUnit artifact
        uses: actions/upload-artifact@v4
        with:
          name: fakemcp-junit
          path: fakemcp-junit.xml

Inline cases instead of suite

If you do not want persisted suites yet, replace the create payload with inline cases and assertions.

Repository merge gate

This repo also ships a full merge-gate workflow at /Users/michael/personal/fakemcp-com/.github/workflows/ci.yml that enforces:

Deployed environment checks are defined in: