Quickstart: Reliability CI Pack

Run deterministic reliability suites and gate regressions before production deploys.

Why this exists

Agent failures are often intermittent in live systems. FakeMCP gives deterministic, replayable outputs so teams can run the same checks on every commit.

When to use this vs DIY script

Use FakeMCP when you need shared run history, assertions, replay fixtures, and CI-friendly reports. Use a DIY script when you only need one-off local experiments with no persistence or reporting.

Start local worker

pnpm install
pnpm --filter @fakemcp/mcp dev

Use this base URL in the commands below:

export BASE_URL="http://127.0.0.1:8787"
export PROJECT_TOKEN="dev-local-token"

1) Create a suite

curl -sS -X POST "$BASE_URL/v1/suites" \
  -H "Content-Type: application/json" \
  -H "X-Project-Token: $PROJECT_TOKEN" \
  -d '{
    "name": "agent-regression-baseline",
    "visibility": "private",
    "cases": [
      { "server": "github", "scenario": "rate_limit" },
      { "server": "slack", "scenario": "auth_expired" }
    ],
    "assertions": [
      { "field": "retryable", "op": "eq", "expected": true, "severity": "error" }
    ]
  }'

Save the returned id as SUITE_ID.

2) Create a run

curl -sS -X POST "$BASE_URL/v1/runs" \
  -H "Content-Type: application/json" \
  -H "X-Project-Token: $PROJECT_TOKEN" \
  -d "{
    \"source\": \"ci\",
    \"suiteId\": \"${SUITE_ID}\",
    \"seed\": \"release-2026-02-21\"
  }"

Save the returned id as RUN_ID.

3) Fetch status and summary

curl -sS "$BASE_URL/v1/runs/${RUN_ID}" \
  -H "X-Project-Token: $PROJECT_TOKEN"

4) Download machine-readable reports

curl -sS "$BASE_URL/v1/runs/${RUN_ID}/report?format=json" \
  -H "X-Project-Token: $PROJECT_TOKEN"
curl -sS "$BASE_URL/v1/runs/${RUN_ID}/report?format=junit" \
  -H "X-Project-Token: $PROJECT_TOKEN"

5) Create replay fixture

curl -sS -X POST "$BASE_URL/v1/replays" \
  -H "Content-Type: application/json" \
  -H "X-Project-Token: $PROJECT_TOKEN" \
  -d "{
    \"runId\": \"${RUN_ID}\"
  }"

The replay response includes replayId, fixtureR2Key, and shareSlug.