Your conformance suite · services.dev

You don't write tests for your API. Your brand publishes its own suite, generated from the same registries every surface renders, so it covers every branch your Services can reach. When a Service changes, its suite changes with it.

What's in it

Each operation row is called with each live credential that may call it, once for every magic value its steps can reach. A magic value is a sandbox input that picks a branch without a person or an outside system: a party who signs, an outside system that rejects once, a review that flags. Here is lingua as File it somewhere leaves it:

Output

magic values: metadata.sandbox.outside = { reject: [missing_certificate] }, metadata.sandbox.outside = { reject: [a_code_no_route_names] }, metadata.sandbox.review = pass, metadata.sandbox.review = flag_once, metadata.sandbox.outside = ack, metadata.sandbox.outside = reject_once, metadata.sandbox.outside = never, metadata.sandbox.outside = alerts
$ curl http://localhost:8787/verify/suite.json
counted  {"rows":42,"cases":144,"skipped":54,"http":121,"mcp":23}

# service.order, the row × its live credential × each magic value its graph reaches
service.order · sk_test_ · metadata.sandbox.outside = { reject: [missing_certificate] }  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.outside = { reject: [a_code_no_route_names] }  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.review = pass  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.review = flag_once  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.outside = ack  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.outside = reject_once  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.outside = never  → OK 202, until delivered
service.order · sk_test_ · metadata.sandbox.outside = alerts  → OK 202, until delivered

# every case, by surface and kind
 63  http · row
 18  mcp · row
 23  http · auth
 13  http · leak
 11  http · next

Beside the row cases, each row gets the cases that prove the rules hold:

Kind	Proves
`row`	the call answers as its row says, and each magic value reaches its branch
`auth`	a key that isn't this brand's gets 401, and a key on a person's act gets 403
`leak`	a buyer reads no owner record on any job, list or event
`replay`	the same Idempotency-Key gets the same status and bytes, and emits nothing twice
`next`	every `next.tool` is in `tools/list`, and the first one runs unchanged
`protocol`	MCP negotiates its version and answers a bad tool or body with the protocol's codes

A case names everything it needs, so any runner can run it. One, whole:

Output

magic values: metadata.sandbox.outside = { reject: [missing_certificate] }, metadata.sandbox.outside = { reject: [a_code_no_route_names] }, metadata.sandbox.review = pass, metadata.sandbox.review = flag_once, metadata.sandbox.outside = ack, metadata.sandbox.outside = reject_once, metadata.sandbox.outside = never, metadata.sandbox.outside = alerts
# service.order · sk_test_ · metadata.sandbox.outside = reject_once
{
  "id": "service.order · sk_test_ · metadata.sandbox.outside = reject_once",
  "op": "service.order",
  "surface": "http",
  "kind": "row",
  "credential": "sk_test_",
  "magic": "metadata.sandbox.outside = reject_once",
  "service": "translation",
  "given": [
    {
      "call": {
        "method": "POST",
        "path": "/test_clocks",
        "body": {
          "frozen_at": "2026-10-25T14:02:11Z"
        }
      },
      "bind": {

Every magic value and the branch it picks is in Magic values.

Run it

Serve your brand with npx services.dev, then, in another terminal:

Output

magic values: metadata.sandbox.outside = { reject: [missing_certificate] }, metadata.sandbox.outside = { reject: [a_code_no_route_names] }, metadata.sandbox.review = pass, metadata.sandbox.review = flag_once, metadata.sandbox.outside = ack, metadata.sandbox.outside = reject_once, metadata.sandbox.outside = never, metadata.sandbox.outside = alerts, metadata.sandbox.talk = open, watch.url = https://fails-once.example.test/suite, watch.url = https://down.example.test/suite, watch.url = https://hooks.example.test/suite

$ npx services.dev verify
lingua · 121 HTTP cases and 23 MCP cases from 42 rows: each row by its live credentials and magic values, with its auth, audience, replay and next cases · http://localhost:8787/verify/suite.json
ran the HTTP cases

✓ service.order · sk_test_ · metadata.sandbox.outside = { reject: [missing_certificate] }    202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.outside = { reject: [a_code_no_route_names] }  202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.review = pass                                  202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.review = flag_once                             202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.outside = ack                                  202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.outside = reject_once                          202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.outside = never                                202 OK → delivered
✓ service.order · sk_test_ · metadata.sandbox.outside = alerts                               202 OK → delivered
✓ job.accept · sk_test_ · none                                                               200 OK
✓ job.accept · auth · a key not this brand's                                                 401 BLOCKED
✓ job.accept · leak · a buyer reads no owner record                                          200 OK
✓ job.accept · replay · the same Idempotency-Key                                             200 OK
✓ job.accept · next · every tool is listed and runs                                          200 OK
121 passed · 0 failed · 54 skipped

That excerpt shows the order row's cases, the five kinds on job.accept, and the end. A case that needs live mode or an outside system's own call is skipped on a test run and listed, never counted as a pass; --live runs them. The same suite's MCP cases run with --surface mcp, and both with --surface all:

Output

magic values: metadata.sandbox.outside = { reject: [missing_certificate] }, metadata.sandbox.outside = { reject: [a_code_no_route_names] }, metadata.sandbox.review = pass, metadata.sandbox.review = flag_once, metadata.sandbox.outside = ack, metadata.sandbox.outside = reject_once, metadata.sandbox.outside = never, metadata.sandbox.outside = alerts, metadata.sandbox.talk = open, watch.url = https://fails-once.example.test/suite, watch.url = https://down.example.test/suite, watch.url = https://hooks.example.test/suite
121 passed · 0 failed · 54 skipped
  a shape: its values are the Service's own: magic value metadata.sandbox.asks = { <step>, or <step>.<item>: after <n>d, or waive } · service.order
  a shape: its values are the Service's own: magic value metadata.sandbox.events = { <event>: { at, data } } · service.order
not in lingua's contract: service.waitlist, job.end, job.again, job.policy, job.signal, member.join, member.verify, member.availability, member.ledger.get, member.ledger.add, member.ledger.checks, member.missed, task.list, task.get, task.sheet, task.pass, task.claim, task.clear, task.packet, task.release, task.offer, task.attest, task.send_back, task.decline, task.recuse, task.extend, task.contest, task.message, subject.list, subject.get, member.subjects, subject.withdraw, clock.list, clock.get, clock.ack, clock.satisfy, clock.choose, clock.assign, subscription.list, subscription.get, subscription.cancel, member.login, ask.revoke
$ npx services.dev verify --surface mcp
lingua · 121 HTTP cases and 23 MCP cases from 42 rows: each row by its live credentials and magic values, with its auth, audience, replay and next cases · http://localhost:8787/verify/suite.json
ran the MCP cases
✓ mcp · initialize echoes a version it speaks                              200 OK
✓ mcp · tools/list is the buyer toolset                                    200 OK
23 passed · 0 failed · 54 skipped

verify exits 1 when a case fails, so it gates a deploy. In CI wires it in beside check.

Against any brand

The suite is public: services.dev verify --url <url> fetches a brand's /verify/suite.json and runs it against that brand. A buyer, an integrator or an outside reviewer gets the same verdict you do, from the same cases. With --live and a live key it runs the same cases with real money, inside the key's cap.

deploy runs this suite before a version reaches buyers: Deploy.

What's in it#

Run it#

Against any brand#

What's in it

Run it

Against any brand