--- title: "Quality" url: https://services.dev/docs/operate/quality captured: [operate-quality.txt] --- # Quality URL: https://services.dev/docs/operate/quality > Each review lens keeps its own numbers on the record, and the outside world's later answers count what each lens missed. Evals score a change against held-out jobs whose real outcome is known, and deploy --live refuses a version on which any lens finds fewer grounds. Quality answers one question: is the work good? It is read from the record, lens by lens, and checked against what the outside world later said. ## Two review patterns, each with its own numbers [Review steps](https://services.dev/docs/service/review.md) come in two patterns. **Adversarial** hardens: each lens attacks the draft and flags it back until it holds. **Affirmative** extends: each lens proposes an addition, an adversarial pass judges it, and only what survives is folded in. Each round writes `review.done` with every finding, so each lens has a find rate, a fix rate and a cost. When the outside world later answers, the record counts what each lens missed. api.lawyer's trademark response, with the trademark office's own answers on the same matters: ```bash npx services.dev quality trademark-response --since 90d --live ``` ```txt title="magic values: none" Trademark Office Action Response · live · 2026-08-04 → 2026-11-02 · 412 matters · rules 2026-10 review attack · adversarial · 3 lenses · 1.4 rounds a matter · AI $0.62 a matter likelihood of confusion found 61 · fixed 58 · the office later cited 4 it missed descriptiveness found 37 · fixed 37 · the office later cited 2 it missed specimen found 88 · fixed 86 · the office later cited 0 it missed people respond · attorney · changed the AI's draft on 23 of 412 (5.6%) · reserved by rules.ts: the line never moves outcome the office accepted 371 · refused 29 · 12 pending ``` The specimen lens finds what the office would cite, and the likelihood-of-confusion lens misses some. That is where to work. ## Score a change before it ships An eval is a set of held-out jobs whose real outcome is known: the buyer accepted, the person changed the draft, or the outside system accepted or refused, and on what grounds. Running one replays the AI steps of each job under a candidate version, in test mode, from the inputs the record hashed, and scores each lens against what really happened: ```bash npx services.dev evals create trademark-response --label office --since 180d --size 200 ``` ```txt title="magic values: none" ← evl_3Lq7 · 200 matters held out · labelled by the office's answer: 171 accepted, 29 refused on 46 grounds, each read from its office action ``` ```bash npx services.dev evals run evl_3Lq7 --against 2026-11-02.1 ``` ```txt title="magic values: none" evl_3Lq7 · 200 matters · live 2026-10-19.3 against 2026-11-02.1 · test mode, each input replayed by its hash · AI $131.20 lens grounds found false flags a matter likelihood of confusion 22 of 25 → 22 0.21 → 0.21 descriptiveness 9 of 11 → 9 0.08 → 0.08 specimen 6 of 6 → 6 0.11 → 0.11 genericness 0 of 4 → 4 — → 0.06 matter 37 of 46 → 41 of 46 · AI $0.62 → $0.71 a matter ✓ 2026-11-02.1 finds 4 more grounds and no lens finds fewer · eval.finished · deploy --live reads it ``` `deploy --live` reads the newest eval of each Service. A version on which any lens finds fewer labelled grounds than live, or the job's catch rate falls, is refused with `eval_regressed` and the lens that fell. An eval reads only your brand's own record, in test mode, under the uses your terms declare. Evals run on [Team](https://services.dev/docs/money/plans.md#team). Where a person still changes the work, and how a step moves from people to AI on the evidence this page's numbers feed, is [Move a step to AI](https://services.dev/docs/operate/line.md). ## Controls | Call (CLI) | API | Event | On the console | Plan | |---|---|---|---|---| | `quality ` | `GET /owner/quality?service=` | `review.done` | `/console/quality` | every plan | | `evals create --label