diff --git a/docs/productivity.md b/docs/productivity.md new file mode 100644 index 0000000..608e7e3 --- /dev/null +++ b/docs/productivity.md @@ -0,0 +1,84 @@ +# Productivity log + +How much of the agents' work reaches main, where the rest goes, and what was changed to improve it. +Each review records the readings, the changes made after them, and what to compare next time. + +Readings come from `td-server stats --hours N` (or `td-server stats json`). The numbers: + +- **submitted / merged**: functions submitted as code in the window, and how many of those merged + (byte-identical vs. peer-confirmed). `mergedShare` is the headline: merged / submitted. +- **attempt states**: where the submitted functions are now. `needs_cleanup`, `returned` and + `rejected` are losses; `withdrawn` (from 2026-10-18) means "claimed but not written", not a loss. +- **top reasons work came back**: the feedback summaries of the losses, grouped. +- **peer review**: verdicts in the window, functions decided by them, verdicts per decision, queue. +- **progress**: byte-identical functions and bytes over all functions (accepted "no decompile" + functions excluded). + +Compare windows of the same length with a similar number of contributors. + +--- + +## Review 1: 2026-10-18 (first day of collaboration) + +### Readings (24 h window, 19 contributors) + +``` +submitted functions: 8680; merged: 337 (3.9%; 206 byte-identical, 131 peer-confirmed); 14.0 merged/h +attempt states: approved=38 in_review=146 merged=337 needs_cleanup=6295 peer_review=756 rejected=244 returned=838 verifying=26 +peer review: 2088 verdicts, 322 functions decided, 6.5 verdicts/decision, 773 waiting +progress: 7193/42856 functions (16.8%), 747236/10520125 bytes (7.1%) +top reasons work came back: + 2176 no compiled function is annotated with // FUNCTION: fn_# + 2022 the submission changes functions it does not hold: fn_# ; claim them ... + 803 your change breaks # function(s) that matched before + 739 internal verifier error; a maintainer has been notified + 702 not byte-identical (score #), sent to peer review: ... (returned with their batch) + 243 rejected: the submission adds comments or strings that are not allowed + 177 compilation failed + 165 the annotated function was not compiled or has no address in the current version +``` + +Last 6 h of the same window (12 contributors): 2408 submitted, 49 merged (2.0%), 8.2/h; among +the losses 123 "byte-identical, approved automatically": good functions returned with their batch. + +### Diagnosis + +Agents produce a lot (about 27 functions per contributor-hour); the pipeline lost about 96%. + +1. **Unwritten functions counted as failures** (2176). `td submit` sends every held function; an + agent claims a group of up to 50, writes 10, and the other 40 failed with "no compiled function + is annotated" and were reserved to the author for 6 hours. +2. **Neighbour rule too strict** (about 2000). Any change to another function's compiled code in + the same file rejected the whole submission, even when nobody else worked on that function. +3. **One failure returned the whole batch at verification** (838 batches; the 702 + 123 good + functions above). Fixed earlier for peer review and merge, not for verification. +4. **Outage** (739 internal errors): the worker ran from a deleted release directory. +5. Peer review costs 6.5 verdicts per decided function (3 agreeing + mixed votes + 4th/5th for + auto-accept). Considered a score gate before peer review: rejected, the data shows reviewers + judge low-score ports "same" nearly as often as high-score ones (78-82% at 40-50%, 85-92% above), + so the score does not predict wrong logic. + +### Changes (deployed 2026-10-18) + +- Verification: a submitted function with no annotation anywhere in the submission is + **withdrawn** (back into the author's claim, no failure, no 6 h reservation). +- Verification: failing functions go back **one by one**; the rest of the batch continues and its + approved part merges without them (the batch is only returned when nothing is left on its way). +- Neighbour rule: a submission is refused only for changing functions **someone else** is working + on (held by another account, or in another account's submission on its way to main). Other + neighbours may change; their status is measured again; the regression check still protects + everything that already matches. +- Same day, earlier: verification compiles only the submitted units (header includers at merge), + merges reuse verification per unit, two lanes for small and big work, grouped claims (file, + class, neighbouring functions), bulk peer reviews with random spread, peer queue before + auto-accept, review-first rule above 100 waiting reviews, "no decompile" claims. + +### What to compare at the next review + +- `mergedShare` (was 3.9%) and merged/h (was 14.0 over 24 h, 8.2 in the last 6 h). +- "no compiled function is annotated" should be near zero; `withdrawn` takes its place. +- "changes functions it does not hold" should drop by most of its 2022. +- `returned` should drop; "byte-identical, approved automatically" should no longer appear among + the losses. +- Verdicts per decision (6.5) and the peer queue (773) after bulk reviews and the review-first rule. +- If the peer queue still grows: lower `peerReviews` from 3 to 2 (decision for the maintainer). diff --git a/src/server/cli.js b/src/server/cli.js index d287374..5d68106 100644 --- a/src/server/cli.js +++ b/src/server/cli.js @@ -11,6 +11,7 @@ import { openServer, startServer, DEFAULT_CONFIG } from './app.js'; import { Store, now } from './db.js'; import { Accounts } from './accounts.js'; import { agentGuidance, setAgentGuidance, resetAgentGuidance } from './guidance.js'; +import { productivity, formatProductivity } from './stats.js'; import { PEImage } from '../shared/pe.js'; const TEMPLATE_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', '..', 'template'); @@ -32,6 +33,7 @@ Usage: td-server [options] (all commands accept --data , defaul user token [label] create an API token for an account version add [--current] register a target executable version version set-current + stats [json] [--hours N] pipeline productivity (submitted, merged, losses, reviews) guidance show|set |off|reset policy note sent to agents with every work response version label readable name shown on the website version map --from @@ -71,6 +73,7 @@ async function main() { current: { type: 'boolean', default: false }, version: { type: 'string' }, from: { type: 'string' }, + hours: { type: 'string', default: '24' }, help: { type: 'boolean', short: 'h' }, }, }); @@ -82,6 +85,13 @@ async function main() { if (cmd === 'init') return init(dataDir); if (cmd === 'start') return startServer(dataDir, { role: sub ?? 'all' }); + if (cmd === 'stats') { + // Pipeline productivity (see docs/productivity.md). + const store = new Store(path.join(dataDir, 'td.sqlite')); + const p = productivity(store, Number(values.hours) || 24); + return console.log(sub === 'json' ? JSON.stringify(p, null, 2) : formatProductivity(p)); + } + if (cmd === 'guidance') { // Reaches running agents on their next call; no restart needed. const store = new Store(path.join(dataDir, 'td.sqlite')); diff --git a/src/server/merge.js b/src/server/merge.js index 74c5491..8e82613 100644 --- a/src/server/merge.js +++ b/src/server/merge.js @@ -249,7 +249,7 @@ export class Merger { noSplit(batch, reason) { this.store.run(`UPDATE batches SET result_json = json_set(COALESCE(result_json, '{}'), '$.noSplit', ?) WHERE id = ?`, reason, batch.id); // With a function sent back it can never merge as a whole, so it all goes back. - if (this.store.get(`SELECT 1 FROM attempts WHERE batch_id = ? AND state IN ('needs_cleanup', 'rejected', 'returned')`, batch.id)) { + if (this.store.get(`SELECT 1 FROM attempts WHERE batch_id = ? AND state IN ('needs_cleanup', 'rejected', 'returned', 'withdrawn')`, batch.id)) { returnBatch(this.store, batch.id, `a function of this batch was sent back and the rest cannot be merged without it: ${reason}`); return this.log.info(`batch ${batch.id} returned: ${reason}`); } @@ -299,7 +299,7 @@ export class Merger { // their source put back as on main); shared changes in the file may change their compiled code. // That is not a change to functions someone else holds; their status is measured again. if (parentId) for (const a of batchAttempts(this.store, parentId)) targets.add(a.symbol_id); - const outside = this.verifier.changedOutside(check, merged.files, targets); + const outside = this.verifier.changedOutside(check, merged.files, targets, batch.user_id); const confirmedIds = new Set(); const results = []; for (const a of attempts) { diff --git a/src/server/stats.js b/src/server/stats.js new file mode 100644 index 0000000..7e1d4c1 --- /dev/null +++ b/src/server/stats.js @@ -0,0 +1,63 @@ +// Pipeline productivity over a time window: how much submitted work reaches main and where the rest +// goes. `td-server stats --hours N` prints it; docs/productivity.md records the readings and what +// was changed in between, so reviews compare like with like. + +export function productivity(store, hours = 24) { + const since = Date.now() - hours * 3600_000; + const q = (sql, ...a) => store.get(sql, ...a); + const v = store.currentVersion(); + const submitted = q(`SELECT COUNT(*) AS n FROM attempts WHERE kind = 'code' AND created_at > ?`, since).n; + const byState = Object.fromEntries(store.all(`SELECT state, COUNT(*) AS n FROM attempts WHERE kind = 'code' AND created_at > ? GROUP BY state`, since).map((r) => [r.state, r.n])); + const merged = byState.merged ?? 0; + const mergedIdentical = q( + `SELECT COUNT(*) AS n FROM attempts WHERE kind = 'code' AND state = 'merged' AND created_at > ? AND json_extract(result_json, '$.feedback.target.status') IN ('exact', 'match_unresolved')`, + since, + ).n; + const reasons = {}; + for (const r of store.all(`SELECT json_extract(result_json, '$.feedback.summary') AS s FROM attempts WHERE kind = 'code' AND created_at > ? AND state IN ('needs_cleanup', 'returned', 'rejected')`, since)) { + const k = String(r.s ?? 'unknown').replace(/fn_\d+/g, 'fn_#').replace(/\d+(\.\d+)?%?/g, '#').replace(/(fn_#(, )?)+/g, 'fn_# ').slice(0, 70); + reasons[k] = (reasons[k] ?? 0) + 1; + } + const verdicts = q('SELECT COUNT(*) AS n FROM peer_reviews WHERE created_at > ?', since).n; + const decided = q( + `SELECT COUNT(DISTINCT a.id) AS n FROM attempts a WHERE a.state IN ('in_review', 'approved', 'merged', 'needs_cleanup') + AND EXISTS (SELECT 1 FROM peer_reviews p WHERE p.attempt_id = a.id AND p.created_at > ?)`, + since, + ).n; + const total = q( + `SELECT COUNT(*) AS n, COALESCE(SUM(a.size), 0) AS bytes FROM symbols s JOIN symbol_addresses a ON a.symbol_id = s.id AND a.version_id = ? WHERE s.kind = 'function' AND s.library = 0 AND s.skip_reason IS NULL`, + v.id, + ); + const done = q( + `SELECT COUNT(*) AS n, COALESCE(SUM(a.size), 0) AS bytes FROM func_status f JOIN symbol_addresses a ON a.symbol_id = f.symbol_id AND a.version_id = ? WHERE f.status IN ('exact', 'match_unresolved')`, + v.id, + ); + return { + hours, + contributors: q('SELECT COUNT(DISTINCT user_id) AS n FROM attempts WHERE created_at > ?', since).n, + submitted, + merged, + mergedIdentical, + mergedPeerConfirmed: merged - mergedIdentical, + mergedShare: submitted ? merged / submitted : 0, + mergedPerHour: merged / hours, + byState, + topReasons: Object.entries(reasons).sort((a, b) => b[1] - a[1]).slice(0, 8), + peer: { verdicts, decided, verdictsPerDecision: decided ? verdicts / decided : null, waiting: q(`SELECT COUNT(*) AS n FROM attempts WHERE state = 'peer_review'`).n }, + progress: { functions: done.n, functionsTotal: total.n, bytes: done.bytes, bytesTotal: total.bytes }, + }; +} + +export function formatProductivity(p) { + const pct = (x) => `${(x * 100).toFixed(1)}%`; + const lines = [ + `window: last ${p.hours} h, ${p.contributors} contributors`, + `submitted functions: ${p.submitted}; merged: ${p.merged} (${pct(p.mergedShare)}; ${p.mergedIdentical} byte-identical, ${p.mergedPeerConfirmed} peer-confirmed); ${p.mergedPerHour.toFixed(1)} merged/h`, + `attempt states: ${Object.entries(p.byState).map(([k, n]) => `${k}=${n}`).join(' ')}`, + `peer review: ${p.peer.verdicts} verdicts, ${p.peer.decided} functions decided, ${p.peer.verdictsPerDecision?.toFixed(1) ?? '-'} verdicts/decision, ${p.peer.waiting} waiting`, + `progress: ${p.progress.functions}/${p.progress.functionsTotal} functions (${pct(p.progress.functions / p.progress.functionsTotal)}), ${p.progress.bytes}/${p.progress.bytesTotal} bytes (${pct(p.progress.bytes / p.progress.bytesTotal)})`, + 'top reasons work came back:', + ...p.topReasons.map(([k, n]) => ` ${String(n).padStart(6)} ${k}`), + ]; + return lines.join('\n'); +} diff --git a/src/server/verify.js b/src/server/verify.js index 2d684c4..5a61c91 100644 --- a/src/server/verify.js +++ b/src/server/verify.js @@ -10,7 +10,7 @@ import { reconcile } from '../shared/reconcile.js'; import { STATUS } from '../shared/compare.js'; import { listSources, isUnit } from '../shared/project.js'; import { now } from './db.js'; -import { returnBatch, batchAttempts } from './batches.js'; +import { returnBatch, batchAttempts, settleBatch } from './batches.js'; import { analyzeInWorkers } from './analyze.js'; // A batch whose verification compiles more units than this is big (it changes a widely used header). @@ -245,7 +245,10 @@ export class Verifier { * part of the submission: a submission may only change the functions it holds the lease for. * Unknown on main (no fingerprint yet, e.g. before the first rebuild) is not counted. */ - changedOutside(check, files, targets) { + // With `userId`, only functions someone else is working on count (held by another account, or in + // another account's submission on its way to main). Other neighbours may change: their status is + // measured again, and the regression check still protects whatever already matches. + changedOutside(check, files, targets, userId = null) { const fileSet = new Set(Object.keys(files)); const seen = new Map(); for (const u of check.units) for (const f of u.functions) if (fileSet.has(f.file) && !targets.has(f.id) && f.codeHash) seen.set(f.id, f); @@ -255,7 +258,17 @@ export class Verifier { if (!main) out.push({ id, file: f.file, why: 'new or newly annotated' }); else if (main.code_hash && main.code_hash !== f.codeHash) out.push({ id, file: f.file, why: 'compiled code changed' }); } - return out; + if (userId == null) return out; + return out.filter((o) => this.workedOnByOthers(o.id, userId)); + } + + workedOnByOthers(symbolId, userId) { + return !!this.store.get( + `SELECT 1 FROM tasks t WHERE t.symbol_id = ? AND ( + (t.state = 'leased' AND t.lease_user != ?) + OR EXISTS (SELECT 1 FROM attempts a WHERE a.task_id = t.id AND a.user_id != ? AND a.state IN ('queued', 'verifying', 'peer_review', 'in_review', 'approved', 'merging')))`, + symbolId, userId, userId, + ); } /** @@ -304,10 +317,18 @@ export class Verifier { // units that only include a changed header are compiled when the batch merges. const { check, rec, changedHeaders } = await this.check(merged.files, `batch-${batch.id}`, { dependents: false }); const targets = new Set(attempts.map((a) => a.symbol_id)); - const outside = this.changedOutside(check, merged.files, targets); + const outside = this.changedOutside(check, merged.files, targets, batch.user_id); let failed = 0; + let withdrawn = 0; for (const a of attempts) { const ev = this.evaluate({ check, rec, targetId: a.symbol_id, files: merged.files, changedHeaders, outside }); + // Held but not written: no annotation for it anywhere in the submission. Not a failure; it + // goes back to the author's claim. + if (!ev.feedback.target && !Object.values(submitted).some((text) => new RegExp(`//\\s*(FUNCTION|STUB)\\s*:\\s*${a.symbol_id}\\b`).test(String(text)))) { + this.withdraw(a); + withdrawn++; + continue; + } ev.feedback.headCommit = this.repo.head(); if (attempts.length > 1) ev.feedback.batch = { id: batch.id, functions: attempts.length }; if (batch.env_digest && this.config.envDigest && batch.env_digest !== this.config.envDigest) { @@ -325,7 +346,18 @@ export class Verifier { now(), JSON.stringify({ mergedFiles: merged.files, check: summarizeCheck(check), changedHeaders, outside }), batch.id, ); this.rememberCheck(batch.id, { check, rec, changedHeaders }); - if (failed) returnBatch(this.store, batch.id, `${failed} of ${attempts.length} functions need work; see each function's feedback`); + // Failing functions went back one by one (finish); the rest of the batch carries on and its + // approved part merges without them. Only a batch with nothing left on its way is over. + if (failed || withdrawn) settleBatch(this.store, batch.id, `${failed} of ${attempts.length} functions need work${withdrawn ? `, ${withdrawn} were not written` : ''}; see each function's feedback`); + } + + // A function submitted without any code for it: back into its author's claim, as if not submitted. + withdraw(attempt) { + const expires = now() + (this.config.leaseMinutes ?? 120) * 60_000; + this.store.tx(() => { + this.store.run(`UPDATE attempts SET state = 'withdrawn', finished_at = ?, result_json = ? WHERE id = ?`, now(), JSON.stringify({ feedback: { summary: 'not written: no annotation for it in the submission; it stays in your claim' } }), attempt.id); + this.store.run(`UPDATE tasks SET state = 'leased', lease_user = ?, lease_expires = ?, updated_at = ? WHERE id = ?`, attempt.user_id, expires, now(), attempt.task_id); + }); } /** Routes a check result: rejected, needs_cleanup or in_review. */ diff --git a/src/server/web/app.js b/src/server/web/app.js index 71610a8..55ff22b 100644 --- a/src/server/web/app.js +++ b/src/server/web/app.js @@ -85,7 +85,20 @@ async function loadProgress() { meter.append(fill); gauge.append(meter); const conf = p.confirmed?.n ?? 0; - gauge.append(el('p', p.functions ? `${exact.n} of ${p.functions} functions match byte for byte${conf ? `, ${conf} more confirmed by peer review` : ''}${p.peerQueue ? ` · ${p.peerQueue} waiting for peer review` : ''}.` : 'No target loaded yet. The machines are warming up.', 'gauge-caption')); + if (!p.functions) gauge.append(el('p', 'No target loaded yet. The machines are warming up.', 'gauge-caption')); + else { + // Both measures: functions (many small ones are done) and bytes of code (how much of the binary). + const share = (a, b) => `${b ? ((a / b) * 100).toFixed(2) : '0.00'}%`; + const kb = (n) => `${(n / 1024).toLocaleString(undefined, { maximumFractionDigits: 0 })} KB`; + const counts = el('div', null, 'gauge-counts'); + counts.append( + el('div', `Functions: ${exact.n.toLocaleString()} of ${p.functions.toLocaleString()} byte-identical (${share(exact.n, p.functions)})`), + el('div', `Code: ${kb(exact.bytes)} of ${kb(p.bytes)} byte-identical (${share(exact.bytes, p.bytes)})`), + ); + gauge.append(counts); + const extra = [conf ? `${conf.toLocaleString()} more confirmed by peer review` : '', p.peerQueue ? `${p.peerQueue.toLocaleString()} waiting for peer review` : ''].filter(Boolean); + if (extra.length) gauge.append(el('p', extra.join(' · '), 'gauge-caption')); + } body.append(gauge); requestAnimationFrame(() => fill.style.setProperty('width', `${Math.min(100, pct)}%`)); diff --git a/src/server/web/style.css b/src/server/web/style.css index 6b364a0..dc71fa2 100644 --- a/src/server/web/style.css +++ b/src/server/web/style.css @@ -530,3 +530,6 @@ details.fold[open] > summary { margin-bottom: 6px; } z-index: 10; } .info:hover::after, .info:focus::after { opacity: 1; } + +/* --- progress: both measures */ +.gauge-counts { grid-column: 1 / -1; display: flex; flex-wrap: wrap; gap: 4px 24px; margin: 2px 0; font: 14px var(--font-mono); } diff --git a/test/server.test.js b/test/server.test.js index ce08468..b01d0d0 100644 --- a/test/server.test.js +++ b/test/server.test.js @@ -693,6 +693,61 @@ test('"no decompile" claims are peer reviewed; accepted ones are skipped and not assert.ok(before >= 1); }); +test('verification: unwritten functions stay claimed, failing ones go back alone, neighbours may change', async () => { + delete ctx.merger.pump; + const author = await activeUser('pvauthor'); + const other = await activeUser('pvother'); + const v = ctx.store.currentVersion(); + const mk = (addr) => ctx.store.ensureSymbol({ kind: 'function', versionId: v.id, addr, size: 8, unit: 'src/pv.cpp' }).symbol.id; + const good = mk(0x412000); + const broken = mk(0x412040); + const unwritten = mk(0x412080); + const neighbour = mk(0x4120c0); + const guarded = mk(0x412100); + ctx.tasks.generate(); + const taskOf = (sym) => ctx.store.get('SELECT id FROM tasks WHERE symbol_id = ?', sym).id; + for (const sym of [good, broken, unwritten]) assert.equal((await call('POST', '/api/tasks/claim', { taskId: taskOf(sym) }, author)).status, 200); + const fn = (id, status, hash) => ({ id, status, score: status === STATUS.EXACT ? 1 : 0.5, file: 'src/pv.cpp', codeHash: hash, relocations: [], size: { ours: 8, original: 8 } }); + let guardedHash = 'g1'; + const real = ctx.verifier.check; + ctx.verifier.check = async (files) => { + const text = files['src/pv.cpp'] ?? ''; + const functions = []; + if (text.includes('int good()')) functions.push(fn(good, STATUS.EXACT, 'h-good')); + // broken is annotated but does not compile into anything: a real failure. + if (text.includes('int nb()')) functions.push(fn(neighbour, STATUS.MISMATCH, 'h-nb-changed')); + if (text.includes('int gd()')) functions.push(fn(guarded, STATUS.MISMATCH, guardedHash)); + if (text.includes('int uw()')) functions.push(fn(unwritten, STATUS.EXACT, 'h-uw')); + return { check: { annotationErrors: [], units: [{ file: 'src/pv.cpp', compiled: true, lintErrors: [], unannotated: [], bindings: {}, inferred: [], includes: [], errors: [], functions }] }, rec: { conflicts: [], unknown: [], learned: [], learnedAddresses: [] }, changedHeaders: [], units: ['src/pv.cpp'] }; + }; + try { + // The neighbour changed and nobody holds it: allowed now. + ctx.store.run('INSERT OR REPLACE INTO func_status (symbol_id, file, status, score, commit_hash, updated_at, code_hash) VALUES (?, ?, ?, 0.5, ?, 0, ?)', neighbour, 'src/pv.cpp', 'mismatch', 'x', 'h-nb-old'); + const files = { 'src/pv.cpp': `// FUNCTION: ${good}\nint good() { return 1; }\n// FUNCTION: ${broken}\n#if 0\nint br() { return 2; }\n#endif\n// FUNCTION: ${neighbour}\nint nb() { return 3; }\n` }; + const sub = await call('POST', '/api/batches', { taskIds: [good, broken, unwritten].map(taskOf), baseCommit: ctx.repo.head(), files }, author); + assert.equal(sub.status, 200, JSON.stringify(sub.body)); + await waitFor(async () => !['queued', 'verifying'].includes(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub.body.batchId).state)); + const state = (sym) => ctx.store.get('SELECT state, lease_user FROM tasks WHERE symbol_id = ?', sym); + assert.equal(state(good).state, 'approved', 'the good one goes on, changing an unheld neighbour is fine'); + assert.equal(state(broken).state, 'needs_cleanup', 'the failing one goes back alone'); + assert.deepEqual([state(unwritten).state, state(unwritten).lease_user], ['leased', ctx.accounts.byName('pvauthor').id], 'not written: still claimed, no failure'); + assert.equal(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub.body.batchId).state, 'open'); + + // Changing a function another account holds is still refused. + ctx.tasks.generate(); + assert.equal((await call('POST', '/api/tasks/claim', { taskId: taskOf(guarded) }, other)).status, 200); + ctx.store.run('INSERT OR REPLACE INTO func_status (symbol_id, file, status, score, commit_hash, updated_at, code_hash) VALUES (?, ?, ?, 0.5, ?, 0, ?)', guarded, 'src/pv.cpp', 'mismatch', 'x', 'g0'); + const sub2 = await call('POST', '/api/batches', { taskIds: [taskOf(unwritten)], baseCommit: ctx.repo.head(), files: { 'src/pv.cpp': `// FUNCTION: ${unwritten}\nint uw() { return 1; }\n// FUNCTION: ${guarded}\nint gd() { return 4; }\n` } }, author); + await waitFor(async () => !['queued', 'verifying'].includes(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub2.body.batchId).state)); + const fb = JSON.parse(ctx.store.get('SELECT result_json FROM attempts WHERE batch_id = ?', sub2.body.batchId).result_json).feedback; + assert.match(fb.summary, /changes functions it does not hold/); + } finally { + ctx.verifier.check = real; + for (const sym of [good, broken, unwritten, neighbour, guarded]) ctx.store.run(`UPDATE tasks SET state = 'done', lease_user = NULL WHERE symbol_id = ?`, sym); + ctx.store.run(`UPDATE batches SET state = 'superseded' WHERE user_id = ?`, ctx.accounts.byName('pvauthor').id); + } +}); + test('the crew view is public and shows no account details', async () => { const token = await activeUser('crewview'); await call('GET', '/api/tasks', null, token); // the agent checks in