Productivity: unwritten functions withdrawn, per-function returns at verification, neighbour rule
- A submitted function without any annotation in the submission was counted as a failure (2176 in a day) and reserved to its author; it is now withdrawn back into the author's claim. - Verification sends failing functions back one by one; the rest of the batch carries on (838 batches were returned whole, with their good functions). - Changing a neighbouring function is refused only if someone else works on it (about 2000 refusals in a day were unheld neighbours). - td-server stats [json] [--hours N] and docs/productivity.md: readings, diagnosis, changes and what to compare at the next review. - Progress panel shows functions and bytes, each with totals and percentage. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RNwtXPYqFu3oKaaXwYwNWW
This commit is contained in:
Saty
parent
d726e97604
commit
1defae5e8b
8 changed files with 268 additions and 8 deletions
84
docs/productivity.md
Normal file
84
docs/productivity.md
Normal file
|
|
@ -0,0 +1,84 @@
|
|||
# Productivity log
|
||||
|
||||
How much of the agents' work reaches main, where the rest goes, and what was changed to improve it.
|
||||
Each review records the readings, the changes made after them, and what to compare next time.
|
||||
|
||||
Readings come from `td-server stats --hours N` (or `td-server stats json`). The numbers:
|
||||
|
||||
- **submitted / merged**: functions submitted as code in the window, and how many of those merged
|
||||
(byte-identical vs. peer-confirmed). `mergedShare` is the headline: merged / submitted.
|
||||
- **attempt states**: where the submitted functions are now. `needs_cleanup`, `returned` and
|
||||
`rejected` are losses; `withdrawn` (from 2026-10-18) means "claimed but not written", not a loss.
|
||||
- **top reasons work came back**: the feedback summaries of the losses, grouped.
|
||||
- **peer review**: verdicts in the window, functions decided by them, verdicts per decision, queue.
|
||||
- **progress**: byte-identical functions and bytes over all functions (accepted "no decompile"
|
||||
functions excluded).
|
||||
|
||||
Compare windows of the same length with a similar number of contributors.
|
||||
|
||||
---
|
||||
|
||||
## Review 1: 2026-10-18 (first day of collaboration)
|
||||
|
||||
### Readings (24 h window, 19 contributors)
|
||||
|
||||
```
|
||||
submitted functions: 8680; merged: 337 (3.9%; 206 byte-identical, 131 peer-confirmed); 14.0 merged/h
|
||||
attempt states: approved=38 in_review=146 merged=337 needs_cleanup=6295 peer_review=756 rejected=244 returned=838 verifying=26
|
||||
peer review: 2088 verdicts, 322 functions decided, 6.5 verdicts/decision, 773 waiting
|
||||
progress: 7193/42856 functions (16.8%), 747236/10520125 bytes (7.1%)
|
||||
top reasons work came back:
|
||||
2176 no compiled function is annotated with // FUNCTION: fn_#
|
||||
2022 the submission changes functions it does not hold: fn_# ; claim them ...
|
||||
803 your change breaks # function(s) that matched before
|
||||
739 internal verifier error; a maintainer has been notified
|
||||
702 not byte-identical (score #), sent to peer review: ... (returned with their batch)
|
||||
243 rejected: the submission adds comments or strings that are not allowed
|
||||
177 compilation failed
|
||||
165 the annotated function was not compiled or has no address in the current version
|
||||
```
|
||||
|
||||
Last 6 h of the same window (12 contributors): 2408 submitted, 49 merged (2.0%), 8.2/h; among
|
||||
the losses 123 "byte-identical, approved automatically": good functions returned with their batch.
|
||||
|
||||
### Diagnosis
|
||||
|
||||
Agents produce a lot (about 27 functions per contributor-hour); the pipeline lost about 96%.
|
||||
|
||||
1. **Unwritten functions counted as failures** (2176). `td submit` sends every held function; an
|
||||
agent claims a group of up to 50, writes 10, and the other 40 failed with "no compiled function
|
||||
is annotated" and were reserved to the author for 6 hours.
|
||||
2. **Neighbour rule too strict** (about 2000). Any change to another function's compiled code in
|
||||
the same file rejected the whole submission, even when nobody else worked on that function.
|
||||
3. **One failure returned the whole batch at verification** (838 batches; the 702 + 123 good
|
||||
functions above). Fixed earlier for peer review and merge, not for verification.
|
||||
4. **Outage** (739 internal errors): the worker ran from a deleted release directory.
|
||||
5. Peer review costs 6.5 verdicts per decided function (3 agreeing + mixed votes + 4th/5th for
|
||||
auto-accept). Considered a score gate before peer review: rejected, the data shows reviewers
|
||||
judge low-score ports "same" nearly as often as high-score ones (78-82% at 40-50%, 85-92% above),
|
||||
so the score does not predict wrong logic.
|
||||
|
||||
### Changes (deployed 2026-10-18)
|
||||
|
||||
- Verification: a submitted function with no annotation anywhere in the submission is
|
||||
**withdrawn** (back into the author's claim, no failure, no 6 h reservation).
|
||||
- Verification: failing functions go back **one by one**; the rest of the batch continues and its
|
||||
approved part merges without them (the batch is only returned when nothing is left on its way).
|
||||
- Neighbour rule: a submission is refused only for changing functions **someone else** is working
|
||||
on (held by another account, or in another account's submission on its way to main). Other
|
||||
neighbours may change; their status is measured again; the regression check still protects
|
||||
everything that already matches.
|
||||
- Same day, earlier: verification compiles only the submitted units (header includers at merge),
|
||||
merges reuse verification per unit, two lanes for small and big work, grouped claims (file,
|
||||
class, neighbouring functions), bulk peer reviews with random spread, peer queue before
|
||||
auto-accept, review-first rule above 100 waiting reviews, "no decompile" claims.
|
||||
|
||||
### What to compare at the next review
|
||||
|
||||
- `mergedShare` (was 3.9%) and merged/h (was 14.0 over 24 h, 8.2 in the last 6 h).
|
||||
- "no compiled function is annotated" should be near zero; `withdrawn` takes its place.
|
||||
- "changes functions it does not hold" should drop by most of its 2022.
|
||||
- `returned` should drop; "byte-identical, approved automatically" should no longer appear among
|
||||
the losses.
|
||||
- Verdicts per decision (6.5) and the peer queue (773) after bulk reviews and the review-first rule.
|
||||
- If the peer queue still grows: lower `peerReviews` from 3 to 2 (decision for the maintainer).
|
||||
|
|
@ -11,6 +11,7 @@ import { openServer, startServer, DEFAULT_CONFIG } from './app.js';
|
|||
import { Store, now } from './db.js';
|
||||
import { Accounts } from './accounts.js';
|
||||
import { agentGuidance, setAgentGuidance, resetAgentGuidance } from './guidance.js';
|
||||
import { productivity, formatProductivity } from './stats.js';
|
||||
import { PEImage } from '../shared/pe.js';
|
||||
|
||||
const TEMPLATE_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), '..', '..', 'template');
|
||||
|
|
@ -32,6 +33,7 @@ Usage: td-server <command> [options] (all commands accept --data <dir>, defaul
|
|||
user token <name> [label] create an API token for an account
|
||||
version add <id> <exe> [--current] register a target executable version
|
||||
version set-current <id>
|
||||
stats [json] [--hours N] pipeline productivity (submitted, merged, losses, reviews)
|
||||
guidance show|set <text>|off|reset policy note sent to agents with every work response
|
||||
version label <id> <text> readable name shown on the website
|
||||
version map <newId> <csv> --from <oldId>
|
||||
|
|
@ -71,6 +73,7 @@ async function main() {
|
|||
current: { type: 'boolean', default: false },
|
||||
version: { type: 'string' },
|
||||
from: { type: 'string' },
|
||||
hours: { type: 'string', default: '24' },
|
||||
help: { type: 'boolean', short: 'h' },
|
||||
},
|
||||
});
|
||||
|
|
@ -82,6 +85,13 @@ async function main() {
|
|||
if (cmd === 'init') return init(dataDir);
|
||||
if (cmd === 'start') return startServer(dataDir, { role: sub ?? 'all' });
|
||||
|
||||
if (cmd === 'stats') {
|
||||
// Pipeline productivity (see docs/productivity.md).
|
||||
const store = new Store(path.join(dataDir, 'td.sqlite'));
|
||||
const p = productivity(store, Number(values.hours) || 24);
|
||||
return console.log(sub === 'json' ? JSON.stringify(p, null, 2) : formatProductivity(p));
|
||||
}
|
||||
|
||||
if (cmd === 'guidance') {
|
||||
// Reaches running agents on their next call; no restart needed.
|
||||
const store = new Store(path.join(dataDir, 'td.sqlite'));
|
||||
|
|
|
|||
|
|
@ -249,7 +249,7 @@ export class Merger {
|
|||
noSplit(batch, reason) {
|
||||
this.store.run(`UPDATE batches SET result_json = json_set(COALESCE(result_json, '{}'), '$.noSplit', ?) WHERE id = ?`, reason, batch.id);
|
||||
// With a function sent back it can never merge as a whole, so it all goes back.
|
||||
if (this.store.get(`SELECT 1 FROM attempts WHERE batch_id = ? AND state IN ('needs_cleanup', 'rejected', 'returned')`, batch.id)) {
|
||||
if (this.store.get(`SELECT 1 FROM attempts WHERE batch_id = ? AND state IN ('needs_cleanup', 'rejected', 'returned', 'withdrawn')`, batch.id)) {
|
||||
returnBatch(this.store, batch.id, `a function of this batch was sent back and the rest cannot be merged without it: ${reason}`);
|
||||
return this.log.info(`batch ${batch.id} returned: ${reason}`);
|
||||
}
|
||||
|
|
@ -299,7 +299,7 @@ export class Merger {
|
|||
// their source put back as on main); shared changes in the file may change their compiled code.
|
||||
// That is not a change to functions someone else holds; their status is measured again.
|
||||
if (parentId) for (const a of batchAttempts(this.store, parentId)) targets.add(a.symbol_id);
|
||||
const outside = this.verifier.changedOutside(check, merged.files, targets);
|
||||
const outside = this.verifier.changedOutside(check, merged.files, targets, batch.user_id);
|
||||
const confirmedIds = new Set();
|
||||
const results = [];
|
||||
for (const a of attempts) {
|
||||
|
|
|
|||
63
src/server/stats.js
Normal file
63
src/server/stats.js
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
// Pipeline productivity over a time window: how much submitted work reaches main and where the rest
|
||||
// goes. `td-server stats --hours N` prints it; docs/productivity.md records the readings and what
|
||||
// was changed in between, so reviews compare like with like.
|
||||
|
||||
export function productivity(store, hours = 24) {
|
||||
const since = Date.now() - hours * 3600_000;
|
||||
const q = (sql, ...a) => store.get(sql, ...a);
|
||||
const v = store.currentVersion();
|
||||
const submitted = q(`SELECT COUNT(*) AS n FROM attempts WHERE kind = 'code' AND created_at > ?`, since).n;
|
||||
const byState = Object.fromEntries(store.all(`SELECT state, COUNT(*) AS n FROM attempts WHERE kind = 'code' AND created_at > ? GROUP BY state`, since).map((r) => [r.state, r.n]));
|
||||
const merged = byState.merged ?? 0;
|
||||
const mergedIdentical = q(
|
||||
`SELECT COUNT(*) AS n FROM attempts WHERE kind = 'code' AND state = 'merged' AND created_at > ? AND json_extract(result_json, '$.feedback.target.status') IN ('exact', 'match_unresolved')`,
|
||||
since,
|
||||
).n;
|
||||
const reasons = {};
|
||||
for (const r of store.all(`SELECT json_extract(result_json, '$.feedback.summary') AS s FROM attempts WHERE kind = 'code' AND created_at > ? AND state IN ('needs_cleanup', 'returned', 'rejected')`, since)) {
|
||||
const k = String(r.s ?? 'unknown').replace(/fn_\d+/g, 'fn_#').replace(/\d+(\.\d+)?%?/g, '#').replace(/(fn_#(, )?)+/g, 'fn_# ').slice(0, 70);
|
||||
reasons[k] = (reasons[k] ?? 0) + 1;
|
||||
}
|
||||
const verdicts = q('SELECT COUNT(*) AS n FROM peer_reviews WHERE created_at > ?', since).n;
|
||||
const decided = q(
|
||||
`SELECT COUNT(DISTINCT a.id) AS n FROM attempts a WHERE a.state IN ('in_review', 'approved', 'merged', 'needs_cleanup')
|
||||
AND EXISTS (SELECT 1 FROM peer_reviews p WHERE p.attempt_id = a.id AND p.created_at > ?)`,
|
||||
since,
|
||||
).n;
|
||||
const total = q(
|
||||
`SELECT COUNT(*) AS n, COALESCE(SUM(a.size), 0) AS bytes FROM symbols s JOIN symbol_addresses a ON a.symbol_id = s.id AND a.version_id = ? WHERE s.kind = 'function' AND s.library = 0 AND s.skip_reason IS NULL`,
|
||||
v.id,
|
||||
);
|
||||
const done = q(
|
||||
`SELECT COUNT(*) AS n, COALESCE(SUM(a.size), 0) AS bytes FROM func_status f JOIN symbol_addresses a ON a.symbol_id = f.symbol_id AND a.version_id = ? WHERE f.status IN ('exact', 'match_unresolved')`,
|
||||
v.id,
|
||||
);
|
||||
return {
|
||||
hours,
|
||||
contributors: q('SELECT COUNT(DISTINCT user_id) AS n FROM attempts WHERE created_at > ?', since).n,
|
||||
submitted,
|
||||
merged,
|
||||
mergedIdentical,
|
||||
mergedPeerConfirmed: merged - mergedIdentical,
|
||||
mergedShare: submitted ? merged / submitted : 0,
|
||||
mergedPerHour: merged / hours,
|
||||
byState,
|
||||
topReasons: Object.entries(reasons).sort((a, b) => b[1] - a[1]).slice(0, 8),
|
||||
peer: { verdicts, decided, verdictsPerDecision: decided ? verdicts / decided : null, waiting: q(`SELECT COUNT(*) AS n FROM attempts WHERE state = 'peer_review'`).n },
|
||||
progress: { functions: done.n, functionsTotal: total.n, bytes: done.bytes, bytesTotal: total.bytes },
|
||||
};
|
||||
}
|
||||
|
||||
export function formatProductivity(p) {
|
||||
const pct = (x) => `${(x * 100).toFixed(1)}%`;
|
||||
const lines = [
|
||||
`window: last ${p.hours} h, ${p.contributors} contributors`,
|
||||
`submitted functions: ${p.submitted}; merged: ${p.merged} (${pct(p.mergedShare)}; ${p.mergedIdentical} byte-identical, ${p.mergedPeerConfirmed} peer-confirmed); ${p.mergedPerHour.toFixed(1)} merged/h`,
|
||||
`attempt states: ${Object.entries(p.byState).map(([k, n]) => `${k}=${n}`).join(' ')}`,
|
||||
`peer review: ${p.peer.verdicts} verdicts, ${p.peer.decided} functions decided, ${p.peer.verdictsPerDecision?.toFixed(1) ?? '-'} verdicts/decision, ${p.peer.waiting} waiting`,
|
||||
`progress: ${p.progress.functions}/${p.progress.functionsTotal} functions (${pct(p.progress.functions / p.progress.functionsTotal)}), ${p.progress.bytes}/${p.progress.bytesTotal} bytes (${pct(p.progress.bytes / p.progress.bytesTotal)})`,
|
||||
'top reasons work came back:',
|
||||
...p.topReasons.map(([k, n]) => ` ${String(n).padStart(6)} ${k}`),
|
||||
];
|
||||
return lines.join('\n');
|
||||
}
|
||||
|
|
@ -10,7 +10,7 @@ import { reconcile } from '../shared/reconcile.js';
|
|||
import { STATUS } from '../shared/compare.js';
|
||||
import { listSources, isUnit } from '../shared/project.js';
|
||||
import { now } from './db.js';
|
||||
import { returnBatch, batchAttempts } from './batches.js';
|
||||
import { returnBatch, batchAttempts, settleBatch } from './batches.js';
|
||||
import { analyzeInWorkers } from './analyze.js';
|
||||
|
||||
// A batch whose verification compiles more units than this is big (it changes a widely used header).
|
||||
|
|
@ -245,7 +245,10 @@ export class Verifier {
|
|||
* part of the submission: a submission may only change the functions it holds the lease for.
|
||||
* Unknown on main (no fingerprint yet, e.g. before the first rebuild) is not counted.
|
||||
*/
|
||||
changedOutside(check, files, targets) {
|
||||
// With `userId`, only functions someone else is working on count (held by another account, or in
|
||||
// another account's submission on its way to main). Other neighbours may change: their status is
|
||||
// measured again, and the regression check still protects whatever already matches.
|
||||
changedOutside(check, files, targets, userId = null) {
|
||||
const fileSet = new Set(Object.keys(files));
|
||||
const seen = new Map();
|
||||
for (const u of check.units) for (const f of u.functions) if (fileSet.has(f.file) && !targets.has(f.id) && f.codeHash) seen.set(f.id, f);
|
||||
|
|
@ -255,7 +258,17 @@ export class Verifier {
|
|||
if (!main) out.push({ id, file: f.file, why: 'new or newly annotated' });
|
||||
else if (main.code_hash && main.code_hash !== f.codeHash) out.push({ id, file: f.file, why: 'compiled code changed' });
|
||||
}
|
||||
return out;
|
||||
if (userId == null) return out;
|
||||
return out.filter((o) => this.workedOnByOthers(o.id, userId));
|
||||
}
|
||||
|
||||
workedOnByOthers(symbolId, userId) {
|
||||
return !!this.store.get(
|
||||
`SELECT 1 FROM tasks t WHERE t.symbol_id = ? AND (
|
||||
(t.state = 'leased' AND t.lease_user != ?)
|
||||
OR EXISTS (SELECT 1 FROM attempts a WHERE a.task_id = t.id AND a.user_id != ? AND a.state IN ('queued', 'verifying', 'peer_review', 'in_review', 'approved', 'merging')))`,
|
||||
symbolId, userId, userId,
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -304,10 +317,18 @@ export class Verifier {
|
|||
// units that only include a changed header are compiled when the batch merges.
|
||||
const { check, rec, changedHeaders } = await this.check(merged.files, `batch-${batch.id}`, { dependents: false });
|
||||
const targets = new Set(attempts.map((a) => a.symbol_id));
|
||||
const outside = this.changedOutside(check, merged.files, targets);
|
||||
const outside = this.changedOutside(check, merged.files, targets, batch.user_id);
|
||||
let failed = 0;
|
||||
let withdrawn = 0;
|
||||
for (const a of attempts) {
|
||||
const ev = this.evaluate({ check, rec, targetId: a.symbol_id, files: merged.files, changedHeaders, outside });
|
||||
// Held but not written: no annotation for it anywhere in the submission. Not a failure; it
|
||||
// goes back to the author's claim.
|
||||
if (!ev.feedback.target && !Object.values(submitted).some((text) => new RegExp(`//\\s*(FUNCTION|STUB)\\s*:\\s*${a.symbol_id}\\b`).test(String(text)))) {
|
||||
this.withdraw(a);
|
||||
withdrawn++;
|
||||
continue;
|
||||
}
|
||||
ev.feedback.headCommit = this.repo.head();
|
||||
if (attempts.length > 1) ev.feedback.batch = { id: batch.id, functions: attempts.length };
|
||||
if (batch.env_digest && this.config.envDigest && batch.env_digest !== this.config.envDigest) {
|
||||
|
|
@ -325,7 +346,18 @@ export class Verifier {
|
|||
now(), JSON.stringify({ mergedFiles: merged.files, check: summarizeCheck(check), changedHeaders, outside }), batch.id,
|
||||
);
|
||||
this.rememberCheck(batch.id, { check, rec, changedHeaders });
|
||||
if (failed) returnBatch(this.store, batch.id, `${failed} of ${attempts.length} functions need work; see each function's feedback`);
|
||||
// Failing functions went back one by one (finish); the rest of the batch carries on and its
|
||||
// approved part merges without them. Only a batch with nothing left on its way is over.
|
||||
if (failed || withdrawn) settleBatch(this.store, batch.id, `${failed} of ${attempts.length} functions need work${withdrawn ? `, ${withdrawn} were not written` : ''}; see each function's feedback`);
|
||||
}
|
||||
|
||||
// A function submitted without any code for it: back into its author's claim, as if not submitted.
|
||||
withdraw(attempt) {
|
||||
const expires = now() + (this.config.leaseMinutes ?? 120) * 60_000;
|
||||
this.store.tx(() => {
|
||||
this.store.run(`UPDATE attempts SET state = 'withdrawn', finished_at = ?, result_json = ? WHERE id = ?`, now(), JSON.stringify({ feedback: { summary: 'not written: no annotation for it in the submission; it stays in your claim' } }), attempt.id);
|
||||
this.store.run(`UPDATE tasks SET state = 'leased', lease_user = ?, lease_expires = ?, updated_at = ? WHERE id = ?`, attempt.user_id, expires, now(), attempt.task_id);
|
||||
});
|
||||
}
|
||||
|
||||
/** Routes a check result: rejected, needs_cleanup or in_review. */
|
||||
|
|
|
|||
|
|
@ -85,7 +85,20 @@ async function loadProgress() {
|
|||
meter.append(fill);
|
||||
gauge.append(meter);
|
||||
const conf = p.confirmed?.n ?? 0;
|
||||
gauge.append(el('p', p.functions ? `${exact.n} of ${p.functions} functions match byte for byte${conf ? `, ${conf} more confirmed by peer review` : ''}${p.peerQueue ? ` · ${p.peerQueue} waiting for peer review` : ''}.` : 'No target loaded yet. The machines are warming up.', 'gauge-caption'));
|
||||
if (!p.functions) gauge.append(el('p', 'No target loaded yet. The machines are warming up.', 'gauge-caption'));
|
||||
else {
|
||||
// Both measures: functions (many small ones are done) and bytes of code (how much of the binary).
|
||||
const share = (a, b) => `${b ? ((a / b) * 100).toFixed(2) : '0.00'}%`;
|
||||
const kb = (n) => `${(n / 1024).toLocaleString(undefined, { maximumFractionDigits: 0 })} KB`;
|
||||
const counts = el('div', null, 'gauge-counts');
|
||||
counts.append(
|
||||
el('div', `Functions: ${exact.n.toLocaleString()} of ${p.functions.toLocaleString()} byte-identical (${share(exact.n, p.functions)})`),
|
||||
el('div', `Code: ${kb(exact.bytes)} of ${kb(p.bytes)} byte-identical (${share(exact.bytes, p.bytes)})`),
|
||||
);
|
||||
gauge.append(counts);
|
||||
const extra = [conf ? `${conf.toLocaleString()} more confirmed by peer review` : '', p.peerQueue ? `${p.peerQueue.toLocaleString()} waiting for peer review` : ''].filter(Boolean);
|
||||
if (extra.length) gauge.append(el('p', extra.join(' · '), 'gauge-caption'));
|
||||
}
|
||||
body.append(gauge);
|
||||
requestAnimationFrame(() => fill.style.setProperty('width', `${Math.min(100, pct)}%`));
|
||||
|
||||
|
|
|
|||
|
|
@ -530,3 +530,6 @@ details.fold[open] > summary { margin-bottom: 6px; }
|
|||
z-index: 10;
|
||||
}
|
||||
.info:hover::after, .info:focus::after { opacity: 1; }
|
||||
|
||||
/* --- progress: both measures */
|
||||
.gauge-counts { grid-column: 1 / -1; display: flex; flex-wrap: wrap; gap: 4px 24px; margin: 2px 0; font: 14px var(--font-mono); }
|
||||
|
|
|
|||
|
|
@ -693,6 +693,61 @@ test('"no decompile" claims are peer reviewed; accepted ones are skipped and not
|
|||
assert.ok(before >= 1);
|
||||
});
|
||||
|
||||
test('verification: unwritten functions stay claimed, failing ones go back alone, neighbours may change', async () => {
|
||||
delete ctx.merger.pump;
|
||||
const author = await activeUser('pvauthor');
|
||||
const other = await activeUser('pvother');
|
||||
const v = ctx.store.currentVersion();
|
||||
const mk = (addr) => ctx.store.ensureSymbol({ kind: 'function', versionId: v.id, addr, size: 8, unit: 'src/pv.cpp' }).symbol.id;
|
||||
const good = mk(0x412000);
|
||||
const broken = mk(0x412040);
|
||||
const unwritten = mk(0x412080);
|
||||
const neighbour = mk(0x4120c0);
|
||||
const guarded = mk(0x412100);
|
||||
ctx.tasks.generate();
|
||||
const taskOf = (sym) => ctx.store.get('SELECT id FROM tasks WHERE symbol_id = ?', sym).id;
|
||||
for (const sym of [good, broken, unwritten]) assert.equal((await call('POST', '/api/tasks/claim', { taskId: taskOf(sym) }, author)).status, 200);
|
||||
const fn = (id, status, hash) => ({ id, status, score: status === STATUS.EXACT ? 1 : 0.5, file: 'src/pv.cpp', codeHash: hash, relocations: [], size: { ours: 8, original: 8 } });
|
||||
let guardedHash = 'g1';
|
||||
const real = ctx.verifier.check;
|
||||
ctx.verifier.check = async (files) => {
|
||||
const text = files['src/pv.cpp'] ?? '';
|
||||
const functions = [];
|
||||
if (text.includes('int good()')) functions.push(fn(good, STATUS.EXACT, 'h-good'));
|
||||
// broken is annotated but does not compile into anything: a real failure.
|
||||
if (text.includes('int nb()')) functions.push(fn(neighbour, STATUS.MISMATCH, 'h-nb-changed'));
|
||||
if (text.includes('int gd()')) functions.push(fn(guarded, STATUS.MISMATCH, guardedHash));
|
||||
if (text.includes('int uw()')) functions.push(fn(unwritten, STATUS.EXACT, 'h-uw'));
|
||||
return { check: { annotationErrors: [], units: [{ file: 'src/pv.cpp', compiled: true, lintErrors: [], unannotated: [], bindings: {}, inferred: [], includes: [], errors: [], functions }] }, rec: { conflicts: [], unknown: [], learned: [], learnedAddresses: [] }, changedHeaders: [], units: ['src/pv.cpp'] };
|
||||
};
|
||||
try {
|
||||
// The neighbour changed and nobody holds it: allowed now.
|
||||
ctx.store.run('INSERT OR REPLACE INTO func_status (symbol_id, file, status, score, commit_hash, updated_at, code_hash) VALUES (?, ?, ?, 0.5, ?, 0, ?)', neighbour, 'src/pv.cpp', 'mismatch', 'x', 'h-nb-old');
|
||||
const files = { 'src/pv.cpp': `// FUNCTION: ${good}\nint good() { return 1; }\n// FUNCTION: ${broken}\n#if 0\nint br() { return 2; }\n#endif\n// FUNCTION: ${neighbour}\nint nb() { return 3; }\n` };
|
||||
const sub = await call('POST', '/api/batches', { taskIds: [good, broken, unwritten].map(taskOf), baseCommit: ctx.repo.head(), files }, author);
|
||||
assert.equal(sub.status, 200, JSON.stringify(sub.body));
|
||||
await waitFor(async () => !['queued', 'verifying'].includes(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub.body.batchId).state));
|
||||
const state = (sym) => ctx.store.get('SELECT state, lease_user FROM tasks WHERE symbol_id = ?', sym);
|
||||
assert.equal(state(good).state, 'approved', 'the good one goes on, changing an unheld neighbour is fine');
|
||||
assert.equal(state(broken).state, 'needs_cleanup', 'the failing one goes back alone');
|
||||
assert.deepEqual([state(unwritten).state, state(unwritten).lease_user], ['leased', ctx.accounts.byName('pvauthor').id], 'not written: still claimed, no failure');
|
||||
assert.equal(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub.body.batchId).state, 'open');
|
||||
|
||||
// Changing a function another account holds is still refused.
|
||||
ctx.tasks.generate();
|
||||
assert.equal((await call('POST', '/api/tasks/claim', { taskId: taskOf(guarded) }, other)).status, 200);
|
||||
ctx.store.run('INSERT OR REPLACE INTO func_status (symbol_id, file, status, score, commit_hash, updated_at, code_hash) VALUES (?, ?, ?, 0.5, ?, 0, ?)', guarded, 'src/pv.cpp', 'mismatch', 'x', 'g0');
|
||||
const sub2 = await call('POST', '/api/batches', { taskIds: [taskOf(unwritten)], baseCommit: ctx.repo.head(), files: { 'src/pv.cpp': `// FUNCTION: ${unwritten}\nint uw() { return 1; }\n// FUNCTION: ${guarded}\nint gd() { return 4; }\n` } }, author);
|
||||
await waitFor(async () => !['queued', 'verifying'].includes(ctx.store.get('SELECT state FROM batches WHERE id = ?', sub2.body.batchId).state));
|
||||
const fb = JSON.parse(ctx.store.get('SELECT result_json FROM attempts WHERE batch_id = ?', sub2.body.batchId).result_json).feedback;
|
||||
assert.match(fb.summary, /changes functions it does not hold/);
|
||||
} finally {
|
||||
ctx.verifier.check = real;
|
||||
for (const sym of [good, broken, unwritten, neighbour, guarded]) ctx.store.run(`UPDATE tasks SET state = 'done', lease_user = NULL WHERE symbol_id = ?`, sym);
|
||||
ctx.store.run(`UPDATE batches SET state = 'superseded' WHERE user_id = ?`, ctx.accounts.byName('pvauthor').id);
|
||||
}
|
||||
});
|
||||
|
||||
test('the crew view is public and shows no account details', async () => {
|
||||
const token = await activeUser('crewview');
|
||||
await call('GET', '/api/tasks', null, token); // the agent checks in
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue