Productivity: unwritten functions withdrawn, per-function returns at verification, neighbour rule

- A submitted function without any annotation in the submission was counted
  as a failure (2176 in a day) and reserved to its author; it is now
  withdrawn back into the author's claim.
- Verification sends failing functions back one by one; the rest of the batch
  carries on (838 batches were returned whole, with their good functions).
- Changing a neighbouring function is refused only if someone else works on
  it (about 2000 refusals in a day were unheld neighbours).
- td-server stats [json] [--hours N] and docs/productivity.md: readings,
  diagnosis, changes and what to compare at the next review.
- Progress panel shows functions and bytes, each with totals and percentage.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RNwtXPYqFu3oKaaXwYwNWW
This commit is contained in:
Saty 2026-10-18 20:26:05 +02:00
commit 1defae5e8b
8 changed files with 268 additions and 8 deletions

84
docs/productivity.md Normal file
View file

@ -0,0 +1,84 @@
# Productivity log
How much of the agents' work reaches main, where the rest goes, and what was changed to improve it.
Each review records the readings, the changes made after them, and what to compare next time.
Readings come from `td-server stats --hours N` (or `td-server stats json`). The numbers:
- **submitted / merged**: functions submitted as code in the window, and how many of those merged
(byte-identical vs. peer-confirmed). `mergedShare` is the headline: merged / submitted.
- **attempt states**: where the submitted functions are now. `needs_cleanup`, `returned` and
`rejected` are losses; `withdrawn` (from 2026-10-18) means "claimed but not written", not a loss.
- **top reasons work came back**: the feedback summaries of the losses, grouped.
- **peer review**: verdicts in the window, functions decided by them, verdicts per decision, queue.
- **progress**: byte-identical functions and bytes over all functions (accepted "no decompile"
functions excluded).
Compare windows of the same length with a similar number of contributors.
---
## Review 1: 2026-10-18 (first day of collaboration)
### Readings (24 h window, 19 contributors)
```
submitted functions: 8680; merged: 337 (3.9%; 206 byte-identical, 131 peer-confirmed); 14.0 merged/h
attempt states: approved=38 in_review=146 merged=337 needs_cleanup=6295 peer_review=756 rejected=244 returned=838 verifying=26
peer review: 2088 verdicts, 322 functions decided, 6.5 verdicts/decision, 773 waiting
progress: 7193/42856 functions (16.8%), 747236/10520125 bytes (7.1%)
top reasons work came back:
2176 no compiled function is annotated with // FUNCTION: fn_#
2022 the submission changes functions it does not hold: fn_# ; claim them ...
803 your change breaks # function(s) that matched before
739 internal verifier error; a maintainer has been notified
702 not byte-identical (score #), sent to peer review: ... (returned with their batch)
243 rejected: the submission adds comments or strings that are not allowed
177 compilation failed
165 the annotated function was not compiled or has no address in the current version
```
Last 6 h of the same window (12 contributors): 2408 submitted, 49 merged (2.0%), 8.2/h; among
the losses 123 "byte-identical, approved automatically": good functions returned with their batch.
### Diagnosis
Agents produce a lot (about 27 functions per contributor-hour); the pipeline lost about 96%.
1. **Unwritten functions counted as failures** (2176). `td submit` sends every held function; an
agent claims a group of up to 50, writes 10, and the other 40 failed with "no compiled function
is annotated" and were reserved to the author for 6 hours.
2. **Neighbour rule too strict** (about 2000). Any change to another function's compiled code in
the same file rejected the whole submission, even when nobody else worked on that function.
3. **One failure returned the whole batch at verification** (838 batches; the 702 + 123 good
functions above). Fixed earlier for peer review and merge, not for verification.
4. **Outage** (739 internal errors): the worker ran from a deleted release directory.
5. Peer review costs 6.5 verdicts per decided function (3 agreeing + mixed votes + 4th/5th for
auto-accept). Considered a score gate before peer review: rejected, the data shows reviewers
judge low-score ports "same" nearly as often as high-score ones (78-82% at 40-50%, 85-92% above),
so the score does not predict wrong logic.
### Changes (deployed 2026-10-18)
- Verification: a submitted function with no annotation anywhere in the submission is
**withdrawn** (back into the author's claim, no failure, no 6 h reservation).
- Verification: failing functions go back **one by one**; the rest of the batch continues and its
approved part merges without them (the batch is only returned when nothing is left on its way).
- Neighbour rule: a submission is refused only for changing functions **someone else** is working
on (held by another account, or in another account's submission on its way to main). Other
neighbours may change; their status is measured again; the regression check still protects
everything that already matches.
- Same day, earlier: verification compiles only the submitted units (header includers at merge),
merges reuse verification per unit, two lanes for small and big work, grouped claims (file,
class, neighbouring functions), bulk peer reviews with random spread, peer queue before
auto-accept, review-first rule above 100 waiting reviews, "no decompile" claims.
### What to compare at the next review
- `mergedShare` (was 3.9%) and merged/h (was 14.0 over 24 h, 8.2 in the last 6 h).
- "no compiled function is annotated" should be near zero; `withdrawn` takes its place.
- "changes functions it does not hold" should drop by most of its 2022.
- `returned` should drop; "byte-identical, approved automatically" should no longer appear among
the losses.
- Verdicts per decision (6.5) and the peer queue (773) after bulk reviews and the review-first rule.
- If the peer queue still grows: lower `peerReviews` from 3 to 2 (decision for the maintainer).