diff --git a/AGENTS.md b/AGENTS.md index 1c9cf88..c138eb5 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -17,7 +17,8 @@ current policy. It is up to date and takes precedence over this file. ## Setup (once per machine) `td_toolchain_install` tool (or `td toolchain install` CLI) downloads the compiler and -sets it up; on Linux it needs wine. Checks compile **on your machine**. The +sets it up. Windows runs it with `--runner native`; Linux needs `--runner wine` or +`--runner docker`. Checks compile **on your machine**. The server only compiles your submissions (to prove they build and to route each function) and once more before a merge. @@ -44,7 +45,7 @@ td sync any function of it, and the first accepted candidate of each function merges (the others are dropped, the rest of their batch goes on). Prefer a region nobody claimed; share only to help with a big or stuck one. -3. **Read the originals**: `td_disassemble` tool (or `td disassemble` CLI) (ground truth) and `td_decompile` tool (or `td decompile` CLI) +3. **Read the originals**: `td_disassemble` tool (or `td disasm` CLI) (ground truth) and `td_decompile` tool (or `td decompile` CLI) (imported IDA pseudocode where available, a starting point only). 4. **Write each function** in its file, directly preceded by its annotation: ```cpp @@ -158,6 +159,41 @@ other contributors' agents: a function merges once "same" leads Implement claimed functions and their single-caller helpers in the unit assigned by the claim; do not split their implementation into separate helper `.cpp` files. +## Multi-agent workspace + +Several agents can work in parallel on one machine, each in its own clone (a "slot"). +The container folder holds the main clone (this repository), `workers//` (one clone +per agent) and `_work//` (check logs and edit backups, outside every repository). + +- Each slot gets a copy of the main clone's `.td/` (`config.json`, `symbols.json`, + `target.exe`), hidden from git. Never print the token in `.td/config.json` or copy it + anywhere except a slot's `.td/`. +- `scripts/workspace/` has a Linux/Git Bash and a Windows (PowerShell 5.1+) version of each helper: + + | Purpose | Linux / Git Bash | Windows PowerShell | + |---|---|---| + | Create or refresh slots from `origin/main`; a dirty slot is skipped | `bash scripts/workspace/slot.sh w1 w2` | `powershell -ExecutionPolicy Bypass -File scripts/workspace/slot.ps1 w1 w2` | + | Save the slot's edits to `_work//pre-check-*.patch`, run `td check` there, print one line per function | `bash scripts/workspace/check.sh w1 [--includers] [--full]` | `powershell -ExecutionPolicy Bypass -File scripts/workspace/check.ps1 w1 [--includers] [--full]` | + | Offline lookup in `.td/symbols.json` | `python3 scripts/workspace/sym.py [--slot w1]` | `python scripts/workspace/sym.py [--slot w1]` | + + Run them from the main clone or from any slot. If the main clone is not named + `rodecomp`, set `RODECOMP_MAIN` to its path. +- MCP: configure `td mcp --workspace
` as a stdio MCP server in your agent's + settings (`td connect` registers it for one client only). MCP tools act on the main + clone; the `td` CLI acts on the clone it runs in. +- Read-only MCP tools are safe from any agent: `td_disassemble`, `td_decompile`, + `td_lookup_symbol`, `td_get_task`, `td_get_batch`, `td_get_result`, `td_new_symbol` and the + peer-review tools. Anything that changes a working tree runs as the CLI from your own slot: + `td region claim`, `td claim`, `td check `, `td submit --files ...`, + `td heartbeat`, `td release`, `td sync`. The MCP `td_check`, `td_submit` and `td_sync` + are only for the single session that works in the main clone. +- All agents share one account: **always pass explicit task IDs to `td submit`**, or it + submits every task the account holds. Never commit or push; the server merges. +- Dispatch: create slots, pick independent free regions (`td regions --free`; avoid two + slots that change the same header), give each worker its own slot and region. A + peer-review worker needs no slot. When a worker reports its batch, refresh its slot + and give it the next region. + ## Structural maintenance (maintainers) For data ownership, annotation moves and new owner units that belong to no function task: @@ -208,7 +244,7 @@ Setup td ghidra download the server's Ghidra project (names applied, refreshed after merges) td ghidra install | build | sync pinned Ghidra: install it, analyze locally, re-apply symbols td update update td now (it also checks by itself once an hour) - td connect [--workspace ] connect your AI (registers with Claude Code, prints MCP settings) + td connect [--workspace ] register the MCP with a supported client, print MCP settings td mcp [--workspace ] run the MCP server for your agent (stdio) Work (a region: a source file or class, done as a whole) diff --git a/scripts/workspace/check.ps1 b/scripts/workspace/check.ps1 new file mode 100644 index 0000000..d3c9f78 --- /dev/null +++ b/scripts/workspace/check.ps1 @@ -0,0 +1,54 @@ +# Back up a slot's edits, then run `td check` from that slot. +# Usage: pwsh scripts/workspace/check.ps1 ... [--includers] [--full] +# --full print td's whole output instead of one line per function +$ErrorActionPreference = 'Stop' + +# Layout: \
\scripts\workspace\, or \workers\\scripts\workspace\. +$Clone = Split-Path -Parent (Split-Path -Parent $PSScriptRoot) +if ((Split-Path -Leaf (Split-Path -Parent $Clone)) -eq 'workers') { + $Root = Split-Path -Parent (Split-Path -Parent $Clone) + $Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { Join-Path $Root 'rodecomp' } +} else { + $Root = Split-Path -Parent $Clone + $Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { $Clone } +} +if ($args.Count -lt 2) { Write-Error 'usage: check.ps1 ... [--includers] [--full]'; exit 2 } +$name = $args[0] +$slot = Join-Path $Root "workers\$name" +if (-not (Test-Path (Join-Path $slot '.git'))) { Write-Error "no slot: $slot (run slot.ps1 $name)"; exit 1 } + +$rest = @($args | Select-Object -Skip 1) +$full = $rest -contains '--full' +$tdArgs = @($rest | Where-Object { $_ -ne '--full' }) + +$work = Join-Path $Root "_work\$name" +New-Item -ItemType Directory -Force $work | Out-Null +$stamp = Get-Date -Format 'yyyyMMdd-HHmmss' +$patch = Join-Path $work "pre-check-$stamp.patch" +$log = Join-Path $work "check-$stamp.log" + +# git writes the patch itself: PowerShell pipelines would re-encode it and break `git apply`. +git -C $slot diff HEAD --binary "--output=$patch" +$before = (git hash-object -- $patch | Out-String).Trim() + +Push-Location $slot +try { + $out = & td check @tdArgs 2>&1 | ForEach-Object { "$_" } + $rc = $LASTEXITCODE +} finally { Pop-Location } +[IO.File]::WriteAllLines($log, [string[]]$out, [Text.UTF8Encoding]::new($false)) + +if ($full) { + $out +} else { + $lines = $out | Where-Object { $_ -match '^\s*(fn|gl|vt|st)_\d+ |error|unannotated' } + if ($lines) { $lines } else { $out } +} + +$afterFile = Join-Path $work "post-check-$stamp.patch" +git -C $slot diff HEAD --binary "--output=$afterFile" +$after = (git hash-object -- $afterFile | Out-String).Trim() +Remove-Item $afterFile +if ($before -ne $after) { Write-Warning "working tree changed during td check; edits saved in $patch" } +Write-Host "log: $log backup: $patch exit: $rc" +exit $rc diff --git a/scripts/workspace/check.sh b/scripts/workspace/check.sh new file mode 100644 index 0000000..5049aa5 --- /dev/null +++ b/scripts/workspace/check.sh @@ -0,0 +1,49 @@ +#!/usr/bin/env bash +# Back up a slot's edits, then run `td check` from that slot. +# Usage: bash scripts/workspace/check.sh ... [--includers] [--full] +# --full print td's whole output instead of one line per function +set -euo pipefail + +# Layout: /
/scripts/workspace/, or /workers//scripts/workspace/. +# Native path (C:/...) on Git Bash/MSYS so Windows tools accept it; POSIX path elsewhere. +CLONE="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && { pwd -W 2>/dev/null || pwd; })" +if [ "$(basename "$(dirname "$CLONE")")" = workers ]; then + ROOT="$(dirname "$(dirname "$CLONE")")"; MAIN="${RODECOMP_MAIN:-$ROOT/rodecomp}" +else + ROOT="$(dirname "$CLONE")"; MAIN="${RODECOMP_MAIN:-$CLONE}" +fi +[ $# -ge 2 ] || { echo "usage: $0 ... [--includers] [--full]" >&2; exit 2; } +name="$1"; shift +slot="$ROOT/workers/$name" +[ -d "$slot/.git" ] || { echo "no slot: $slot (run slot.sh $name)" >&2; exit 1; } + +full=0; args=() +for a in "$@"; do + if [ "$a" = "--full" ]; then full=1; else args+=("$a"); fi +done + +work="$ROOT/_work/$name"; mkdir -p "$work" +stamp="$(date +%Y%m%d-%H%M%S)" +patch="$work/pre-check-$stamp.patch" +git -C "$slot" diff HEAD --binary > "$patch" +before="$(git -C "$slot" diff HEAD --binary | git hash-object --stdin)" + +log="$work/check-$stamp.log" +set +e +(cd "$slot" && td check "${args[@]}") > "$log" 2>&1 +rc=$? +set -e + +if [ $full -eq 1 ]; then + cat "$log" +else + # One line per function (stable ID lines) plus errors and the unannotated list. + grep -E '^\s*(fn|gl|vt|st)_[0-9]+ |error|unannotated' "$log" || cat "$log" +fi + +after="$(git -C "$slot" diff HEAD --binary | git hash-object --stdin)" +if [ "$before" != "$after" ]; then + echo "WARNING: working tree changed during td check; edits saved in $patch" >&2 +fi +echo "log: $log backup: $patch exit: $rc" +exit $rc diff --git a/scripts/workspace/slot.ps1 b/scripts/workspace/slot.ps1 new file mode 100644 index 0000000..4b46acc --- /dev/null +++ b/scripts/workspace/slot.ps1 @@ -0,0 +1,62 @@ +# Create or refresh private clones (slots) under workers\. +# Usage: pwsh scripts/workspace/slot.ps1 w1 [w2 ...] (also works in Windows PowerShell 5.1) +# A dirty slot is never touched. +$ErrorActionPreference = 'Stop' + +# Layout: \
\scripts\workspace\, or \workers\\scripts\workspace\. +$Clone = Split-Path -Parent (Split-Path -Parent $PSScriptRoot) +if ((Split-Path -Leaf (Split-Path -Parent $Clone)) -eq 'workers') { + $Root = Split-Path -Parent (Split-Path -Parent $Clone) + $Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { Join-Path $Root 'rodecomp' } +} else { + $Root = Split-Path -Parent $Clone + $Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { $Clone } +} +if ($args.Count -eq 0) { Write-Error 'usage: slot.ps1 ...'; exit 2 } +if (-not (Test-Path (Join-Path $Main '.git'))) { Write-Error "main clone not found: $Main"; exit 1 } +$Url = (git -C $Main remote get-url origin).Trim() + +function Invoke-Git { + git @args + if ($LASTEXITCODE -ne 0) { throw "git $($args -join ' ') failed ($LASTEXITCODE)" } +} + +function Copy-Td([string]$Slot) { + $dst = Join-Path $Slot '.td' + New-Item -ItemType Directory -Force $dst | Out-Null + foreach ($f in 'config.json', 'symbols.json', 'symbols.json.etag', 'target.exe') { + $src = Join-Path $Main ".td\$f" + if (Test-Path $src) { Copy-Item -Force $src (Join-Path $dst $f) } + } + $ex = Join-Path $Slot '.git\info\exclude' + if (-not ((Test-Path $ex) -and (Select-String -Path $ex -SimpleMatch -Pattern '.td/' -Quiet))) { + Add-Content -Path $ex -Value '.td/' -Encoding ascii + } +} + +$rc = 0 +foreach ($name in $args) { + if ($name -match '[\\/]' -or $name -in '', '.', '..') { Write-Warning "[$name] invalid slot name"; $rc = 1; continue } + $slot = Join-Path $Root "workers\$name" + New-Item -ItemType Directory -Force (Join-Path $Root 'workers'), (Join-Path $Root "_work\$name") | Out-Null + if (-not (Test-Path (Join-Path $slot '.git'))) { + try { + Invoke-Git clone -q $Main $slot + Invoke-Git -C $slot remote set-url origin $Url + Invoke-Git -C $slot fetch -q origin + Invoke-Git -C $slot checkout -q -B main origin/main + } catch { + if (Test-Path $slot) { Remove-Item -Recurse -Force $slot } + Write-Warning "[$name] creation failed, removed partial clone: $_"; $rc = 1; continue + } + Copy-Td $slot + Write-Host "[$name] created at $(git -C $slot rev-parse --short HEAD)" + continue + } + if (git -C $slot status --porcelain) { Write-Warning "[$name] dirty, skipped"; $rc = 1; continue } + Invoke-Git -C $slot fetch -q origin + Invoke-Git -C $slot checkout -q -B main origin/main + Copy-Td $slot + Write-Host "[$name] refreshed at $(git -C $slot rev-parse --short HEAD)" +} +exit $rc diff --git a/scripts/workspace/slot.sh b/scripts/workspace/slot.sh new file mode 100644 index 0000000..8955d6e --- /dev/null +++ b/scripts/workspace/slot.sh @@ -0,0 +1,55 @@ +#!/usr/bin/env bash +# Create or refresh private clones (slots) under workers/. +# Usage: bash scripts/workspace/slot.sh w1 [w2 ...] +# A dirty slot is never touched. +set -euo pipefail + +# Layout: /
/scripts/workspace/, or /workers//scripts/workspace/. +# Native path (C:/...) on Git Bash/MSYS so Windows tools accept it; POSIX path elsewhere. +CLONE="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && { pwd -W 2>/dev/null || pwd; })" +if [ "$(basename "$(dirname "$CLONE")")" = workers ]; then + ROOT="$(dirname "$(dirname "$CLONE")")"; MAIN="${RODECOMP_MAIN:-$ROOT/rodecomp}" +else + ROOT="$(dirname "$CLONE")"; MAIN="${RODECOMP_MAIN:-$CLONE}" +fi +[ $# -gt 0 ] || { echo "usage: $0 ..." >&2; exit 2; } +[ -d "$MAIN/.git" ] || { echo "main clone not found: $MAIN" >&2; exit 1; } +URL="$(git -C "$MAIN" remote get-url origin)" + +copy_td() { + local dst="$1/.td" + mkdir -p "$dst" + for f in config.json symbols.json symbols.json.etag target.exe; do + if [ -f "$MAIN/.td/$f" ]; then cp -f "$MAIN/.td/$f" "$dst/$f"; fi + done + # Keep .td/ out of `git status`, as in the main clone. + local ex="$1/.git/info/exclude" + grep -qxF '.td/' "$ex" 2>/dev/null || echo '.td/' >> "$ex" +} + +rc=0 +for name in "$@"; do + case "$name" in */*|*\\*|..|.|"") echo "[$name] invalid slot name" >&2; rc=1; continue;; esac + slot="$ROOT/workers/$name" + mkdir -p "$ROOT/workers" "$ROOT/_work/$name" + if [ ! -d "$slot/.git" ]; then + if ! { git clone -q "$MAIN" "$slot" && + git -C "$slot" remote set-url origin "$URL" && + git -C "$slot" fetch -q origin && + git -C "$slot" checkout -q -B main origin/main; }; then + rm -rf -- "$slot" + echo "[$name] creation failed, removed partial clone" >&2; rc=1; continue + fi + copy_td "$slot" + echo "[$name] created at $(git -C "$slot" rev-parse --short HEAD)" + continue + fi + if [ -n "$(git -C "$slot" status --porcelain)" ]; then + echo "[$name] dirty, skipped" >&2; rc=1; continue + fi + git -C "$slot" fetch -q origin + git -C "$slot" checkout -q -B main origin/main + copy_td "$slot" + echo "[$name] refreshed at $(git -C "$slot" rev-parse --short HEAD)" +done +exit $rc diff --git a/scripts/workspace/sym.py b/scripts/workspace/sym.py new file mode 100644 index 0000000..e42f1a7 --- /dev/null +++ b/scripts/workspace/sym.py @@ -0,0 +1,52 @@ +#!/usr/bin/env python3 +"""Offline lookup in the main clone's .td/symbols.json (or a slot's, with --slot). + +Usage: sym.py [--slot wN] [--limit N] +An address matches the symbol that contains it. +""" +import argparse +import json +import os +import sys +from pathlib import Path + +CLONE = Path(__file__).resolve().parents[2] +ROOT = CLONE.parent.parent if CLONE.parent.name == "workers" else CLONE.parent +MAIN = Path(os.environ["RODECOMP_MAIN"]) if os.environ.get("RODECOMP_MAIN") else ( + ROOT / "rodecomp" if CLONE.parent.name == "workers" else CLONE) + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("query") + ap.add_argument("--slot", help="read the slot's snapshot instead of the main clone's") + ap.add_argument("--limit", type=int, default=20) + a = ap.parse_args() + + base = ROOT / "workers" / a.slot if a.slot else MAIN + path = base / ".td" / "symbols.json" + data = json.loads(path.read_text(encoding="utf-8")) + cols = data["columns"] + rows = [dict(zip(cols, r)) for r in data["symbols"]] + + q = a.query + if q.lower().startswith("0x"): + addr = int(q, 16) + hits = [r for r in rows if r["addr"] is not None and r["addr"] <= addr < r["addr"] + max(r["size"] or 1, 1)] + elif any(r["id"] == q for r in rows): + hits = [r for r in rows if r["id"] == q] + else: + ql = q.lower() + hits = [r for r in rows if ql in (r["name"] or "").lower() + or ql in (r["decorated"] or "").lower()] + + for r in hits[: a.limit]: + print(f'{r["id"]:<10} {r["kind"]:<9} {("0x%08X" % r["addr"]) if r["addr"] is not None else "-":<10} size={r["size"] if r["size"] is not None else "-":<6} ' + f'{r["name"] or "-"} unit={r["unit"] or "-"}') + if len(hits) > a.limit: + print(f"... {len(hits) - a.limit} more", file=sys.stderr) + return 0 if hits else 1 + + +if __name__ == "__main__": + sys.exit(main())