Document multi-agent workspace and add Linux/Windows slot, check and symbol helpers; fix td disasm and runner notes
This commit is contained in:
ragnarok-clanker
parent
2c7250f0c8
commit
d33f0e43de
6 changed files with 311 additions and 3 deletions
42
AGENTS.md
42
AGENTS.md
|
|
@ -17,7 +17,8 @@ current policy. It is up to date and takes precedence over this file.
|
|||
## Setup (once per machine)
|
||||
|
||||
`td_toolchain_install` tool (or `td toolchain install` CLI) downloads the compiler and
|
||||
sets it up; on Linux it needs wine. Checks compile **on your machine**. The
|
||||
sets it up. Windows runs it with `--runner native`; Linux needs `--runner wine` or
|
||||
`--runner docker`. Checks compile **on your machine**. The
|
||||
server only compiles your submissions (to prove they build and to route each
|
||||
function) and once more before a merge.
|
||||
|
||||
|
|
@ -44,7 +45,7 @@ td sync
|
|||
any function of it, and the first accepted candidate of each function merges
|
||||
(the others are dropped, the rest of their batch goes on). Prefer a region
|
||||
nobody claimed; share only to help with a big or stuck one.
|
||||
3. **Read the originals**: `td_disassemble` tool (or `td disassemble` CLI) (ground truth) and `td_decompile` tool (or `td decompile` CLI)
|
||||
3. **Read the originals**: `td_disassemble` tool (or `td disasm` CLI) (ground truth) and `td_decompile` tool (or `td decompile` CLI)
|
||||
(imported IDA pseudocode where available, a starting point only).
|
||||
4. **Write each function** in its file, directly preceded by its annotation:
|
||||
```cpp
|
||||
|
|
@ -158,6 +159,41 @@ other contributors' agents: a function merges once "same" leads
|
|||
Implement claimed functions and their single-caller helpers in the unit assigned by the claim;
|
||||
do not split their implementation into separate helper `.cpp` files.
|
||||
|
||||
## Multi-agent workspace
|
||||
|
||||
Several agents can work in parallel on one machine, each in its own clone (a "slot").
|
||||
The container folder holds the main clone (this repository), `workers/<slot>/` (one clone
|
||||
per agent) and `_work/<slot>/` (check logs and edit backups, outside every repository).
|
||||
|
||||
- Each slot gets a copy of the main clone's `.td/` (`config.json`, `symbols.json`,
|
||||
`target.exe`), hidden from git. Never print the token in `.td/config.json` or copy it
|
||||
anywhere except a slot's `.td/`.
|
||||
- `scripts/workspace/` has a Linux/Git Bash and a Windows (PowerShell 5.1+) version of each helper:
|
||||
|
||||
| Purpose | Linux / Git Bash | Windows PowerShell |
|
||||
|---|---|---|
|
||||
| Create or refresh slots from `origin/main`; a dirty slot is skipped | `bash scripts/workspace/slot.sh w1 w2` | `powershell -ExecutionPolicy Bypass -File scripts/workspace/slot.ps1 w1 w2` |
|
||||
| Save the slot's edits to `_work/<slot>/pre-check-*.patch`, run `td check` there, print one line per function | `bash scripts/workspace/check.sh w1 <files> [--includers] [--full]` | `powershell -ExecutionPolicy Bypass -File scripts/workspace/check.ps1 w1 <files> [--includers] [--full]` |
|
||||
| Offline lookup in `.td/symbols.json` | `python3 scripts/workspace/sym.py <id\|name\|0xaddr> [--slot w1]` | `python scripts/workspace/sym.py <id\|name\|0xaddr> [--slot w1]` |
|
||||
|
||||
Run them from the main clone or from any slot. If the main clone is not named
|
||||
`rodecomp`, set `RODECOMP_MAIN` to its path.
|
||||
- MCP: configure `td mcp --workspace <main clone>` as a stdio MCP server in your agent's
|
||||
settings (`td connect` registers it for one client only). MCP tools act on the main
|
||||
clone; the `td` CLI acts on the clone it runs in.
|
||||
- Read-only MCP tools are safe from any agent: `td_disassemble`, `td_decompile`,
|
||||
`td_lookup_symbol`, `td_get_task`, `td_get_batch`, `td_get_result`, `td_new_symbol` and the
|
||||
peer-review tools. Anything that changes a working tree runs as the CLI from your own slot:
|
||||
`td region claim`, `td claim`, `td check <files>`, `td submit <taskIds> --files ...`,
|
||||
`td heartbeat`, `td release`, `td sync`. The MCP `td_check`, `td_submit` and `td_sync`
|
||||
are only for the single session that works in the main clone.
|
||||
- All agents share one account: **always pass explicit task IDs to `td submit`**, or it
|
||||
submits every task the account holds. Never commit or push; the server merges.
|
||||
- Dispatch: create slots, pick independent free regions (`td regions --free`; avoid two
|
||||
slots that change the same header), give each worker its own slot and region. A
|
||||
peer-review worker needs no slot. When a worker reports its batch, refresh its slot
|
||||
and give it the next region.
|
||||
|
||||
## Structural maintenance (maintainers)
|
||||
|
||||
For data ownership, annotation moves and new owner units that belong to no function task:
|
||||
|
|
@ -208,7 +244,7 @@ Setup
|
|||
td ghidra download the server's Ghidra project (names applied, refreshed after merges)
|
||||
td ghidra install | build | sync pinned Ghidra: install it, analyze locally, re-apply symbols
|
||||
td update update td now (it also checks by itself once an hour)
|
||||
td connect [--workspace <dir>] connect your AI (registers with Claude Code, prints MCP settings)
|
||||
td connect [--workspace <dir>] register the MCP with a supported client, print MCP settings
|
||||
td mcp [--workspace <dir>] run the MCP server for your agent (stdio)
|
||||
|
||||
Work (a region: a source file or class, done as a whole)
|
||||
|
|
|
|||
54
scripts/workspace/check.ps1
Normal file
54
scripts/workspace/check.ps1
Normal file
|
|
@ -0,0 +1,54 @@
|
|||
# Back up a slot's edits, then run `td check` from that slot.
|
||||
# Usage: pwsh scripts/workspace/check.ps1 <slot> <file>... [--includers] [--full]
|
||||
# --full print td's whole output instead of one line per function
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Layout: <root>\<main clone>\scripts\workspace\<this>, or <root>\workers\<slot>\scripts\workspace\<this>.
|
||||
$Clone = Split-Path -Parent (Split-Path -Parent $PSScriptRoot)
|
||||
if ((Split-Path -Leaf (Split-Path -Parent $Clone)) -eq 'workers') {
|
||||
$Root = Split-Path -Parent (Split-Path -Parent $Clone)
|
||||
$Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { Join-Path $Root 'rodecomp' }
|
||||
} else {
|
||||
$Root = Split-Path -Parent $Clone
|
||||
$Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { $Clone }
|
||||
}
|
||||
if ($args.Count -lt 2) { Write-Error 'usage: check.ps1 <slot> <file>... [--includers] [--full]'; exit 2 }
|
||||
$name = $args[0]
|
||||
$slot = Join-Path $Root "workers\$name"
|
||||
if (-not (Test-Path (Join-Path $slot '.git'))) { Write-Error "no slot: $slot (run slot.ps1 $name)"; exit 1 }
|
||||
|
||||
$rest = @($args | Select-Object -Skip 1)
|
||||
$full = $rest -contains '--full'
|
||||
$tdArgs = @($rest | Where-Object { $_ -ne '--full' })
|
||||
|
||||
$work = Join-Path $Root "_work\$name"
|
||||
New-Item -ItemType Directory -Force $work | Out-Null
|
||||
$stamp = Get-Date -Format 'yyyyMMdd-HHmmss'
|
||||
$patch = Join-Path $work "pre-check-$stamp.patch"
|
||||
$log = Join-Path $work "check-$stamp.log"
|
||||
|
||||
# git writes the patch itself: PowerShell pipelines would re-encode it and break `git apply`.
|
||||
git -C $slot diff HEAD --binary "--output=$patch"
|
||||
$before = (git hash-object -- $patch | Out-String).Trim()
|
||||
|
||||
Push-Location $slot
|
||||
try {
|
||||
$out = & td check @tdArgs 2>&1 | ForEach-Object { "$_" }
|
||||
$rc = $LASTEXITCODE
|
||||
} finally { Pop-Location }
|
||||
[IO.File]::WriteAllLines($log, [string[]]$out, [Text.UTF8Encoding]::new($false))
|
||||
|
||||
if ($full) {
|
||||
$out
|
||||
} else {
|
||||
$lines = $out | Where-Object { $_ -match '^\s*(fn|gl|vt|st)_\d+ |error|unannotated' }
|
||||
if ($lines) { $lines } else { $out }
|
||||
}
|
||||
|
||||
$afterFile = Join-Path $work "post-check-$stamp.patch"
|
||||
git -C $slot diff HEAD --binary "--output=$afterFile"
|
||||
$after = (git hash-object -- $afterFile | Out-String).Trim()
|
||||
Remove-Item $afterFile
|
||||
if ($before -ne $after) { Write-Warning "working tree changed during td check; edits saved in $patch" }
|
||||
Write-Host "log: $log backup: $patch exit: $rc"
|
||||
exit $rc
|
||||
49
scripts/workspace/check.sh
Normal file
49
scripts/workspace/check.sh
Normal file
|
|
@ -0,0 +1,49 @@
|
|||
#!/usr/bin/env bash
|
||||
# Back up a slot's edits, then run `td check` from that slot.
|
||||
# Usage: bash scripts/workspace/check.sh <slot> <file>... [--includers] [--full]
|
||||
# --full print td's whole output instead of one line per function
|
||||
set -euo pipefail
|
||||
|
||||
# Layout: <root>/<main clone>/scripts/workspace/<this>, or <root>/workers/<slot>/scripts/workspace/<this>.
|
||||
# Native path (C:/...) on Git Bash/MSYS so Windows tools accept it; POSIX path elsewhere.
|
||||
CLONE="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && { pwd -W 2>/dev/null || pwd; })"
|
||||
if [ "$(basename "$(dirname "$CLONE")")" = workers ]; then
|
||||
ROOT="$(dirname "$(dirname "$CLONE")")"; MAIN="${RODECOMP_MAIN:-$ROOT/rodecomp}"
|
||||
else
|
||||
ROOT="$(dirname "$CLONE")"; MAIN="${RODECOMP_MAIN:-$CLONE}"
|
||||
fi
|
||||
[ $# -ge 2 ] || { echo "usage: $0 <slot> <file>... [--includers] [--full]" >&2; exit 2; }
|
||||
name="$1"; shift
|
||||
slot="$ROOT/workers/$name"
|
||||
[ -d "$slot/.git" ] || { echo "no slot: $slot (run slot.sh $name)" >&2; exit 1; }
|
||||
|
||||
full=0; args=()
|
||||
for a in "$@"; do
|
||||
if [ "$a" = "--full" ]; then full=1; else args+=("$a"); fi
|
||||
done
|
||||
|
||||
work="$ROOT/_work/$name"; mkdir -p "$work"
|
||||
stamp="$(date +%Y%m%d-%H%M%S)"
|
||||
patch="$work/pre-check-$stamp.patch"
|
||||
git -C "$slot" diff HEAD --binary > "$patch"
|
||||
before="$(git -C "$slot" diff HEAD --binary | git hash-object --stdin)"
|
||||
|
||||
log="$work/check-$stamp.log"
|
||||
set +e
|
||||
(cd "$slot" && td check "${args[@]}") > "$log" 2>&1
|
||||
rc=$?
|
||||
set -e
|
||||
|
||||
if [ $full -eq 1 ]; then
|
||||
cat "$log"
|
||||
else
|
||||
# One line per function (stable ID lines) plus errors and the unannotated list.
|
||||
grep -E '^\s*(fn|gl|vt|st)_[0-9]+ |error|unannotated' "$log" || cat "$log"
|
||||
fi
|
||||
|
||||
after="$(git -C "$slot" diff HEAD --binary | git hash-object --stdin)"
|
||||
if [ "$before" != "$after" ]; then
|
||||
echo "WARNING: working tree changed during td check; edits saved in $patch" >&2
|
||||
fi
|
||||
echo "log: $log backup: $patch exit: $rc"
|
||||
exit $rc
|
||||
62
scripts/workspace/slot.ps1
Normal file
62
scripts/workspace/slot.ps1
Normal file
|
|
@ -0,0 +1,62 @@
|
|||
# Create or refresh private clones (slots) under workers\<name>.
|
||||
# Usage: pwsh scripts/workspace/slot.ps1 w1 [w2 ...] (also works in Windows PowerShell 5.1)
|
||||
# A dirty slot is never touched.
|
||||
$ErrorActionPreference = 'Stop'
|
||||
|
||||
# Layout: <root>\<main clone>\scripts\workspace\<this>, or <root>\workers\<slot>\scripts\workspace\<this>.
|
||||
$Clone = Split-Path -Parent (Split-Path -Parent $PSScriptRoot)
|
||||
if ((Split-Path -Leaf (Split-Path -Parent $Clone)) -eq 'workers') {
|
||||
$Root = Split-Path -Parent (Split-Path -Parent $Clone)
|
||||
$Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { Join-Path $Root 'rodecomp' }
|
||||
} else {
|
||||
$Root = Split-Path -Parent $Clone
|
||||
$Main = if ($env:RODECOMP_MAIN) { $env:RODECOMP_MAIN } else { $Clone }
|
||||
}
|
||||
if ($args.Count -eq 0) { Write-Error 'usage: slot.ps1 <slot>...'; exit 2 }
|
||||
if (-not (Test-Path (Join-Path $Main '.git'))) { Write-Error "main clone not found: $Main"; exit 1 }
|
||||
$Url = (git -C $Main remote get-url origin).Trim()
|
||||
|
||||
function Invoke-Git {
|
||||
git @args
|
||||
if ($LASTEXITCODE -ne 0) { throw "git $($args -join ' ') failed ($LASTEXITCODE)" }
|
||||
}
|
||||
|
||||
function Copy-Td([string]$Slot) {
|
||||
$dst = Join-Path $Slot '.td'
|
||||
New-Item -ItemType Directory -Force $dst | Out-Null
|
||||
foreach ($f in 'config.json', 'symbols.json', 'symbols.json.etag', 'target.exe') {
|
||||
$src = Join-Path $Main ".td\$f"
|
||||
if (Test-Path $src) { Copy-Item -Force $src (Join-Path $dst $f) }
|
||||
}
|
||||
$ex = Join-Path $Slot '.git\info\exclude'
|
||||
if (-not ((Test-Path $ex) -and (Select-String -Path $ex -SimpleMatch -Pattern '.td/' -Quiet))) {
|
||||
Add-Content -Path $ex -Value '.td/' -Encoding ascii
|
||||
}
|
||||
}
|
||||
|
||||
$rc = 0
|
||||
foreach ($name in $args) {
|
||||
if ($name -match '[\\/]' -or $name -in '', '.', '..') { Write-Warning "[$name] invalid slot name"; $rc = 1; continue }
|
||||
$slot = Join-Path $Root "workers\$name"
|
||||
New-Item -ItemType Directory -Force (Join-Path $Root 'workers'), (Join-Path $Root "_work\$name") | Out-Null
|
||||
if (-not (Test-Path (Join-Path $slot '.git'))) {
|
||||
try {
|
||||
Invoke-Git clone -q $Main $slot
|
||||
Invoke-Git -C $slot remote set-url origin $Url
|
||||
Invoke-Git -C $slot fetch -q origin
|
||||
Invoke-Git -C $slot checkout -q -B main origin/main
|
||||
} catch {
|
||||
if (Test-Path $slot) { Remove-Item -Recurse -Force $slot }
|
||||
Write-Warning "[$name] creation failed, removed partial clone: $_"; $rc = 1; continue
|
||||
}
|
||||
Copy-Td $slot
|
||||
Write-Host "[$name] created at $(git -C $slot rev-parse --short HEAD)"
|
||||
continue
|
||||
}
|
||||
if (git -C $slot status --porcelain) { Write-Warning "[$name] dirty, skipped"; $rc = 1; continue }
|
||||
Invoke-Git -C $slot fetch -q origin
|
||||
Invoke-Git -C $slot checkout -q -B main origin/main
|
||||
Copy-Td $slot
|
||||
Write-Host "[$name] refreshed at $(git -C $slot rev-parse --short HEAD)"
|
||||
}
|
||||
exit $rc
|
||||
55
scripts/workspace/slot.sh
Normal file
55
scripts/workspace/slot.sh
Normal file
|
|
@ -0,0 +1,55 @@
|
|||
#!/usr/bin/env bash
|
||||
# Create or refresh private clones (slots) under workers/<name>.
|
||||
# Usage: bash scripts/workspace/slot.sh w1 [w2 ...]
|
||||
# A dirty slot is never touched.
|
||||
set -euo pipefail
|
||||
|
||||
# Layout: <root>/<main clone>/scripts/workspace/<this>, or <root>/workers/<slot>/scripts/workspace/<this>.
|
||||
# Native path (C:/...) on Git Bash/MSYS so Windows tools accept it; POSIX path elsewhere.
|
||||
CLONE="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && { pwd -W 2>/dev/null || pwd; })"
|
||||
if [ "$(basename "$(dirname "$CLONE")")" = workers ]; then
|
||||
ROOT="$(dirname "$(dirname "$CLONE")")"; MAIN="${RODECOMP_MAIN:-$ROOT/rodecomp}"
|
||||
else
|
||||
ROOT="$(dirname "$CLONE")"; MAIN="${RODECOMP_MAIN:-$CLONE}"
|
||||
fi
|
||||
[ $# -gt 0 ] || { echo "usage: $0 <slot>..." >&2; exit 2; }
|
||||
[ -d "$MAIN/.git" ] || { echo "main clone not found: $MAIN" >&2; exit 1; }
|
||||
URL="$(git -C "$MAIN" remote get-url origin)"
|
||||
|
||||
copy_td() {
|
||||
local dst="$1/.td"
|
||||
mkdir -p "$dst"
|
||||
for f in config.json symbols.json symbols.json.etag target.exe; do
|
||||
if [ -f "$MAIN/.td/$f" ]; then cp -f "$MAIN/.td/$f" "$dst/$f"; fi
|
||||
done
|
||||
# Keep .td/ out of `git status`, as in the main clone.
|
||||
local ex="$1/.git/info/exclude"
|
||||
grep -qxF '.td/' "$ex" 2>/dev/null || echo '.td/' >> "$ex"
|
||||
}
|
||||
|
||||
rc=0
|
||||
for name in "$@"; do
|
||||
case "$name" in */*|*\\*|..|.|"") echo "[$name] invalid slot name" >&2; rc=1; continue;; esac
|
||||
slot="$ROOT/workers/$name"
|
||||
mkdir -p "$ROOT/workers" "$ROOT/_work/$name"
|
||||
if [ ! -d "$slot/.git" ]; then
|
||||
if ! { git clone -q "$MAIN" "$slot" &&
|
||||
git -C "$slot" remote set-url origin "$URL" &&
|
||||
git -C "$slot" fetch -q origin &&
|
||||
git -C "$slot" checkout -q -B main origin/main; }; then
|
||||
rm -rf -- "$slot"
|
||||
echo "[$name] creation failed, removed partial clone" >&2; rc=1; continue
|
||||
fi
|
||||
copy_td "$slot"
|
||||
echo "[$name] created at $(git -C "$slot" rev-parse --short HEAD)"
|
||||
continue
|
||||
fi
|
||||
if [ -n "$(git -C "$slot" status --porcelain)" ]; then
|
||||
echo "[$name] dirty, skipped" >&2; rc=1; continue
|
||||
fi
|
||||
git -C "$slot" fetch -q origin
|
||||
git -C "$slot" checkout -q -B main origin/main
|
||||
copy_td "$slot"
|
||||
echo "[$name] refreshed at $(git -C "$slot" rev-parse --short HEAD)"
|
||||
done
|
||||
exit $rc
|
||||
52
scripts/workspace/sym.py
Normal file
52
scripts/workspace/sym.py
Normal file
|
|
@ -0,0 +1,52 @@
|
|||
#!/usr/bin/env python3
|
||||
"""Offline lookup in the main clone's .td/symbols.json (or a slot's, with --slot).
|
||||
|
||||
Usage: sym.py <fn_00123 | name substring | 0xaddr> [--slot wN] [--limit N]
|
||||
An address matches the symbol that contains it.
|
||||
"""
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
CLONE = Path(__file__).resolve().parents[2]
|
||||
ROOT = CLONE.parent.parent if CLONE.parent.name == "workers" else CLONE.parent
|
||||
MAIN = Path(os.environ["RODECOMP_MAIN"]) if os.environ.get("RODECOMP_MAIN") else (
|
||||
ROOT / "rodecomp" if CLONE.parent.name == "workers" else CLONE)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument("query")
|
||||
ap.add_argument("--slot", help="read the slot's snapshot instead of the main clone's")
|
||||
ap.add_argument("--limit", type=int, default=20)
|
||||
a = ap.parse_args()
|
||||
|
||||
base = ROOT / "workers" / a.slot if a.slot else MAIN
|
||||
path = base / ".td" / "symbols.json"
|
||||
data = json.loads(path.read_text(encoding="utf-8"))
|
||||
cols = data["columns"]
|
||||
rows = [dict(zip(cols, r)) for r in data["symbols"]]
|
||||
|
||||
q = a.query
|
||||
if q.lower().startswith("0x"):
|
||||
addr = int(q, 16)
|
||||
hits = [r for r in rows if r["addr"] is not None and r["addr"] <= addr < r["addr"] + max(r["size"] or 1, 1)]
|
||||
elif any(r["id"] == q for r in rows):
|
||||
hits = [r for r in rows if r["id"] == q]
|
||||
else:
|
||||
ql = q.lower()
|
||||
hits = [r for r in rows if ql in (r["name"] or "").lower()
|
||||
or ql in (r["decorated"] or "").lower()]
|
||||
|
||||
for r in hits[: a.limit]:
|
||||
print(f'{r["id"]:<10} {r["kind"]:<9} {("0x%08X" % r["addr"]) if r["addr"] is not None else "-":<10} size={r["size"] if r["size"] is not None else "-":<6} '
|
||||
f'{r["name"] or "-"} unit={r["unit"] or "-"}')
|
||||
if len(hits) > a.limit:
|
||||
print(f"... {len(hits) - a.limit} more", file=sys.stderr)
|
||||
return 0 if hits else 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Loading…
Add table
Add a link
Reference in a new issue