gitoriaLog in with ident

antcolony

All repositories: gitoria

ReadmeCodePull requestsReleasesTicketsSettings
Address
https://antcolony.gitoria.worldapi.org/
Owner
Caramboleyo
Created

antcolony-scheduler

The colony's deterministic part (ticket antcolony#1, concept: byrodin /CONTAINERS/projects/antcolony/docs/scheduler-agent.md + README.md). It reads and writes tickets.worldapi.org only through its JSON API and holds no state of its own.

v0 (mission 015) = build-order step 2: a command-line program run BY HAND that replaces the architect's manual loop (tools/t.py). Agent v0 (mission 016, ticket antcolony#2) = work: one headless Claude Code worker per call, run by hand — no daemon, heartbeat, WebSocket or quota control yet. Controller + cycle (mission 019, antcolony#1 build order 4 part 1) = after a schema-valid report a second one-shot Claude session checks it (verdict: pass|fail); posting follows the verdict; cycle = next → work → controller → post, once. Daemon (mission 020, build order 4 part 2) = run: the loop — leases that expire (heartbeat), Claude quota reserve, parking + claude --resume, a port range per session, clean SIGTERM. Still ONE host, no WebSocket / agent split. Scheduler + agents per host (mission 035, antcolony#14) = run --serve (scheduler, starts no worker) + agent (per host, runs the workers, owns quota + ports); HTTPS request/answer because Hybriel has no WebSocket client (section "Scheduler + agents per host"). Relations, questions, rejections (mission 022, antcolony#1) = blocked / parent tickets are not work; every report question becomes its own creator-question ticket (parent = the ticket); a rejected ticket is work again, its brief starts with the rejection (section "Relations, questions, rejections" below; lib/relations.hl). Readable ticket texts, weekly limit (mission 023, antcolony#1) = everything the scheduler writes into tickets follows antcolony README "Ticket texts are written for the creator" (≤ 5 short lines, plain words, ONE short machine line last — section "Ticket texts" below); the report gets result + test (they ARE the comment); run starts nothing while the weekly usage is above --week-limit (84 %); hand-filed Question… children count as creator questions. Finalize step, budget in the brief, salvage (mission 025, antcolony#1) = a worker that runs out of steps / time without a report gets its SAME Claude session resumed once to write the report (then controller → post as usual); the brief states the step budget; ./colony salvage <run> does the same for a finished run (section "Finalize step + salvage" below). Resumed sessions report CUMULATIVE cost — the scheduler now counts only the new part. Continue, clean up, cheaper finalize (mission 026, antcolony#1) = the report has complete (true only if the goal is reached AND what the creator tests is really in place); complete: false → "Half done: …; the next worker continues." + back to open, never awaiting creator; the brief of a ticket with an earlier colony session starts with "Previous attempt" (how it ended, result, open, work copy, report path); after every worker / finalize / controller part the scheduler stops what it left LISTENING on its port range (started during that part, never others); the finalize resume has no --disallowedTools (prompt cache: real finalize $0.03 instead of $1.05); salvage takes a lease like work (section "Mission 026" below). After the first live batch (mission 027, antcolony#1) = result ≤ 120 characters, test = 2–4 steps ≤ 90 characters shown as a NUMBERED list, written for the creator on the LIVE site (never a work copy / local address); a deploy gate: a project with live gets "Built — goes live with the next deploy." and stays in progress until ./colony deployed <project> <n> (./colony ready lists them); new report key decided (small choices → one "Decided: …" bullet), ≤ 3 short questions; every brief carries the antcolony "Conventions for all apps" as TEXT and the project's own tickets; a refused report gets ONE fix step (section "Mission 027" below). Permanent operation (mission 028, antcolony#1) = ./colony run --live as the Docker container antcolony-scheduler on Loreana (restart: unless-stopped, starts with the system, no systemd unit); section "Operations" below. Librarian (mission 029, antcolony#21) = ./colony librarian: the creator's own words from tickets (+ the architect's chat seed once) → one model step per new text → DECISIONS.md per project (by topic, current only; superseded → DECISIONS-archive.md) and templates/decisions-global.md; every brief carries them after the conventions; the librarian runs FIRST in every run iteration and before brief/work/cycle; UI projects' briefs name the layout conventions (section "Librarian" below). Lease expiry race (mission 031, antcolony#1) = the daemon never acts on its survey alone: before it expires a lease or resumes a parked session it RE-READS the ticket and writes nothing unless it is STILL in progress with the same lease; a lease ENDS with colony-deployed:, the session's own report / verdict / give-back, or any state change — such a ticket is never expired (section "Mission 031" below).

Keywords in comments (antcolony#17) = the creator's short commands, code only (no model): /hold, /prio, /confirm, /reject <why> as the FIRST word of a comment by the creator (COLONY_CREATOR_NAMES); run handles them at the start of every iteration (no quota needed), by hand ./colony keywords [--post] (dry run lists them; section "Keywords" below).

Status + daily summary (antcolony#19) = ./colony status [--write] (per project: tickets by state, what waits for the creator) and ./colony digest [--post] (one short summary of what waits for the creator); both code only, built from the tickets — section "Status and daily summary" below.

Small local models (antcolony#20) = the librarian's routine decision (is a creator text a decision, for which project) can run on a small model on a local llama-server instead of Claude — section "Small local models" below.

Operations (mission 028) — the container antcolony-scheduler on Loreana

What it is: docker-compose.yml here. A busybox container bind-mounts the host's / at /host, docker/pivot.sh (the only root part) makes its mounts private, pivot_roots into the host filesystem and drops to uid 1000 (mre) with all of mre's groups → docker/entrypoint.sh → ./colony run [--live]. So the scheduler and every worker see the host's toolchain unchanged: claude (~/.local/bin/claude + login in ~/.claude), node, google-chrome-stable (with its sandbox), rsync, ssh, /media/STORAGE/projects, ~/.config/antcolony/tickets-token. Host network, host PID namespace (lib/cleanup.sh reads /proc + ss), cap SYS_ADMIN + seccomp=unconfined (needed for pivot_root and Chrome's user namespaces; uid 1000 has no effective capabilities). Why not an image with the tools installed: Chrome (AUR), node and claude versions would drift from the host, and Chrome's sandbox would still need the same security options. Why not chroot: the kernel refuses user namespaces to chrooted processes → Chrome's sandbox fails.

cd /media/STORAGE/projects/antcolony-scheduler
cp .env.example .env && $EDITOR .env      # once: settings (creator 2026-09-24: COLONY_RESERVE=100, COLONY_WEEK_LIMIT=100)
docker compose up -d                      # start (creates/recreates the container; also after every .env change)
docker stop antcolony-scheduler           # stop (see "Stop" below); stays stopped, also after a reboot
docker start antcolony-scheduler          # start again with the same settings (parked sessions resume)
docker logs -f antcolony-scheduler        # RUN / ITER / SESSION END lines (json-file, 3 × 10 MB)
tail -f logs/colony.log                   # the same lines with a UTC time, rotates at 10 MB, keeps colony.log.1 … .5
docker compose ps                         # state; `docker inspect -f '{{.RestartCount}}' antcolony-scheduler`
  • Settings: env in docker-compose.yml, overridden by .env (template .env.example, all commented with the defaults): COLONY_LIVE (1 = --live), COLONY_RESERVE (→ COLONY_QUOTA_RESERVE, 70), COLONY_WEEK_LIMIT (84), COLONY_RUN_INTERVAL (60), COLONY_PARALLEL (1), COLONY_MODEL, COLONY_MAX_TURNS, COLONY_WORK_TIMEOUT, COLONY_CONTROLLER_*, COLONY_MAX_BUDGET_USD, COLONY_PORT_POOL, COLONY_PORTS_PER_SESSION, COLONY_RUN_ARGS (extra run options), COLONY_STOP_FINISH_SECONDS (20), COLONY_LOG_MAX_BYTES / COLONY_LOG_KEEP. Change → docker compose up -d (a plain docker start keeps the old values). The first log lines print every COLONY_* value in use.
  • Stop: docker sends ONE SIGTERM; the entrypoint maps it onto the wrapper's two signals: 1st TERM → finish (nothing new starts; idle → stops in ~1 s); still a session running after COLONY_STOP_FINISH_SECONDS (20 s) → 2nd TERM → park (worker/controller killed, colony-parked marker, ticket stays leased; took < 1 s in the test) → exit. stop_grace_period: 75s covers it and stays below systemd's 90 s stop timeout of docker.service at shutdown. A docker kill / SIGKILL takes the running session with it (no park; the lease expires after --lease-ttl).
  • Reboot: docker is enabled at boot; unless-stopped starts the container again unless it was stopped by hand (docker stop). At shutdown docker stops it like docker stop → parked; after the boot run resumes the parked sessions (claude --resume). If the scheduler process dies, the container exits and docker restarts it.
  • One-off colony commands next to the daemon: either on the host as mre (same filesystem, same tools) — COLONY_TOKEN_FILE=~/.config/antcolony/tickets-token ./colony leases --live — or inside the container with ITS settings (docker exec lands in the host filesystem because of the pivot; numeric user only, no supplementary groups): docker exec -u 1000:1000 -e HOME=/home/mre -e PATH=/home/mre/.local/bin:/usr/bin:/bin -w /media/STORAGE/projects/antcolony-scheduler antcolony-scheduler ./colony leases --live. Don't run work/cycle by hand on a ticket the daemon may pick (leases protect in-progress tickets only).
  • Ports: default pool 8700-8799; on Loreana 8765, 8766, 8787, 8795 belong to other services (2026-09-24) — a session given 8760-8769 / 8780-8789 / 8790-8799 may find ports busy.
  • Test instance (never live): .scratch/m028/dc.sh up -d | stop | down = the same compose file + .scratch/m028/compose.test.yml (container antcolony-scheduler-m028, own tickets 8752 + ident 8751 from node .scratch/m028/own-tickets.mjs 6, fake claude, pool 8770-8779, logs in .scratch/m028/logs/); section "Tests of the container" below.

Small local models (antcolony#20) — routine decisions without Claude

Which routine decisions call a model today: only the librarian (classify: decision or not, project, topic, reading). "Which ticket next" (next) and keywords (/hold, /confirm …) are code, no model; the controller (does the report hold?) stays on Claude — it is the check on the worker, not a routine call.

  • Switch: COLONY_LIBRARIAN_MODEL=local (or local:<alias> = the model name sent to the server; --model / --librarian-model) — default stays sonnet. The same prompt goes to an OpenAI-compatible POST <COLONY_LOCAL_URL>/chat/completions (default http://127.0.0.1:8080/v1, COLONY_LOCAL_TIMEOUT 120 s) with response_format: json_schema = templates/librarian.schema.json, temperature 0 — the server enforces the schema. The call files are librarian/calls/<event>-local-{prompt.md,result.json}.
  • Escalation, never a silent wrong answer: the local answer passes the same checkAnswer as Claude's (project allowed, topic, verbatim quote, known supersedes). Server unreachable / not 200 / answer cut off / checks refuse → the same text goes to Claude (COLONY_LIBRARIAN_FALLBACK, default sonnet, off = no fallback → counts as "no answer", retried next pass). Cost of a local answer: $0.
  • The server is not started by the scheduler: one llama-server per GPU (concept: exactly one on the Arc at a time) belongs to the host's agent/operator; the scheduler only needs the URL. tests : local-test.hl (root, so ./templates resolves) runs one canned text through classify: COLONY_HOME=$PWD COLONY_LOCAL_URL=http://127.0.0.1:PORT/v1 bin/hybriel local-test.hl -- DIR local off.
  • Measured (this host, CPU only — the box sees no GPU; 184 real librarian prompts with Claude's answers as reference): see STATUS.

Librarian (mission 029, ticket antcolony#21) — the creator's decisions in one place per project

Why: a worker sees its ticket, the concept and the TITLES of other tickets; the creator's answers and decisions in comments never reached it (gitoria#3 asked about "shared components live in layouts.worldapi.org", decided long before). The librarian collects the creator's own words and every brief carries them.

Where the files go (mission 030): <dev folder>/DECISIONS.md on this host, else the code folder, else librarian/projects/<p>/DECISIONS.md — and ALWAYS librarian/projects/<p>/ for a project with "workers": false (hybriel: its own process in its own repo; the librarian never writes into it). The hybriel file lives in librarian/projects/hybriel/DECISIONS.md.

Sources (only texts BY THE CREATOR — never the architect's or the scheduler's). Creator 2026-09-24: "the librarian should be enough. i have to adopt to the workflow" → tickets only (+ the chat seed once); no chat transcripts.

  1. tickets (READ only, GET /api/tickets + each ticket updated since the last pass): comments, state-change texts (incl. answers on Question… tickets — the question + the history before it go to the model as context), the subject + summary of tickets the creator opened, his edits. The creator = the event's author is one of COLONY_CREATOR_NAMES (default Caramboleyo,creator; creator = the free-text author of events from before the login). The tickets API shows display names only, no user ids — see STATUS "open".
  2. the architect's chat SEED (decisions made in chat, once): ./colony librarian --post-files --seed /media/STORAGE/projects/antcolony-docs/docs/global-decisions-seed.md — no model call; ## Topic headings, bullets - "quote" · "quote" · reading: … (wrapped lines joined with one space), every "…" part is a quote, the text before the first quote is kept as context, the reading is marked as the ARCHITECT's; a bullet without a quote is skipped (printed); idempotent (sha256 of the quotes).

One model step per new text (lib/librarian.hl classify): claude -p --output-format json --json-schema templates/librarian.schema.json --model sonnet --max-turns 4 --permission-mode default --permission-prompts none --strict-mcp-config --tools "" --no-session-persistence, cwd librarian/, prompt = templates/librarian.md + the input as JSON (the text, the ticket, the history before it, the allowed projects = registry + the ticket's project + all, the existing topics, every current entry). Answer: {decision, project, topic, quote, means, supersedes, why}. The code decides what is stored: project must be allowed, topic 1–40 characters; means = ONE line (longer than 300 characters → clipped + note); quote must be "" (the whole text) or a VERBATIM excerpt (checked character for character — else the whole text is stored, note: the excerpt … is not verbatim); supersedes only ids of current entries. The creator's words always come from the ticket, never from the model. A refused answer is retried once (next pass), no answer up to 3 times, then given up (--retry-refused takes the refused ones again; --redo EVENT[,EVENT] classifies texts again and removes the librarian's own entry for them — for a wrong reading, not a creator correction). No model call when nothing is new. A bare "yes"/"no" answers the ticket's SUBJECT (prompt rule, after the first live pass read "yes" on ident#13 as the summary's suggestion).

Files (rendered from librarian/decisions.json, never edited by hand — header says so; rewritten only when their text changes):

  • per project DECISIONS.md in its dev folder when that is on this host, else its code folder on this host (antcolony → the scheduler folder), else librarian/projects/<name>/DECISIONS.md (a project without a registry file); all → templates/decisions-global.md (COLONY_DECISIONS_GLOBAL).
  • CURRENT state only, by TOPIC (the model picks a short topic, reusing existing ones), ## Contents at the top, newest first inside a topic. Entry: - <date> · **"<the creator's exact words>"** · [<project #n — subject>](<ticket url>) (seed: · source: architect session 2026-09-23/24, chat) + - means (the librarian's reading, not the creator's words): … (seed: the architect's reading). Multi-line texts as a blockquote.
  • a correction ("i never said …") → the model lists the entries it supersedes → they move to DECISIONS-archive.md (decisions-global-archive.md) beside it, kept with quote + link + superseded on <date> by <id> — "<new quote>".

In every brief (brief.hl, after "Conventions for all apps", before "Where"): ## Decisions of the creator — a decision here overrides anything else; ask only if it is not decided here · both file paths · "Search both files … before you ask a question" · ### For all projects + ### For <project>: the whole file while it has ≤ 150 lines (COLONY_DECISIONS_FULL_LINES), else its contents list + only the topic sections that share a word with the ticket (subject + summary; words ≥ 4 letters without stop words, compared by their first 5 letters, against the topic name and the entries' "means" lines) + "This file is long … Search the whole file <path> before you ask anything". Read-first item 5 names the decisions; item 6 (registry "ui": true: tickets, ident, gitoria) names the creator's markup/CSS rules /media/STORAGE/projects/antcolony-docs/docs/layouts-conventions.md (COLONY_LAYOUTS_CONVENTIONS).

ORDER — the librarian always runs BEFORE a brief is built (a decision the creator wrote in ANY ticket a moment ago is in DECISIONS.md when the brief is assembled):

  • run: every iteration starts with ONE librarian pass and WAITS for it (daemon.hl tick → librarianPass → only then iterate: leases, candidates, quota, starts). A pass that failed or left a creator text unfiled (e.g. --librarian-max reached, a text without an answer yet) → ITER … · librarian: n creator text(s) not filed yet → no start this iteration (or librarian failed (…) → no start this iteration) — nothing starts until the texts are filed / given up (≤ 3 tries).
  • by hand: brief, work, cycle await one pass first (librarianFirst, scheduler.hl); not for --dry-run (no model call there) or --librarian off. A pass that fails / finds the lock taken says so (librarian: brief goes on WITHOUT a finished librarian pass (…)) and the command goes on with the files as they are.
  • Remaining gap: a text written while the pass runs or after it (seconds) reaches the NEXT iteration's briefs.
  • Usage limit (mission 034, antcolony#25) — see "Usage limit" under run: the quota is read BEFORE every librarian pass of run (at or above the start threshold → no librarian call, no start); a librarian call that still hits the limit prints → USAGE LIMIT — <Claude's text> · resets <time | at an unknown time> → the pass stops, the text stays unfiled (no try counted) and run pauses. By hand (brief/work/cycle): librarian: the Claude usage limit is reached (resets …) — n creator text(s) not filed; <cmd> goes on with the files as they are.
./colony librarian                                  # dry run: the new creator texts; no model call, nothing written
./colony librarian --post-files [--since 2026-09-24|ms] [--max N] [--model M] [--retry-refused] [--redo EVENT,EVENT]
./colony librarian --post-files --seed FILE [--seed-source TEXT]
./colony librarian --render                         # all files again from librarian/decisions.json (no tickets, no model)
python3 -m json.tool librarian/state.json | head    # cursorMs (newest ticket updatedMs read; stays while texts are left), floorMs (--since), done, tries
python3 -c "import json;[print(e['id'],e['status'],e['project'],e['topic'],e['quotes']) for e in json.load(open('librarian/decisions.json'))['entries']]"

Output: librarian: n new creator text(s) …, per text LIBRARIAN TEXT <ref> · <what> · <author> · <when> · "<text>" → LIBRARIAN USAGE: … → → DECISION <id> · <project> · <topic> · quote "…" · means: … / → supersedes <id> / → not a decision — … / → REFUSED (k of 2 tries) — … / → NO ANSWER (k of 3 tries) — … / → note: …; LIBRARIAN FILE <path> (n current · <where>); LIBRARIAN: … classified …; LIBRARIAN COST: $x for n model call(s). librarian: BUSY — … = another pass holds librarian/lock (stale after 15 min — a pass killed mid-way blocks starts of run for at most that long). In run the pass is quiet when nothing is new; --librarian off / COLONY_LIBRARIAN=off, --librarian-model, --librarian-max (0 = all). The RUN: line says librarian on, first in every iteration (sonnet). The live container runs the old code until it is restarted (docker compose up -d) — until then run the librarian by hand (as mre on Loreana, see "Operations"; no token needed, it only reads; CLAUDE_CONFIG_DIR=/home/mre/.claude-colony = the colony's own Claude subscription like the container).

Scheduler + agents per host (mission 035, ticket antcolony#14; concept docs/scheduler-agent.md §1–§3)

Two programs instead of one. ./colony run alone is still the all-in-one loop of missions 020–034 (one host, starts its workers itself — the live container on Loreana runs like this until the architect switches it). New:

commandwheredoes
scheduler./colony run --serve PORTByrodin, next to ticketspicks work, assembles the brief, leases/expiry, librarian; starts no worker, holds no data (tickets only)
agent./colony agent --scheduler URLevery host that has dev folders (Loreana, Byrodin)runs the workers the scheduler sends, owns quota + ports of its host, reports back

Connection (why HTTP, not a WebSocket). The concept says the agent "connects OUT over one WebSocket". Hybriel has no WebSocket client (hybriel#109, open) and a JS bridge is forbidden, so the agent asks and the scheduler answers with plain HTTPS: every --poll s (default 5) POST <scheduler>/agent/sync, JSON, Authorization: Bearer <agent token> (COLONY_AGENT_TOKEN_FILE, a secret both sides read; scheduler and agent refuse to start without it; a wrong / missing token → 401, other path → 404, bad JSON → 400):

  • agent → { host, parallel, accepting, why, running: [{ref, session, folder, phase, ports}], events: [not yet acked] }
  • scheduler → { ack: [event ids], commands: [{kind: start|resume, project, number, session, folder, brief}] } Same effects as the socket: the agent reconnects by itself; a dropped connection breaks nothing — sessions keep running (their lease heartbeat and their report go to tickets directly), SESSION END / refused events are buffered until the scheduler acks them (delivered after the reconnect, printed once — ids dedupe); a command sent twice is ignored (session id); a command whose answer was lost is sent again at the next round. When hybriel#109 is fixed only agent.hl (client) and AgentServer.hl (server) change.

Capabilities (antcolony#15). Each agent announces what its computer offers: --capabilities gpu,chrome,deploy / COLONY_CAPABILITIES (comma list, lower-cased; default nothing) — sent in every sync, shown in AGENT: H connected · offers …. A project's registry file may say "needs": ["chrome", "gpu"]; the scheduler starts a NEW ticket of it only on a host whose agent offers every need, otherwise the iteration says P#n waits — H does not offer: gpu and the ticket stays open. Names are free words (nothing is detected automatically). The single-host run takes the same --capabilities for its own host. A parked session's resume is not checked (it already started there). Test: tests/e2e-agent.mjs (project gp needs a gpu, ag needs chrome).

Scheduler side (daemon.hl, AgentServer.hl; --serve PORT, --serve-host (default 127.0.0.1 — behind TLS/nginx on Byrodin), --agent-ttl (30 s), no --once): the state of the agents (host, last sync, parallel, accepting, running) lives in memory only — each sync rebuilds it, so a restart of the scheduler needs no recovery. A candidate is a ticket whose registry dev.host has an ONLINE agent that is accepting with a free slot; otherwise waits — no agent online on H / agent H is not accepting work (why) / has all N slot(s) busy. A start = the scheduler builds the brief (brief.hl, with the creator's decisions it holds; ports are the marker @AGENT@), queues start, sends it with the agent's next sync. A parked session on host H is resumed by H's agent (resume; the mission-031 re-read before it is unchanged). Lines: AGENT: H connected · parallel n · …, AGENT: H is OFFLINE (no sync for S s) — n session(s) were on it; their leases in tickets decide (the workers keep renewing; a lease nobody renews expires as before → back to open), SESSION END <ref> <session> · <outcome> · cost … · on H · ports …, ITER … · agents: H 1/2, …. A stop (SIGTERM) ends the scheduler at once: sessions keep running on their agents.

Agent side (agent.hl; --scheduler URL / COLONY_SCHEDULER_URL, --poll / COLONY_AGENT_POLL, --parallel, --reserve, --week-limit, --port-pool, --ports-per-session, --quota-every / COLONY_AGENT_QUOTA_EVERY (60 s), the worker options of run; needs COLONY_TOKEN_FILE — its workers lease and post to tickets — and COLONY_AGENT_TOKEN_FILE): it runs the very same work (lease, worker, controller, post) with the brief it was sent (work --brief-file; the ticket checks still run on this host and its own port range replaces the marker). Quota is the agent's: claude -p /usage at most every --quota-every s while a slot is free; at/above the reserve, above the weekly limit, reached limit, or a session parked on the usage limit → accepting: false + why (AGENT: not accepting work — …, LIMIT: …), a command that arrives anyway is refused (event refused). Ports: its own pool, one disjoint range per session. Stop (via the colony wrapper, like run): 1st signal → accepts nothing, sessions finish, AGENT STOPPED; 2nd → they are parked. Every line starts with AGENT / SESSION END / LIMIT / work:.

What each host needs: this folder (code + projects/ registry + templates/), bin/hybriel with plugins/http1 (server) and fetch, the tickets token, the agent token; agent hosts additionally claude and the dev folders. The scheduler host needs claude only for the librarian (COLONY_LIBRARIAN=off where there is none — the decisions then stop updating).

Not decided by this ticket / open: the container files for Byrodin (scheduler) and per-host agent containers — the architect's deploy; the librarian still runs on the scheduler host (needs Claude there); the quota is a shared per-account view since antcolony#16 (next section).

Claude quota per account, shared across computers (antcolony#16; concept §3 "Claude quota per account, not per host")

The quota belongs to the Claude ACCOUNT. Hosts that are logged in to the same account name it the same: ./colony agent --account NAME / COLONY_CLAUDE_ACCOUNT (default = the host's own name = its own account, nothing shared; nothing is detected automatically). Every sync now also carries account, quota (the agent's last /usage reading: percent, weekly, resetsMs, at) and limitUntil (its usage-limit pause, if any). The scheduler pools them per account (daemon.hl accountBlock):

  • the FRESHEST reading of any online agent of the account counts — a host that has not read the quota itself starts on another host's reading (no reading, or one older than 5 min → the quota is not read yet / older than 5 min → waits);
  • same rules as one host: 5 h ≥ --reserve, week > --week-limit (or unknown), 100 % reached → the ticket waits (P#n waits — Claude account A: quota 5 h 80% ≥ reserve 70%);
  • a usage limit hit on ANY host of the account holds every host of it (… usage limit reached until T (hit on H));
  • one new start per account at a time: after a start (or resume) no further start on that account until a reading taken --account-settle s (default 60, COLONY_ACCOUNT_SETTLE) AFTER it is in — otherwise two hosts read 69 % at once and both start. Also within one iteration (… a start on it (T) is not in the quota reading yet — shared by H1, H2). The ITER line lists account A 12% week 41% [h1+h2]; the AGENT: H connected line names the account. Disconnected agent: the concept's "conservative fixed share" needs nothing extra — an agent starts only what the scheduler sends and refuses on its own reading (reserve/week/limit) too, so a host without the scheduler starts nothing new. Test: tests/e2e-agent.mjs section 5b (hand-made syncs of two hosts on one account; +5 checks).

Run

cd /media/STORAGE/projects/antcolony-scheduler          # (Loreana; Hybriel 1 binary vendored)
./colony next [--post]
./colony brief <project> <n> [--out FILE] [--ports A-B] [--session ID]
./colony report <file> [--post]
./colony work <project> <n> [--post] [--dry-run] [--model M] [--max-turns N] [--permission-mode MODE]
              [--timeout SECONDS] [--max-budget-usd X] [--ports A-B] [--session ID]
              [--controller-model M] [--controller-max-turns N] [--controller-timeout S] [--max-fails N]
              [--finalize-max-turns N] [--finalize-timeout S]
./colony check <runs/session dir> [--post] [--again] [controller options]
./colony salvage <runs/session dir> [--post] [--finalize-max-turns N] [--finalize-timeout S] [--ports A-B] [controller options]
./colony cycle [--post] [--dry-run] [work options]
./colony run [--once] [--interval S] [--parallel K] [--reserve P] [--week-limit P] [--port-pool A-B] [--ports-per-session N]
             [--lease-ttl S] [--heartbeat S] [--quota-claude BIN] [--live] [work options]
./colony leases [--expire] [--live]
./colony ready                              # mission 027: tickets built on a project with a live site, waiting for the deploy
./colony deployed <project> <n>             # mission 027: after the deploy — readable comment + awaiting creator
./colony librarian [--post-files] [--since D|ms] [--max N] [--model M] [--retry-refused] [--redo EVENT,…]   # mission 029 (section "Librarian")
./colony librarian --post-files --seed FILE | --render

./colony = bin/hybriel scheduler.hl -- "$@" — the -- is needed: the hybriel binary refuses any --option after the script unless it follows -- (Hybriel issue, STATUS). Output is plain text; the exit status is always 0 (Hybriel has no way to set it) — read the first word: NEXT:, brief:, REPORT OK / REPORT REFUSED, POSTED / POST REFUSED / POST FAILED, work: REFUSED, WORK DONE / WORK OK / WORK FAILED (last line of work; CHECK … for check), VERDICT: PASS|FAIL, CYCLE COST: + CYCLE END — / CYCLE DRY RUN / CYCLE — (last lines of cycle); run: RUN:, ITER, SESSION END, PARKED —, RUN STOPPED —; leases: LEASE / LEASES:; finalize (mission 025): work: FINALIZE —, FINALIZE USAGE:, FINALIZE COST:, work: FINALIZE FAILED —; salvage: salvage: REFUSED —, SALVAGE DONE / SALVAGE OK / SALVAGE FAILED, SALVAGE COST:, SALVAGE END —; mission 026: WORK HALF DONE / CHECK HALF DONE / SALVAGE HALF DONE (a passed report with complete: false), work|finalize|controller: CLEANUP — stopped pid … / skipped pid … / nothing listening on A-B, salvage: <ref> leased until ….

next

Reads every ticket (GET /api/tickets), applies the rules and prints NEXT: <project>#<n> plus why the others wait:

  • eligible = state open + the project has a registry file + that file names a concept + it does not say "workers": false (such a project — hybriel — is listed "takes no workers", no question); v0 priority = oldest (stored createdMs, ties → lower number).
  • not eligible, listed with the reason: no registry file / empty concept (→ question for the creator), in progress (= leased — v0 has no agent, the state is the lease), on hold, blocked by rel#2 (open) (any blocker not confirmed), a parent (n child tickets, k confirmed) — not work itself, its children are (mission 022; the scheduler's own creator-question children don't count).
  • rejected with a rejection newer than the last colony session → a candidate again (same rules), marked — REJECTED by the creator <when> (newer than the last colony session) → rework (mission 022).
  • awaiting creator / confirmed / other rejected are only counted.
  • --post: every project with open tickets but no metadata / no concept gets ONE creator question (ticket in project antcolony, state awaiting creator; idempotent via source colony:no-metadata:<p> / colony:no-concept:<p> → a re-run prints HAVE, writes nothing).

brief

Writes a worker brief for one ticket (default briefs/<project>-<n>.md; --out relative to the cwd). Refused for a project without metadata or concept. Parts, in the order of the architect's missions/*.md: [REJECTED by the creator: <reason> FIRST while a rejection is unanswered — mission 022] · header (ticket URL, session id, UTC time) · Read first (ticket, concept path, project README/STATUS, antcolony README) · Where (dev/code/live folders, deploy note, port range, default 8700-8749 or COLONY_PORTS) · Relations (parent / children — creator questions marked / blocked by — not confirmed = blocking / blocks; mission 022) · Hybriel block (templates/hybriel-block.md, only if dependsOn has hybriel) · open tickets of every project it depends on (all not confirmed / rejected) · rules (templates/rules.md) · report section = the JSON schema (templates/report.md, filled with ticket, session, known projects) · the ticket + history: tickets' Markdown read view (Accept: text/markdown, tickets#6) or, when tickets does not serve it, rendered from the JSON. Session id: --session or generated s-<UTC yyyymmddThhmm>-<6 hex>.

report

Validates a worker's JSON report (schema: concept §4) and prints EVERY problem with its path, e.g. verified[0]: 'output' is missing (every verified entry needs command AND output): all 12 keys ticket session result complete test done verified open issues questions running refused required (mission 027: + decided, optional here, required in the schema — see "Mission 027" for the tightened limits) (mission 026: complete = true / false; lists may be [], not null), no unknown keys at any level, result = one line ≤ 200 characters, test = 1–4 steps, each one line ≤ 200 characters (mission 023 — result + test are what the creator reads), ticket = <project>#<n>, strings non-empty, verified[] = {claim, command, output}, issues[] = {project, subject, repro, observed, expected, blocks: bool}, running[] = {what, url, pid: number, log}. An issues[].project not in the registry is a note, not a refusal (→ creator question on --post, concept: "else → creator's inbox"). --post (only for a valid report):

  1. the ticket must exist; a comment with colony-report: <session> · sha256 <hash> already there → same hash: nothing is written again; different hash: refused (one session, one report);
  2. each issue → a ticket in its project (summary: link back, Repro / Observed / Expected, "Blocks" when blocks), source colony-report:<session>:issue:<i>; unknown project → creator question (antcolony, awaiting creator, source colony-report:<session>:question:<i>, parent = the ticket); an issue with blocks: true → the ticket gets it as a blocker (POST …/blocked-by, once — mission 022); 2b. (mission 022) each questions entry → its own ticket in the TICKET's project: subject Question (<ticket>): <title> (antcolony#23: the entry is { title, text }, the worker writes the short title, ≤ 80 chars; an old plain string still works, its first line is the title), summary = the question + where it came from, awaiting creator, parent = the ticket, source colony-report:<session>:questions:<i> (HAVE on a re-post);
  3. ONE short comment on the ticket (mission 023, section "Ticket texts"): result · **Test:** 1) … 2) … · at most 3 detail bullets picked by the code (questions filed → problems filed elsewhere → first open entry → running → refused) · the machine line colony-report: <S> · sha256 <12 hex>[ · controller pass]. done, verified etc. are NOT in the ticket — the full report stays in runs/<S>/report.json (+ verdict.json);
  4. ticket → awaiting creator (state text "Ready for you to test — see the last comment."; never confirmed), only if the comment was new and the ticket is not already awaiting / confirmed / rejected. Re-posting the same report writes nothing (sources dedupe tickets, the marker dedupes the comment). A run that failed half-way can simply be repeated.

work (agent v0)

Runs ONE worker on one ticket, on the host that holds the project's dev folder (dev.host in the registry must equal this host — /etc/hostname, case-insensitive, or COLONY_HOST):

  1. refuses before any write (work: REFUSED — …): no token file, project without metadata / concept / with "workers": false, ticket not open (in progress = leased; rejected is accepted when its rejection is newer than the last colony session — mission 022), blocked / a parent (… is not eligible: blocked by …), dev folder on another host or missing, --permission-mode bypassPermissions (only auto acceptEdits default dontAsk plan manual), bad --max-turns / --timeout, runs/<session> exists;
  2. writes runs/<session>/brief.md (the brief text) and session.json;
  3. lease: ticket → in progress with the text session <id> started on <host> + the marker line colony-lease: <id> · host <h> · started · ms <expiry> (one state event, text "A worker started on this ticket."); a heartbeat renews it while worker + controller run (see "Leases");
  4. runs, in the dev folder, claude -p --output-format json --json-schema <templates/report.schema.json> --model M --max-turns N --permission-mode MODE --permission-prompts none --strict-mcp-config --session-id <uuid> with the brief on stdin (umask 077, COLONY_TOKEN_FILE unset for the worker); --timeout kills it;
  5. on exit: claude-result.json (raw stdout), claude-stderr.log, report.json (= Claude's structured_output), prints USAGE: $cost · turns · s · tokens … · model · subtype · claude session, keeps cost/usage/turns/permission modes/denials/outcome in session.json; then the report validation (+ report ticket/session must be this session's); 5b. (mission 025) out of steps (error_max_turns) or time (--timeout) → the finalize step (section "Finalize step + salvage") resumes the same Claude session once to write the report; its report goes on to 6./7. like any other;
  6. no valid report (no structured output e.g. max turns, bad JSON, timeout, validation refused — after the finalize step, if there was one) → ticket back to open with the reason (+ validation problems, run folder) → WORK FAILED;
  7. a valid report → the controller (below) → --post posts by the verdict. Without --post: WORK OK — valid report, controller: pass|fail, NOT posted, ticket stays in progress → post later with ./colony check runs/<session> --post (reuses verdict.json). ./colony report runs/<session>/report.json --post still posts WITHOUT a controller (architect's override). --dry-run: every refusal check + the brief (GETs only), then work: DRY RUN — would lease … run …; nothing written, no Claude session.

The controller (mission 019; concept §1 + §2 "Report handling")

A second one-shot Claude Code session, started by work right after a schema-valid report and BEFORE anything is posted, in the SAME dev folder, with the worker's safety settings (--permission-mode auto by default, never bypass, --permission-prompts none, --strict-mcp-config, no COLONY_TOKEN_FILE) plus --disallowedTools Edit,Write,NotebookEdit (it only reads and re-runs).

  • Input (stdin, kept as runs/<session>/controller-prompt.md): templates/controller.md (what to check: every verified entry — re-run where cheap and safe, compare outputs; every done entry against the files; flag claims without evidence; don't re-run writes / deploys / POSTs), the report, the worker's session as a compact transcript (text, tool calls, tool results from the verbose claude-result.json, each clipped, whole ≤ 80 000 characters), the brief.
  • Answer (--json-schema templates/verdict.schema.json): { ticket, session, verdict: pass|fail, findings: [{ claim, about: verified|done|other, status: ok|false|no evidence|not checked, check }], summary } → runs/<session>/verdict.json; controller-result.json, controller-stderr.log; session.json.controller = cost, usage, turns, verdict, effective, overruled. Prints CONTROLLER USAGE: … and VERDICT: PASS|FAIL + one line per finding.
  • The code decides: lib/controller.hl validates the verdict; a pass with any false / no evidence finding counts as fail (overruled, said in the comment).
  • pass → report --post as before, the machine line ends with · controller pass (mission 023: summary + findings stay in verdict.json), awaiting creator, issues filed.
  • fail → ONE comment ## Controller: FAIL — worker session … (fail k of N): summary, findings, run folder, the checked report in <details> (its issues are NOT filed), marker colony-verdict: <session> · fail · k of N → ticket back to open (the next worker reads it in the history). Mission 023: the fail comment is SHORT (what happened + ≤ 3 false / no-evidence findings, "Not true: …" / "Not shown: …"); the checked report and all findings stay in runs/<S>/.
  • Nth fail (k ≥ N; N = --max-fails / COLONY_MAX_FAILS, default 2; k = distinct sessions with a fail marker in the ticket's comments) → a creator question (antcolony, awaiting creator, source colony-verdict:<session>:question) instead, the comment links it, ticket → on hold (next never picks it; the creator sets it back to open).
  • Idempotent: a session's fail comment is written once (HAVE); pass = the report's own marker.
  • No verdict (controller timed out / no structured output / verdict refused) → nothing posted, ticket stays in progress: WORK OK, CONTROLLER FAILED — … Retry: ./colony check <dir> --post.

run (mission 020) — the loop

./colony run (ALWAYS through ./colony: the wrapper catches the signals). Refuses the live tickets server (COLONY_TICKETS_URL unset or host tickets.worldapi.org) unless --live. Every --interval s (default 60) + once at the start, ONE iteration (lib/daemon.hl):

  1. survey + the lease of every in progress ticket: expired colony lease → ticket back to open with a comment (colony-lease-expired: <session>); a parked session of THIS host whose resume time is over → resume candidate; a live lease → skipped (a restarted run never starts a second session on it); in progress WITHOUT a colony lease (architect missions, v0 leases) → listed, never expired; mission 031: a lease that has ENDED → listed, never expired, and the expiry / the resume RE-READ the ticket right before they write (section "Mission 031");
  2. candidates: due resumes first, then eligible open tickets whose registry dev.host is this host (oldest first); a ticket whose dev folder is in use by a running session of this run waits (same dev folder);
  3. free slots = --parallel (default 1) − running; nothing to start / no slot → no quota probe (mission 034: with the librarian on, the probe runs FIRST in every iteration — see "Usage limit" below — and its reading is reused here);
  4. quota: claude -p /usage (below) — percent ≥ --reserve (default 70) of the 5 h window → no start (resumes too); probe failure → no start; weekly (mission 023, creator): weekly_all percent above --week-limit (default 84) or missing in the answer → no start (… week 85% > week limit 84% → no start (waiting: …) / week UNKNOWN → no start); --reserve 100 / --week-limit 100 switch the check off (5 h reserve off (100%), week limit off (100%) in the ITER line — for the first live runs after a quota reset); mission 034: a REACHED limit (5 h or week at 100 %) holds every start also with the checks off (… → the 5 h limit is reached (100%) → no start);
  5. start: a port range per session from --port-pool (default 8700-8799) in blocks of --ports-per-session (default 10; disjoint while running; --parallel > blocks → refused) → work … --post --session S --ports A-B (brief says the range; worker + controller get env COLONY_PORTS, COLONY_PORT_FROM, COLONY_PORT_TO) or the resume. run always posts (by the verdict).
  6. ONE line: ITER <n> <time> · quota 5% of 5 h (resets …), week 41% < reserve 70% · running 1/1 · started d1#1 (s-…, ports 8780-8789) · expired … · skipped: …. A finished session: SESSION END <ref> <session> · posted | sent back | question | parked | no valid report | … · cost $x (worker $a + controller $b) · ports … freed. Stop (Hybriel has no signal handlers): the wrapper runs the scheduler with setsid (a terminal's Ctrl-C reaches only the wrapper), traps SIGTERM/SIGINT and writes $COLONY_STOP_FILE (a mktemp name in $TMPDIR//tmp, removed on exit); the run checks it twice a second. 1st signal → RUN: stop requested: nothing new starts, running sessions FINISH (post), then RUN STOPPED. 2nd signal → the running worker/controller processes are killed and parked (PARKED — … the scheduler run was stopped, resume = now) → the next run resumes them. --once = one iteration, wait for what it started, exit. A SIGKILLed / crashed run takes its sessions with it: hl:proc sets PR_SET_PDEATHSIG = SIGTERM on every child (plugins/proc/proc.zig:305) — the lease then expires. Wrapper prints colony run: wrapper pid <p>, scheduler pid <q>, stop file <f> (kill the WRAPPER pid).

Usage limit (mission 034, ticket antcolony#25; daemon.hl tick / pauseForLimit, claude.hl rateLimitOf)

Why (live 2026-09-25 19:58): the 5 h limit was reached (a worker parked "resets 8:50pm"), yet EVERY iteration ran the librarian first: its model call came back empty (LIBRARIAN USAGE: $0 · 1 turns · 0.466 s · tokens 0 · success, 1 without an answer), the text stayed unfiled → librarian: 1 creator text(s) not filed yet → no start — every minute until the reset. The answer's shape (librarian/calls/0muha3gdyxbp-result.json, kept in .scratch/m034/livecheck/): rate_limit_event status rejected (resetsAt), a SYNTHETIC assistant message with error: "rate_limit" + "You've hit your session limit · resets 8:50pm (Europe/Vienna)", then a result with subtype success, is_error: true, api_error_status: 429, 0 tokens, $0, no structured output. The librarian never looked for the limit.

  1. Quota FIRST (the creator: "the usage command is model independent? it always works"): with the librarian on, every iteration starts with claude -p /usage (local, $0) BEFORE the librarian pass. At or above the start threshold (--reserve of the 5 h window, --week-limit of the week; a REACHED limit = 100 % always counts, also with the checks off) → no librarian call and no start this iteration; the ITER line has the reading with the reset time and librarian not run: quota at or above the start threshold (5 h 75% ≥ reserve 70% | the 5 h limit is reached (100%) | week …). A failed probe / unknown week → the librarian runs (starts are held as before). The reading is reused for the start check (one probe per iteration) unless the pass made model calls (then read again right before a start).
  2. Safety net — a call that still hits the limit: rateLimitOf recognises the live shape (and also a synthetic error: "rate_limit" message alone, a 429 without is_error, "session limit" / "weekly limit"). The librarian pass stops at that text (no try counted, text stays unfiled) and reports limited + the reset time; a worker / controller parked on the limit reports it the same way (parkRun → limited, resetMs when known). run then remembers "limited until": LIMIT: usage limit reached — librarian and starts paused until <ISO> · <who hit it> (ONE line); until then no librarian pass, no quota probe, no start or resume, the iterations are silent (leases are still looked after; a line only if one was expired); a later limit with a later reset → LIMIT: the pause is extended until …. After it: LIMIT: resumed — the pause until <ISO> is over; the quota is read first, then the librarian and starts (ONE line) → normal iteration.
  3. No reset time known → paused 15 min (… (no reset time known → 15 min, then try once)), then the quota is read first.
  4. Parked sessions resume as before (their resume time = the reset; after the pause the iteration resumes them). The pause lives in the running process only (a restarted run starts with the quota probe, which sees the limit).

Leases (mission 020, lib/lease.hl)

tickets is the only state, so a lease is marker text: the → in progress state event of work (above), heartbeat comments Still working. + colony-lease: <S> · host <H> · renewed · ms <expiry> every --heartbeat s (default ttl/2) while the session runs, … · resumed (park k) … on resume, and on a park colony-parked: <S> · host <H> · phase worker|controller · claude <uuid> · resume after <ISO> · resume-ms <ms> · lease until <ISO> · ms <resume + ttl>. The LAST marker of the lease's session decides; past ms the lease is dead. --lease-ttl default 7200 s, heartbeat ttl/2 = 1 h (mission 023: "heartbeats as few as possible" — a session under 1 h writes none, the default worker 1 h + controller 15 min at most one; was 3600 / ttl/3 = 3 per hour; a crash puts the ticket back to open ≤ 2 h later). Park marker since mission 023: colony-parked: <S> · host <H> · phase … · resume-ms <ms> · ms <expiry> (the Claude session id lives in session.json); older markers with lease until / claude / resume after parts are still parsed. ./colony leases lists every in-progress ticket's lease; --expire gives the dead ones back.

Mission 031 (ticket antcolony#1): lease expiry never touches a finished ticket (lease.hl endOf / stillLeased / expireLease)

Why (live 2026-09-24 21:05Z, logs/colony.log ITER 45): ident.worldapi.org#19 was built (ready to deploy), the architect deployed it and ran ./colony deployed ident.worldapi.org 19 (comment colony-deployed: S at 21:05:18.683, → awaiting creator 146 ms later). The daemon's iteration had read the ticket list while it was still in progress; its lease (session s-20260924T1852-496f93, dead since 20:52) was no longer "built, waiting for the deploy" (the deployed marker voids that), so leaseOf returned the old dead lease and expireLease wrote → open "the worker stopped answering" 1.9 s after the deploy. The finished ticket left the creator's inbox and a second worker ran on it (s-20260924T2108-d3b3f0, changed nothing). Same iteration: ident#15 was in the same gap and only escaped because its lease was still alive.

  • A lease ENDS (leaseOf kind ended, never expired) when after its → in progress event: the ticket changed state (to anything but in progress, by anyone), or a colony-deployed: line appears (any session), or the lease's OWN session wrote colony-report: (not the · ready to deploy one — that is kind ready), colony-verdict:, colony-run: or colony-lease-expired:. run: skipped: … <ref> in progress, but the lease of S has ended (<why>) → never expired; leases: LEASE <ref> — the lease of session S has ended (<why>): never expires. A ticket left like that (e.g. deployed wrote its comment but not the state) is completed by running ./colony deployed <p> <n> again (HAVE + state).
  • Re-read before acting: expireLease (used by run and leases --expire) and the resume of a parked session (run) first GET the ticket again (stillLeased) and act only if it is STILL in progress with the SAME lease (session, expiry, parked state) and still dead (expire) — else NOTHING is written and the reason is printed: lease: NOT expired <ref> (session S): re-read right before — it is "awaiting creator" now → nothing written, in the ITER line NOT expired <ref> (session S, lease ended …): re-read right before — <why> → nothing written; resume: NOT resumed <ref> (S): …; leases --expire: … EXPIRED at … → NOT expired, re-read right before: <why> (nothing written). Reasons: it is "<state>" now, the lease of S has ended: …, another session holds it now (…), the lease of S changed (now until …) (a heartbeat / park after the survey), it is built now, waiting for the deploy, could not re-read it (…).
  • Remaining gap: re-read and write are two requests a few ms apart; tickets has no conditional state change (e.g. POST …/state {state, expect: "in progress"}) — that would close it (question for the creator, STATUS).
  • Test hook COLONY_TEST_HOOK_URL (tests only, never set live): right before the re-read the scheduler GETs <url>?action=expire|resume&project=…&number=…&session=… and waits for the answer — the e2e changes the ticket there, which reproduces the race deterministically. Unset = no request.

Quota + parking (mission 020, lib/quota.hl, work.hl parkRun / runResume)

Reading: claude -p /usage --output-format json --verbose --model haiku --max-turns 1 --permission-mode plan --permission-prompts none --strict-mcp-config (--quota-claude / COLONY_QUOTA_CLAUDE, default the worker's binary). It is a LOCAL command of Claude Code 2.1.280: $0, 0 turns, 0 tokens, ~1–4 s (proved 2026-09-24, .scratch/m020/usage-probe/); the assistant message carries usage_report.rate_limits.limits[] with kind: session → percent, resets_at (fallback: the text Current session: N% used). The account = whatever claude is logged in as on this host (per host = per account today). A probe that cost > 0 is printed (PROBE COST). Every session's verbose output also has rate_limit_events (unifiedWindows.five_hour.utilization) — kept in session.json.rateLimit. Evicted by the limit (a rate_limit_event with status rejected, or an error result with api_error_status 429 / "hit your limit" / "usage limit"; mission 034: also a synthetic error: "rate_limit" message, "session limit" / "weekly limit"): the session is parked, not given back — comment **Parked** — Claude usage limit (…) + colony-parked marker, ticket stays in progress, session.json.parked = { phase, reason, resumeMs, count }. Resume time = the event's resetsAt, else …|<epoch> in the text, else now + 1 h; run resumes only on the host that parked (the Claude session lives there) and only below the reserve. Resume: phase worker → claude -p --resume <claudeSession> (same flags, no --session-id) with runs/<S>/resume-<k>.md (continue, the NEW port range, the report's ticket + session); the earlier output is kept as claude-result.parked-<k>.json (+ stderr) and the controller sees all parts; phase controller (stopped after a valid report, or the controller hit the limit) → a new controller session on the kept report.json. Costs of all parts are summed (costEarlier, controllerCostEarlier).

Mission 027 (ticket antcolony#1): after the first live batch — readable test steps, deploy gate, fewer questions

Why (batch 2026-09-24: ident#2, tickets#12, gitoria#2, all PASS, $6.36): the Test steps were one run-on line pointing at work copies on Loreana; "Ready for you to test" was posted before anything was deployed (the creator can only test on the live site); too many / too long questions, some trivial, one already decided (only in a ticket, not in a doc the worker could read — workers on Loreana cannot read Byrodin's antcolony README at all: the gitoria worker tried ssh and was refused).

  • Limits (report.hl validate; told in templates/report.md + the schema descriptions): result one line ≤ 120; test 2–4 steps, one line ≤ 90 each, no .scratch, and on a project with live no localhost / 127.0.0.1 / 0.0.0.0; questions (antcolony#24: no count limit), each ≤ 3 non-empty lines and ≤ 300 characters; decided = list of one-line strings ≤ 120 (key OPTIONAL in the validation so older reports stay valid; REQUIRED in report.schema.json so workers give it). The brief's report part says where the creator tests: on the live site <live.url> (after the deploy …) or, without live, on the real result (… never a work copy).
  • Numbered Test list: every comment with steps renders **Test:** + 1. … lines (was **Test:** 1) … 2) … on one line). The README "≤ 5 short lines" limit is counted WITHOUT the step lines (the **Test:** header counts as one) — e2e readableLines.
  • decided → ONE detail bullet Decided: a; b. (after questions + problems filed, before "Not done").
  • Deploy gate (report.hl deployGate/readyCommentOf, lib/deploy.hl, lease.hl readyOf): a passed, complete report on a project whose registry has live → questions / issues filed as usual, then ONE comment Built — goes live with the next deploy. + colony-report: S · sha256 H · controller pass · ready to deploy; the ticket stays in progress (from open / rejected → in progress "Built — waiting for the next deploy."). Outcome ready to deploy (WORK DONE — x built, waiting for the deploy …, SESSION END … · ready to deploy). readyOf(events) = the last ready marker's session unless a later colony-deployed: S, a lease of ANOTHER session or a state change away from in progress voids it; leaseOf → kind ready (never expires; run notes "built, waiting for the deploy"; leases: LEASE x — built, waiting for the deploy (session S): never expires); next lists it built — waiting for the deploy (session S; ./colony ready).
  • ./colony ready (GETs only): READY <ref> "<subject>" — built by session S · live <url> · deploy: <note> · after the deploy: ./colony deployed <p> <n> · <url> + READY: n ticket(s) waiting for a deploy.
  • ./colony deployed <project> <n> (token; on the host that holds runs/<S>/report.json, COLONY_RUNS_DIR): refused when the ticket waits for no deploy or the report is not on this host (nothing written). Else ONE comment Now live on <site> — <result> · **Test:** + numbered steps · ≤ 3 details (question / problem links found by their source colony-report:S:…, Decided, Not done …) · colony-deployed: S · sha256 H, then in progress → awaiting creator ("Ready for you to test — see the last comment."). Again → HAVE, nothing written (a missing state change is repaired). Last line DEPLOYED — <ref> (session S, live on <site>)[: nothing new].
  • Projects without live (gitoria research, the e2e's dummy) → awaiting creator at once, as before.
  • Brief (brief.hl): after "Read first" the section ## Conventions for all apps — already decided by the creator, don't ask about them = templates/conventions.md (a COPY of antcolony README "Conventions for all apps" + "Design: colors" — keep in step by hand); ## Tickets of <project> — decisions may already be there (every other ticket of the project, all states, newest first, ≤ 40; an answered / confirmed question is followed by the creator's own comments on it — antcolony#22) before the Budget; "Where" names the live site as the place the creator tests; the concept line says "list it under decided". templates/rules.md: small choices → decided, ask only what changes what gets built.
  • One fix step (work.hl finish/fixPrompt; concept §2 "reject incomplete reports back to the worker (one retry)"): a report the validation refuses → the SAME Claude session is resumed once like the finalize step (--resume, same flags, --finalize-max-turns, runs/<S>/finalize.md = # FIX YOUR REPORT — the scheduler refused it + the problems + "do not continue, change nothing" + ticket/session + colony-finalize: S) → valid → controller → post; refused again → back to open once ("its report was incomplete"). work: FINALIZE — the worker had its report refused (…); resuming claude session … once to FIX it. session.json.finalize.why = "had its report refused". --finalize-max-turns 0 switches it off too. Salvage treats such a run as "did not run out of steps or time".
  • Controller (templates/controller.md 3): on a live project the steps describe the live site after the deploy — check them against the code, never open the live site; a work-copy / local step is false.
  • Tests: section "Tests of mission 027" below. Pre-027 code: .scratch/m027/orig/.

Mission 026 (ticket antcolony#1): finished or not, "Previous attempt", cleanup, cheaper finalize, salvage lease

Why (first live run, ident.worldapi.org#2): the salvaged report said "built in a work copy, not copied in, test unfinished, docs not updated" — posting it as "Ready for you to test" was wrong; the worker left two dev servers running (8700/8701); the finalize resume cost $1.05 because it missed the prompt cache.

  • complete (report key, boolean, report.hl + templates/report.schema.json + templates/report.md): true ONLY if the goal is reached AND what the creator should test is in place (copied into the app, docs updated). The controller checks it (templates/controller.md 3b: a true whose work is only in a work copy → false finding → fail). complete: false + valid report + pass → postReport writes ONE comment Half done: <result>; the next worker continues. + - Still open: <open[0]> (and n more) + questions / problems filed / running / refused (≤ 3 bullets, no Test line) + machine line colony-report: S · sha256 … · controller pass · half done, then the state in progress → open ("Back to open: half done — the next worker continues."; any other state is left alone). Questions and issues are filed as usual. Outcome half done (WORK HALF DONE — …, SESSION END … · half done). A half-done report does NOT answer a rejection (relations.hl): the next brief still starts with it. Half-done sessions are not counted as controller fails (no limit on half-done rounds yet).
  • "Previous attempt" (lib/previous.hl, in the brief after the intro, before "Read first"): only when the ticket's history has a colony lease event (colony-lease: S — work or salvage — or v0 session S started on H). The LAST such session: how it ended (last marker of S: colony-report [· half done], colony-verdict … fail, colony-run (no report), colony-lease-expired, colony-parked), and from runs/<S>/ on THIS host (COLONY_RUNS_DIR / $COLONY_HOME/runs): its result (+ "not complete"), up to 5 open items, its work copy = every .scratch/<name> named in the report (done / open / verified commands / running logs / result; relative → the dev folder, nested ones dropped, trailing ./, stripped), its report + run folder paths; no report → session.json's outcome + the run folder. Run folder on another host → says so. Example (e2e): .scratch/m026/briefs/brief-m026-rej.md.
  • Cleanup (claude.hl runClaude/cleanupPorts + lib/cleanup.sh): every worker / resumed / finalize / controller session gets COLONY_SESSION=<S> in its env; when the part exits (any outcome — report, no report, timeout kill, park) bash lib/cleanup.sh <from> <to> <since-ms> <S> stops each process that LISTENS (TCP, ss -ltnpH) on the session's range AND started > 50 ms after the part began (start = now − (uptime − /proc starttime); btime+starttime was up to 1 s off) AND belongs to this user AND is not an ancestor of the script (the scheduler) AND does not carry ANOTHER COLONY_SESSION. SIGTERM, ≤ 5 s, then SIGKILL. Every stopped / skipped process is printed (work: CLEANUP — stopped pid P · port N · started <ISO> · SIGTERM · <cmd>) and kept in session.json.cleanup[{part, at, lines}] / session.json.controller.cleanup. Not stopped: children that do not listen (e.g. a node --watch parent), processes of other users (no pid visible), anything outside the range. rules.md tells the worker.
  • Cheaper finalize: the finalize resume has NO --disallowedTools any more (it changed the tool list = the cached prefix); the "change nothing" instruction (now "do not use the Edit/Write tools"), auto mode and the rest stay. Real run (2026-09-24, .scratch/m026/real-work.txt): Sonnet worker --max-turns 2 → out of steps ($0.1866, cache write 41 438) → finalize resume $0.0301, cache read 83 909 / write 1 259, changed no file (mtimes) — vs. mission 025's finalize with the flag: $1.0528, cache read 0 (207 k context). The controller keeps --disallowedTools (a new session).
  • Salvage lease: salvage now needs the token always and a ticket in state open; it leases it (open → in progress, text A worker started on this ticket: it writes the report of an earlier session. + colony-lease: S · host H · salvage · ms N, heartbeat) → posted by the verdict, or no report → the lease goes back (→ open, the worker's reason, SALVAGE FAILED — … → <ref> back to open); without --post the ticket stays leased (./colony check <dir> --post).
  • The real salvaged run of ident.worldapi.org#2 predates complete: to post it as half done, a COPY with "complete": false was posted to OWN tickets (.scratch/m026/halfdone-real.sh → halfdone-real.txt, halfdone-real-ticket.md). For the live post the architect adds "complete": false to runs/s-20260924T1527-525807/report.json (changes its sha256; no comment of it exists live yet) and runs ./colony check runs/s-20260924T1527-525807 --post (reuses verdict.json).

Finalize step + salvage (mission 025, ticket antcolony#1; work.hl startFinalize / runSalvage)

Why: the first live cycle (ident.worldapi.org#2, 2026-09-24) — the Sonnet worker did the work (6 gates green in its dev copy), then hit --max-turns 120 before copying back + reporting → error_max_turns, no report, $4.67 lost.

  • Budget in the brief: section ## Budget — leave room for the report (before ## Rules): "Steps: at most N … Write your report before step R" (R = N − min(10, ⌊N/3⌋), ≥ 1: 120 → 110, 40 → 30, 12 → 8), the time limit (--timeout), and "running short: stop, report done / verified / open". N / time = the --max-turns / --timeout (or env) of work, brief, run.
  • Finalize (inside work / a resumed run session): the worker ended with subtype error_max_turns, or was killed by --timeout, and no park applies → runs/<S>/claude-result.json → claude-result.before-finalize.json (+ stderr), then claude -p --resume <claude session> --output-format json --json-schema templates/report.schema.json --model <same> --max-turns <--finalize-max-turns, 8> --permission-mode <same> --permission-prompts none --strict-mcp-config (mission 026: --disallowedTools Edit,Write,NotebookEdit removed — prompt cache) in the dev folder, stdin runs/<S>/finalize.md ("STOP WORKING — write your report now": do not continue, change nothing, report done/verified/open honestly, ticket + session, last line colony-finalize: <S>). Timeout --finalize-timeout (default 600 s, never more than the worker's --timeout). Valid report → controller → post by the verdict (the controller prompt shows "Part 1 — the worker, until it ran out" + "Part 2 — the finalize step"). No report again → work: FINALIZE FAILED — …, then the normal give-back (open, ONE state event, the WORKER's reason — "it ran out of steps / time"). Nothing about the finalize step is written to the ticket. --finalize-max-turns 0 = off (old behaviour). ONE finalize per session.
  • session.json.finalize = { why, workerReason, workerSubtype, workerTurns, workerCostUsd, maxTurns, timeoutSeconds, started, ended, costUsd, numTurns, subtype, outcome (valid report / report refused by the validation / no report: …) }; costEarlier = the worker part, costUsd = the finalize part.
  • Cost of a resumed session (found on the real salvage): Claude Code 2.1.280 reports the WHOLE session's cost in total_cost_usd / modelUsage of a --resume call (cumulative: 5.7198 = worker 4.6670 + resume 1.0528; usage is per call). finish now takes total − meta.claudeCostSeen for any resumed part (finalize AND mission 020's parked resume — that one used to count the first part twice) and prints FINALIZE COST: / RESUME COST: $x for this part (Claude reports $y for the whole session so far).
  • ./colony salvage <runs/S> [--post] [--ports A-B] [--finalize-max-turns N] [--finalize-timeout S] [controller options]: the finalize step for a FINISHED run: refused unless the worker ran out of steps (claude-result subtype error_max_turns) or time (outcome "timed out after"), the run is on THIS host (the Claude session lives there), no finalize yet, no report.json, not parked; --post needs the token and a ticket that is not in progress / confirmed (posts to COLONY_TICKETS_URL!). Mission 026: takes a LEASE (token always, ticket must be open) — see "Mission 026". Controller gets an extra line: "SALVAGED run — re-run NOTHING that writes …, only read". Valid report + verdict → with --post posted by the verdict like check, else SALVAGE OK — … NOT posted; later ./colony check runs/<S> --post posts it (reuses verdict.json — that is how the architect posts a salvage to the live tickets). No report → SALVAGE FAILED, nothing written to tickets. Last lines SALVAGE COST: finalize $a + controller $b (the worker before: $c; whole session now $d), SALVAGE END — <ref>: <outcome>.
  • Controller transcript (mission 025): a transcript over 80 000 characters now keeps its first 30 000 AND its last 50 000 characters (was: only the start — the end of a long session, where the final checks are, was lost).

check

./colony check runs/<session> [--post] [--again]: the controller for a finished work run (valid report.json + session.json needed). Uses an existing verdict.json (no new session) unless --again; then, with --post, posts by the verdict (same as work). Last line CHECK DONE / OK / FAILED ….

cycle

./colony cycle [--post] [--dry-run] [work/controller options]: ONE pass for ONE ticket — 1 next (the oldest eligible ticket whose registry dev.host is THIS host; eligible tickets of other hosts are printed as skipped; next --post's creator questions are NOT filed by cycle) → 2 work → 3 controller → 4 post by the verdict (only with --post) → CYCLE COST: $sum (worker $a + controller $b) and CYCLE END — <ref>: <outcome> (posted / sent back / question / no verdict / verdict …, not posted / no valid report). --dry-run stops before any Claude session (and before the lease): nothing written. Options after cycle go to work unchanged.

Permissions (tested 2026-09-24, Claude Code 2.1.280 on Loreana). Default --permission-mode auto (the classifier lets a worker edit and run commands in its folder, blocks production actions), with --permission-prompts none (anything that would ask is denied, shows in permission_denials). Never bypass. Haiku is allowed as worker model (creator 2026-09-24; nothing refuses it), the default stays Sonnet + auto. But auto mode needs a model that supports it: Haiku does NOT — Claude then silently falls back to default (debug log: auto mode disabled: model claude-haiku-4-5… does not support auto mode), and with prompts off the worker can edit/run nothing. work prints WARNING — asked for permission mode "auto", Claude reported "default" (it reads the system messages). So the default model is sonnet (cheapest with auto). Loreana's ~/.claude/settings.json has defaultMode: bypassPermissions — work always passes --permission-mode explicitly. --output-format json gives a message LIST on Loreana (verbose: true in the user settings); work takes the result entry (or a lone object). Cost seen: Sonnet, 4 turns, trivial file task: $0.18; Haiku 1 turn: $0.03–0.06 (≈28 k tokens of system prompt; --strict-mcp-config halves it by dropping the user's MCP servers).

Relations, questions, rejections (mission 022, lib/relations.hl; concept §2 "Creator questions are tickets", "Rejections are work")

Needs tickets' relations (tickets#4, mission 017: parent, children, blockedBy, blocks on every row). A tickets without them (no children key) is tolerated: no blocking, no parents, no links; the brief says "serves no relations".

  • Blocked: any blockedBy entry whose state is not confirmed (also awaiting creator, rejected) → not eligible.
  • Parent: a ticket with children is not work — except children that are creator questions: the scheduler's own (source starts with colony and contains :question: report questions, unknown-project issues, the Nth controller fail) and (mission 023) the architect's hand-filed ones: subject starts with the WORD Question (Question: …, Question (x#1): …, Questions … — not Questionnaire …). Without that exception a ticket that ever asked a question could never be reworked.
  • Questions: see report --post 2b. Every creator question the scheduler files (fileQuestionIn in lib/tickets.hl) is idempotent by source, sets awaiting creator only on a new ticket or one that never had a state event (repairs a half-done run, never undoes the creator), and sets the parent only while there is none. next --post's project-level questions (no metadata / no concept) stay in antcolony without a parent.
  • Rejections: the last → rejected event vs. the last colony session start (= work's lease event: → in progress with a colony-lease: marker, or v0's session <S> started on <H>). Newer → eligible (next, cycle, run, work). The brief starts with it while no colony-report: comment came after it (so a rework that ended without a valid report still carries it): REJECTED by the creator: <the rejection's text> · who/when · "The creator's comments since the last session" (comments by the rejecting user after the last session start before the rejection, quoted) · Build what was described, not a workaround; if impossible, stop and explain · --- · then the normal brief. The lease moves rejected → in progress (Colony may: only → confirmed/rejected is creator-only).
  • A question ticket is NEVER work, in no state (mission 030, antcolony#1). The pre-030 live run treated rejected question tickets ("already answered") as rework: ident#17 once, gitoria#3 twice, gitoria#4 five times. Now a ticket that is a creator question (isCreatorQuestion: colony question source, or subject starting with the word Question) is: next / cycle / run → listed a question for the creator — the librarian takes its answer (<state>) — never work (open, rejected, on hold; checked BEFORE the rework-on-rejection logic, not counted as "rejected"); brief / work → REFUSED — <ref> "<subject>" is a question for the creator — the librarian takes its answer (state …); never work — no brief, no worker (nothing written); salvage → REFUSED; a session PARKED on one (pre-030) → run does not resume it (<ref> parked, but a question for the creator … → not resumed); leased in progress → next adds …: never resumed. Its answer reaches workers only through the librarian (DECISIONS.md in every brief). Not guarded: ready / deployed (a question that a pre-030 session left "built — waiting for the deploy", e.g. ident#17, is still listed there).

Ticket texts (mission 023)

antcolony README "Ticket texts are written for the creator" — hard limits: ≤ 5 short lines, plain words, what is done / what to do first. Every text the scheduler writes = plain lines + at most ONE machine line, always the last (tickets' Markdown shows raw HTML as typed, so no <!-- --> / <details>). All of them (e2e examples: .scratch/m023/texts-2.md):

whentext (machine line last)
lease (→ in progress)A worker started on this ticket. · colony-lease: S · host H · started · ms N
heartbeat (comment)Still working. · colony-lease: S · host H · renewed · ms N
parked (comment)Paused: the Claude usage limit is reached — work continues after 2026-09-24 18:19 UTC. (or the scheduler was stopped) · colony-parked: …
resumed (comment)Work continues after a pause. · colony-lease: S · host H · resumed (park k) · ms N
no valid report (→ open)Back to open: the worker stopped without a usable result (it ran out of steps / time, its report was incomplete, it ended without a result) — the next worker tries again. · colony-run: S · host H · no report (reason + validation problems in session.json)
lease expired (→ open)Back to open: the worker stopped answering (its computer crashed or went offline) — a new worker will pick it up. · colony-lease-expired: S · host H
report (comment)<result> · **Test:** + 1. … lines (m027; was **Test:** 1) … 2) …) · ≤ 3 bullets (A question for you: [alpha #6](…), Problems found elsewhere: [hybriel #3](…) (blocks this ticket), Not done: …, Still running: …, The safety check refused n actions.) · colony-report: S · sha256 <12 hex> · controller pass
report (→ awaiting creator)Ready for you to test — see the last comment.
report on a project with live (m027, comment)Built — goes live with the next deploy. · colony-report: S · sha256 <12 hex> · controller pass · ready to deploy
same, ticket was open (m027, → in progress)Built — waiting for the next deploy.
deployed (m027, comment)Now live on <site> — <result> · **Test:** · 1. … (2–4) · ≤ 3 bullets (question / problems / Decided: … / Not done …) · colony-deployed: S · sha256 <12 hex>
deployed (m027, → awaiting creator)Ready for you to test — see the last comment.
half done (m026, comment)Half done: <result>; the next worker continues. · - Still open: … · ≤ 2 more bullets · colony-report: S · sha256 <12 hex> · controller pass · half done
half done (m026, in progress → open)Back to open: half done — the next worker continues.
salvage lease (m026, → in progress)A worker started on this ticket: it writes the report of an earlier session. · colony-lease: S · host H · salvage · ms N
controller fail (comment)The result did not hold up when it was checked — back to open for another try. (overruled: could not be confirmed; Nth: failed its check 2 times — on hold until you answer [antcolony #4](…)) · ≤ 3 Not true: … / Not shown: … · colony-verdict: S · fail · k of N[ · overruled]
controller fail (state)Back to open: the check failed (1 of 2 tries). / On hold: the check failed 2 times — waiting for your answer.
creator question: reportsubject Question (alpha#3): <first line ≤ 100> · summary = the question + Asked while working on [alpha #3](…). **Answer** with a comment here — the next worker reads it.
creator question: Nth failQuestion (dummy#6): the work failed its check 2 times — how should it go on? · what happened · **What to do:** … · - Last check: …
creator question: unknown projectQuestion (alpha#3): where should "<issue>" go? · A problem found while working on … belongs to "zeta", a project the colony does not know. · **Answer:** … · Repro / Observed / Expected
creator question: next --postQuestion: project beta has no written concept yet — can we write one? / … is not set up for the colony yet — where does its code live? · reason · **What to do:** … · - Waiting: [beta #1](…) (≤ 3 links)
any question (→ awaiting creator)Waiting for your answer.
issue in another projectsubject = the worker's · Found while working on [alpha #3](…) — it blocks that ticket. · **Repro:** / **Observed:** / **Expected:**

The worker is told the limits in templates/report.md; report.hl refuses a result / test that breaks them. Free text written by workers (open, questions, issue repro) is clipped to one line where the code quotes it.

Config (env)

VarDefault
COLONY_TICKETS_URLhttps://tickets.worldapi.orgtickets base URL
COLONY_TOKEN_FILE—file holding a tickets API token (tkt_…); needed only for --post; never printed
COLONY_PROJECTS_DIRprojects/ beside scheduler.hlregistry: one <name>.json per project
COLONY_INBOX_PROJECTantcolonywhere creator questions go
COLONY_DIGEST_HOUR / COLONY_STATUS_DIR7 / statusantcolony#19: UTC hour of the daily summary (off = never) / where STATUS.md is written
COLONY_PORTS8700-8749default port range for briefs
COLONY_SERVE / --serve—mission 035: the scheduler proper — port of the agent door (no worker of its own); COLONY_SERVE_HOST (127.0.0.1), COLONY_AGENT_TTL (30 s)
COLONY_AGENT_TOKEN_FILE—mission 035: the secret scheduler and agents share (both refuse to start without it)
COLONY_SCHEDULER_URL / --scheduler—mission 035: the agent's scheduler, e.g. https://…
COLONY_AGENT_POLL / COLONY_AGENT_QUOTA_EVERY5 / 60agent: seconds between syncs / between quota reads
COLONY_USER_AGENTantcolony-scheduler/0sent on every request (Cloudflare refuses script defaults)
COLONY_HOMEset by ./colonyabsolute path of this folder (work refuses without it)
COLONY_HOST/etc/hostnamethis host's name, compared with the registry's dev.host
COLONY_RUNS_DIRruns/ beside scheduler.hlwork's per-session files
COLONY_SANDBOXonoff = sessions run with host access (only the tests' fake claude needs it) — see "Sandbox"
COLONY_SANDBOX_ROthe docs folderextra read-only paths in the box, :-separated
COLONY_MODEL / --modelsonnetworker model; haiku is allowed, but has no auto mode (see Permissions)
COLONY_MAX_TURNS / --max-turns40
COLONY_PERMISSION_MODE / --permission-modeautonever bypassPermissions (refused)
COLONY_WORK_TIMEOUT / --timeout3600seconds, then the worker is killed
COLONY_MAX_BUDGET_USD / --max-budget-usd—passed to claude --max-budget-usd when set
COLONY_CLAUDE / --claudeclaudethe Claude Code binary (tests: tests/fake-claude.mjs)
COLONY_CONTROLLER_CLAUDE / --controller-claude= the worker'sthe controller's binary (tests: a fake worker + the real controller)
COLONY_CONTROLLER_MODEL / --controller-modelsonnet
COLONY_CONTROLLER_MAX_TURNS / --controller-max-turns20
COLONY_CONTROLLER_TIMEOUT / --controller-timeout900seconds, then the controller is killed (= no verdict)
COLONY_MAX_FAILS / --max-fails2the Nth controller fail on a ticket → creator question + on hold
COLONY_FINALIZE_MAX_TURNS / --finalize-max-turns8mission 025: steps of the finalize step; 0 = no finalize step
COLONY_FINALIZE_TIMEOUT / --finalize-timeout600 (≤ the worker's --timeout)seconds, then the finalize step is killed
COLONY_LEASE_TTL / --lease-ttl7200seconds a lease lives without a heartbeat (work, cycle, run)
COLONY_HEARTBEAT / --heartbeatttl / 2seconds between heartbeat comments
COLONY_RUN_INTERVAL / --interval60run: seconds between iterations
COLONY_PARALLEL / --parallel1run: sessions at once
COLONY_QUOTA_RESERVE / --reserve70run: no start at/above this % of the 5 h window; 100 = never blocks
COLONY_WEEK_LIMIT / --week-limit84run: no start (resumes too) while the WEEKLY usage is ABOVE this % or unknown; 100 = never blocks (mission 023)
COLONY_QUOTA_CLAUDE / --quota-claude= --clauderun: binary for the /usage probe
COLONY_PORT_POOL / --port-pool8700-8799run: ports handed out to sessions
COLONY_PORTS_PER_SESSION / --ports-per-session10run: size of one session's range
COLONY_STOP_FILEset by ./colony runthe signal channel wrapper → scheduler (don't set by hand)
COLONY_SESSIONset by the scheduler for every Claude session partmission 026: the colony session id; the cleanup never stops a process carrying ANOTHER one (don't set by hand)
COLONY_LIBRARIAN / --librarianonmission 029: the librarian runs first in every run iteration and before brief/work/cycle; off = never
COLONY_LIBRARIAN_MODEL / --librarian-model (librarian: --model)sonnetthe librarian's model
COLONY_LIBRARIAN_MAX / --librarian-max (librarian: --max)0 (= all)creator texts classified per pass; texts left over → run starts nothing that iteration
COLONY_LIBRARIAN_CLAUDE / --librarian-claude= --claudethe librarian's Claude binary (tests: the fake)
COLONY_LIBRARIAN_TIMEOUT, COLONY_LIBRARIAN_MAX_TURNS300, 4per model call
COLONY_LIBRARIAN_DIRlibrarian/ beside scheduler.hlstate.json, decisions.json, calls/, lock, projects/<name>/DECISIONS.md (fallback)
COLONY_DECISIONS_GLOBALtemplates/decisions-global.mdthe decisions for all projects (+ -archive.md beside it)
COLONY_CREATOR_NAMESCaramboleyo,creatorthe creator's display names in tickets (exact, case-sensitive)
COLONY_DECISIONS_FULL_LINES150a decisions file up to this many lines goes into the brief whole, a longer one by matching topics
COLONY_LAYOUTS_CONVENTIONS/media/STORAGE/projects/antcolony-docs/docs/layouts-conventions.mdnamed in briefs of "ui": true projects
COLONY_TEST_HOOK_URL—mission 031, TESTS ONLY: GET before the re-read of an expiry / resume (the e2e reproduces the race with it); never set live

Relative paths in env/arguments are resolved against the cwd. projects/ here is THE project registry (single source since 2026-09-24; one <name>.json each: name, dev {host, folder}, concept, dependsOn, code?, live?, deploy?, note?, workers?, ui? — mission 029: ui: true = an app with pages/styles → its briefs name the layout conventions). Live use needs the "Colony" identity + token (created by the creator, concept "Decided" 5); until then the architect's token file works: COLONY_TOKEN_FILE=/root/.config/antcolony/tickets-token.

Test

COLONY_E2E_PORT_BASE=8750 node tests/e2e.mjs      # ~4 min, 343 checks (mission 034: + 13); 330 checks (mission 031: + 10); 320 checks (mission 030: + 10); 310 checks (mission 029: + 44 librarian); before: 266 checks (234 up to mission 026 − 3 moved + 35 mission 027); ident base+1, tickets base+2
                        # (base 8700–8797); `work` sessions get base+3..base+9 (COLONY_PORTS — m026: the cleanup stops leftovers
                        # there, so it must be a range only the e2e uses), `run` workers 8780-8799, salvage 8770-8779
COLONY_E2E_PORT_BASE=8750 COLONY_E2E_TICKETS_DIR=/media/STORAGE/projects/tickets.worldapi.org/.scratch/dev-m012 node tests/e2e.mjs
                        # 222 checks — a tickets WITHOUT the Markdown read view and WITHOUT relations: JSON fallback,
                        # relation checks skipped (the e2e prints `tickets serves relations …: false`)

The whole process against its OWN instances: tickets' and ident's code are COPIED (no .env, storage, sessions) to .scratch/e2e/ and started there (ident via tickets' tests/identkit.mjs, mail to a sink). User "Colony" logs in via ident's code exchange, sets its name, makes an API token (→ .scratch/e2e/colony-token). A registry fixture (alpha concept + deps hybriel/beta, beta no concept, gamma no file, delta no deps, hybriel, antcolony) and 11 seeded tickets. Then ./colony runs as a user would (cwd .scratch/e2e) and every outcome is read back over the API: next picks alpha#3 and explains each other ticket; next --post files 2 questions, a second run leaves the whole store identical (snapshot of every ticket + event); brief contains all 16 parts, refusals (no concept, no metadata, 404, bad ports); report refuses 17 kinds of bad reports with the exact message; report --post → exactly comment + state on alpha#3, hybriel#3 issue, antcolony question; again → store identical; different report same session → refused; second session on an awaiting ticket → comment only. Logs: .scratch/e2e/{tickets,ident}.log. Nothing stays running.

Test isolation (mission 030) — tests never depend on the live configuration. The Hybriel interpreter loads the .env BESIDE THE ENTRY SCRIPT (scheduler.hl; not the cwd; the real environment wins, an empty variable counts as set — probe .scratch/m030/envprobe/). Run from this folder, the e2e's ./colony calls got the live .env (max turns 120, week limit, token file, CLAUDE_CONFIG_DIR) → 6 failures. Now tests/e2e.mjs copies this folder WITHOUT .env, .scratch, runs, sessions, briefs, logs, librarian, projects to .scratch/e2e/app/ and runs every ./colony from there (chosen over "set every setting explicitly": a copy also covers keys added to .env later); every call gets a defined environment (inherited COLONY_* / FAKE_* dropped except COLONY_E2E_*) and — unless COLONY_E2E_REAL=1 — a claude guard first on PATH that refuses and logs to .scratch/e2e/claude-guard.log (last check: never called). Prove it both ways: node tests/e2e.mjs here (live .env present) and in a copy without .env — both must be fully green.

Tests of mission 035 (own file tests/e2e-agent.mjs, FAKE claude only, 39 checks, ~3 min; COLONY_E2E_PORT_BASE=8710 node tests/e2e-agent.mjs)

Own tickets + ident (like e2e.mjs; ident = base+1, tickets = base+2, scheduler door = base+3, the agent's port pool base+4…base+9), work dir .scratch/e2e-agent/. Covers: scheduler alone (no agent → the ticket waits, no Claude call); the door (401 wrong / no token, 404, 400); agent connects → the ticket is worked in the AGENT's runs folder and posted (controller pass), the brief carries the agent's port range and no marker, the scheduler ran no Claude; scheduler killed mid-session → the worker still posts, the agent buffers the end and delivers it once after the reconnect, nothing runs twice; quota 80 % (agent side) → not accepting, the start waits, then resumes at 5 %; agent stop → OFFLINE on the scheduler; the four refusals. Both suites need a tickets/ident copy that logs in (COLONY_E2E_TICKETS_DIR / COLONY_E2E_IDENT_DIR).

Tests of mission 034 (in tests/e2e.mjs, FAKE claude only, 13 checks; block "mission 034" at the end of mission 029)

Project lim (dev folder .scratch/e2e/limdev, host e2e-host; every other open ticket put on hold first), the m029 librarian folder, FAKE_USAGE_FILE .scratch/e2e/m034-usage, FAKE_LIBRARIAN_MODE_FILE .scratch/e2e/m034-librarian-mode (re-read per call — the test switches the librarian's answer while run runs), FAKE_LOG_DIR .scratch/e2e/m034-calls (every probe with its time). fake-claude.mjs librarian mode ratelimit = the LIVE answer's shape (resetsAt = now + FAKE_RATELIMIT_RESET), ratelimit-noreset = without the rate_limit_event and the 429. Outputs .scratch/e2e/m034-*.txt. (A) quota 100 % / 75 % (reserve 70) / 100 % with --reserve 100 --week-limit 100: run --once → no librarian call, no start, ITER line with the reset time and librarian not run: …, one probe. (B) quota 5 %, the librarian's call answers the limit (reset in 8 s) → → USAGE LIMIT … resets <that time>, the pass stops, 0 without an answer; ONE LIMIT: … paused until <reset>; during the pause NO librarian call, NO probe, no start, no ITER line; ONE LIMIT: resumed; then the SAME text is filed (tries = 1) and lim#1 is started → posted. (C) the limit without a reset time → recognised, paused 15 min, the next pass files the text (tries = 1). (D) a WORKER parked on the limit (reset 8 s) → LIMIT: … paused until <its reset> · lim#3 <session> was parked on it; a creator text written during the pause gets no call until LIMIT: resumed, then it is filed and the parked session is resumed (--resume) → posted. Changed old checks (behaviour changed on purpose): m020 "parked: …" now expects the ONE LIMIT: line instead of ITER lines "parked until" during the pause; m023 "both checks off" runs at 99 % (100 % = limit reached now holds starts); m029 "I" holds starts by an UNKNOWN week instead of 5 h 99 % ≥ reserve 50 % (that would now skip the librarian). Regression proof: the pre-034 code (.scratch/m034/orig/lib) with the new tests → 13 FAILED (.scratch/m034/e2e-origcode-newtests.txt).

Tests of mission 031 (in tests/e2e.mjs, FAKE claude only, 10 checks; block "mission 031" after the m027 deploy gate)

Own registry .scratch/e2e/projects-m031/ with only project lr (live site https://lr.example, dev folder on e2e-host), COLONY_LIBRARIAN=off. (A) work on lr with --lease-ttl 2 → ready to deploy; after 2.5 s the FIRST half of deployed is posted by hand (the comment with colony-deployed: S, no state change — the 146 ms gap of the live case, frozen) → leases --expire and run --once list "the lease of S has ended (deployed …)", write nothing, start nothing; deployed then completes it (HAVE, → awaiting creator). (B–E) one run --once with COLONY_TEST_HOOK_URL = an HTTP server of the e2e on base+8 (8758), four lr tickets leased by hand: B dead lease — the hook moves it to awaiting creator (the live case) → "NOT expired … it is "awaiting creator" now", only that event; C dead lease — the hook posts a heartbeat renewal → "NOT expired … the lease of S changed"; D dead lease, hook does nothing → still expired (control); E parked + due (runs/s-m031-race-e/ session.json) — the hook moves it to awaiting creator → "NOT resumed", no claude call, no lease comment. Then leases --expire with the hook putting C on hold → "NOT expired … it is "on hold" now". Run outputs .scratch/e2e/run-logs-m031/. Proof the test catches the bug: the same e2e against the pre-031 code + ONLY the hook call (.scratch/m031/orighook/) → 322 passed / 8 failed (.scratch/m031/e2e-orighook.txt): A expired → open and run started a SECOND worker on it; B expired lr#2 … → open after the move to awaiting creator (the live bug); C expired despite the renewal; E resumed.

Tests of mission 030 (in tests/e2e.mjs, FAKE claude only, 10 checks; block "mission 030" at the end of mission 022)

Isolation (copy without .env; the guard never called). Questions in project rej: the report's question rej#q0 REJECTED by the creator ("already answered" — newer than any session, the pre-030 rework trigger), a hand-filed Question: … left OPEN, one ON HOLD → next lists all three "a question for the creator — the librarian takes its answer (<state>) — never work", 0 rejected; work ×3 and brief ×2 REFUSED, store snapshot unchanged, no run folder, no fake-claude call; cycle --post → "nothing to do"; run --once → "nothing to start"; a session PARKED on a question (lease + park markers written by the test, runs/s-m030-parked-q/session.json) → run --once does not resume it; next shows it "…: never resumed"; salvage of a max-turns run on the question → REFUSED, nothing written. m029's project lib (workers: false) now gets its DECISIONS files in the librarian folder and NOTHING in its dev folder (3 m029 checks changed accordingly). Logs: .scratch/m030/e2e-*.txt (see STATUS).

Tests of work (in tests/e2e.mjs)

With a FAKE claude (tests/fake-claude.mjs, no quota; answers like Claude 2.1.280 incl. the verbose message list): registry dummy (dev folder .scratch/e2e/dummy, host e2e-host) + omega ("workers": false); next lists omega "takes no workers"; 7 refusals write nothing; a good run → lease event text, comment, awaiting creator, claude's cwd = dev folder, brief on stdin, every flag (no bypass), no COLONY_TOKEN_FILE in the worker, all run files, session.json cost/usage, USAGE line; no structured output → open + reason; invalid report → open + problem; --timeout 2 on a hanging worker → killed, open, nothing left running; without --post → stays in progress, then report … --post.

Tests of the controller / check / cycle (mission 019, in tests/e2e.mjs)

The fake claude is ALSO the controller (it sees the verdict schema in --json-schema; FAKE_CONTROLLER_MODE pass | fail | noevidence | nostructured | hang, FAKE_CONTROLLER_LOG); in pass it really looks at the files the report cats. Checked: the controller runs after the worker in the dev folder with every flag (verdict schema, --disallowedTools Edit,Write,NotebookEdit, no bypass, no token), its stdin = controller-prompt.md = instructions + report + transcript (the worker's tool call/result) + brief; verdict.json, session.json.controller; pass → the comment's controller section, marker order. fail: fake worker falseclaim (report claims missing.txt) → VERDICT: FAIL, findings comment, issues NOT filed, → open; check --post again → uses verdict.json, store unchanged; 2nd fail → creator question naming both sessions, ticket on hold, next skips it; missing evidence: controller says pass + a no evidence finding → the code fails it (overruled); --max-fails 1; no verdict → stays in progress, then check --post → controller → posted; cycle: nothing on another host, --dry-run (no Claude, no run folder, store unchanged), cycle --post → every step + CYCLE COST: $0.0168 (worker $0.0123 + controller $0.0045). 119 checks (118 with the m012 tickets).

Tests of mission 029 — the librarian (in tests/e2e.mjs, FAKE librarian model only, 44 checks; block "mission 029" before the m023 sweep)

tests/fake-claude.mjs: a call whose --json-schema has "supersedes" is the LIBRARIAN — it reads the prompt's JSON input and answers by markers in the creator's text: #not / "test" → not a decision, #global → all, #project:<p>, #topic:<a-b>, #excerpt:"…" (verbatim), #paraphrase (a quote NOT in the text), #longmeans (a 2-line reading of ~400 characters), i never said "X" → supersedes the entries quoting X; FAKE_LIBRARIAN_MODE good | nostructured | badproject | hang; FAKE_LIBRARIAN_LOG (argv, cwd, input per call). Project lib (dev folder .scratch/e2e/libdev on e2e-host, workers: false, ui: true), project lord (a ticket run can start), creator texts by the e2e's CREATOR (display name Creator, COLONY_CREATOR_NAMES=Creator), --since = the block's start; every e2e call has its own COLONY_LIBRARIAN_DIR / COLONY_DECISIONS_GLOBAL under .scratch/e2e/ (never the real librarian/ or templates/). Checked: dry run (lists 8, no call, nothing written); first pass (8 calls, argv --tools "", mode default, no session, cwd librarian/, no token; quote = the API text exactly; topics + contents; not-a-decision and Colony's text in no file; #global → the global file; verbatim excerpt alone / paraphrase → whole text + note; "yes" on a question ticket with the question as link + context; a ticket the creator opened (subject + summary); multi-line → blockquote; newest first; every registered project's file); nothing new → no call, no file changed; correction → archive; refused answer → retried once → given up → --retry-refused; no answer → retried; long reading → one line, clipped to 300; "i dont care" on two questions → two entries; --redo; brief (section after the conventions, before Where; both paths; whole short files; UI rules for ui: true only; a LONG file (COLONY_DECISIONS_FULL_LINES=5) → contents + only the matching topic + the search note); ORDER: brief files a decision written a moment ago before building (in the brief), work --dry-run / --librarian off call no model, run --once files the new text BEFORE the ITER line that starts a worker and that worker's runs/<S>/brief.md has it, a pass with a text left over (--librarian-max 1, 2 new) → "not filed yet → no start this iteration", the next iteration files it and starts (brief has both); run quiet when nothing is new, --librarian off, bad value refused; seed (refused without --post-files, 4 entries, wrapped quote joined, architect's reading, item without quote skipped, HAVE on re-run); lock → BUSY (and brief goes on, saying so); --render identical; nothing written to tickets. Logs: .scratch/m029/e2e-fake-final.txt (310/310 = 266 + 44; the one changed old check: the Hybriel block path, changed by the architect after mission 027 — the baseline before this mission was 265/266 because of it: .scratch/m029/e2e-baseline.txt), e2e-fake-m012.txt (m012 tickets). Pre-029 code: .scratch/m029/orig/.

Tests of the container (mission 028, by hand, FAKE claude + own tickets; outputs in .scratch/m028/*.txt)

cd /media/STORAGE/projects/antcolony-scheduler/.scratch/m028
setsid nohup node own-tickets.mjs 6 > own-tickets.log 2>&1 &   # own ident 8751 + tickets 8752, tiny#1..6; kill -TERM it afterwards
./dc.sh up -d; docker logs -f antcolony-scheduler-m028          # A: RUN + ITER lines, tiny#1..6 leased → fake worker → PASS → posted
./t.sh states                                                     #    → all `awaiting creator`
./test-stop-park.sh      # B+C: docker stop during a 60 s session → finish → 20 s → PARKED (stop took 21 s, nothing left); docker start → resumed → posted
./test-finish-kill.sh    # D: stop during a short session → it finishes + posts, no park (9 s); E: docker kill → no process left;
                         # F: kill -9 of the scheduler → container exits → docker restarts it (RestartCount 1)
./test-exec.sh           # G: docker exec as uid 1000 = host fs; REAL claude --version + `claude -p /usage` (cost 0, 0 turns); ./colony leases
./dc.sh down             # remove the test container; kill -TERM $(cat own-tickets.pid)

Processes checked with ps//proc/<pid>/status: entrypoint, scheduler and fake claude run as uid 1000, groups of mre, CapEff 0. Log rotation: COLONY_LOG_MAX_BYTES=6000 in the test → colony.log, .1, .2.

Tests of mission 027 (in tests/e2e.mjs, FAKE claude only, 35 new checks; 3 old ones removed/moved)

tests/fake-claude.mjs: FAKE_DECIDED (the report's decided), FAKE_CLAUDE_MODE=longresult (result 124 characters + a test step on .scratch/ → refused). Registry fixture alpha has live (https://alpha.example) → alpha is the deploy-gate project; new project lv (host e2e-host, live https://lv.example). Checked: 10 new report refusals (result 121, 1 step, step 91, step on .scratch, step on 127.0.0.1 for a live project, 4 questions, a 4-line question, a 301-character question, decided on two lines / not a list) + a report without decided accepted; brief: the conventions section (after "Read first"), the shared-components decision, "Tickets of alpha" (newest first, without the ticket itself), where to test (live URL / real result), the new limits, decided; deploy gate via report --post (alpha#3): open → in progress "Built — waiting for the next deploy.", ONE-line comment + · ready to deploy, next "built — waiting for the deploy", ready lists exactly it, leases "never expires", deployed refused without the run folder / on a ticket that waits for nothing, then the deployed comment (numbered steps, question / problems / Decided bullets, machine line) + awaiting creator, again → nothing new, ready empty; deploy gate via work (lv: worker → controller → WORK DONE — lv#1 built, waiting for the deploy), lease ttl 2 s long past → leases --expire / run --once leave it alone, deployed → awaiting creator; fix step: refused report → FIX YOUR REPORT resume of the same session → posted with the Decided bullet; refused again → open once; --finalize-max-turns 0 → no fix step. Changed on purpose: the good report's test steps use the live URL; alpha#3 checks now go through ready/deployed (the comment-link checks moved to the deployed comment); the dummy comment's Test steps are a numbered list; readableLines does not count numbered step lines; refusal texts for 2–4 steps / 120 characters. Logs: .scratch/m027/e2e-fake-3.txt (266/266), e2e-fake-m012.txt (254/254, m012 tickets); texts .scratch/m027/texts.md, briefs .scratch/m027/briefs/ (incl. live-tickets-12.md: a brief built from the LIVE tickets, GETs only). Pre-027 code .scratch/m027/orig/, baseline before the change .scratch/m027/e2e-baseline.txt (234/234).

Tests of mission 026 (in tests/e2e.mjs, FAKE claude only, 19 checks; block "mission 026" after mission 022)

tests/fake-claude.mjs: FAKE_COMPLETE=false; FAKE_LEAVE_SERVER=worker,finalize,controller + FAKE_SERVER_LOG (detached node servers after 300 ms: worker → own on COLONY_PORT_FROM, another session's (COLONY_SESSION=s-foreign-e2e) on +2, a SIGTERM-ignoring one on +3; finalize → +5; controller → +4); a SIMULATED prompt cache (a resume with another --disallowedTools than the session's first call: cache read 0 / write 20000 / 9× cost). Checked: report refuses a missing / non-boolean complete; complete:false → REPORT OK "NOT complete", PASS, WORK HALF DONE, ticket = lease + the half-done comment (exact text, ≤ 5 lines, question linked, no Test line) + → open; next lists it; its brief has "Previous attempt" (session, HALF DONE, result, open items, work copy <dev>/.scratch/dev-m026, report path); a fresh ticket's brief has none; the next worker's stdin has it → complete → awaiting creator; the section then names the LAST session ("2 colony sessions"); a rejected rework that ends half done → open, next brief still starts with the rejection + Previous attempt; after a no-report session: "stopped without a usable report", run files. Cleanup: the e2e's own server started BEFORE the session (skipped, alive), the worker's own (stopped, SIGTERM), the stubborn one (SIGKILL), another session's (skipped, alive), the controller's (stopped after the controller), the finalize step's, a timed-out worker's; session.json cleanup lines; the e2e kills the skipped ones itself, nothing of ours listens afterwards. Cheaper finalize (fake model): old flags $0.1107 / cache read 0 vs. new $0.0123 / cache read 20000. m025 checks changed on purpose: finalize argv now WITHOUT --disallowedTools; salvage → lease + give-back / lease + comment + awaiting creator. Logs: .scratch/m026/e2e-fake-5.txt (234/234), e2e-fake-m012.txt (222/222), texts .scratch/m026/texts.md, briefs .scratch/m026/briefs/. Pre-026 code: .scratch/m026/orig/. Real run: .scratch/m026/real.sh (own tickets 8753/8754 via own-tickets.mjs, Sonnet worker --max-turns 2 + FAKE controller) → real-work.txt, run folder .scratch/m026/real-runs/, show.py <run dir> prints the transcript + cache numbers.

Tests of mission 025 (in tests/e2e.mjs, FAKE claude only, 28 checks)

tests/fake-claude.mjs: a --resume whose stdin has colony-finalize: is the finalize call (FAKE_FINALIZE_MODE good | nostructured | badreport | hang, default = the worker's mode; FAKE_FINALIZE_LOG; kind finalize in FAKE_LOG_DIR); it keeps each Claude session's total cost in .fake-claude-sessions/ of its cwd (FAKE_STATE_DIR) and reports it cumulatively on --resume like the real Claude; hang writes the worker's file before hanging. Checked: brief Budget section (120 → 110, 40 → 30, 7 → 5, minutes); out of steps → finalize → report → controller PASS → posted (lease + comment + awaiting creator only; finalize argv: --resume <the worker's session>, same schema/model/mode, --max-turns 8, --disallowedTools Edit,Write,NotebookEdit, no --session-id, no bypass, no token; finalize.md text; run files; costs $0.0123 + $0.0123 with Claude's cumulative $0.0246 not double counted; controller sees both parts); finalize fails too → back to open ONCE (2 events, the worker's reason); out of time → finalize (2 s) → posted; finalize hangs too → killed, open, nothing left running; --finalize-max-turns 0 = old behaviour, bad values refused; salvage: 5 refusals (ended well, report refused, other host, no token, no folder) write nothing, --post on an in progress ticket refused, failed finalize → SALVAGE FAILED, store identical, 2nd salvage refused; a salvage --post --ports 8770-8779 → finalize → controller (SALVAGED note) → posted (open → awaiting creator, no lease), SALVAGE COST line, ports in both env; check --post afterwards → nothing new. Logs: .scratch/m025/e2e-fake-4.txt (215/215), e2e-fake-m012.txt (203/203).

Tests of mission 023 (in tests/e2e.mjs, FAKE claude only)

Readable texts: every check that asserted an old text now asserts the NEW text exactly (lease, heartbeat, park, resume, give-back, expiry, report comment, awaiting-creator state, fail comment + state, all creator questions, issue summaries); 6 new report refusals (result on two lines / > 200 characters / missing, test with 5 or 0 steps, a step on two lines); the report comment has result + Test first, ≤ 3 bullets, no verified commands, the machine line last and short. Sweep at the end: every text the scheduler wrote in the whole store (92 in the last run; seeds skipped) has ≤ 5 lines the creator reads, at most ONE machine line and it is the last, none of the old jargon, every filed creator question's subject starts with "Question"; all texts are dumped to .scratch/e2e/texts.md (copy: .scratch/m023/texts-2.md). Weekly limit: week 85 % → no start, no weekly number → week UNKNOWN → no start, --week-limit 101 refused, --reserve 100 --week-limit 100 at 5 h 100 % and no weekly number → starts (fake /usage reads FAKE_WEEK_FILE, none = no weekly limit). Hand-filed question: a child Question (rel#5): which colour? (no colony source) does not make rel#5 a parent, the brief lists it as creator question; a child Questionnaire page does. Haiku: work --model haiku runs (not refused).

Tests of mission 022 (in tests/e2e.mjs, FAKE claude only, ~25 checks)

The e2e now starts tickets with a CREATOR ([email protected] in our ident; its per-app id = TICKETS_CREATOR_IDENTITY; logs in as "Creator", token CTOKEN, helper capi) — only the creator rejects/confirms. Checked: Colony may not reject (403); report --post of good(): alpha#3 blocked by hybriel#3 (link event), the question "Is small enough?" → alpha ticket Question (alpha#3): …, awaiting creator, parent alpha#3, linked in the comment, the zeta question has parent alpha#3; project rel (host loreana): rel#1 blocked by rel#2 → next "blocked by rel#2 (open)", rel#3 with child rel#4 → "a parent …", work refuses both (store unchanged), brief Relations sections (blocking / parent / children), blocker awaiting creator still blocks, confirmed by the creator → rel#1 eligible; project rej (host e2e-host): fake worker with 2 questions (FAKE_QUESTIONS) → 2 tickets (subjects, summary, parent, links), re-post (check --post, report --post) → HAVE, store identical; creator comment + Colony comment + rejection → next "REJECTED … → rework" (2 question children don't make it a parent), brief starts with the rejection + the creator's comment only + the rule; work → rejected → in progress → the worker's stdin starts with it → posted; afterwards the brief has no rejection; 2nd rejection → only the comments since the last session; a failed rework → open, next brief still starts with it; 3rd rejection → run --once picks it → posted. Briefs of the last run: .scratch/m022/briefs/ (.scratch/e2e/ is wiped by every e2e).

Tests of run (mission 020, in tests/e2e.mjs, FAKE claude only, 25 checks, ~40 s)

Projects d1, d2 (host e2e-host, own dev folders), worker port pool 8780-8799; tests/fake-claude.mjs also answers -p /usage (percent from FAKE_USAGE_FILE, re-read per call), FAKE_CLAUDE_MODE=ratelimit (rejected rate_limit_event + "You've hit your limit", 429; --resume then answers good), FAKE_CLAUDE_SLEEP, FAKE_LOG_DIR (one JSON per call: argv, pid, start/end, COLONY_PORTS*). Checked: live URL (unset / any spelling) → refused; --parallel 3 > pool → refused; quota 80% → no start, store unchanged, probe flags (plan, never bypass); --reserve 90 → starts, ports 8780-8789 in brief + env of worker AND controller; crash: SIGKILL of the scheduler while a 60 s worker runs (the worker dies with it, PDEATHSIG) → restart skips the live lease → after ttl 6 s expired … → open with the reason → ONE new session → posted; alpha#1 (no colony lease) never expired; SIGTERM idle → stops; leases; SIGTERM while running → finishes + posts, the other open ticket not started; 2nd SIGTERM → worker killed, PARKED (marker with the Claude session id), next run --once → claude -p --resume <id> → posted; usage limit → PARKED until the reset, quota 90% holds the resume, 10% → resumed → posted (cost of both parts summed); --parallel 2 → d1 + d2 started in ONE iteration with 8780-8789 / 8790-8799, really overlapping in time, the third ticket in d1's folder waits; one ITER line per iteration; nothing left running. Run outputs: .scratch/e2e/run-logs/*.txt (wiped by every e2e; last copy .scratch/m020/run-logs-2/). Last runs: .scratch/m020/e2e-fake-2.txt 144/144, e2e-fake-m012.txt 143/143 (COLONY_E2E_PORT_BASE=8760).

Real Claude (costs quota — 3 sessions, ≈ $0.37): COLONY_E2E_REAL=1 COLONY_E2E_PORT_BASE=8760 node tests/e2e.mjs adds (briefs get ports 8770–8779): the open dummy tickets are parked (on hold), then ONE cycle --post with a Sonnet worker (--max-turns 12) creating hello.txt and a Sonnet controller (--controller-max-turns 10) → the controller re-ran the cat, PASS, report comment with the controller section, awaiting creator, both sessions in mode auto; then a FAKE worker claiming missing.txt checked by the REAL controller → FAIL (both findings false), back to open. 124 checks. COLONY_E2E_REAL_MAXTURNS=1 adds mission 016's Haiku --max-turns 1 run (no report → open, no controller). Last real run (2026-09-24): .scratch/m019/e2e-real.txt, run folders kept in .scratch/m019/real-runs/ (.scratch/e2e/ is wiped by every run) — worker $0.087 + controller $0.175 (pass cycle), controller $0.104 (fail).

Files

colonywrapper (--); for run: setsid + SIGTERM/SIGINT → stop file
scheduler.hlentry, dispatch
lib/next.hl, lib/brief.hl, lib/report.hl, lib/work.hlthe commands (rules in their headers); work.hl also has check and cycle
lib/sandbox.hlthe bubblewrap box of every Claude session + the deploy mark (README "Sandbox"); tests/e2e-sandbox.mjs, tests/sandbox-mark.hl
lib/controller.hlthe controller: prompt, transcript, verdict validation, posting by the verdict
lib/status.hlantcolony#19: status, digest, the daily summary tick of run; tests/e2e-status.mjs
lib/daemon.hlrun (the loop, slots, ports, stop) and leases
lib/lease.hllease markers, parsing, expiry, heartbeat; mission 031: endOf (a lease that ended), stillLeased (re-read before acting)
lib/quota.hlthe /usage probe + its parser (ISO time → ms)
lib/claude.hlone headless Claude session (spawn, timeout, output parsing, usage) — worker + controller
lib/tickets.hlthe API client (UA, token, fileQuestion)
lib/registry.hlproject metadata
lib/util.hlsort / list / path helpers (Hybriel workarounds)
lib/previous.hlmission 026: the brief's "Previous attempt" (last colony session, how it ended, its report / work copy)
lib/deploy.hlmission 027: ready + deployed (the deploy gate's commands)
templates/conventions.mdmission 027: COPY of antcolony README "Conventions for all apps" (+ colours) — every brief carries it as text
lib/cleanup.shmission 026: stops what a session part left listening on its port range (called by claude.hl runClaude)
lib/relations.hlmission 022: blocked / parent rules, creator-question children, rejection vs. last session, the brief's rejection + Relations parts
lib/jsoncheck.hlcopy of tickets' JSON syntax check (hybriel#6: JSON.parse cannot fail softly)
templates/brief parts: Hybriel block (copy of antcolony missions/TEMPLATE-hybriel-block.md), rules, report schema; report.schema.json (JSON Schema for claude --json-schema, keep in step with report.hl); controller.md (the controller's instructions), verdict.schema.json (keep in step with controller.hl)
runs/<session>/work's files: brief.md, claude-result.json, claude-stderr.log, report.json, session.json; the controller's controller-prompt.md, controller-result.json, controller-stderr.log, verdict.json (0600/0700); mission 025: finalize.md, claude-result.before-finalize.json, claude-stderr.before-finalize.log
projects/the project registry
tests/e2e.mjsthe test; tests/fake-claude.mjs the claude stand-in
docker-compose.yml, .env.examplemission 028: the permanent container antcolony-scheduler + its settings template (a .env is never committed)
docker/pivot.shmission 028: root part in the busybox container — private mounts, pivot_root into the host fs, drop to uid 1000
docker/entrypoint.shmission 028: uid 1000 main process — mre's environment, ./colony run [--live], docker stop → finish/park, file log
logs/colony.logmission 028: the container's rotating file log (.1 … .5)
lib/librarian.hlmission 029: the librarian — sources (tickets, seed), the model step + the code's checks, rendering, the brief's decisions part, librarianFirst
templates/librarian.md, templates/librarian.schema.jsonmission 029: the librarian's instructions + answer schema (keep in step with librarian.hl checkAnswer)
templates/decisions-global.md (+ -archive.md)mission 029: GENERATED — the creator's decisions for all projects (every brief carries it)
librarian/mission 029: decisions.json (all entries, the source of every DECISIONS file), state.json, calls/<event>-{prompt.md,result.json,stderr.log}, lock, projects/<name>/DECISIONS.md for projects without a folder on this host

Vendored Hybriel: bin/hybriel + plugins/{core,crypto,data,fetch,fs,http,proc,time} copied from tickets.worldapi.org 2026-09-24 (sha256 c51d163d…5805, hybriel e565176b — see tickets' README).

Keywords (antcolony#17) — lib/keywords.hl

The creator writes a command as the first word of a comment; the scheduler acts and (for /prio and refusals) answers with a comment that ends in colony-keyword: <event id> · <keyword> — that marker is the only memory (nothing else is stored; an event with the marker in its history is done). Only comments whose author is in COLONY_CREATOR_NAMES count.

  • /hold → state on hold ("On hold, as you asked.").
  • /prio high|normal|low|-9..9 → the last valid /prio of the ticket decides; next / run pick the highest priority first, then the oldest (next lists priority high per ticket). Not understood → one refusing comment. Tickets has no priority field, so the history is the store.
  • /confirm, /reject <why> (the reason = the rest of the comment = the state text, so a rework brief quotes it). Tickets lets ONLY the creator confirm/reject (403 for the Colony token) — the scheduler needs the creator's token in COLONY_CREATOR_TOKEN_FILE (a tickets API token of the creator, ~/.config/antcolony/, 0600). Without it the scheduler answers once "I cannot confirm for you … use the button" and changes nothing. /reject without a reason is refused in a comment.
  • A state keyword is ignored when a state change came after it (not tickets' own automatic answered after the creator's comment) or the ticket is already in that state; a keyword that is not the first word is text.
  • run keeps an in-memory updatedMs per ticket and re-reads only changed tickets; keywords (CLI) reads all.
  • Test: COLONY_E2E_PORT_BASE=8700 node tests/e2e-keywords.mjs (own ident + tickets, 15 checks). If the current ident code of the dev folder is mid-change, point COLONY_E2E_IDENT_DIR at an older ident copy (e.g. .scratch/m028/own/ident/ident-code).

New ticket states and roles (antcolony#27; tickets#20) — lib/tickets.hl

tickets#20 renamed the states: awaiting creator → review (a creator question → pending), confirmed → done, rejected → reopened, in progress → progress, on hold → pending (canceled is new; answered is gone). One vocabulary inside, translated at the edge. All scheduler logic keeps the old names; tickets.hl is the only file that knows both:

  • everything READ (getJson, and the answer of postJson) has its state / to / from turned into the old names (normalizeStates) — an old server is left as it is;
  • everything WRITTEN is turned into what the server speaks: serverIsNew() asks GET /api/projects once (states has review); new server → new names (writeState), old server → old names. A built ticket → review; a question ticket (fileQuestionIn) → pending; /hold → pending; /confirm → done; /reject → reopened; the lease → progress. A reopened ticket is read as rejected, so it is picked up like a rejected one ("Rejections are work"); done counts as confirmed; review / pending are not work.
  • Roles. The scheduler's token (and the architect's) is a project member with the role edit (may move a ticket to any state); use may only move open ⇄ review. Somebody with the role admin must add it: POST /api/projects/<p>/members { user: "<name>", role: "edit" }.
  • The creator = the project's admin, not a name: adminsOf(project) reads the members (GET /api/projects/<p>, 5 min cache), isCreatorEvent decides for the librarian (creator texts) and the keywords (/prio /hold /confirm /reject). An event is matched by user id when the API shows one (userId), else by the admin's display name — the tickets API never shows the ident short id (it stays server side). A server without members (old tickets) → COLONY_CREATOR_NAMES as before.
  • Test: COLONY_E2E_PORT_BASE=8714 COLONY_E2E_IDENT_DIR=… node tests/e2e-states.mjs (own tickets #20 copy) — never together with another e2e (the full suite kills listeners on 8710–8719).

Sandbox — each worker in its own sealed box (antcolony#18) — lib/sandbox.hl

Every Claude session the scheduler starts (worker, its finalize / resume step, the controller) runs inside a bubblewrap box (/usr/bin/bwrap), built from an allowlist — not the whole host filesystem any more:

in the boxhow
the system: /usr /bin /lib /lib64 /sbin /etc /opt /sys, /proc + /dev of its own, private empty /tmp /var /runread-only (+ /run/systemd/resolve for DNS)
home = an empty tmpfs with only ~/.local/bin, ~/.local/share/claude, ~/.hybriel, ~/.gitconfig (ro) and the Claude login folder (CLAUDE_CONFIG_DIR, else ~/.claude + ~/.claude.json, rw)no ~/.config/antcolony (tokens), ~/.ssh, shell history, other projects
the project's own dev folderread-write; .env* files (not .env.*.example) replaced by an empty file
the dev folders of the projects it dependsOn (this host only) + paths its concept names + the docs folder (COLONY_SANDBOX_RO)read-only
the controller additionallythe run folder, read-only
the scheduler's own folder (project antcolony)runs/ logs/ sessions/ briefs/ are empty inside — no sight of other projects' briefs and reports

Own process namespace (--unshare-pid), dies with the scheduler (--die-with-parent); COLONY_TOKEN_FILE, COLONY_CREATOR_TOKEN_FILE, COLONY_AGENT_TOKEN_FILE, COLONY_STOP_FILE are removed from the environment. The network is shared (Claude API, the tickets copy, dev servers on the session's ports) — not restricted. Chrome, node, git, the project's own bin/hybriel work inside.

Deploy rights only for tickets marked for deploy. A ticket is marked by the line colony-deploy: yes in its opening text or in a comment of the creator. Only then the box also holds the project's live folder (if it is on this host) read-write and ~/.ssh read-only. The mark is read when the session starts and kept in session.json (deploy); a resume / finalize step uses the same box; the controller never gets it. (Deploys are still the architect's job — this is the door for the day an agent may deploy.)

No box possible (bwrap missing) → work REFUSES before it leases the ticket; a finalize / resume step whose box cannot be built runs /usr/bin/false (never a session with host access). ./colony box <project> [--deploy] [--folder F] prints the box's command prefix, one argument per line — "$(./colony box p)"-style: run <prefix> sh -c 'ls /media/STORAGE/projects' to see what a worker of p sees. Tested live: bwrap works as uid 1000 inside the antcolony-scheduler container (docker exec --user 1000 … bwrap …, no extra capabilities needed).

Test: node tests/e2e-sandbox.mjs (19 checks, no tickets, no Claude, ~10 s). The other suites set COLONY_SANDBOX=off themselves (their fake claude lives outside any box); tests/e2e.mjs has one block "antcolony#18" that runs a whole work (worker + controller, fake claude) with the box ON.

Status and daily summary (antcolony#19) — lib/status.hl

Built from the tickets, never written by hand (concept: "STATUS is generated from tickets"; timer "daily digest").

  • ./colony status [--write] — Markdown: a table per project (open / in progress / awaiting creator / on hold / rejected / confirmed; answered counts with awaiting creator), what waits for the creator, what is being worked on. --write also writes status/STATUS.md (COLONY_STATUS_DIR, default status/ beside the scheduler; regenerated, never edited).
  • ./colony digest [--post] — the daily summary: what waits for the creator = tickets in awaiting creator, split into questions to answer and built work to check (max 8 listed each, then "and N more" + a count per project), plus one line "working on N, M in the queue". Dry run prints; --post files it as ONE ticket in the inbox project (COLONY_INBOX_PROJECT), subject Daily summary <UTC day> — N tickets wait for you, source colony:digest:<day> (once per day, also after a restart; the creator's inbox is the only delivery channel until ident's daily mail exists), and writes status/STATUS.md. The summary tickets themselves are never counted as waiting.
  • run files it once a day: first iteration after COLONY_DIGEST_HOUR (UTC hour, default 7; off = never), log line DIGEST <day> — N waiting · filed|already filed <ticket>. No quota, no model.
  • Test: COLONY_E2E_PORT_BASE=8710 node tests/e2e-status.mjs (own tickets + ident, 12 checks).

Only real questions reach the creator (antcolony#24)

The check after each job (the controller) is the only filter for a worker's questions — there is no count limit any more (lib/report.hl no longer refuses more than 3). The verdict has a new list questions: one ruling { index, ruling, reason, answer } per question of the report (refused when one is missing, lib/controller.hl validateVerdict): ask (changes what gets built, nothing decided answers it → its own creator ticket as before) · decided / trivial / internal (dropped, no ticket; answer = the decision, added to the report's decided line) · packed (more than one decision → the verdict becomes a fail, decide, the next worker splits it). filterQuestions applies the rulings before postReport; templates/controller.md tells the check to search the brief's decisions first. Test: bin/hybriel tests/questions-test.hl; tests/fake-claude.mjs answers ask for all (FAKE_QUESTION_RULINGS=ask,decided,… to vary). A report posted by hand (report --post) is not filtered.

A ticket waiting for a creator question (antcolony#30)

A ticket with a creator question (child ticket) that is not answered (confirmed) or refused (rejected) is not work: next lists it as "waiting for the creator's answer on <question>", work refuses it. Without an open question a half-done ticket continues as before; a newer rejection outranks a pending question. Safety net: more than COLONY_MAX_STARTS_PER_HOUR (default 4; 0 = off) session starts on one ticket within an hour → not eligible until the hour has passed.

A report refused only on length keeps its work (antcolony#37)

The fix step (work.hl, mission 027) now gets up to MAX_FIX_TRIES (3) tries instead of one: each refused report resumes the SAME Claude session again with the exact problems (field, length, limit) listed once more (fixPrompt, title now "… (try N of 3)"); only after 3 refused fix tries does the ticket go back to open with "its report was incomplete" — one work session, one lease, the whole time. Before this, a report refused only on a line 5 characters too long threw the finished work away and restarted the ticket from nothing (seen 3× on 2026-09-27: tracker#1 twice, tracker#3). session.json.fixTries counts the tries made; meta.finalize still holds the LAST fix round (salvage still refuses a session that already had its fix/finalize step). Test (tests/e2e.mjs, fake claude): a worker whose first two reports are too long, fixed on the third try (2 fix tries, ONE lease, no second work session) → posted; a worker whose report stays too long for all 3 fix tries → back to open, no controller run.