fuzzyworld

fuzzyworld-tools / files / PLAN.md

PLAN.md

40.5 KB ยท history ยท blame

Plan

Where fuzzyworld-tools goes from here: from "a static mirror of my repos" to a small, open-source code host built on one rule: DreamHost only serves files; other machines do the work.

This is a planning document. Nothing in it is promised; the order is the current priority. Last updated 2026-11-03.

Who it is for. First, me: I build the site I want and need (no GitLab, no WordPress, as few machines as possible). Second, anyone who wants to get going with the fewest resources. Anything that only makes sense for my setup is tagged ๐Ÿ”Œ plugin: it should be optional and separable, so that the core stays small enough for most people.

The machines

Machine Does Can't do
DreamHost (shared hosting) serves static files with no rate limits; git over ssh; hourly cron; post-receive hooks long-running processes, Chromium, V8's JIT (Node runs --jitless), native addons
TrueNAS (home, always on) Docker: CI runner, headless browsers, screenshots, code-server; already mirrors every repo be reached from the internet. It pulls work from DreamHost over ssh instead
Laptop fw deploy, interactive work; can act as a second runner stay on

Everything on the sites stays pre-generated: no app server, no database, no accounts, repo content is never executed. Where a feature needs JavaScript, it is one script we ship (CSP script-src 'self', never anything from a repo).

Now (prioritized)

All done: โœ… live since 2026-10-27 (v1.1.0), unless noted.

  1. โœ… Apps index: newest deploy first (2026-11-04: now newest code commit first, and both the code and apps indexes are tables, not tiles). (It once listed the commits of each deploy; removed 2026-10-27 as noise.)
  2. โœ… LICENSE file (MIT) in this repo. Each repo's license on its summary page.
  3. โœ… Principles in four places: /why/, /how/, a new /principles/ page, and a "Philosophy" section in the README. /how/ is still a draft that install.sh doesn't install (see Backlog); the principle is in it for when it goes live.
  4. โœ… File navigation
  5. "Open in VS Code" link on every summary page (vscode://vscode.git/clone?url=โ€ฆ).
  6. Syntax highlighting at generation time (Pygments in ~/fw/pylib; no JavaScript).
  7. Per-file history, blame, and browsing every branch not yet merged into the default branch.
  8. VS Code in the browser at apps.fuzzyworld.net/vscode/?repo=<name>: the vscode-web build plus a read-only file-system extension reading files.json and objects/<sha> from code.fuzzyworld.net.
  9. โœ… CI runner on TrueNAS. See CI. Verified live with nordic-skating and tower.
  10. โœ… Screenshots as a built-in runner job. See Screenshots. Live for nordic-skating and tower.

Next (in this order)

The order goes from cheap to expensive, and each step reuses what the one before it built.

  1. Runners without TrueNAS (decided 2026-10-28; see Runners): runner tags, the laptop as a runner when idle and plugged in, secrets in a folder on each runner, and "no runners" as a normal state. Goes first. Built and committed 2026-10-31 (a499715); deployed 2026-10-31 (server, then the TrueNAS runner, then the laptop runner with --authorize, in that order: a new runner against the old server has every heartbeat refused). Smoke-tested: both runners poll DreamHost and find nothing to run; no real job has gone through them yet. Review follow-ups (claim check on done, tolerant release) deployed the same day.
  2. Feeds: Atom per repo (commits, releases), for app deploys, and site-wide. Smallest step: the data is already gathered for the log pages and the apps index. The blog reuses it. 1b. Blog at blog.fuzzyworld.net (see Blog): a generator that turns a folder or repo of Markdown into a blog from one JSON file, plus a one-time WordPress import. Replaces WordPress.
  3. Releases: a page per tag (the tag's message as notes), git archive .zip/.tar.gz, and files the runner builds (APKs, extensions). Import the release assets already in ~/code/gitlab-export/*/release-assets/. Comes early because some of those files exist nowhere else. Adds a releases feed to step 1.
  4. Stats and heatmap, built together, since both are computed from the same pass over the history when pages are generated: - stats: languages by bytes, size, commit count, share of commits made with AI (see Provenance) - heatmap: a commit-activity SVG per repo and site-wide
  5. Wiki: render a repo's docs/ (or wiki/) folder with a sidebar. Import the GitLab wikis in ~/code/gitlab-export/*/wikis.json. Static rendering only; editing comes with pull requests.
  6. Issues (built 2026-11-03): see Issues. The first feature that reads email; builds the mailbox plumbing. 5b. Blog comments by email (see Comments): right after Issues, because it reuses the mailbox plumbing.
  7. Pull requests: see Pull requests. Last, because it reuses the issues mailbox plumbing and needs the CI runner (approved CI runs on contributed code).

Runners

Goal (2026-10-28): TrueNAS becomes optional (๐Ÿ”Œ plugin), and the laptop can do the work. Running jobs needs no always-on machine; a small setup is a DreamHost account and a laptop.

Blog

blog.fuzzyworld.net is WordPress today (theme Scrawl, 144 posts, 4 pages, 45 comments, 15 media files, 3 categories, 16 tags, one author). Goal: the same site, generated by this repo, with no WordPress.

Built and live 2026-10-31 at writing.fuzzyworld.net (queue item Q4; not linked from anywhere yet). The generator is server/blog.py (+ server/atom.py, server/site-blog/), the importer lib/wp_import.py; commands and behavior are in docs/blog.md. Decisions taken: build on the laptop from Dropbox and upload only the audited output folder (never the source); footer "All rights reserved" notice with an AI-training line and a robots.txt blocking AI crawlers; the WordPress text wins when a source file differs (similarity under 0.95: 100 of the 142 posts are written from WordPress into ~/Blog/wordpress/); the old site's custom CSS is carried over (hidden <br>, dotted scene breaks, underlined chapter headings); a light/dark switch and reading-progress bar added (they needed script-src 'self'). The Atom writer exists now, so Next 1 (Feeds) can reuse it. Done 2026-11-01 (Q5/Q6): blog.fuzzyworld.net 301-redirects every path (query kept) to writing.; til.fuzzyworld.net answers 410 Gone. Both WordPress folders stay on the server, unserved; the original .htaccess of each is saved as .htaccess.wp-20261101. Backup: a WordPress export of the blog (Pavel, ~/Downloads). The ancient old.blog.fuzzyworld.net copy was moved to ~/old/ on the server (it was not served). Still to do: building from the push hook after the move to a private repo, the til blog: built 2026-11-01 (Q14), config ~/til/fw-blog.json (4 posts, WordPress slugs kept, one image in ~/til/media/), same generator; uploaded the same day (the WordPress folder was moved to ~/old/ on the server first, because fw blog deploy runs rsync --delete). Below, the plan as written before the build: it still says blog.fuzzyworld.net in places; read that as writing.fuzzyworld.net.

What is there (read from its public API, 2026-10-28): - Posts at /YYYY/MM/DD/<slug>/, an RSS feed at /feed/, and monthly archives. Categories hardly matter ("Uncategorized" has 138 posts). Tags are the real structure (a story series is a tag, e.g. Dice, Walkabout, Writing Excuses). - Pages: Highlights (the front page: a chronological list of the work I consider good, with an AI-use note), Blog (all posts), Collections (series in reading order, a hand-written list), Upcoming. - 2 posts are password-protected (Rasputin, Belfast). They are not carried over (see Protected posts below). - A reading time on each post, media (images, plus epub/kpf/pdf downloads of Wick Canyon), and comments. - Comments: 45 (36 top-level, 8 replies). They get imported, as static text with the author's name (no emails).

Source: ~/Blog (Dropbox for now; a private git repo later): 286 Markdown files, YYYY-MM-DD-name.md, no front matter. The title is the first # heading. The post is only the ## Writing section; ## Spelling (cspell words) and ## Removed writing (cut text) are private notes and are never published. 164 files have a ## Writing section; the others are notes. WordPress turns single line breaks into <br>, so the renderer must too.

Findings that shape the design: - File names do not match published URLs. Only 13 of 144 posts match date-slug: the publish date differs from the draft date, and some names differ (belfast-porn.md is published as /belfast/). The import must record the mapping (published date, slug, tags, highlighted) somewhere. - So the generator must keep the old URLs working: same path pattern and the same slugs.

Post metadata (decided 2026-10-28): front matter and config, so the tool does not force edits to source files. Precedence, highest first:

  1. front matter in the file (date, slug, tags, highlight, published);
  2. per-file overrides in the blog's JSON file, keyed by file name (a "sidecar" for people who will not or cannot edit their files);
  3. defaults from the file: date and name from YYYY-MM-DD-name.md, title from the first # heading.

Nothing is required: a plain folder of Markdown works. Nothing is published unless marked (published in front matter or an entry in the config): ~/Blog has 286 files and 144 posts are live. This is the same rule as repo visibility (private unless exactly public). For ~/Blog, the import writes the sidecar (your files stay untouched; easy to undo), and a later command can convert it to front matter, e.g. when the posts move to a private git repo.

Protected posts (decided 2026-10-28): not carried over. The importer skips password-protected posts (Rasputin, Belfast) and the comments on them, and says so. They stay drafts in ~/Blog. ๐Ÿ”Œ plugin idea for later: .htaccess basic auth first; client-side encryption only if needed (it adds a script and allows offline guessing).

Design: - One JSON file describes a blog (title, base URL, source folder, which section is the post body, URL pattern, tag list, highlight list, collections, feed). The tool is gen.py run on a repo of Markdown, from the push hook. - Reuse what exists: Python-Markdown, Pygments, the site CSS, the feed code from step 1. - Import (๐Ÿ”Œ plugin): one-time script that reads the WordPress API: posts, comments, media, and a redirect map. - The editor and the Dropbox โ†’ private-git-repo move come after the generator, not with it. - Built 2026-11-02 (Q91/Q92): fw blog watch <config> (server/blog_watch.py; docs/blog.md, "The watcher"): a laptop launchd agent (fw blog watch install) runs one tick every 60 s per blog config. After a quiet period it mirrors the Dropbox source into a local git repo (snapshot commits on dropbox-mirror, never main) and, with autodeploy, deploys edits only when the set of output files, the generator (code/templates), the config and the deploy target are unchanged (anything else is held back with a notification; the baseline is deployed.json, also written by a manual fw blog deploy and guarded by the state folder's lock). Run on ONE machine only. Not built yet: the cutover fw blog move <config> --to repo|dropbox (copy, build both, require identical output, repoint the ~/Blog symlink, keep the old copy), then Dropbox is dropped and the Air uses a clone plus a simple fw blog sync; the server remote and the push-hook build (Q25). - Needs the blog.fuzzyworld.net subdomain pointed at a folder (DreamHost panel).

Comments

Decided 2026-10-28: by email, no accounts. Revised 2026-11-01: a web form (server/comments/post.php, written, never run or installed) was designed too, but only the email path is enabled (Pavel: a form means code that runs on every web request, which is more attack surface than he wants; see the tiers in the README, form = T2, email = T1). The form stays in the repo, off by default. - A "comment by email" link on each post is a mailto: to the comments mailbox, with the post in the subject (Comment [writing:/2024/05/01/slug/]). A cron job (server/comments.py ingest) reads the comments mailbox, over IMAP (a hosted DreamHost mailbox; mail arrives through MailChannels) or from a local Maildir, and puts each comment in a pending queue. - Identity is the sender's verified address (the topmost Authentication-Results header must show DMARC pass, or DKIM pass for the From domain). The address is never shown or stored, only a hash; only the display name is shown. - Moderation: a person's first comment waits for my approval (fw comments list|approve|reject|remove|untrust); with approve --trust that sender is published on arrival from then on. Limits: 4000 characters, 5 per sender per 24 hours, 200 pending. A deletion request is honoured. - Approved comments are written as one static JSON file per post (/_comments/<post path>.json) which blog.js renders with textContent, so a comment appears without a rebuild. fw blog deploy excludes /_comments/. Imported WordPress comments stay baked into the pages. Readers without JavaScript see only the imported ones. - The form (off): server/comments/post.php on its own subdomain, comments_form in the blog config; every form comment is moderated. Enabling it is a decision for later; the upsides are no email client needed, the downsides are a public PHP endpoint, spam, and unverified identities. - ๐Ÿ”Œ Google sign-in plugin: would need an endpoint that checks the Google token (T2), so it stays a plugin.

CI dashboard

โœ… Built 2026-10-27 as proposed (docs/ci.md: "Watching CI"):

Backlog

Principles

Wording approved 2026-10-26:

Things made with AI should be shared freely with the world. The code and apps here were made with AI, so they are MIT-licensed: use them, change them, share them.

My conversations with the AI are different. My words and my way of working are mine to publish or not. I will publish a conversation only when I choose to.

CI

โœ… Live since 2026-10-27. docs/ci.md is the full reference; this is the summary. It is the same pattern as GitLab runners: a runner registers, pulls jobs, runs them in containers, and reports back. The difference is that there's no coordinator service: the coordinator is files on DreamHost.

  1. Queue. post-receive runs fw ci-queue, which writes ~/fw/queue/pending/<id>.json (repo, ref, sha, default branch, time) for each pushed branch or tag with a .fw-ci.yml, .gitlab-ci.yml or package.json.
  2. Claim. The runner (runner/fw-runner.py, TrueNAS cron every 2 minutes) runs ssh dh claim <runner> through server/runner-api. The server moves the job to running/ with one rename, which is atomic, so two runners can never take the same job. The same step puts jobs running for over 2 hours back in pending/.
  3. Registration = an ssh key. The runner's key is limited in authorized_keys to command=".../server/runner-api": it can claim jobs, fetch repos and upload results, and can't get a shell.
  4. Run. The runner clones over ssh and runs each job in a throwaway Docker container from the job's image. Inside jobs, origin is the read-only https clone URL; the runner's key never enters a container.
  5. Report. Logs and pipeline.json are uploaded to ~/fw/ci/<repo>/<sha>/ (a running job's log every 15 seconds); gen.py renders โœ“/โœ— on the log and summary pages, a page per pipeline with its logs, a badge, and the dashboard at /ci/.
  6. Updates. The runner restarts itself between jobs when runner/install.sh has replaced it.
  7. Two runners. Tags, the heartbeat lease and the laptop runner are deployed (2026-10-31; see Runners).

Job definitions: .fw-ci.yml (decided 2026-10-27: our own name; the format stays a subset of GitLab CI's, so moving a repo over is a rename; the old .gitlab-ci.yml name still works until then). 42 repos have one, and they use only image, stages, script, before_script, variables, rules (branch conditions), needs, artifacts, cache. 36 of them are GitLab Pages jobs; a pages job maps to fw deploy. An unsupported keyword fails the job with a clear message; it is never silently ignored. Repos without the file fall back to fuzzyworld.checks from package.json. engye (20 KB: Android builds, Play Store, Firefox extension) is the hard case and the test of how far the subset needs to go.

Security. Secrets (/mnt/main-2025/fw-runner/secrets/<repo>.env on TrueNAS) are given only to jobs on the default branch and tags. None are set yet (see Backlog). Code from contributors will run only after I approve it, because it runs on a machine in my home (part of Pull requests).

Screenshots

โœ… Live since 2026-10-27. A built-in runner job, after a successful pipeline on the default branch of an app (fuzzyworld.app in package.json):

Issues

Report replays (Q104; planned 2026-11-03, built 2026-11-03 in this repo)

Idea from the Explodorama bug mails: a report can carry a machine-readable payload (for the game: the run record, a few KB of JSON: seed, rules version and actions), and the issue page can then show what the reporter saw as a looping animation. The game side (the replay page, its limits, how old rules are loaded) is in ~/code/explodorama/PLAN.md ยง20. This repo's part:

This repo's task list (2026-11-03; Pavel: "add to the plan, start implementing"). Items 1-4 were built on 2026-11-03 (server/issues.py, lib/deploy.mjs); the first line of each says so. Not yet committed or deployed is deliberately not recorded here. In dependency order:

  1. Issues (built 2026-11-03; Next 5): cron ingest of the bugs mailbox over IMAP (reuse the comments.py / comments-ingest.sh plumbing), one static page per thread under โ€ฆ/<repo>/issues/<n>/, fw issues <repo> public|reviewed|private (anything else = private), reporter addresses never shown, and a "mark public" action for reviewed mode.
  2. Record extraction (built 2026-11-03): decode the quoted-printable mail, find the Run record: {โ€ฆ} line, validate (โ‰ค 64 KB, valid JSON, expected keys), write record.json beside the issue under the issue's visibility rule; device string stays out of the page. Test with a real report (Explodorama's test/fixtures/q101-bomb-pass.json and its .eml).
  3. App opt-in (built 2026-11-03): a replay key in the "fuzzyworld" block of package.json ("replay": "replay.html?report={record}"); the issue-page generator reads it and shows "Watch what happened". No key, no change.
  4. fw deploy carries extra files, kept forever (built 2026-11-03): a per-app glob list ("extra": ["sim/*.js"]) published with the deploy; extra files are protected from --delete through an rsync filter (uploaded and updated, never removed), so a later deploy never deletes old rules versions.

No CSP change (the viewer is the app's own script) and no server code on DreamHost. The admin panel (Q102, local-only) comes after all four.

Order: (1) the game's production replay page and limits (its PLAN ยง20.2 steps 1-2), by hand, with a stored record; (2) the cron extracting and publishing record.json and the replay key; (3) per-rules-version sim files in fw deploy; (4) the panel. Each step is useful alone.

Decided 2026-11-03 (Pavel): the viewer opens as its own page (a plain link, no iframe); a reviewed report's record is published at the same moment as its text; old sim/ files are kept forever; compact records in the game's mailto: link (game PLAN ยง20.2 step 4) come after the cron (step 2); the published page defaults to the plain recorded run, with comparison as an option (game PLAN ยง20.2 step 5). Nothing open here.

Admin panel (fw admin, Q102; designed 2026-11-04, decisions by Pavel)

Version 1 built 2026-11-04 (lib/admin.py, tests/test_admin.py; server side: issues list|show --json, issues record, comments list --json). Not yet tried against the real server (tested only with a fake runner). Decisions made while building: - Login: a one-time login URL (random key) that sets a session cookie; everything else needs the cookie, plus the per-launch POST token and Host / Origin / Sec-Fetch-Site checks. - Watch = dev replay: a link to the deployed viewer when the record is public, "Download record" always, and "Prepare dev replay" (runs the app's report-replay npm script with the record and --rules <commit>; needs npm run dev running in the app). - Recently approved comments (the 20 newest) with a Remove button are in v1, not later. - FW_CODE_DIR (also --code-dir, default ~/code) is configurable: where the apps' working copies live. - Deploy tab (Q111 stage 1, built 2026-11-04): read-only; fw deploys --json (server/deploys.py) joins the deploy ledger (~/fw/state/deployed.json, written by bin/deploy-tools, keyed <repo>/<target>) and each app's .fuzzyworld.json with the repo's HEAD. Targets are declared in package.json fuzzyworld.deploy. Next: stage 2, a "Deploy now" button that deploys origin/main from a clean export (confirm page with details, background run, progress page); stage 3, any repo declares targets, apps included, tested on nordic-skating. - Still not in v1: "make this a regression fixture", pull requests, CI, blog state.

A local-only web page over fw commands (see "Report replays": never hosted on DreamHost). fw admin starts a small Python (stdlib) server on 127.0.0.1 and opens the browser; every button runs the same fw ... command over ssh that Pavel would type, so the panel holds no state and has no authority beyond his ssh key.

Pull requests

Provenance

Source
1# Plan
2
3Where fuzzyworld-tools goes from here: from "a static mirror of my repos" to a small, open-source
4code host built on one rule: **DreamHost only serves files; other machines do the work.**
5
6This is a planning document. Nothing in it is promised; the order is the current priority.
7Last updated 2026-11-03.
8
9**Who it is for.** First, me: I build the site I want and need (no GitLab, no WordPress, as few machines as possible).
10Second, anyone who wants to get going with the fewest resources. Anything that only makes sense for my setup is
11tagged ๐Ÿ”Œ **plugin**: it should be optional and separable, so that the core stays small enough for most people.
12
13## The machines
14
15| Machine | Does | Can't do |
16|---|---|---|
17| DreamHost (shared hosting) | serves static files with no rate limits; git over ssh; hourly cron; `post-receive` hooks | long-running processes, Chromium, V8's JIT (Node runs `--jitless`), native addons |
18| TrueNAS (home, always on) | Docker: CI runner, headless browsers, screenshots, code-server; already mirrors every repo | be reached from the internet. It **pulls** work from DreamHost over ssh instead |
19| Laptop | `fw deploy`, interactive work; can act as a second runner | stay on |
20
21Everything on the sites stays pre-generated: no app server, no database, no accounts, repo content is
22never executed. Where a feature needs JavaScript, it is one script **we** ship (CSP `script-src 'self'`,
23never anything from a repo).
24
25## Now (prioritized)
26
27All done: โœ… live since 2026-10-27 (v1.1.0), unless noted.
28
291. โœ… Apps index: newest deploy first (2026-11-04: now newest code commit first, and both the code and apps indexes are tables, not tiles). (It once listed the commits of each deploy; removed 2026-10-27 as noise.)
302. โœ… **LICENSE file** (MIT) in this repo. Each repo's license on its summary page.
313. โœ… **Principles** in four places: `/why/`, `/how/`, a new `/principles/` page, and a "Philosophy" section in the
32 README. `/how/` is still a draft that `install.sh` doesn't install (see Backlog); the principle is in it for
33 when it goes live.
344. โœ… **File navigation**
35 1. "Open in VS Code" link on every summary page (`vscode://vscode.git/clone?url=โ€ฆ`).
36 2. Syntax highlighting at generation time (Pygments in `~/fw/pylib`; no JavaScript).
37 3. Per-file history, blame, and browsing every branch not yet merged into the default branch.
38 4. VS Code in the browser at `apps.fuzzyworld.net/vscode/?repo=<name>`: the `vscode-web` build plus a
39 read-only file-system extension reading `files.json` and `objects/<sha>` from code.fuzzyworld.net.
405. โœ… **CI runner** on TrueNAS. See [CI](#ci). Verified live with nordic-skating and tower.
416. โœ… **Screenshots** as a built-in runner job. See [Screenshots](#screenshots). Live for nordic-skating and tower.
42
43## Next (in this order)
44
45The order goes from cheap to expensive, and each step reuses what the one before it built.
46
470. **Runners without TrueNAS** (decided 2026-10-28; see [Runners](#runners)): runner tags, the laptop as a runner when
48 idle and plugged in, secrets in a folder on each runner, and "no runners" as a normal state. Goes first.
49 **Built and committed 2026-10-31 (a499715); deployed 2026-10-31** (server, then the TrueNAS runner, then the laptop runner with `--authorize`,
50 in that order: a new runner against the old server has every heartbeat refused). Smoke-tested: both runners poll
51 DreamHost and find nothing to run; no real job has gone through them yet. Review follow-ups (claim check on `done`,
52 tolerant `release`) deployed the same day.
531. **Feeds**: Atom per repo (commits, releases), for app deploys, and site-wide. Smallest step: the data is already
54 gathered for the log pages and the apps index. The blog reuses it.
551b. **Blog** at `blog.fuzzyworld.net` (see [Blog](#blog)): a generator that turns a folder or repo of Markdown into a
56 blog from one JSON file, plus a one-time WordPress import. Replaces WordPress.
572. **Releases**: a page per tag (the tag's message as notes), `git archive` `.zip`/`.tar.gz`, and files the
58 runner builds (APKs, extensions). Import the release assets already in `~/code/gitlab-export/*/release-assets/`.
59 Comes early because some of those files exist nowhere else. Adds a releases feed to step 1.
603. **Stats and heatmap**, built together, since both are computed from the same pass over the history when pages are generated:
61 - stats: languages by bytes, size, commit count, share of commits made with AI (see [Provenance](#provenance))
62 - heatmap: a commit-activity SVG per repo and site-wide
634. **Wiki**: render a repo's `docs/` (or `wiki/`) folder with a sidebar. Import the GitLab wikis in
64 `~/code/gitlab-export/*/wikis.json`. Static rendering only; editing comes with pull requests.
655. **Issues** (built 2026-11-03): see [Issues](#issues). The first feature that reads email; builds the mailbox plumbing.
665b. **Blog comments** by email (see [Comments](#comments)): right after Issues, because it reuses the mailbox plumbing.
676. **Pull requests**: see [Pull requests](#pull-requests). Last, because it reuses the issues mailbox plumbing
68 and needs the CI runner (approved CI runs on contributed code).
69
70## Runners
71
72Goal (2026-10-28): TrueNAS becomes optional (๐Ÿ”Œ plugin), and the laptop can do the work. Running jobs needs no
73always-on machine; a small setup is a DreamHost account and a laptop.
74
75- **No runners is a normal state.** Jobs stay in `pending/` until a runner claims them (the queue is files, so this
76 already works). The dashboard says "waiting for a runner" instead of looking broken (a runner counts as gone after
77 15 minutes without a claim; each claim writes `~/fw/runners/<name>`).
78- **Laptop runner** (launchd via `runner/install-laptop.sh`, every 2 minutes), only when it will not hurt my work: on AC
79 power, CPU at least half idle, Docker running, containers at a low CPU share, no new claims once I get busy (checked
80 between jobs; a job already running is not stopped). Everything scheduled runs then. It assumes the laptop
81 is on a few hours a day; that is enough for me.
82- **Lease.** A claimed job needs a heartbeat (`beat`, every 30 seconds, from a thread in the runner). A job whose runner
83 went quiet for ~2 minutes (lid closed) goes back to `pending/` instead of waiting 2 hours. If that runner wakes up and
84 finishes anyway, its results are uploaded too, and the job may have run twice: accepted. Its `done` is refused,
85 so it cannot remove the new holder's claim. Runners that send no
86 heartbeat (old versions) keep the 2-hour rule.
87- **Tags** (`docker`, `chrome`, `android`): job `tags:` are honoured, and screenshots need `chrome`. The runner learns what
88 a pipeline needs only after cloning, so a runner that lacks a tag gives the job back (`release`); the server
89 remembers and does not offer it to that runner again. Simple, at the cost of one clone per declined job.
90- **Secrets.** Each runner has a folder (`โ€ฆ/secrets/<repo>.env`; on TrueNAS `/mnt/main-2025/fw-runner/secrets/`).
91 A deploy job tagged `secrets` (a reserved tag: "this repo's secrets") runs only on a runner that has the file; no repo
92 uses it yet. Only default-branch and tag jobs get secrets. Put them on the laptop and TrueNAS is not needed. Laptop theft means rotating them (FileVault on).
93- **Stretch goal: a secure secrets API** (๐Ÿ”Œ): `runner-api` hands each job its secrets over ssh, per runner key, so
94 no runner stores them. Wait for user requests or downtime.
95
96## Blog
97
98`blog.fuzzyworld.net` is WordPress today (theme Scrawl, 144 posts, 4 pages, 45 comments, 15 media files, 3 categories,
9916 tags, one author). Goal: the same site, generated by this repo, with no WordPress.
100
101**Built and live 2026-10-31 at `writing.fuzzyworld.net`** (queue item Q4; not linked from anywhere yet). The generator is
102`server/blog.py` (+ `server/atom.py`, `server/site-blog/`), the importer `lib/wp_import.py`; commands and behavior are in
103docs/blog.md. Decisions taken: build on the laptop from Dropbox and upload only the audited output folder (never
104the source); footer "All rights reserved" notice with an AI-training line and a `robots.txt` blocking AI crawlers; the
105WordPress text wins when a source file differs (similarity under 0.95: 100 of the 142 posts are written from WordPress
106into `~/Blog/wordpress/`); the old site's custom CSS is carried over (hidden `<br>`, dotted scene breaks, underlined
107chapter headings); a light/dark switch and reading-progress bar added (they needed `script-src 'self'`). The Atom
108writer exists now, so Next 1 (Feeds) can reuse it. **Done 2026-11-01 (Q5/Q6):** `blog.fuzzyworld.net` 301-redirects every path (query kept) to `writing.`; `til.fuzzyworld.net` answers 410 Gone. Both WordPress folders stay on the server, unserved; the original `.htaccess` of each is saved as `.htaccess.wp-20261101`. Backup: a WordPress export of the blog (Pavel, `~/Downloads`). The ancient `old.blog.fuzzyworld.net` copy was moved to `~/old/` on the server (it was not served). **Still to do:** building from the push hook after the move to a private repo,
109the `til` blog: **built 2026-11-01 (Q14)**, config `~/til/fw-blog.json` (4 posts, WordPress slugs kept, one image in `~/til/media/`), same generator; uploaded the same day (the WordPress folder was moved to `~/old/` on the server first, because `fw blog deploy` runs rsync `--delete`). Below, the plan as written before the
110build: it still says `blog.fuzzyworld.net` in places; read that as `writing.fuzzyworld.net`.
111
112**What is there** (read from its public API, 2026-10-28):
113- Posts at `/YYYY/MM/DD/<slug>/`, an RSS feed at `/feed/`, and monthly archives. Categories hardly matter ("Uncategorized"
114 has 138 posts). Tags are the real structure (a story series is a tag, e.g. Dice, Walkabout, Writing Excuses).
115- Pages: **Highlights** (the front page: a chronological list of the work I consider good, with an AI-use note),
116 **Blog** (all posts), **Collections** (series in reading order, a hand-written list), **Upcoming**.
117- 2 posts are password-protected (Rasputin, Belfast). They are not carried over (see Protected posts below).
118- A reading time on each post, media (images, plus epub/kpf/pdf downloads of Wick Canyon), and comments.
119- Comments: 45 (36 top-level, 8 replies). They get imported, as static text with the author's name (no emails).
120
121**Source: `~/Blog`** (Dropbox for now; a private git repo later): 286 Markdown files, `YYYY-MM-DD-name.md`, no front
122matter. The title is the first `# heading`. The post is only the **`## Writing`** section; `## Spelling` (cspell words)
123and `## Removed writing` (cut text) are private notes and are never published. 164 files have a `## Writing` section;
124the others are notes. WordPress turns single line breaks into `<br>`, so the renderer must too.
125
126**Findings that shape the design:**
127- File names do **not** match published URLs. Only 13 of 144 posts match `date-slug`: the publish date differs from the
128 draft date, and some names differ (`belfast-porn.md` is published as `/belfast/`). The import must record the mapping
129 (published date, slug, tags, highlighted) somewhere.
130- So the generator must keep the old URLs working: same path pattern and the same slugs.
131
132**Post metadata** (decided 2026-10-28): front matter *and* config, so the tool does not force edits to source files.
133Precedence, highest first:
134
1351. front matter in the file (`date`, `slug`, `tags`, `highlight`, `published`);
1362. per-file overrides in the blog's JSON file, keyed by file name (a "sidecar" for people who will not or cannot edit
137 their files);
1383. defaults from the file: date and name from `YYYY-MM-DD-name.md`, title from the first `# heading`.
139
140Nothing is required: a plain folder of Markdown works. **Nothing is published unless marked** (`published` in front
141matter or an entry in the config): `~/Blog` has 286 files and 144 posts are live. This is the same rule as repo
142visibility (private unless exactly `public`). For `~/Blog`, the import writes the sidecar (your files stay untouched;
143easy to undo), and a later command can convert it to front matter, e.g. when the posts move to a private git repo.
144
145**Protected posts** (decided 2026-10-28): not carried over. The importer skips password-protected posts (Rasputin,
146Belfast) and the comments on them, and says so. They stay drafts in `~/Blog`. ๐Ÿ”Œ plugin idea for later: `.htaccess`
147basic auth first; client-side encryption only if needed (it adds a script and allows offline guessing).
148
149**Design:**
150- One JSON file describes a blog (title, base URL, source folder, which section is the post body, URL pattern, tag
151 list, highlight list, collections, feed). The tool is `gen.py` run on a repo of Markdown, from the push hook.
152- Reuse what exists: Python-Markdown, Pygments, the site CSS, the feed code from step 1.
153- **Import (๐Ÿ”Œ plugin):** one-time script that reads the WordPress API: posts, comments, media, and a redirect map.
154- The editor and the Dropbox โ†’ private-git-repo move come after the generator, not with it.
155- **Built 2026-11-02 (Q91/Q92):** `fw blog watch <config>` (`server/blog_watch.py`; docs/blog.md, "The watcher"): a laptop
156 launchd agent (`fw blog watch install`) runs one tick every 60 s per blog config. After a quiet period it mirrors the
157 Dropbox source into a local git repo (snapshot commits on `dropbox-mirror`, never main) and, with `autodeploy`, deploys
158 edits only when the set of output files, the generator (code/templates), the config and the deploy target are
159 unchanged (anything else is held back with a notification; the baseline is `deployed.json`, also written by a manual `fw blog deploy` and guarded by the
160 state folder's lock). Run on ONE machine only. **Not built yet:**
161 the cutover `fw blog move <config> --to repo|dropbox` (copy, build both, require identical output, repoint the `~/Blog`
162 symlink, keep the old copy), then Dropbox is dropped and the Air uses a clone plus a simple `fw blog sync`; the server
163 remote and the push-hook build (Q25).
164- Needs the `blog.fuzzyworld.net` subdomain pointed at a folder (DreamHost panel).
165
166## Comments
167
168Decided 2026-10-28: by email, no accounts. Revised 2026-11-01: a web form (`server/comments/post.php`, written, never run
169or installed) was designed too, but **only the
170email path is enabled** (Pavel: a form means code that runs on every web request, which is more attack surface than
171he wants; see the tiers in the README, form = T2, email = T1). The form stays in the repo, off by default.
172- A "comment by email" link on each post is a `mailto:` to the comments mailbox, with the post in the subject
173 (`Comment [writing:/2024/05/01/slug/]`). A cron job (`server/comments.py ingest`) reads the comments mailbox, over IMAP
174 (a hosted DreamHost mailbox; mail arrives through MailChannels) or from a local Maildir, and puts each comment in a pending queue.
175- Identity is the sender's verified address (the topmost Authentication-Results header must show DMARC pass, or DKIM
176 pass for the From domain). The address is never shown or stored, only a hash; only the display name is shown.
177- Moderation: a person's first comment waits for my approval (`fw comments list|approve|reject|remove|untrust`);
178 with `approve --trust` that sender is published on arrival from then on. Limits: 4000 characters, 5 per sender per 24 hours, 200 pending.
179 A deletion request is honoured.
180- Approved comments are written as one static JSON file per post (`/_comments/<post path>.json`) which `blog.js`
181 renders with `textContent`, so a comment appears without a rebuild. `fw blog deploy` excludes `/_comments/`.
182 Imported WordPress comments stay baked into the pages. Readers without JavaScript see only the imported ones.
183- **The form (off):** `server/comments/post.php` on its own subdomain, `comments_form` in the blog config; every
184 form comment is moderated. Enabling it is a decision for later; the upsides are no email client needed, the
185 downsides are a public PHP endpoint, spam, and unverified identities.
186- **๐Ÿ”Œ Google sign-in plugin:** would need an endpoint that checks the Google token (T2), so it stays a plugin.
187
188## CI dashboard
189
190โœ… Built 2026-10-27 as proposed (docs/ci.md: "Watching CI"):
191
192- `code.fuzzyworld.net/ci/`: one static page for all public repos: queued jobs, running pipelines (which job,
193 for how long), and the last 50 results. Rebuilt on every queue change and every upload from a runner.
194- Live logs: while a job runs, the runner uploads its log every ~15 seconds, so the job's page shows it growing.
195- Pages that show something running reload themselves every 15 seconds (`<meta http-equiv="refresh">`), so no
196 JavaScript is needed and the site's `script-src 'none'` stays.
197- Private repos: never named on the public page. For those, `fw ci-list` in the terminal (and a `fw ci-watch`
198 that follows a run and prints each job's result).
199
200## Backlog
201
202- **URGENT (Pavel, 2026-11-04): AI usage analysis per repo, on the CI/usage pages.** There is a lot of public chatter
203 about Claude's usage limits, so each repo should show what its AI use looks like over time. It builds on the
204 existing per-commit "Usage" tab (`repo_usage` / `usage_costs` in `server/gen.py`, from the `FW-Tokens-*` commit
205 trailers) and on the planned "Stats and heatmap" step (Next 3). Candidate views, as static SVG charts (no JS):
206 AI usage across time (output tokens, cached reads, cost per week or per session); AI share of commits and of pull
207 requests over time (`Co-Authored-By`, with the Provenance caveats); lines of code added and removed over time;
208 **usage per line of code** (output tokens, and estimated cost, per added line, per week and per commit, with the
209 median and the hot commits already marked on the Usage tab); and a site-wide roll-up across repos. Open points:
210 how to price tokens (a per-model price table kept in the repo, stated as an estimate); what a "line" is (added
211 lines in non-generated, non-vendored files); sessions that span several repos; commits without trailers (the
212 ChatGPT era) shown as unknown, never as zero. Data stays derived from git; no transcripts are published.
213 **Q107 stage 1 (done 2026-11-04):** trailers that make cost computable (`FW-Tokens-Method: 2`, per-model
214 `FW-Model-Tokens`, written by `fw usage --trailers`; each message id counted once, subagents included); per-commit
215 estimated cost from `server/prices.json` (list prices, `as_of`; unknown stays unknown); one weekly "output tokens"
216 SVG chart on the Usage tab with a hatched strip for commits without data. Old method-1 trailers are left alone
217 (they ran about 1.5-2x high).
218 **Q107 stages 2 and 3 (done 2026-11-04):** the Usage tab also charts estimated cost, cached reads, average cost per
219 commit and AI-marked commits (two lines) per week or month, and the Sessions table has an est. cost column; the
220 site-wide `/usage/` page (`build_site_usage`, called from `build_index`, public published repos only) has the same
221 charts plus a per-repo table (unknown costs are hatched, never zero; the page is rebuilt on every `fw publish`
222 from a per-repo summary cached in the state dir by HEAD sha). Reserved top-level names (`usage`, `ci`, `projects`,
223 ...: `SITE_RESERVED` in `server/gen.py`) are never treated as repos by the index or the roll-up.
224 Remaining: (4) optional: lines added/removed per week and usage per added line.
225- File finder and code search (one shipped script plus a prebuilt index per repo)
226- code-server on TrueNAS, for editing in the browser (private, over Tailscale)
227- Provenance stats and transcript publishing (see below)
228- An app at every commit: `apps.fuzzyworld.net/<app>/@<sha>/` (a service worker at `/<app>/` would cover those paths; must be solved first)
229- OpenGraph images (link previews) from the screenshots
230- Deploy automatically when CI passes on `main`
231- Run GitHub Actions workflows through `act` (10 repos have `.github/workflows`)
232- Publish the shared-origin deploy checks as a reusable lint
233- Discuss: an Advent of Code wrapper app (idea added 2026-10-28; Pavel thinks it has merit). A static page: sign in
234 with GitHub, fetch the user's own puzzle data from adventofcode.com (it differs per user), show the puzzle beside
235 an in-browser editor holding a TypeScript function that takes the logically named test data, run the code on a
236 button press, submit the answer to adventofcode and show whether it is right. To settle: how to sign in to
237 adventofcode from a static page (its session cookie, CORS, and its automation rules), where the code runs
238 (a worker in the browser), and how it relates to the existing AoC harness
239- Found while building "Now" (2026-10-27):
240 - VS Code in the browser: Markdown preview and other webviews need a separate domain per view
241 (a wildcard subdomain, e.g. `*.vscode.fuzzyworld.net`, set up in the DreamHost panel)
242 - VS Code in the browser: build from current VS Code source instead of the `vscode-web` 1.91.1 package (2024)
243 - VS Code in the browser: project-wide TypeScript IntelliSense (turned off; the web TS server rejects our URI scheme)
244 - โœ… Renamed `.gitlab-ci.yml` โ†’ `.fw-ci.yml` in 37 repos (2026-10-27; blister's is only local, never pushed).
245 READMEs that said "deployed to GitLab Pages" are fixed in 11 repos (2026-10-28; edits waiting to be committed in each).
246 Still to do: the same in basin, chance-it, flight, herd, nuts, takeover (their local copies are behind, see below),
247 the `.md` plans/designs that still say it (e.g. tower's design.md, decisions.md), and dropping `pages` jobs that only
248 published to GitLab Pages
249 - Old `.gitlab-ci.yml` files: `node:*-alpine` images have no `git`, so installing the
250 `fuzzyworld-tools` git dependency fails (e.g. bubbles). Switch those jobs to a `node:22` image, or delete
251 `pages` jobs that `fw deploy` replaced
252 - โœ… The 34 merged `fuzzyworld-deploy` branches on DreamHost were deleted (2026-10-28). Local clones and the TrueNAS
253 mirror may still list `origin/fuzzyworld-deploy` until pruned (`git fetch --prune`)
254 - CI `cache:` support (a per-repo volume on TrueNAS), and `services:`
255 - Branch pages: link to the default branch's page when a file is identical, instead of a copy
256- Found in review (2026-10-27): discussed, not done
257 - Site pages: all now in `server/content/` (`gen.py content`), including `404.html`; `/how/` is `_how.html` (draft, not built)
258 - `/how/` goes live: finish the draft and rename `server/content/_how.html` to `how.html` (`/why/` already says it is coming)
259 - CI secrets: none are set, so jobs that deploy elsewhere fail (`dreamhost`'s rsync, engye's store uploads).
260 They go in `<runner>/secrets/<repo>.env` (TrueNAS: `/mnt/main-2025/fw-runner/secrets/`; default branch and tag jobs only)
261 - Look at the results of the 37 rename pushes' CI runs (north-star and nuts failed; not looked at yet)
262 - explodorama: `test/demo.test.ts` "decoy" timed out at vitest's 5-second default on TrueNAS (passed the day
263 before); a longer `testTimeout` for that test, or a faster test
264 - tower: its screenshot caught the loading screen. `"screenshots": [{ "path": "", "size": "phone", "wait": 8000 }]`
265 is added to `fuzzyworld` in its `package.json` (2026-10-28); it takes effect once committed and pushed, and the
266 next CI run takes the new shot
267 - Local copies behind DreamHost, with uncommitted edits in the same files as the incoming commits:
268 basin, chance-it, flight, herd; takeover has staged changes. Pull them by hand
269- Housekeeping (noticed 2026-10-27, never discussed): nothing is cleaned up yet
270 - `~/fw/ci/<repo>/<sha>/` on DreamHost keeps every pipeline (pages show only the newest 100 per repo)
271 - `runner.log` on TrueNAS grows forever
272 - the runner's Docker images on TrueNAS (about 5 GB so far); a periodic `docker image prune`
273
274## Principles
275
276Wording approved 2026-10-26:
277
278> Things made with AI should be shared freely with the world. The code and apps here were made with
279> AI, so they are MIT-licensed: use them, change them, share them.
280>
281> My conversations with the AI are different. My words and my way of working are mine to publish or not.
282> I will publish a conversation only when I choose to.
283
284## CI
285
286โœ… Live since 2026-10-27. docs/ci.md is the full reference; this is the summary. It is the same
287pattern as GitLab runners: **a runner registers, pulls jobs, runs them in containers, and reports back.** The
288difference is that there's no coordinator service: the coordinator is files on DreamHost.
289
2901. **Queue.** `post-receive` runs `fw ci-queue`, which writes `~/fw/queue/pending/<id>.json` (repo, ref, sha,
291 default branch, time) for each pushed branch or tag with a `.fw-ci.yml`, `.gitlab-ci.yml` or `package.json`.
2922. **Claim.** The runner (`runner/fw-runner.py`, TrueNAS cron every 2 minutes) runs `ssh dh claim <runner>`
293 through `server/runner-api`. The server moves the job to `running/` with one `rename`, which is atomic, so
294 two runners can never take the same job. The same step puts jobs running for over 2 hours back in `pending/`.
2953. **Registration = an ssh key.** The runner's key is limited in `authorized_keys` to
296 `command=".../server/runner-api"`: it can claim jobs, fetch repos and upload results, and can't get a shell.
2974. **Run.** The runner clones over ssh and runs each job in a throwaway Docker container from the job's
298 `image`. Inside jobs, `origin` is the read-only https clone URL; the runner's key never enters a container.
2995. **Report.** Logs and `pipeline.json` are uploaded to `~/fw/ci/<repo>/<sha>/` (a running job's log every 15
300 seconds); `gen.py` renders โœ“/โœ— on the log and summary pages, a page per pipeline with its logs, a badge,
301 and the dashboard at `/ci/`.
3026. **Updates.** The runner restarts itself between jobs when `runner/install.sh` has replaced it.
3037. **Two runners.** Tags, the heartbeat lease and the laptop runner are deployed (2026-10-31; see [Runners](#runners)).
304
305**Job definitions: `.fw-ci.yml`** (decided 2026-10-27: our own name; the format stays a subset of GitLab CI's,
306so moving a repo over is a rename; the old `.gitlab-ci.yml` name still works until then). 42 repos have one, and they use only `image`,
307`stages`, `script`, `before_script`, `variables`, `rules` (branch conditions), `needs`, `artifacts`, `cache`.
30836 of them are GitLab Pages jobs; a `pages` job maps to `fw deploy`. An unsupported keyword fails the job with
309a clear message; it is never silently ignored. Repos without the file fall back to `fuzzyworld.checks` from
310`package.json`. engye (20 KB: Android builds, Play Store, Firefox extension) is the hard case and the test of
311how far the subset needs to go.
312
313**Security.** Secrets (`/mnt/main-2025/fw-runner/secrets/<repo>.env` on TrueNAS) are given only to jobs on the
314default branch and tags. None are set yet (see Backlog). Code from contributors will run only after I approve it,
315because it runs on a machine in my home (part of Pull requests).
316
317## Screenshots
318
319โœ… Live since 2026-10-27. A built-in runner job, after a successful pipeline on the default branch of an app
320(`fuzzyworld.app` in `package.json`):
321
322- builds the latest `main` the way `fw deploy` does (`fw deploy --here --dry-run`), serves it at `/<app>/` in a
323 Playwright container with no network, and photographs it. This is the commit CI just tested, which may be newer
324 than what is deployed
325- configured in `package.json`:
326 `"screenshots": [{ "path": "", "size": "phone", "wait": 1500 }, { "path": "?demo", "size": "desktop" }]`,
327 or a command for apps that need clicks first (`"screenshots": "pnpm shots"`, writing PNGs to `$FW_SHOTS_DIR`),
328 or `false`. With no config: one phone shot of the start page
329- apps without runner screenshots show the images in the repo's `screenshots/` folder instead (public repos)
330- shown on the apps index and repo summary pages; OpenGraph images are in the Backlog
331
332## Issues
333
334- Reports arrive at `bugs-k7q4+<repo>@fuzzyworld.net` (only the address was live before 2026-11-03; the ingest and
335 the pages were built that day, see the task list under Report replays).
336- A cron job reads the mailbox (IMAP, Python standard library) and renders each thread as static pages.
337- Reporters' email addresses are never shown.
338- **Configurable per repo**, the same way as visibility: `git config fuzzyworld.issues public|reviewed|private`,
339 set with `fw issues <repo> <mode>`:
340 - `public`: a report whose sender passed email authentication is published as it arrives (an unauthenticated one waits for approval)
341 - `reviewed`: a report is published only after I mark it public
342 - `private`: nothing is published; reports are still stored in `~/fw/issues` (and listed by `fw issues list`)
343 - anything other than exactly `public` or `reviewed` counts as `private`, so nothing is published by accident.
344 A private repo's issues are always private, whatever this says.
345- Import the GitLab issues and notes in `~/code/gitlab-export/*/issues.json`.
346
347### Report replays (Q104; planned 2026-11-03, built 2026-11-03 in this repo)
348
349Idea from the Explodorama bug mails: a report can carry a **machine-readable payload** (for the game: the run
350record, a few KB of JSON: seed, rules version and actions), and the issue page can then show *what the reporter
351saw* as a looping animation. The game side (the replay page, its limits, how old rules are loaded) is in
352`~/code/explodorama/PLAN.md` ยง20. This repo's part:
353
354- **Intake stays email.** DreamHost only serves files, so no upload endpoint (no abuse surface, no server code).
355 The existing mailbox cron reads `Run record: {โ€ฆ}` (or whatever line an app agrees on) out of a report.
356 The mail's quoted-printable text is decoded first (the game's `tools/report-replay.mjs` already does it in
357 Node; port the same rule to Python, and keep a test with a real report).
358- **Store it beside the issue.** For a repo whose issue mode allows publishing, the cron writes the payload as
359 a static file next to the issue page (`โ€ฆ/issues/<n>/record.json`), under the same visibility rule as the
360 report itself: `public` as it arrives, `reviewed` only after I mark it public, `private` never. The payload
361 is validated first (size โ‰ค 64 KB, parses as finite JSON, an integer `seed` and `simVersion`, a list `actions`) and only the keys
362 `simVersion, seed, startLevel, startCharge, options, actions` are kept; a record with an address-like string is refused.
363 Anything else stays only in my mailbox.
364 Reporter emails are still never shown; the device string stays out of the page by default.
365- **Link a viewer, don't build one.** An app announces its viewer in `package.json`, for example
366 `"fuzzyworld": { "replay": "replay.html?report={record}" }`; with it, an issue page that has a payload shows
367 "Watch what happened", a plain link to that page of the app, with `{record}` set
368 to the stored file. Without the key, nothing changes. This keeps the core free of any game knowledge (๐Ÿ”Œ
369 plugin-shaped: an app opts in).
370- **CSP.** The viewer is the app's own script on the app's own origin, so the site-wide `script-src 'self'`
371 policy (Q99) holds; the record is data, never inserted as HTML. The issue page itself needs no script.
372- **Old rules.** A viewer may need the sim of the report's own rules version. An app can publish those as
373 static files at deploy time (Explodorama: `/explodorama/sim/<simVersion>.js`); fuzzyworld-tools only has to
374 let `fw deploy` carry such extra files (a per-app list of globs in `package.json`) and keep old ones. The
375 runner (TrueNAS) can build them on a tag; DreamHost has no Node.
376- **Admin panel (Q102).** On the laptop, the inbox lists reports with a Replay link per report (the same viewer,
377 served locally with the dev page's `&rules=` mode, which can use any commit from git), and the buttons that
378 mark a report public, close it, or turn it into a regression fixture in the app's repo (`test/fixtures/`).
379 The panel is a thin local web page over `fw` commands; it holds no state of its own.
380 **Decided 2026-11-03: local-only, never hosted on DreamHost.** Every panel action is privileged (publish,
381 merge, deploy), and a login form on shared hosting would make the most valuable target on the site; the
382 local panel's security is that it is unreachable, and its authority is the ssh key already held. It also
383 keeps "DreamHost only serves files". Cost: no triage from a phone (reports still arrive as email, and the
384 replay link works anywhere because it is a static page). Possible later add, not planned: one narrow
385 endpoint behind a per-report signed link in the report mail ("mark this report public"), with no login
386 and no general admin surface.
387
388**This repo's task list (2026-11-03; Pavel: "add to the plan, start implementing").** Items 1-4 were built
389on 2026-11-03 (`server/issues.py`, `lib/deploy.mjs`); the first line of each says so. Not yet committed or
390deployed is deliberately not recorded here. In dependency order:
391
3921. **Issues** (built 2026-11-03; Next 5): cron ingest of the bugs mailbox over IMAP (reuse the `comments.py` /
393 `comments-ingest.sh` plumbing), one static page per thread under `โ€ฆ/<repo>/issues/<n>/`,
394 `fw issues <repo> public|reviewed|private` (anything else = private), reporter addresses never shown, and
395 a "mark public" action for `reviewed` mode.
3962. **Record extraction** (built 2026-11-03): decode the quoted-printable mail, find the `Run record: {โ€ฆ}` line, validate
397 (โ‰ค 64 KB, valid JSON, expected keys), write `record.json` beside the issue under the issue's visibility
398 rule; device string stays out of the page. Test with a real report (Explodorama's
399 `test/fixtures/q101-bomb-pass.json` and its `.eml`).
4003. **App opt-in** (built 2026-11-03): a `replay` key in the `"fuzzyworld"` block of `package.json`
401 (`"replay": "replay.html?report={record}"`); the issue-page generator reads it and shows "Watch what
402 happened". No key, no change.
4034. **`fw deploy` carries extra files, kept forever** (built 2026-11-03): a per-app glob list
404 (`"extra": ["sim/*.js"]`) published with the deploy; extra files are protected from `--delete` through an rsync
405 filter (uploaded and updated, never removed), so a later deploy never deletes old rules versions.
406
407No CSP change (the viewer is the app's own script) and no server code on DreamHost. The admin panel (Q102,
408local-only) comes after all four.
409
410**Order:** (1) the game's production replay page and limits (its PLAN ยง20.2 steps 1-2), by hand, with a stored
411record; (2) the cron extracting and publishing `record.json` and the `replay` key; (3) per-rules-version sim
412files in `fw deploy`; (4) the panel. Each step is useful alone.
413
414**Decided 2026-11-03 (Pavel):** the viewer opens as its own page (a plain link, no iframe); a `reviewed`
415report's record is published at the same moment as its text; old `sim/` files are kept forever; compact records
416in the game's `mailto:` link (game PLAN ยง20.2 step 4) come after the cron (step 2); the published page defaults
417to the plain recorded run, with comparison as an option (game PLAN ยง20.2 step 5). Nothing open here.
418
419## Admin panel (`fw admin`, Q102; designed 2026-11-04, decisions by Pavel)
420
421**Version 1 built 2026-11-04** (`lib/admin.py`, `tests/test_admin.py`; server side: `issues list|show --json`, `issues record`, `comments list --json`). Not yet tried against the real server (tested only with a fake runner). Decisions made while building:
422- **Login:** a one-time login URL (random key) that sets a session cookie; everything else needs the cookie, plus the per-launch POST token and Host / Origin / Sec-Fetch-Site checks.
423- **Watch = dev replay:** a link to the deployed viewer when the record is public, "Download record" always, and "Prepare dev replay" (runs the app's `report-replay` npm script with the record and `--rules <commit>`; needs `npm run dev` running in the app).
424- **Recently approved comments** (the 20 newest) with a Remove button are in v1, not later.
425- **`FW_CODE_DIR`** (also `--code-dir`, default `~/code`) is configurable: where the apps' working copies live.
426- **Deploy tab (Q111 stage 1, built 2026-11-04):** read-only; `fw deploys --json` (server/deploys.py) joins the deploy ledger (~/fw/state/deployed.json, written by `bin/deploy-tools`, keyed `<repo>/<target>`) and each app's `.fuzzyworld.json` with the repo's HEAD. Targets are declared in package.json `fuzzyworld.deploy`. Next: stage 2, a "Deploy now" button that deploys `origin/main` from a clean export (confirm page with details, background run, progress page); stage 3, any repo declares targets, apps included, tested on nordic-skating.
427- Still not in v1: "make this a regression fixture", pull requests, CI, blog state.
428
429A **local-only** web page over `fw` commands (see "Report replays": never hosted on DreamHost). `fw admin` starts a
430small Python (stdlib) server on `127.0.0.1` and opens the browser; every button runs the same `fw ...` command over
431ssh that Pavel would type, so the panel holds no state and has no authority beyond his ssh key.
432
433- **Decided:** version 1 is **reports and blog comments** together (both are "read it, then approve or reject");
434 **no JavaScript at first** (server-rendered pages and `POST` forms, a full reload after each action); the command
435 is `fw admin`.
436- **Safety:** bind to 127.0.0.1 only; check the `Host` header (DNS rebinding); every action is a `POST` carrying a
437 random token made at launch; a confirm step for destructive actions; never show passwords.
438- **Version 1:** (1) one list of reports across repos (repo, number, title, state, visibility, record, age;
439 filters "waiting for approval" and "has a replay"); (2) a report page with the full stored text, **including
440 unpublished reports**, the details table and thread, and Approve / Hide / Close / Reopen and the repo's mode
441 switch; (3) blog comments: pending ones with approve, reject, trust sender; (4) a Watch link per report with a
442 record (the dev replay page until the game's production viewer exists).
443- **Needs from the existing code:** `fw issues list --json`; `fw issues record <repo> <n>` (read an unpublished
444 record); a machine-readable `fw comments list`.
445- **Later, each useful alone:** pull requests (after contrib intake), CI and runners, blog build/deploy and watcher
446 state (Q100), "make this report a regression fixture" (writes `test/fixtures/` in the app's repo).
447- **Backlog (Q108, Pavel 2026-11-04): the admin page on the live site, working with the local `fw admin`.** A page on
448 code.fuzzyworld.net checks whether the visitor's machine runs `fw admin` (a probe of `http://127.0.0.1:<port>`) and,
449 if so, shows the admin UI there while all data and actions go browser to localhost; the live site never sees
450 reports, addresses or tokens (the hosted page is a static shell). Feasible, with conditions: (1) it needs JavaScript
451 on that one page (allowed by the "local JS is OK" policy, nothing else on the site changes); (2) `fw admin` gains a
452 JSON API with CORS for exactly the live origin, a bearer token (the current cookie login does not work
453 cross-site), strict Origin / Host checks and the Private Network Access preflight header; (3) browsers differ:
454 Chrome and Firefox allow an https page to call 127.0.0.1, newer Chrome asks the user once for "local network
455 access", Safari may block it as mixed content, so test all three and keep the local page (`fw admin` as today) as
456 the fallback; (4) the token reaches the live page through a link `fw admin` prints (token in the URL fragment,
457 which is never sent to a server), kept in `sessionStorage`. Not started.
458
459## Pull requests
460
461- Contributions arrive at `contrib-68uq+<repo>@fuzzyworld.net` as a branch URL or patches (already live).
462 **Forward-only today** (2026-11-03): like `bugs-k7q4`, it must be recreated as a fully hosted mailbox before a
463 cron job can read it over IMAP, and so that the receiving server's `Authentication-Results` header is trusted
464 (a forwarded copy carries the destination's header instead). Give it its own account, never the comments one.
465- A cron job fetches each one into `refs/contrib/<n>` in the bare repo, never onto `main`.
466- It renders a review page: the diff, CI results (after I approve the run), and the discussion from email replies.
467- **Transcripts.** A submitter can attach the AI conversation behind the change, so the reviewer sees exactly
468 where they're coming from. It is the submitter's choice, published under the submitter's terms, and only on that pull request.
469- I merge locally, as now.
470- Import the GitLab merge requests in `~/code/gitlab-export/*/merge_requests.json`.
471
472## Provenance
473
474- **AI share.** Count commits with an AI `Co-Authored-By:` trailer, per repo and site-wide.
475- **Transcripts** (Claude Code session โ†’ rendered page):
476 - opt-in, one conversation at a time
477 - with a redaction pass (paths, email addresses, tokens) and my review before anything is published
478 - each published transcript has a consent record: who gave permission, when, for which conversation
479 - `fw transcript withdraw <id>` removes it from the sites
480 - transcripts are **not** MIT; they say "shown with permission, all rights reserved"
481- **First candidate:** the Firefox bug-fix conversation. Permission was given 2026-10-26 and can be withdrawn.
482 It is in a different session, so it still goes through redaction and review first.
483- **My own conversations** (like the ones that built this tool) are not published.