fuzzyworld-tools / commit
PLAN: statuses, CI and Screenshots as built, review findings in the backlog
PLAN.md | 83 +++++++++++++++++++++++++++++++++++++++++++---------------------- 1 file changed, 55 insertions(+), 28 deletions(-) diff --git a/PLAN.md b/PLAN.md index 7e87d05..9d6ad75 100644 --- a/PLAN.md +++ b/PLAN.md @@ -20,21 +20,21 @@ never anything from a repo). ## Now (prioritized) -Status: ✅ live · 🧪 built and tested, not installed yet · ⏳ not started. +All done: ✅ live since 2026-10-27 (v1.1.0), unless noted. 1. ✅ Apps index: newest deploy first, with what changed. Live since 2026-10-26. -2. 🧪 **LICENSE file** (MIT) in this repo. Each repo's license on its summary page. -3. 🧪 **Principles** in four places: `/why/`, `/how/`, a new `/principles/` page, and a "Philosophy" section in the - README. `/how/` is still a draft that `install.sh` doesn't install; the principle is in it for when it goes live. -4. 🧪 **File navigation** +2. ✅ **LICENSE file** (MIT) in this repo. Each repo's license on its summary page. +3. ✅ **Principles** in four places: `/why/`, `/how/`, a new `/principles/` page, and a "Philosophy" section in the + README. `/how/` is still a draft that `install.sh` doesn't install (see Backlog); the principle is in it for + when it goes live. +4. ✅ **File navigation** 1. "Open in VS Code" link on every summary page (`vscode://vscode.git/clone?url=…`). 2. Syntax highlighting at generation time (Pygments in `~/fw/pylib`; no JavaScript). 3. Per-file history, blame, and browsing every branch not yet merged into the default branch. 4. VS Code in the browser at `apps.fuzzyworld.net/vscode/?repo=<name>`: the `vscode-web` build plus a read-only file-system extension reading `files.json` and `objects/<sha>` from code.fuzzyworld.net. -5. 🧪 **CI runner** on TrueNAS. See [CI](#ci). Tested on TrueNAS against the git mirrors (explodorama passed, - bubbles failed for a real reason: see below). -6. 🧪 **Screenshots** as a built-in runner job. See [Screenshots](#screenshots). Tested: explodorama. +5. ✅ **CI runner** on TrueNAS. See [CI](#ci). Verified live with nordic-skating and tower. +6. ✅ **Screenshots** as a built-in runner job. See [Screenshots](#screenshots). Live for nordic-skating and tower. ## Next (in this order) @@ -90,6 +90,23 @@ The order goes from cheap to expensive, and each step reuses what the one before - 34 repos still have a `fuzzyworld-deploy` branch from the migration, fully merged; they can be deleted - CI `cache:` support (a per-repo volume on TrueNAS), and `services:` - Branch pages: link to the default branch's page when a file is identical, instead of a copy +- Found in review (2026-10-27): discussed, not done + - `/how/` goes live: finish the draft, add it to `server/install.sh` (`/why/` already says it is coming) + - CI runner tags (`docker`, `chrome`, `android`) and the laptop as a second runner (`fw-runner.py --once`): + designed, not built; there is one runner, and it takes every job + - CI secrets: none are set, so jobs that deploy elsewhere fail (`dreamhost`'s rsync, engye's store uploads). + They go in `/mnt/main-2025/fw-runner/secrets/<repo>.env` on TrueNAS (default branch and tag jobs only) + - Look at the results of the 37 rename pushes' CI runs (north-star and nuts failed; not looked at yet) + - explodorama: `test/demo.test.ts` "decoy" timed out at vitest's 5-second default on TrueNAS (passed the day + before); a longer `testTimeout` for that test, or a faster test + - tower: its screenshot caught the loading screen; add + `"screenshots": [{ "path": "", "size": "phone", "wait": 8000 }]` to `fuzzyworld` in its `package.json` + - Local copies behind DreamHost, with uncommitted edits in the same files as the incoming commits: + basin, chance-it, flight, herd; takeover has staged changes. Pull them by hand +- Housekeeping (noticed 2026-10-27, never discussed): nothing is cleaned up yet + - `~/fw/ci/<repo>/<sha>/` on DreamHost keeps every pipeline (pages show only the newest 100 per repo) + - `runner.log` on TrueNAS grows forever + - the runner's Docker images on TrueNAS (about 5 GB so far); a periodic `docker image prune` ## Principles @@ -108,19 +125,24 @@ not something a license can enforce. The wording is written to be honest about t ## CI -Built as described here; the README's CI section is the reference. Yes, this is the same pattern as GitLab runners: **a runner registers, pulls jobs, runs them in containers, -and reports back.** The difference is that there's no coordinator service: the coordinator is files on DreamHost. - -1. **Queue.** `post-receive` writes `~/fw/queue/pending/<repo>@<sha>.json` (repo, ref, sha, time). -2. **Claim.** Each runner polls `ssh dh fw runner claim --tags docker,chrome`. The server moves the job to - `running/` with one `rename`, which is atomic, so two runners can never take the same job. -3. **Registration = an ssh key.** A runner's key is limited in `authorized_keys` with `command="fw runner-api"`: - it can claim jobs and upload results, and can't get a shell. -4. **Run.** The runner clones over ssh and runs each job in a throwaway Docker container from the job's `image`. -5. **Report.** Logs and `status.json` are rsynced to `~/fw/ci/<repo>/<sha>/`; `gen.py` renders ✅/❌ on the - log and commit pages, a page per run with its log, and a badge. -6. **Stale jobs.** A cron job puts jobs that have been running too long back into `pending/`. -7. **More runners.** The laptop can run `fw runner --once`. Tags say what a runner can do (`docker`, `chrome`, `android`). +✅ Live since 2026-10-27. The README's CI section is the full reference; this is the summary. It is the same +pattern as GitLab runners: **a runner registers, pulls jobs, runs them in containers, and reports back.** The +difference is that there's no coordinator service: the coordinator is files on DreamHost. + +1. **Queue.** `post-receive` runs `fw ci-queue`, which writes `~/fw/queue/pending/<id>.json` (repo, ref, sha, + default branch, time) for each pushed branch or tag with a `.fw-ci.yml`, `.gitlab-ci.yml` or `package.json`. +2. **Claim.** The runner (`runner/fw-runner.py`, TrueNAS cron every 2 minutes) runs `ssh dh claim <runner>` + through `server/runner-api`. The server moves the job to `running/` with one `rename`, which is atomic, so + two runners can never take the same job. The same step puts jobs running for over 2 hours back in `pending/`. +3. **Registration = an ssh key.** The runner's key is limited in `authorized_keys` to + `command=".../server/runner-api"`: it can claim jobs, fetch repos and upload results, and can't get a shell. +4. **Run.** The runner clones over ssh and runs each job in a throwaway Docker container from the job's + `image`. Inside jobs, `origin` is the read-only https clone URL; the runner's key never enters a container. +5. **Report.** Logs and `pipeline.json` are uploaded to `~/fw/ci/<repo>/<sha>/` (a running job's log every 15 + seconds); `gen.py` renders ✓/✗ on the log and summary pages, a page per pipeline with its logs, a badge, + and the dashboard at `/ci/`. +6. **Updates.** The runner restarts itself between jobs when `runner/install.sh` has replaced it. +7. **One runner so far.** Runner tags and a second runner on the laptop are in the Backlog. **Job definitions: `.fw-ci.yml`** (decided 2026-10-27: our own name; the format stays a subset of GitLab CI's, so moving a repo over is a rename; the old `.gitlab-ci.yml` name still works until then). 42 repos have one, and they use only `image`, @@ -130,19 +152,24 @@ a clear message; it is never silently ignored. Repos without the file fall back `package.json`. engye (20 KB: Android builds, Play Store, Firefox extension) is the hard case and the test of how far the subset needs to go. -**Security.** Secrets (`~/.config/<project>.env` on the runner) are given only to jobs on branches I push. -Code from contributors runs only after I approve it, because it runs on a machine in my home. +**Security.** Secrets (`/mnt/main-2025/fw-runner/secrets/<repo>.env` on TrueNAS) are given only to jobs on the +default branch and tags. None are set yet (see Backlog). Code from contributors will run only after I approve it, +because it runs on a machine in my home (part of Pull requests). ## Screenshots -A built-in runner job (tag `chrome`), run after a successful build: +✅ Live since 2026-10-27. A built-in runner job, after a successful pipeline on the default branch of an app +(`fuzzyworld.app` in `package.json`): -- photographs the **exact build** being deployed, served locally at `/<app>/`, so it matches the deploy +- builds the latest `main` the way `fw deploy` does (`fw deploy --here --dry-run`), serves it at `/<app>/` in a + Playwright container with no network, and photographs it. This is the commit CI just tested, which may be newer + than what is deployed - configured in `package.json`: `"screenshots": [{ "path": "", "size": "phone", "wait": 1500 }, { "path": "?demo", "size": "desktop" }]`, - or a script name for apps that need clicks first (`"screenshots": "pnpm shots"`) -- with no config: falls back to a `screenshots/` folder in the repo, or the README's first image -- used on the apps index, repo summary pages, and (later) OpenGraph images + or a command for apps that need clicks first (`"screenshots": "pnpm shots"`, writing PNGs to `$FW_SHOTS_DIR`), + or `false`. With no config: one phone shot of the start page +- apps without runner screenshots show the images in the repo's `screenshots/` folder instead (public repos) +- shown on the apps index and repo summary pages; OpenGraph images are in the Backlog ## Issues