fuzzyworld-tools / commit
Plan: runners without TrueNAS, blog, email comments, plugin tag; quick fixes done
PLAN.md | 93 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 86 insertions(+), 7 deletions(-) diff --git a/PLAN.md b/PLAN.md index 6d972a8..d16e2ec 100644 --- a/PLAN.md +++ b/PLAN.md @@ -4,7 +4,11 @@ Where fuzzyworld-tools goes from here: from "a static mirror of my repos" to a s code host built on one rule: **DreamHost only serves files; other machines do the work.** This is a planning document. Nothing in it is promised; the order is the current priority. -Last updated 2026-10-27. +Last updated 2026-10-28. + +**Who it is for.** First, me: I build the site I want and need (no GitLab, no WordPress, as few machines as possible). +Second, anyone who wants to get going with the fewest resources. Anything that only makes sense for my setup is +tagged 🔌 **plugin**: it should be optional and separable, so that the core stays small enough for most people. ## The machines @@ -40,8 +44,12 @@ All done: ✅ live since 2026-10-27 (v1.1.0), unless noted. The order goes from cheap to expensive, and each step reuses what the one before it built. +0. **Runners without TrueNAS** (decided 2026-10-28; see [Runners](#runners)): runner tags, the laptop as a runner when + idle and plugged in, secrets in a folder on each runner, and "no runners" as a normal state. Goes first. 1. **Feeds**: Atom per repo (commits, releases), for app deploys, and site-wide. Smallest step: the data is already - gathered for the log pages and the apps index. + gathered for the log pages and the apps index. The blog reuses it. +1b. **Blog** at `blog.fuzzyworld.net` (see [Blog](#blog)): a generator that turns a folder or repo of Markdown into a + blog from one JSON file, plus a one-time WordPress import. Replaces WordPress. 2. **Releases**: a page per tag (the tag's message as notes), `git archive` `.zip`/`.tar.gz`, and files the runner builds (APKs, extensions). Import the release assets already in `~/code/gitlab-export/*/release-assets/`. Comes early because some of those files exist nowhere else. Adds a releases feed to step 1. @@ -51,9 +59,76 @@ The order goes from cheap to expensive, and each step reuses what the one before 4. **Wiki**: render a repo's `docs/` (or `wiki/`) folder with a sidebar. Import the GitLab wikis in `~/code/gitlab-export/*/wikis.json`. Static rendering only; editing comes with pull requests. 5. **Issues**: see [Issues](#issues). The first feature that reads email; builds the mailbox plumbing. +5b. **Blog comments** by email (see [Comments](#comments)): right after Issues, because it reuses the mailbox plumbing. 6. **Pull requests**: see [Pull requests](#pull-requests). Last, because it reuses the issues mailbox plumbing and needs the CI runner (approved CI runs on contributed code). +## Runners + +Goal (2026-10-28): TrueNAS becomes optional (🔌 plugin), and the laptop can do the work. Running jobs needs no +always-on machine; a small setup is a DreamHost account and a laptop. + +- **No runners is a normal state.** Jobs stay in `pending/` until a runner claims them (the queue is files, so this + already works). The dashboard says "waiting for a runner" instead of looking broken. +- **Laptop runner** (launchd, every 2 minutes), only when it will not hurt my work: on AC power, CPU mostly idle, + containers at low priority, no new claims once I get busy. Everything scheduled runs then. It assumes the laptop + is on a few hours a day; that is enough for me. +- **Lease.** A claimed job needs a heartbeat (uploaded with the log). A job whose runner went quiet for ~2 minutes + (lid closed) goes back to `pending/` instead of waiting 2 hours. +- **Tags** (`docker`, `chrome`, `android`): a runner takes only jobs it can run. +- **Secrets.** Each runner has a folder (`…/secrets/<repo>.env`; on TrueNAS `/mnt/main-2025/fw-runner/secrets/`). + A runner without a repo's secrets leaves that repo's deploy jobs for one that has them. Only default-branch and + tag jobs get secrets. Put them on the laptop and TrueNAS is not needed. Laptop theft means rotating them (FileVault on). +- **Stretch goal: a secure secrets API** (🔌): `runner-api` hands each job its secrets over ssh, per runner key, so + no runner stores them. Wait for user requests or downtime. + +## Blog + +`blog.fuzzyworld.net` is WordPress today (theme Scrawl, 144 posts, 4 pages, 45 comments, 15 media files, 3 categories, +16 tags, one author). Goal: the same site, generated by this repo, with no WordPress. + +**What is there** (read from its public API, 2026-10-28): +- Posts at `/YYYY/MM/DD/<slug>/`, an RSS feed at `/feed/`, and monthly archives. Categories hardly matter ("Uncategorized" + has 138 posts). Tags are the real structure (a story series is a tag, e.g. Dice, Walkabout, Writing Excuses). +- Pages: **Highlights** (the front page: a chronological list of the work I consider good, with an AI-use note), + **Blog** (all posts), **Collections** (series in reading order, a hand-written list), **Upcoming**. +- 2 posts are password-protected (Rasputin, Belfast). Static answer: HTTP basic auth in `.htaccess`, or + client-side encryption; to be chosen. +- A reading time on each post, media (images, plus epub/kpf/pdf downloads of Wick Canyon), and comments. +- Comments: 45 (36 top-level, 8 replies). They get imported, as static text with the author's name (no emails). + +**Source: `~/Blog`** (Dropbox for now; a private git repo later): 286 Markdown files, `YYYY-MM-DD-name.md`, no front +matter. The title is the first `# heading`. The post is only the **`## Writing`** section; `## Spelling` (cspell words) +and `## Removed writing` (cut text) are private notes and are never published. 164 files have a `## Writing` section; +the others are notes. WordPress turns single line breaks into `<br>`, so the renderer must too. + +**Findings that shape the design:** +- File names do **not** match published URLs. Only 13 of 144 posts match `date-slug`: the publish date differs from the + draft date, and some names differ (`belfast-porn.md` is published as `/belfast/`). The import must record the mapping + (published date, slug, tags, highlighted, password) somewhere. Proposal: add small front matter to each file once. +- So the generator must keep the old URLs working: same path pattern and the same slugs. + +**Design:** +- One JSON file describes a blog (title, base URL, source folder, which section is the post body, URL pattern, tag + list, highlight list, collections, feed). The tool is `gen.py` run on a repo of Markdown, from the push hook. +- Reuse what exists: Python-Markdown, Pygments, the site CSS, the feed code from step 1. +- **Import (🔌 plugin):** one-time script that reads the WordPress API: posts, comments, media, and a redirect map. +- The editor and the Dropbox → private-git-repo move come after the generator, not with it. +- Needs the `blog.fuzzyworld.net` subdomain pointed at a folder (DreamHost panel). + +## Comments + +Decided 2026-10-28: **by email first, no accounts.** Google sign-in is a 🔌 plugin for later. +- A "comment" link on each post is a `mailto:` to the comments mailbox, with the post in the subject. Same plumbing as + [Issues](#issues): a cron job reads the mailbox and renders threads as static pages. +- Identity is the sender's verified address (DKIM/DMARC must pass), which proves they control it, so no login. Only the + display name is shown, never the address. +- Moderation, light: a person's first comment waits for my approval; after that they are trusted and published on + arrival. One command removes a comment. Length and rate limits apply. A deletion request is honoured. +- **🔌 Google sign-in plugin:** needs an endpoint that checks the Google token and stores the comment (a small CGI on + DreamHost or a Cloudflare Worker), and a redirect flow so the site's CSP (`script-src 'self'`) stays. It breaks + "DreamHost only serves files", which is why it is a plugin. + ## CI dashboard ✅ Built 2026-10-27 as proposed (README: "Watching CI"): @@ -88,12 +163,15 @@ The order goes from cheap to expensive, and each step reuses what the one before - VS Code in the browser: build from current VS Code source instead of the `vscode-web` 1.91.1 package (2024) - VS Code in the browser: project-wide TypeScript IntelliSense (turned off; the web TS server rejects our URI scheme) - ✅ Renamed `.gitlab-ci.yml` → `.fw-ci.yml` in 37 repos (2026-10-27; blister's is only local, never pushed). - Still to do: READMEs and plans that say "deployed to GitLab Pages by `.gitlab-ci.yml`" (stale since the move), - and dropping `pages` jobs that only published to GitLab Pages + READMEs that said "deployed to GitLab Pages" are fixed in 11 repos (2026-10-28; edits waiting to be committed in each). + Still to do: the same in basin, chance-it, flight, herd, nuts, takeover (their local copies are behind, see below), + the `.md` plans/designs that still say it (e.g. tower's design.md, decisions.md), and dropping `pages` jobs that only + published to GitLab Pages - Old `.gitlab-ci.yml` files: `node:*-alpine` images have no `git`, so installing the `fuzzyworld-tools` git dependency fails (e.g. bubbles). Switch those jobs to a `node:22` image, or delete `pages` jobs that `fw deploy` replaced - - 34 repos still have a `fuzzyworld-deploy` branch from the migration, fully merged; they can be deleted + - ✅ The 34 merged `fuzzyworld-deploy` branches on DreamHost were deleted (2026-10-28). Local clones and the TrueNAS + mirror may still list `origin/fuzzyworld-deploy` until pruned (`git fetch --prune`) - CI `cache:` support (a per-repo volume on TrueNAS), and `services:` - Branch pages: link to the default branch's page when a file is identical, instead of a copy - Found in review (2026-10-27): discussed, not done @@ -106,8 +184,9 @@ The order goes from cheap to expensive, and each step reuses what the one before - Look at the results of the 37 rename pushes' CI runs (north-star and nuts failed; not looked at yet) - explodorama: `test/demo.test.ts` "decoy" timed out at vitest's 5-second default on TrueNAS (passed the day before); a longer `testTimeout` for that test, or a faster test - - tower: its screenshot caught the loading screen; add - `"screenshots": [{ "path": "", "size": "phone", "wait": 8000 }]` to `fuzzyworld` in its `package.json` + - tower: its screenshot caught the loading screen. `"screenshots": [{ "path": "", "size": "phone", "wait": 8000 }]` + is added to `fuzzyworld` in its `package.json` (2026-10-28); it takes effect once committed and pushed, and the + next CI run takes the new shot - Local copies behind DreamHost, with uncommitted edits in the same files as the incoming commits: basin, chance-it, flight, herd; takeover has staged changes. Pull them by hand - Housekeeping (noticed 2026-10-27, never discussed): nothing is cleaned up yet