Operations: installing, layout and jobs
Requirements
| Machine | Needs |
|---|---|
| DreamHost | git, Python 3.12, flock, setsid |
| Laptop | Node ≥ 18, git, rsync, ssh with the dh alias |
| TrueNAS (runner) | Docker, Python 3 with PyYAML (both part of TrueNAS SCALE), and the truenas ssh alias on the laptop |
| Laptop as a runner (optional) | Docker (Docker Desktop, OrbStack, ...) and Python 3; the installer makes a venv with PyYAML |
Install and update
server/install.sh # copy server/ to dh, install site files and hooks
server/install.sh --publish # ...and republish every repo
runner/install.sh # CI runner on TrueNAS: copy it, make its ssh key, add the cron job
runner/install.sh --authorize # ...and allow that key on DreamHost (runner-api only)
runner/install-laptop.sh # the laptop as a polite second runner (launchd, Docker, its own venv)
vscode-web/install.sh # VS Code in the browser at apps.fuzzyworld.net/vscode/
install.sh copies server/ from the working copy, not from a commit, so uncommitted changes there go live too.
npm run deploy (the usual way)
| Script | Runs |
|---|---|
npm run deploy |
bin/deploy-tools: install the server and republish every repo, but only if server/ changed since the last deploy |
npm run deploy-status |
bin/deploy-tools status: what is deployed where, and what is pending |
npm run list-repos |
bin/fw list |
bin/deploy-tools [server|runner|laptop-runner|vscode|all] [--force] wraps the install scripts with a ledger
kept on the server, ~/fw/state/deployed.json.
- Targets are declared in this repo's
package.json:"fuzzyworld": {"deploy": {"<target>": {"paths": [folders it ships], "command": [script, args], "description": "...", "perMachine": true}}}.
| Target | Ships | Runs |
|---|---|---|
server |
server/ |
server/install.sh --publish (includes the republish, since pages are generated by the installed gen.py) |
runner |
runner/ |
runner/install.sh |
laptop-runner |
runner/ |
runner/install-laptop.sh (per machine) |
vscode |
vscode-web/ |
vscode-web/install.sh |
- The ledger records, for each
<repo>/<target>, the commit and a content hash of the folders that target ships. A target is redeployed only when that hash differs. So uncommitted edits count, unrelated commits (README, tests) do not, and committing the files you already deployed does not trigger a second deploy. Gitignored files in those folders are not hashed. - Ledger keys used to be bare (
server,laptop-runner@<hostname>).bin/deploy-toolsrenames such a key to<repo>/<target>in memory when it loads the ledger (an existing new key wins), and the next successful deploy saves the result. Times in the ledger are the laptop's local time. - The ledger is written only after the script succeeds, and any laptop sees the same state. The exception is
laptop-runner, which is installed per machine and so is recorded per hostname (key<repo>/laptop-runner@<hostname>). fw deploysand the admin panel read the same ledger.- For flags
deploy-toolsdoes not pass (--authorize,--remove,--out) run the install script directly. To regenerate pages without installing, runbin/fw publish-all.
Layout on DreamHost
Sites and code
| Path | Contents |
|---|---|
~/git/<name>.git |
canonical bare repos (push here; not web-visible) |
~/code.fuzzyworld.net/<name>/ |
generated pages for public repos |
~/code.fuzzyworld.net/<name>.git |
public clone copy (dumb HTTP), refreshed on push |
~/apps.fuzzyworld.net/<app>/ |
deployed apps (fw deploy); index.html at the root is generated |
~/apps.fuzzyworld.net/_shots/<app>/ |
screenshots of each app, uploaded by runners (shots.json lists them) |
~/apps.fuzzyworld.net/vscode/ |
VS Code in the browser (not listed on the apps index) |
~/writing.fuzzyworld.net/ |
the blog's built site, uploaded by fw blog deploy from the laptop (never edited there) |
Tools and state (outside every web root)
| Path | Contents |
|---|---|
~/fw/tools/server/ |
this repo's server/ directory, installed by server/install.sh |
~/bin/fw |
symlink to ~/fw/tools/server/fw |
~/fw/state/ |
per-repo page manifests and locks; deployed.json is the deploy ledger |
~/fw/logs/<name>.log |
output of each push's publish run |
~/fw/pylib/ |
Python-Markdown 3.7 (README rendering) and Pygments 2.21.0 (syntax highlighting), pip install --target; install.sh adds both (the landing page needs Markdown) |
~/fw/queue/pending/, running/ |
CI jobs, one JSON file each (ci.md). running/<job>.beat is the runner's heartbeat; declined/ holds the runners that gave a job back |
~/fw/runners/<runner> |
one file per runner: its tags, and (by its time) when it last checked in |
~/fw/ci/<name>/<sha>/ |
CI results uploaded by runners: pipeline.json and one log per job |
~/fw/issues/ |
bug-report issues: stored reports, secret, imap.json, locks (issues.md) |
~/fw/comments/ |
blog comments state and imap.json (blog.md) |
Scheduled jobs
Jobs live in server/jobs/. install.sh writes server/jobs/crontab into the DreamHost crontab, between
# fw:begin and # fw:end; it replaces only that block. Cron wakes each job hourly. DreamHost's clock is Pacific
time, so each job checks its own UTC schedule, and daylight saving can't shift it.
| Job | Schedule | What it does | Log |
|---|---|---|---|
north-star-cleanup.sh |
Tuesdays 06:00 UTC (was the GitLab schedule) | updates ~/fw/jobs/north-star to main, then runs scripts/clean-old-firebase.sh --days=30 as a preview and then --apply |
~/fw/logs/cron-north-star-cleanup.log |
comments-ingest.sh |
hourly | comments.py ingest: turns new mail in the comments mailbox into blog comments (quiet unless something happened) |
~/fw/logs/cron-comments-ingest.log |
issues-ingest.sh |
hourly | issues.py ingest: turns new mail in the bugs mailbox into issues and republishes the touched repos (quiet unless something happened) |
~/fw/logs/cron-issues-ingest.log |
Run one by hand: ssh dh '~/fw/tools/server/jobs/north-star-cleanup.sh --now --preview-only' (without
--preview-only for the real thing).
Secrets are in ~/.config/<project>.env on DreamHost (mode 600, never in a repo or the web root).
north-star.env holds FIREBASE_TOKEN.
Node on DreamHost
The system Node is 18. Jobs use Node 22 from ~/fw/node (the official linux-x64 build; checksum-verified), always
with NODE_OPTIONS=--jitless: DreamHost's grsec kernel refuses executable memory, so V8's JIT crashes without it.
npm installs use --ignore-scripts, because native addons can't be built there. jq 1.7.1 (the official static
build) is in ~/bin.
VS Code in the browser
apps.fuzzyworld.net/vscode/?repo=<name> opens a public repo in VS Code. It is linked from each summary page.
| Parameter | Effect |
|---|---|
repo=<name> |
the repo |
branch=<branch> |
an unmerged branch |
file=<path> |
open a file |
file=<path>:<line> |
jump to a line |
What it is: the open-source VS Code web build (the vscode-web npm package, 1.91.1, MIT) plus vscode-web/fw-fs/,
an extension that reads files.json and objects/<sha> from code.fuzzyworld.net. Everything is static files.
- Works: quick open (Ctrl/Cmd+P), and search across files (text files up to 256 KB).
- Read-only: nothing can be changed; clone to edit.
- Not yet: Markdown preview and other webviews (they need a separate domain per view), and project-wide TypeScript IntelliSense (turned off; syntax colors work).
The plain "Open in VS Code" link (vscode://vscode.git/clone?url=…) clones the repo into the visitor's own VS Code.