fuzzyworld-tools / commit
Add scheduled jobs: north-star weekly cleanup on DreamHost
README.md | 14 ++++++++++ server/install.sh | 3 +++ server/jobs/crontab | 4 +++ server/jobs/north-star-cleanup.sh | 56 +++++++++++++++++++++++++++++++++++++++ server/site/why/index.html | 3 +-- 5 files changed, 78 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 68ba297..b8f3d50 100644 --- a/README.md +++ b/README.md @@ -114,6 +114,20 @@ The site never executes anything from a repo: - inside public clone copies, `config`, `description`, `hooks/` and `logs/` return 404 - apps.fuzzyworld.net gets the same no-execution rules. HTML, service workers and manifests are sent with `no-cache` (a deploy is seen immediately); `/<app>/assets/` (hashed by vite) is cached for a year. +## Scheduled jobs + +Jobs live in `server/jobs/`. `install.sh` writes `server/jobs/crontab` into the DreamHost crontab, between `# fw:begin` and `# fw:end`; it replaces only that block. Cron wakes each job hourly. DreamHost's clock is Pacific time, so **each job checks its own UTC schedule**, and daylight saving can't shift it. + +| Job | Schedule | What it does | Log | +|---|---|---|---| +| `north-star-cleanup.sh` | Tuesdays 06:00 UTC (was the GitLab schedule) | updates `~/fw/jobs/north-star` to `main`, then runs `scripts/clean-old-firebase.sh --days=30` as a preview and then `--apply` | `~/fw/logs/cron-north-star-cleanup.log` | + +Run one by hand: `ssh dh '~/fw/tools/server/jobs/north-star-cleanup.sh --now --preview-only'` (or without `--preview-only` for the real thing). + +Secrets are in `~/.config/<project>.env` on DreamHost (mode 600, never in a repo or the web root): `north-star.env` holds `FIREBASE_TOKEN`. + +**Node on DreamHost:** the system Node is 18. Jobs use Node 22 from `~/fw/node` (the official linux-x64 build; checksum-verified), always with `NODE_OPTIONS=--jitless`. DreamHost's grsec kernel refuses executable memory, so V8's JIT crashes without it. npm installs use `--ignore-scripts`, because native addons can't be built there. `jq` 1.7.1 (the official static build) is in `~/bin`. + ## Install / update ```bash diff --git a/server/install.sh b/server/install.sh index 34338fc..abcffaa 100755 --- a/server/install.sh +++ b/server/install.sh @@ -29,6 +29,9 @@ cp ~/fw/tools/server/site-apps/htaccess "$apps/.htaccess" cp ~/fw/tools/server/site/style.css "$apps/" ~/bin/fw install-hooks ~/bin/fw apps-index +# Scheduled jobs: replace only our own block in the crontab, keeping anything else. +chmod +x ~/fw/tools/server/jobs/*.sh +{ { crontab -l 2>/dev/null || true; } | sed '/^# fw:begin/,/^# fw:end/d'; cat ~/fw/tools/server/jobs/crontab; } | crontab - EOF if [ "${1:-}" = --publish ]; then ssh "$HOST" '~/bin/fw publish-all'; fi echo "installed on $HOST" diff --git a/server/jobs/crontab b/server/jobs/crontab new file mode 100644 index 0000000..cbd0837 --- /dev/null +++ b/server/jobs/crontab @@ -0,0 +1,4 @@ +# fw:begin (managed by fuzzyworld-tools server/install.sh; edit server/jobs/crontab, then reinstall) +# Each job script checks its own UTC schedule; cron only wakes it hourly. +0 * * * * $HOME/fw/tools/server/jobs/north-star-cleanup.sh >> $HOME/fw/logs/cron-north-star-cleanup.log 2>&1 +# fw:end diff --git a/server/jobs/north-star-cleanup.sh b/server/jobs/north-star-cleanup.sh new file mode 100755 index 0000000..06a6d8c --- /dev/null +++ b/server/jobs/north-star-cleanup.sh @@ -0,0 +1,56 @@ +#!/bin/bash +# Weekly cleanup of stale north-star rooms in Firebase RTDB. Replaces the GitLab pipeline +# schedule "drop month-old or older rooms" (cron 0 6 * * 2, UTC) and runs the same two steps. +# +# Cron starts this every hour (see server/jobs/crontab). DreamHost's clock is Pacific time, +# so the script itself checks for Tuesday 06:00 UTC; daylight saving can't move the job. +# +# north-star-cleanup.sh scheduled use: act only on Tuesday 06 UTC +# north-star-cleanup.sh --now run now +# north-star-cleanup.sh --now --preview-only list what would be removed, remove nothing +# +# Needs ~/.config/north-star.env (mode 600) with FIREBASE_TOKEN=... +set -euo pipefail + +NOW=false +PREVIEW_ONLY=false +for arg in "$@"; do + case "$arg" in + --now) NOW=true ;; + --preview-only) PREVIEW_ONLY=true ;; + *) echo "unknown option $arg" >&2; exit 2 ;; + esac +done + +if ! $NOW && [ "$(date -u +%u%H)" != "206" ]; then exit 0; fi # %u: 2 = Tuesday + +mkdir -p "$HOME/fw/state/locks" +exec 9>"$HOME/fw/state/locks/job-north-star-cleanup.lock" +flock -n 9 || { echo "already running"; exit 0; } + +echo "== $(date -u '+%F %T UTC') north-star cleanup$($PREVIEW_ONLY && echo ' (preview only)')" + +# Node 22 needs --jitless here: DreamHost's grsec kernel refuses executable memory for V8's JIT. +export PATH="$HOME/fw/node/bin:$HOME/bin:/usr/bin:/bin" +export NODE_OPTIONS=--jitless +set -a; . "$HOME/.config/north-star.env"; set +a +export FIREBASE_PROJECT=north-star-e5dee FIREBASE_RTDB_INSTANCE=north-star-e5dee-default-rtdb + +JOB="$HOME/fw/jobs/north-star" +[ -d "$JOB" ] || git clone --quiet --branch main "$HOME/git/north-star.git" "$JOB" +cd "$JOB" +git fetch --quiet origin main +git reset --quiet --hard origin/main +echo "north-star at $(git rev-parse --short HEAD)" + +# Reinstall only when the lockfile changes. --ignore-scripts: native addons (re2) can't be +# built on shared hosting in jitless mode, and the database commands don't need them. +lock_hash=$(sha256sum package-lock.json | cut -d' ' -f1) +if [ ! -x node_modules/.bin/firebase ] || [ "$(cat node_modules/.fw-lock-hash 2>/dev/null)" != "$lock_hash" ]; then + npm ci --ignore-scripts --no-audit --no-fund --loglevel=error + echo "$lock_hash" > node_modules/.fw-lock-hash +fi + +bash scripts/clean-old-firebase.sh --days=30 --verbose +$PREVIEW_ONLY || bash scripts/clean-old-firebase.sh --days=30 --apply --verbose +echo "== done $(date -u '+%F %T UTC')" diff --git a/server/site/why/index.html b/server/site/why/index.html index 0ef46df..cf0190d 100644 --- a/server/site/why/index.html +++ b/server/site/why/index.html @@ -21,8 +21,7 @@ projects are hosted on Dreamhost, where I already keep a blog and a lightweight webpage.</p> <ul> - <li>code: <a href="https://code.fuzzyworld.net/">https://code.fuzzyworld.net/</a> (browse, or - <code>git clone https://code.fuzzyworld.net/<repo>.git</code>) + <li>code: <a href="https://code.fuzzyworld.net/">https://code.fuzzyworld.net/</a> </li> <li>apps: <a href="https://apps.fuzzyworld.net/">https://apps.fuzzyworld.net/</a></li> </ul>