fuzzyworld

fuzzyworld-tools / files / docs / issues.md

issues.md

8.9 KB · history · blame

Issues (bug reports by email)

Bug reports arrive by email at bugs-k7q4+<repo>@fuzzyworld.net. The address is on every repo's summary page. The repo name is part of it, and a repo that does not exist under ~/git sends the mail to the bad folder.

server/issues.py (fw issues ..., Python standard library only) turns each report into an issue and, when allowed, a static page. It reuses the comments plumbing in comments.py (IMAP or Maildir source, Authentication-Results check, locks, atomic JSON files).

Moderate from the terminal with fw issues, or from the local web page (admin.md).

Modes

fw issues <repo> public|reviewed|private sets git config fuzzyworld.issues in the bare repo and republishes. Anything other than exactly public or reviewed is private, and a repo that is itself private (fuzzyworld.visibility) never shows issues, whatever its mode.

Mode An issue's page appears
private never; reports are stored and listed by fw issues list only
reviewed after fw issues approve <repo> <n>
public as soon as its first message passes email authentication (unauthenticated mail is stored with authenticated: false and waits for approve)

In public and reviewed mode each sender may send 5 messages per 24 hours (the rest goes to the bad folder).

Moderating

Command Effect
approve <repo> <n> marks the messages the issue has at that moment as shown
hide <repo> <n> takes the issue off the site in any mode and keeps it off until approve
close / reopen change the state shown on the page
list, show read stored issues (plain text)
publish, publish-all rebuild the pages
ingest, imap-check read the mailbox / check the mailbox settings

Rules for messages:

Pages

/<repo>/issues/ lists the visible issues; /<repo>/issues/<n>/ shows one. Every publish rebuilds the whole issues/ folder, so hiding removes the page. When the folder appears or disappears, the summary page is rebuilt so its "Issues" link follows.

An issue page has:

Senders are labelled "Reporter A", "Reporter B", ... by first appearance within that one issue. The stored author stays "Reporter"; hashes are never written.

Privacy rules for published text

Never written to a page: any address, name, Message-ID or sender hash.

Replaced or dropped:

Device lines that wrap: following lines are dropped while they are indented or look like a user agent (token/version, parentheses or semicolons). A blank line, a template label such as Seed ..., or plain prose ends the block, so the reporter's own words are kept.

Normalisation before filtering: format=flowed text is re-joined first (an empty line always ends a paragraph; delsp=yes is honoured), and the text is NFKC-normalised, so fullwidth Device: or @ count.

Quoted replies are cut, so quoted names never publish. The text is cut at the first of:

The last cut is never made when it would leave nothing, and prose such as I pressed <Enter> and wrote: stays.

Output: all text is HTML-escaped inside <pre class="issue-text">, and no page has a script.

Run records

If the text part holds a line Run record: {json} (the Explodorama bug-report mail does), it is read and checked.

Rule Detail
Source the full decoded part (the text shown is cut to 4000 characters; the record is not). If the decoded text has none, the raw mail is decoded as quoted-printable and searched again
Validity at most 64 KB, valid JSON, nesting at most 32 deep, an object with integer seed and simVersion and a list actions, finite numbers only
Kept keys only simVersion, seed, startLevel, startCharge, options and actions; all others are dropped
Refused a record with an address-like string anywhere in it. Every refusal is logged in rejected.log
Not recognised a record split over several lines by a mail client
Which one wins the first line that passes (so the mail's later "too long for an email link" footer is ignored). Only the first record of an issue is kept; a record in an unauthenticated reply is ignored
Storage re-serialised, never as mail text

Publishing. A record is as public as the message that carried it: one in a reply is published only once that reply is shown (after approve in reviewed mode; at once in public mode for an authenticated sender). It is published as /<repo>/issues/<n>/record.json at the same moment as the issue page (approve publishes both in reviewed mode; hide removes both). server/site/htaccess lets https://apps.fuzzyworld.net read it (CORS) and sends no-cache.

Replay link

If a deployed app of the repo has a replay template, a visible issue with a record shows Watch what happened.

Machine-readable commands

Used by fw admin. The text of unpublished reports is included (it is a private tool); sender hashes, mkey and mid never are.

Command Prints
fw issues list [repo] --json every stored issue (repo, n, title, created, updated, state, approved, hidden, public, record, messages, shown, waiting) and each repo's mode and public_repo. waiting means a public or reviewed repo's issue that is not hidden and has messages not yet shown
fw issues show <repo> <n> --json one issue with the full text of every message (label, date, authenticated, approved, shown), the details-table rows, previous/next numbers, the public page URL (only when public) and the replay URL (only when the record is shown)
fw issues record <repo> <n> record.json as stored, even for an unpublished issue (error no record for repo#n when there is none)

Plain list and show are unchanged.

State and mail source

State is in ~/fw/issues/ (override with FW_ISSUES_DIR), outside every web root:

File Purpose
repos/<repo>/<n>/issue.json, record.json the stored issue and its run record
secret salt for sender hashes (mode 600)
msgids.json threading
keys.json a mail is never stored twice
rejected.log every refusal
locks

Mail source: ~/fw/issues/imap.json (mode 600, the same format as the comments one; finished mail goes to issues-done / issues-bad), or, without it, the Maildir ~/Maildir/.bugs (.bugs.done, .bugs.bad).

Requirements:

Source
1# Issues (bug reports by email)
2
3Bug reports arrive by email at `bugs-k7q4+<repo>@fuzzyworld.net`. The address is on every repo's summary page. The
4repo name is part of it, and a repo that does not exist under `~/git` sends the mail to the bad folder.
5
6`server/issues.py` (`fw issues ...`, Python standard library only) turns each report into an issue and, when
7allowed, a static page. It reuses the comments plumbing in `comments.py` (IMAP or Maildir source,
8`Authentication-Results` check, locks, atomic JSON files).
9
10Moderate from the terminal with `fw issues`, or from the local web page ([admin.md](admin.md)).
11
12## Modes
13
14`fw issues <repo> public|reviewed|private` sets `git config fuzzyworld.issues` in the bare repo and republishes.
15Anything other than exactly `public` or `reviewed` is `private`, and a repo that is itself private
16(`fuzzyworld.visibility`) never shows issues, whatever its mode.
17
18| Mode | An issue's page appears |
19|---|---|
20| `private` | never; reports are stored and listed by `fw issues list` only |
21| `reviewed` | after `fw issues approve <repo> <n>` |
22| `public` | as soon as its first message passes email authentication (unauthenticated mail is stored with `authenticated: false` and waits for `approve`) |
23
24In `public` and `reviewed` mode each sender may send 5 messages per 24 hours (the rest goes to the bad folder).
25
26## Moderating
27
28| Command | Effect |
29|---|---|
30| `approve <repo> <n>` | marks the messages the issue has at that moment as shown |
31| `hide <repo> <n>` | takes the issue off the site in any mode and keeps it off until `approve` |
32| `close` / `reopen` | change the state shown on the page |
33| `list`, `show` | read stored issues (plain text) |
34| `publish`, `publish-all` | rebuild the pages |
35| `ingest`, `imap-check` | read the mailbox / check the mailbox settings |
36
37Rules for messages:
38
39- A reply (its `In-Reply-To` or `References` matches an earlier message of an issue of the same repo) is added to that issue.
40- A message that arrives after `approve` is shown only in `public` mode, and only if its sender passed authentication, or after another `approve`. An unauthenticated reply is never shown automatically.
41- A mail that repeats a stored Message-ID or a stored byte-for-byte copy is ignored.
42
43## Pages
44
45`/<repo>/issues/` lists the visible issues; `/<repo>/issues/<n>/` shows one. Every publish rebuilds the whole
46`issues/` folder, so hiding removes the page. When the folder appears or disappears, the summary page is rebuilt so
47its "Issues" link follows.
48
49An issue page has:
50
51- an open/closed badge (also in the list),
52- a small details table (version, rules, commit, seed, level, turns, lives, status, modes), filled from the game's own report lines when they match exactly,
53- the messages in wrapping text boxes (a line of three or more dashes becomes a thin rule between boxes),
54- links to the previous and next visible issue.
55
56Senders are labelled "Reporter A", "Reporter B", ... by first appearance within that one issue. The stored author
57stays "Reporter"; hashes are never written.
58
59## Privacy rules for published text
60
61**Never written to a page:** any address, name, Message-ID or sender hash.
62
63**Replaced or dropped:**
64
65- every address-shaped string in the title and text becomes `[address hidden]`;
66- lines starting `Device:` or `Run record:` are left out.
67
68**Device lines that wrap:** following lines are dropped while they are indented or look like a user agent
69(`token/version`, parentheses or semicolons). A blank line, a template label such as `Seed ...`, or plain prose ends
70the block, so the reporter's own words are kept.
71
72**Normalisation before filtering:** format=flowed text is re-joined first (an empty line always ends a paragraph; `delsp=yes` is honoured), and the text is NFKC-normalised, so fullwidth `Device:` or `@` count.
73
74**Quoted replies are cut,** so quoted names never publish. The text is cut at the first of:
75
76- an attribution line in any language: a colon line with an address, or an `On`/`Am`/`Le`/`El`/`Op`... line with a date that says wrote/schrieb/a écrit/schreef/escribió; any such line directly followed by a quoted `>` line;
77- a wrapped attribution: up to three lines ending in a colon, directly followed by a quoted line, or ending in a separate `wrote:` line;
78- a `-----Original Message-----` line;
79- a block of at least three `Label: value` lines that is a mail header (`From:`/`Von:`/`De :`..., with an address or three header labels; an Outlook `____` line before it is dropped);
80- a colon line followed only by quoted lines at the end of the text.
81
82The last cut is never made when it would leave nothing, and prose such as `I pressed <Enter> and wrote:` stays.
83
84**Output:** all text is HTML-escaped inside `<pre class="issue-text">`, and no page has a script.
85
86## Run records
87
88If the text part holds a line `Run record: {json}` (the Explodorama bug-report mail does), it is read and checked.
89
90| Rule | Detail |
91|---|---|
92| Source | the full decoded part (the text shown is cut to 4000 characters; the record is not). If the decoded text has none, the raw mail is decoded as quoted-printable and searched again |
93| Validity | at most 64 KB, valid JSON, nesting at most 32 deep, an object with integer `seed` and `simVersion` and a list `actions`, finite numbers only |
94| Kept keys | only `simVersion`, `seed`, `startLevel`, `startCharge`, `options` and `actions`; all others are dropped |
95| Refused | a record with an address-like string anywhere in it. Every refusal is logged in `rejected.log` |
96| Not recognised | a record split over several lines by a mail client |
97| Which one wins | the first line that passes (so the mail's later "too long for an email link" footer is ignored). Only the first record of an issue is kept; a record in an unauthenticated reply is ignored |
98| Storage | re-serialised, never as mail text |
99
100**Publishing.** A record is as public as the message that carried it: one in a reply is published only once that
101reply is shown (after `approve` in `reviewed` mode; at once in `public` mode for an authenticated sender). It is published as `/<repo>/issues/<n>/record.json` at the same moment as the issue page
102(`approve` publishes both in `reviewed` mode; `hide` removes both). `server/site/htaccess` lets
103`https://apps.fuzzyworld.net` read it (CORS) and sends `no-cache`.
104
105## Replay link
106
107If a deployed app of the repo has a `replay` template, a visible issue with a record shows **Watch what happened**.
108
109- "A deployed app of the repo" means `apps.fuzzyworld.net/<app>/.fuzzyworld.json` has `repo` equal to the repo, and the app has an `index.html`.
110- The template comes from the `"fuzzyworld"` block of the app's `package.json`, for example `"replay": "replay.html?report={record}"` ([apps.md](apps.md)).
111- The link goes to `https://apps.fuzzyworld.net/<app>/<replay>` with `{record}` replaced by the URL-encoded address of `record.json`.
112- The app named like the repo is preferred, so a `-next` copy is not chosen.
113- It is a plain link, not an iframe (the site's CSP blocks framing). No `replay` key, no link.
114
115## Machine-readable commands
116
117Used by [`fw admin`](admin.md). The text of unpublished reports **is** included (it is a private tool); sender
118hashes, `mkey` and `mid` never are.
119
120| Command | Prints |
121|---|---|
122| `fw issues list [repo] --json` | every stored issue (`repo`, `n`, `title`, `created`, `updated`, `state`, `approved`, `hidden`, `public`, `record`, `messages`, `shown`, `waiting`) and each repo's `mode` and `public_repo`. `waiting` means a public or reviewed repo's issue that is not hidden and has messages not yet shown |
123| `fw issues show <repo> <n> --json` | one issue with the full text of every message (label, date, authenticated, approved, shown), the details-table rows, previous/next numbers, the public page URL (only when public) and the replay URL (only when the record is shown) |
124| `fw issues record <repo> <n>` | `record.json` as stored, even for an unpublished issue (error `no record for repo#n` when there is none) |
125
126Plain `list` and `show` are unchanged.
127
128## State and mail source
129
130State is in `~/fw/issues/` (override with `FW_ISSUES_DIR`), outside every web root:
131
132| File | Purpose |
133|---|---|
134| `repos/<repo>/<n>/issue.json`, `record.json` | the stored issue and its run record |
135| `secret` | salt for sender hashes (mode 600) |
136| `msgids.json` | threading |
137| `keys.json` | a mail is never stored twice |
138| `rejected.log` | every refusal |
139| locks | |
140
141**Mail source:** `~/fw/issues/imap.json` (mode 600, the same format as the comments one; finished mail goes to
142`issues-done` / `issues-bad`), or, without it, the Maildir `~/Maildir/.bugs` (`.bugs.done`, `.bugs.bad`).
143
144**Requirements:**
145
146- The bugs mailbox needs its own account or its own `folder` in `imap.json`. Ingest and `imap-check` refuse to run when it has the same host, user and folder as the comments `imap.json` (the two jobs would move each other's mail to their bad folders).
147- Ingest refuses to run until `authserv_id` is set in `server/comments/sites.json`.
148- The hourly job is `issues-ingest.sh` ([operations.md](operations.md#scheduled-jobs)).