fuzzyworld-tools / commit
Issue pages: state badge, details table, wrapped text, Reporter A/B labels, separators, prev/next links
README.md | 2 +- server/issues.py | 70 ++++++++++++++++++++++++++++++++++++----- server/site/style.css | 27 ++++++++++++++++ tests/test_issues.py | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 177 insertions(+), 9 deletions(-) diff --git a/README.md b/README.md index 54bbb26..2f4c23e 100644 --- a/README.md +++ b/README.md @@ -315,7 +315,7 @@ In `public` and `reviewed` mode each sender may send 5 messages per 24 hours (th `fw issues hide <repo> <n>` takes an issue off the site in any mode and keeps it off until `approve`. `close` / `reopen` change the state shown on the page. A reply (its `In-Reply-To` or `References` matches an earlier message of an issue of the same repo) is added to that issue. `approve` marks the messages the issue has at that moment; a message that arrives later is shown only in `public` mode and only if its sender passed authentication, or after another `approve`. An unauthenticated reply is never shown automatically. A mail that repeats a stored Message-ID or a stored byte-for-byte copy is ignored. -**Pages.** `/<repo>/issues/` lists the visible issues, `/<repo>/issues/<n>/` shows one. Every publish rebuilds the whole `issues/` folder, so hiding removes the page; when the folder appears or disappears the summary page is rebuilt so its "Issues" link follows. The reporter is always shown as "Reporter". **Privacy rules:** no address, name, Message-ID or sender hash is written to a page; every address-shaped string in the title and text is replaced by `[address hidden]`; lines starting `Device:` or `Run record:` are left out, and so are the lines that wrap a long `Device:` value (following lines are dropped while they are indented or look like a user agent: `token/version`, parentheses or semicolons; a blank line, a template label such as `Seed ...`, or plain prose ends the block, so the reporter's own words are kept); format=flowed text is re-joined first (an empty line always ends a paragraph) and the text is NFKC-normalized before any filtering (so fullwidth `Ｄevice:` or `＠` count); the text is cut at the start of a quoted reply so quoted names never publish: an attribution line in any language (a colon line with an address, or an `On`/`Am`/`Le`/`El`/`Op`... line with a date that says wrote/schrieb/a écrit/schreef/escribió; any such line directly followed by a quoted `>` line), a wrapped attribution (up to three lines ending in a colon, directly followed by a quoted line, or ending in a separate `wrote:` line), a `-----Original Message-----` line, or a block of at least three `Label: value` lines that is a mail header (`From:`/`Von:`/`De :`..., with an address or three header labels; an Outlook `____` line before it is dropped); a colon line followed only by quoted lines at the end of the text is cut too, but never when it would leave nothing, and prose such as `I pressed <Enter> and wrote:` stays; `delsp=yes` is honoured when re-joining flowed text; all text is HTML-escaped inside `<pre class="issue-text">` and no page has a script. +**Pages.** `/<repo>/issues/` lists the visible issues, `/<repo>/issues/<n>/` shows one. Every publish rebuilds the whole `issues/` folder, so hiding removes the page; when the folder appears or disappears the summary page is rebuilt so its "Issues" link follows. Each issue page has an open/closed badge (also in the list), a small details table (version, rules, commit, seed, level, turns, lives, status, modes) filled from the game's own report lines when they match exactly, the messages in wrapping text boxes (a line of three or more dashes becomes a thin rule between boxes), and links to the previous and next visible issue. Senders are labelled "Reporter A", "Reporter B", ... by first appearance within that one issue (the stored author stays "Reporter"; hashes are never written). **Privacy rules:** no address, name, Message-ID or sender hash is written to a page; every address-shaped string in the title and text is replaced by `[address hidden]`; lines starting `Device:` or `Run record:` are left out, and so are the lines that wrap a long `Device:` value (following lines are dropped while they are indented or look like a user agent: `token/version`, parentheses or semicolons; a blank line, a template label such as `Seed ...`, or plain prose ends the block, so the reporter's own words are kept); format=flowed text is re-joined first (an empty line always ends a paragraph) and the text is NFKC-normalized before any filtering (so fullwidth `Ｄevice:` or `＠` count); the text is cut at the start of a quoted reply so quoted names never publish: an attribution line in any language (a colon line with an address, or an `On`/`Am`/`Le`/`El`/`Op`... line with a date that says wrote/schrieb/a écrit/schreef/escribió; any such line directly followed by a quoted `>` line), a wrapped attribution (up to three lines ending in a colon, directly followed by a quoted line, or ending in a separate `wrote:` line), a `-----Original Message-----` line, or a block of at least three `Label: value` lines that is a mail header (`From:`/`Von:`/`De :`..., with an address or three header labels; an Outlook `____` line before it is dropped); a colon line followed only by quoted lines at the end of the text is cut too, but never when it would leave nothing, and prose such as `I pressed <Enter> and wrote:` stays; `delsp=yes` is honoured when re-joining flowed text; all text is HTML-escaped inside `<pre class="issue-text">` and no page has a script. **Run records.** If the text part holds a line `Run record: {json}` (the Explodorama bug-report mail does), it is read from the full decoded part (the text shown is cut to 4000 characters, the record is not) and checked: at most 64 KB, valid JSON, nesting at most 32 deep, an object with integer `seed` and `simVersion` and a list `actions`, finite numbers only. Only the keys `simVersion`, `seed`, `startLevel`, `startCharge`, `options` and `actions` are kept (all others are dropped), and a record with an address-like string anywhere in it is refused. A run record split over several lines by a mail client is not recognised. The first line that passes wins, so the mail's later "too long for an email link" footer is ignored; if the decoded text has none, the raw mail is decoded as quoted-printable and searched again. The record is stored re-serialized, never as mail text, and only the first record of an issue is kept (a record in an unauthenticated reply is ignored; every refusal is logged in `rejected.log`). A record is as public as the message that carried it: one in a reply is published only once that reply is shown (after `approve` in `reviewed` mode; at once in `public` mode for an authenticated sender). It is published as `/<repo>/issues/<n>/record.json` at the same moment as the issue page (`approve` publishes both in `reviewed` mode; `hide` removes both). `server/site/htaccess` lets `https://apps.fuzzyworld.net` read it (CORS) and sends `no-cache`. diff --git a/server/issues.py b/server/issues.py index baa97be..37ae727 100644 --- a/server/issues.py +++ b/server/issues.py @@ -304,17 +304,68 @@ def record_shown(issue, mode): return 0 <= i < len(issue["messages"]) and issue["messages"][i] in shown_messages(issue, mode) -def issue_page(repo, issue, with_record, mode): +VERSION_RE = re.compile(r"Version (\S+) \(rules (\d+)\), commit ([0-9a-f]{7,40})") +SEED_RE = re.compile(r"Seed (\d+), level (\d+), (\d+) turns taken, (\d+) lives, ([a-z ]+); modes: (.*)") +SEPARATOR_RE = re.compile(r"^\s*-{3,}\s*$", re.M) +DETAIL_MAX = 40 + + +def badge(state): + return f'<span class="badge{" badge-success" if state == "open" else ""}">{gen.esc(state)}</span>' + + +def details_table(text): + """A small table of the game's own report lines (strict patterns); "" when none matched.""" + rows = [] + v, d = VERSION_RE.search(text), SEED_RE.search(text) + if v: + rows += list(zip(("Version", "Rules", "Commit"), v.groups())) + if d: + rows += list(zip(("Seed", "Level", "Turns", "Lives", "Status", "Modes"), d.groups())) + if not rows: + return "" + cells = "".join(f"<tr><th>{k}</th><td>{gen.esc(val.strip()[:DETAIL_MAX])}</td></tr>" for k, val in rows) + return f'<table class="issue-details">{cells}</table>' + + +def text_blocks(text): + """The message as <pre> blocks; a line of three or more dashes becomes a thin rule between them.""" + blocks = [b.strip("\n") for b in SEPARATOR_RE.split(text)] + return '<hr class="issue-sep">'.join(f'<pre class="issue-text">{gen.esc(b)}</pre>' for b in blocks if b.strip()) + + +def reporter_labels(messages): + """"Reporter A", "Reporter B", ... by first appearance of each sender within this one issue (hashes never written).""" + seen, labels = {}, [] + for m in messages: + h = m.get("sender_hash") + if not h: + labels.append("Reporter") + continue + if h not in seen: + seen[h] = len(seen) + i = seen[h] + labels.append("Reporter " + (chr(ord("A") + i) if i < 26 else str(i + 1))) + return labels + + +def issue_page(repo, issue, with_record, mode, prev=None, next_=None): n = issue["n"] - parts = [f'<h1>#{n} {gen.esc(issue["title"])}</h1>'] - meta = f'Opened {gen.esc(issue["created"][:10])} · {gen.esc(issue["state"])}' + parts = [f'<h1>#{n} {gen.esc(issue["title"])} {badge(issue["state"])}</h1>'] + meta = f'Opened {gen.esc(issue["created"][:10])}' if with_record: link = replay_link(repo, n) meta += (" · " + link if link else "") parts.append(f'<p class="meta">{meta}</p>') - for m in shown_messages(issue, mode): - parts.append(f'<p class="meta">{gen.esc(m["author"])} · {gen.esc(m["date"][:10])}</p>' - f'<pre class="issue-text">{gen.esc(m["text"])}</pre>') + shown = shown_messages(issue, mode) + if shown: + parts.append(details_table(shown[0]["text"])) + for label, m in zip(reporter_labels(shown), shown): + parts.append(f'<p class="meta">{gen.esc(label)} · {gen.esc(m["date"][:10])}</p>{text_blocks(m["text"])}') + nav = (f'<a href="{gen.url(repo, "issues", str(prev))}/">\u2190 #{prev}</a>' if prev else "<span></span>") + \ + (f'<a href="{gen.url(repo, "issues", str(next_))}/">#{next_} \u2192</a>' if next_ else "<span></span>") + if prev or next_: + parts.append(f'<p class="issue-nav">{nav}</p>') crumbs = [f'<a href="{gen.url(repo)}/">{gen.esc(repo)}</a>', f'<a href="{gen.url(repo, "issues")}/">Issues</a>', f"#{n}"] return gen.page(f"{repo} issue #{n}", "".join(parts), crumbs=crumbs) @@ -322,7 +373,7 @@ def issue_page(repo, issue, with_record, mode): def index_page(repo, issues): rows = "".join( f'<tr><td class="num"><a href="{gen.url(repo, "issues", str(i["n"]))}/">#{i["n"]}</a></td>' - f'<td>{gen.esc(i["title"])}</td><td class="date">{gen.esc(i["created"][:10])}</td><td>{gen.esc(i["state"])}</td></tr>' + f'<td>{gen.esc(i["title"])}</td><td class="date">{gen.esc(i["created"][:10])}</td><td>{badge(i["state"])}</td></tr>' for i in sorted(issues, key=lambda i: -i["n"])) body = f'<h1>{gen.esc(repo)} issues</h1><table><tr><th>#</th><th>Title</th><th>Opened</th><th>State</th></tr>{rows}</table>' return gen.page(f"{repo} issues", body, crumbs=[f'<a href="{gen.url(repo)}/">{gen.esc(repo)}</a>', "Issues"]) @@ -349,9 +400,12 @@ def write_site(repo): return had, False tmp = os.path.join(site_repo, ".issues-new") shutil.rmtree(tmp, ignore_errors=True) + numbers = sorted(i["n"] for i in shown) for i in shown: + before = [x for x in numbers if x < i["n"]] + after = [x for x in numbers if x > i["n"]] has_record = record_shown(i, mode) and os.path.isfile(record_path(repo, i["n"])) - gen.write(os.path.join(tmp, str(i["n"]), "index.html"), issue_page(repo, i, has_record, mode)) + gen.write(os.path.join(tmp, str(i["n"]), "index.html"), issue_page(repo, i, has_record, mode, before[-1] if before else None, after[0] if after else None)) if has_record: with open(record_path(repo, i["n"])) as f: gen.write(os.path.join(tmp, str(i["n"]), "record.json"), f.read()) diff --git a/server/site/style.css b/server/site/style.css index eb69758..d28556c 100644 --- a/server/site/style.css +++ b/server/site/style.css @@ -268,6 +268,33 @@ pre.message, pre.issue-text { white-space: pre-wrap; overflow-wrap: anywhere; } +pre.issue-text { + max-width: 100%; + box-sizing: border-box; + overflow-x: hidden; +} +hr.issue-sep { + border: 0; + border-top: 1px solid var(--site-line); + margin: var(--spacing-sm) 0; +} +table.issue-details { + width: auto; + max-width: 100%; + font-size: var(--font-size-xs); + margin: var(--spacing-sm) 0; +} +table.issue-details th { + text-align: left; + padding-right: var(--spacing-md); + color: var(--site-muted); + font-weight: normal; +} +.issue-nav { + display: flex; + justify-content: space-between; + margin-top: var(--spacing-md); +} .clone { user-select: all; diff --git a/tests/test_issues.py b/tests/test_issues.py index ea8434e..ea083e0 100644 --- a/tests/test_issues.py +++ b/tests/test_issues.py @@ -775,6 +775,93 @@ class Review4(Base): self.assertEqual(self.R("\uff32un record: {}\nafter"), "after") +class PageLayout(Base): + def test_state_badge_in_header_and_list(self): + self.ingest(mail()) + self.assertIn('<span class="badge badge-success">open</span>', rd(self.page("demo", "1", "index.html"))) + self.assertIn('<span class="badge badge-success">open</span>', rd(self.page("demo", "index.html"))) + self.cli("close", "demo", "1") + self.assertIn('<span class="badge">closed</span>', rd(self.page("demo", "1", "index.html"))) + self.assertIn(">closed</span>", rd(self.page("demo", "index.html"))) + + def test_details_table_from_the_game_template(self): + body = ("What happened:\nIt broke.\n\n---\nVersion 0.2.0 (rules 43), commit 805e7a9\n" + "Seed 1464004408, level 4, 66 turns taken, 3 lives, playing; modes: none\nErrors caught: none\n") + self.ingest(mail(body=body)) + page = rd(self.page("demo", "1", "index.html")) + for want in ("<th>Version</th><td>0.2.0</td>", "<th>Rules</th><td>43</td>", "<th>Commit</th><td>805e7a9</td>", + "<th>Seed</th><td>1464004408</td>", "<th>Level</th><td>4</td>", "<th>Turns</th><td>66</td>", + "<th>Lives</th><td>3</td>", "<th>Status</th><td>playing</td>", "<th>Modes</th><td>none</td>"): + self.assertIn(want, page) + self.assertIn("Errors caught: none", page) # the full text stays below + self.assertNotIn("issue-details", rd(self.page("demo", "index.html"))) + + def test_details_table_only_for_matched_fields_escaped_and_capped(self): + t = issues.details_table("Seed 5, level 2, 9 turns taken, 1 lives, dead; modes: <b>" + "x" * 80) + self.assertNotIn("<th>Version</th>", t) + self.assertIn("<th>Seed</th><td>5</td>", t) + self.assertIn("<b>", t) + self.assertNotIn("<b>", t) + self.assertNotIn("x" * 41, t) + self.assertEqual(issues.details_table("Seed five, level two"), "") + self.ingest(mail(body="Plain report with no template.\n")) + self.assertNotIn("issue-details", rd(self.page("demo", "1", "index.html"))) + + def test_dash_line_becomes_a_thin_rule(self): + out = issues.text_blocks("First part.\n------------\nSecond <b>part</b>.\n---\nThird.") + self.assertEqual(out.count("<hr"), 2) + self.assertEqual(out.count('<pre class="issue-text">'), 3) + self.assertNotIn("---", out) + self.assertIn("Second <b>part</b>.", out) + self.assertEqual(issues.text_blocks("no separator"), '<pre class="issue-text">no separator</pre>') + self.assertNotIn("<hr", issues.text_blocks("a-b -- c\n- item")) + self.ingest(mail(body="Above.\n---\nBelow.\n")) + page = rd(self.page("demo", "1", "index.html")) + self.assertIn('<hr class="issue-sep">', page) + self.assertNotIn("\n---\n", page) + + def test_reporters_are_labelled_by_first_appearance_without_hashes(self): + self.ingest(mail(mid="<a@x>", sender="A <alice@example.org>", auth=("mx; dkim=pass header.d=example.org",)), + mail(mid="<b@x>", sender="B <bob@example.org>", auth=("mx; dkim=pass header.d=example.org",), + body="Bob here.\n", extra_headers=("In-Reply-To: <a@x>",)), + mail(mid="<c@x>", sender="A <alice@example.org>", auth=("mx; dkim=pass header.d=example.org",), + body="Alice again.\n", extra_headers=("In-Reply-To: <a@x>",))) + page = rd(self.page("demo", "1", "index.html")) + self.assertLess(page.index("Reporter A"), page.index("Reporter B")) + self.assertEqual(page.count("Reporter A"), 2) + self.assertEqual(page.count("Reporter B"), 1) + stored = issues.load_issue("demo", 1) + for m in stored["messages"]: + self.assertEqual(m["author"], "Reporter") + self.assertNotIn(m["sender_hash"], page) + self.assertEqual(issues.reporter_labels([{"sender_hash": "h1"}, {"sender_hash": "h2"}, {"sender_hash": "h1"}, {}]), + ["Reporter A", "Reporter B", "Reporter A", "Reporter"]) + + def test_previous_and_next_links_skip_hidden_issues(self): + self.ingest(mail(subject="One"), mail(subject="Two"), mail(subject="Three"), mail(subject="Four")) + self.cli("hide", "demo", "3") + p1, p2, p4 = (rd(self.page("demo", str(n), "index.html")) for n in (1, 2, 4)) + self.assertNotIn("\u2190 #", p1) + self.assertIn("#2 \u2192", p1) + self.assertIn("\u2190 #1", p2) + self.assertIn("#4 \u2192", p2) # #3 is hidden + self.assertIn("\u2190 #2", p4) + self.assertNotIn("#5", p4) + self.assertNotIn("#3", p2.split('<p class="issue-nav">')[1]) + + def test_single_issue_has_no_nav_and_css_wraps_text(self): + self.ingest(mail()) + self.assertNotIn("issue-nav", rd(self.page("demo", "1", "index.html"))) + css = rd(os.path.join(ROOT, "server", "site", "style.css")) + self.assertIn("pre.message, pre.issue-text {\n white-space: pre-wrap;\n overflow-wrap: anywhere;", css) + self.assertIn("max-width: 100%", css) + + def test_pages_still_hold_no_script_or_hash(self): + self.ingest(mail(subject="<script>x</script>", body="<script>y</script>\n---\n<script>z</script>\n")) + for p in (self.page("demo", "index.html"), self.page("demo", "1", "index.html")): + self.assertNotIn("<script", rd(p).replace("<script", "")) + + class FwScript(Base): """server/fw publish_one and `fw issues <repo> <mode>` against temp directories (flock is faked: macOS has none)."""