ADR 0024: Architecture and quality assessment documents (audit format)

Ratification

This ADR sets process and format for assessment documents under docs/audit/. Per ADR 0018, add the discussion Issue (title starts with [ADR]), merge PR, and acceptance date when you have a ratification thread. Until then, git history on main is the record.

• Discussion Issue: optional — link when opened with [ADR] title
• Merge PR: see git history for this file
• Accepted: 2026-04-14

Context

Why this matters: ADRs record what we chose and why. Runbooks record how to fix an incident. We also need a third thing: periodic checks that compare a subsystem to common reference practices — a dated report (scope, method, scores, gaps, mitigations) that you can reopen after big refactors or releases.

Without a shared format, assessments become random pages or chat logs. We already treat docs as a first-class artifact (ADR 0001) and govern HTTP contracts (ADR 0007). Assessment documents add to that: they measure fit to reference practices and record what we reviewed and when. They do not replace ADRs.

Relationship to other doc types (summary)

Industry context and applicability to this project

This section is informative (not a normative copy of any one company’s process). It explains why we keep assessments in-repo and which “big company” ideas we intentionally lighten or skip for Study App.

What large technology organizations typically do

Names vary by employer, but patterns recur at scale:

• Design docs / RFCs / API proposals — written before major changes; reviewed for trade-offs. Closest to our ADRs, not to maturity assessments.
• Production or launch readiness reviews — checklists for observability, rollback, capacity, dependencies before wide release. Heavier and more gate-like than our HTML audits.
• Security and privacy review programs — mandatory for products and external APIs; often rubric-driven with tooling. We do not replicate that program here; penetration tests and formal compliance are out of scope for these assessment pages (see each report’s “what this is not”).
• Internal developer portals and API catalogues — org-wide discovery, keys, usage, sometimes billing. Table 1 may reference those patterns; Table 2 should say honestly when the repo is PET-scale and does not need a full portal.
• Platform standards for public APIs — linting (e.g. Spectral), versioning and deprecation policy, SLOs for documentation. Our assessments align vocabulary with that world while scoring what is actually implemented in git.

How repository assessments map to that landscape

Our format is a lightweight, versioned rubric: Table 1 states reference practices; Table 2 scores this codebase against them and points to evidence. That is analogous in spirit to internal maturity conversations and launch readiness themes, but it is not a substitute for organizational gates, headcount review boards, or automated org-wide scorecards. It is cheap to maintain for a small service and stays next to ADRs in docs/audit/.

PET scale: when “full” industry standards are unnecessary or deferred

Study App is framed as a small / PET-scale service unless an assessment explicitly widens scope. Many Table 1 rows describe practices that matter at platform or multi-team scale (e.g. distributed rate limits across regions, full developer portals, multi-language SDK tabs, 24/7 doc analytics programs). Scores in the 1–6 band often mean “not built yet” or “not worth the cost at current stage,” not necessarily “the team failed.”

Each assessment should make that explicit in Scope and methodology (see published backbone): name the deployment assumptions, call out rows that are intentionally out of scope for now, and use 5.3 Nice-to-have (or equivalent) for product-dependent items. Readers should never infer that every row must reach 9–10 for the project to be legitimate.

Decision

We adopt the following norms:

1. Location. Stand-alone assessment reports live under docs/audit/ as static HTML (same stack as other docs: docs/assets/docs.css, docs/assets/docs-nav.js).
2. Naming. Put the formation / assessment date first in the filename (ISO YYYY-MM-DD), then the topic slug, e.g. YYYY-MM-DD-short-topic-assessment.html, all lowercase with hyphens (example: 2026-04-14-rest-api-assessment.html under docs/audit/api/). The date is the assessment date (when the review was performed or the document was last materially refreshed), not necessarily the file creation date in git.
3. Format. Each assessment follows the published backbone below unless the author documents a deliberate deviation in Overview (e.g. a one-page addendum).
4. Purpose. Assessments are for shared maturity understanding, traceability of what was checked, prioritization of follow-ups, and onboarding of new owners. They are not a substitute for automated tests or OpenAPI contract checks — they interpret results in a broader reference frame.
5. Scoring. When scores are used, they are indicative (1–10), defined in the document, and tied to rows in a reference table. They are not performance ratings for individuals.
6. History. Each file includes a short Document history table so readers know what changed between revisions without relying on git archaeology alone.

Assessment score scale (1–10) and shared legend

Table 2 score cells use a single shared palette and legend so readers see the same semantics on every assessment page. Colours and layout live in docs/assets/docs.css (variables --audit-score-* and classes below). The only copy of the legend markup is docs/assets/audit-score-legend-fragment.html. In assessment pages, place a placeholder immediately before Table 2: <div class="audit-score-legend-include" data-legend-id="suffix"></div> — docs/assets/docs-nav.js fetches the fragment and injects it. Use a short suffix (e.g. api, dx) so the heading id becomes audit-score-legend-title-suffix when one document needs distinct labels; omit data-legend-id to keep the default id audit-score-legend-title. For readers without JavaScript, add a <noscript> note pointing to this ADR section.

Score column CSS classes (apply to <td> in table.audit-score-table):

Published assessment backbone (normative)

The working templates in this repository are docs/audit/AUDIT_TEMPLATE.html (skeleton), docs/audit/api/2026-04-14-rest-api-assessment.html (API), and docs/audit/docs/2026-04-14-documentation-experience-assessment.html (documentation DX). New assessments should match their section order and intent (anchors and numbering may differ slightly, e.g. optional “beyond baseline” only where useful).

1. Title, nav chrome, <section id="overview" class="card">. Put the formation / assessment date in both <title> and <h1> (e.g. — formed 14 Apr 2026 in the title and (formed 14 Apr 2026) in the heading) so the browser tab matches the page; repeat or extend dates in Overview. Do not use a top class="lead" paragraph (metadata belongs in Overview only).
2. Navigation. Rely on <div class="docs-inpage-toc-mount" data-inpage-toc="auto"></div> (injected sidebar TOC) and section ids — do not add a separate “Table of contents” block after Overview unless there is a documented exception.
3. Scope and methodology — what was examined, method (often a short numbered list), what the assessment is not, and a short industry / PET applicability subsection (why some reference practices are deferred or not targeted — see Industry context and applicability above).
4. Table 1 — Reference practices. Neutral rubric: #, Category, Practice, Reference description, Typical benchmark (wording may vary by topic).
5. Table 2 — Mapping and scores. One row per Table 1 practice being scored; include a Justification column (why this score vs the reference row); shared legend placeholder immediately above the table (see score scale); score cells use the CSS classes above — no duplicated score rules in page-local <style>.
6. Scoring summary — narrative overall; optional explicit arithmetic (weighted model or Table 2 mean) in pre.audit-scoring-formula; Top‑5 gaps in table.audit-gap-table with status (TODO / IN PROGRESS / DONE) and started / closed / PR columns.
7. Gaps, refactors, and recommendations — usually 5.1 should add, 5.2 refactor/tighten, 5.3 nice-to-have; link to table rows where helpful.
8. Mitigation plan — phased or dated actions (indicative dates are fine).
9. Optional — Beyond baseline / industry context — e.g. API assessment §7; omit when it would only repeat Table 1.
10. Actionable checklist — concise bullets tied to rows or sections for backlog/issues.
11. Page history — Date | Change | Author (see Documentation style guide); substantive edits only (not trivial typos).

Scope of this ADR

• In scope: Files under docs/audit/ that follow this backbone; cross-links from docs/index.html or ADRs when an assessment is cited.
• Out of scope: Confidential penetration-test reports (keep non-sensitive summaries elsewhere if needed). Incident postmortems follow the incident process, not this ADR.

Goals and process

Goals: institutional memory; shared vocabulary from Table 1; prioritization via Table 2 and mitigations; onboarding without a live walkthrough; traceability via Overview + history.

• When to refresh: Major releases, security or API reviews, annual hygiene — or whenever scores or conclusions change (update document history).
• Ownership: The lead or an early section should imply an owner for follow-up triage (items may land in backlog).
• Linking: Prefer ADRs and runbooks over duplicating policy; assessments summarize and map.
• Changelog: Add docs/CHANGELOG.md when adding or materially revising an assessment (ADR 0013).

Alternatives considered

1. Markdown-only in docs/. Faster to draft; this site is HTML-first for many handbooks. Companions are OK per developer guide 0008; the canonical published assessment is HTML under docs/audit/.
2. Wiki or external tool only. Weakens docs-as-code review and exact commit/ADR linkage.
3. No separate assessments (ADRs only). ADRs do not naturally hold “we scored N practices”; mixing would blur decisions vs evaluations.

Consequences

Positive

• Clear place and shape for maturity snapshots.
• Easier prioritization with a visible rubric.
• Consistent reader experience next to other docs/ pages.

Trade-offs

• Authors must maintain tables and history or the document misleads.
• Scores invite debate; mitigate with scale definition, evidence, and no person-level use.

Compatibility and migration

No runtime impact. Existing docs/audit/ files should converge on this backbone when edited. Rollback: revert this ADR; assessments remain ordinary HTML.

Rollout and validation

• Link this ADR from the ADR index; link docs/audit/README.html from docs/index.html and other doc hubs so readers can browse all assessments.
• New assessments: copy a current audit file and replace Table 1/2 and narrative sections.
• Technical: make docs-check passes after HTML changes.
• Human: A new reader can state what was reviewed, top gaps, and next steps from the assessment and linked ADRs alone.
• Documentation: docs/CHANGELOG.md updated for new or materially revised assessments.

References

• ADR 0001 — Docs as Code.
• ADR 0018 — ADR lifecycle and ratification.
• ADR 0013 — Changelog policy (root vs docs).
• Developer guide 0008 — Documentation pipeline.
• audit-score-legend-fragment.html — legend markup (injected via .audit-score-legend-include in docs-nav.js).
• docs-nav.js — loads the fragment into assessment pages.
• docs/assets/docs.css — --audit-score-* and .audit-score-* rules.
• API assessment — reference layout.
• DX assessment — same backbone, Documentation Experience topic.

HTML formatting note (repository tooling)

This ADR, docs/assets/audit-score-legend-fragment.html, and the assessment HTML files under docs/audit/ may be listed in SKIP_HTML_INDENT_NORMALIZE in scripts/format_docs_html.py so the line-based HTML indenter does not rewrite them. That indenter can mis-count nesting when block tags span multiple lines. Stylesheet and nav normalization still apply when you run make docs-fix / format_docs_html.py.

Page history