Maintenance Audit
Audit date: July 5, 2026
Audit Metadata
- Target repo root: current Git repository root. Exact local filesystem path is intentionally omitted from this public report.
- Prompt-pack root: user-supplied external v5 deep-evidence repo-health prompt pack containing an entrypoint named RUN_THIS_FIRST.md and the required pass files. Exact local filesystem path is intentionally omitted from this public report.
- Git state at start of this continuation:
main...origin/mainwith maintenance edits already present in.env.example,README.md,docs/**,src/ha_backend/config.py,tests/test_config.py, and an untrackeddocs/maintenance-audit.md. - Prompt-pack handling: read-only. No files under the prompt-pack root were modified or copied into the repo.
Repo Profile
HealthArchive is a public app monorepo for HealthArchive.ca. It contains:
- Python/FastAPI/SQLAlchemy backend code under
src/ha_backend/; - an in-tree crawler/orchestrator package under
src/archive_tool/; - Alembic migrations under
alembic/; - a Next.js/TypeScript frontend under
frontend/; - MkDocs documentation under
docs/; - CI, verification, and operations scripts under
.github/,scripts/, and related ops config trees.
Risk level for unattended changes: moderate to high. Public APIs, admin and metrics auth, crawl job lifecycle, WARC/replay paths, local/production config, database migrations, and public/private documentation boundaries all affect operator safety or public behavior. Changes in this pass were kept local, reversible, repo-native, and verified with existing commands.
Instructions And Conventions Considered
- Repo-local instructions:
AGENTS.md,CLAUDE.md,GEMINI.md. - Frontend-specific instructions:
frontend/AGENTS.md. - Canonical docs:
README.md,docs/architecture.md,docs/documentation-guidelines.md,docs/development/live-testing.md,src/archive_tool/docs/documentation.md,frontend/docs/development/bilingual-dev-guide.md. - Tooling/config:
Makefile,pyproject.toml,requirements.txt,mkdocs.yml,.pre-commit-config.yaml,.nvmrc,frontend/package.json,frontend/package-lock.json. - CI/workflows:
.github/workflows/backend-ci.yml,.github/workflows/backend-ci-full.yml,.github/workflows/frontend-ci.yml,.github/workflows/docs.yml,.github/workflows/workflow-lint.yml,.github/workflows/platform-ops-integration.yml,.github/workflows/production-smoke.yml. - Public/private boundary: production host paths, private inventories, exact operator runbooks, credentials, private network details, and shared host state are not to be added to public docs.
Closeout Maintenance Continuation
After the initial audit pass, a maintenance closeout pass inspected the repository state, project process docs, branch/PR/CI state, generated artifacts, roadmap requirements, and attribution policy before push.
Closeout findings:
CLAUDE.mdandGEMINI.mdare relative symlinks toAGENTS.md.- GitHub PR, branch, and CI state was inspected through
gitand the GitHub CLI available in the local environment. - Open PRs: none.
- Remote branches:
mainandgh-pages;mainis protected.gh-pagesis the documentation publishing branch and was intentionally left in place. - Latest pushed
mainCI for commitd6df53c2was green for Backend CI, Frontend CI, platform validation, and documentation deployment. The documentation publishing branch was treated as intentional generated output and left in place. - Roadmap process was re-read. No active implementation plan corresponded to this maintenance audit, so no plan was moved to
docs/planning/implemented/and no roadmap item was silently expanded. Nonessential findings remain in this report's recommendations. - Ignored local environment assets such as
.env,.venv/,node_modules/,private/, andfrontend/.env.localwere intentionally preserved. Safe generated cache/database artifacts were cleaned during closeout. - The maintenance changes were collected in human-authored local commits with no co-author trailer.
Reviewable-File Inventory And Coverage Tier
Inventory evidence:
git ls-files | wc -l->742tracked files.- Reviewable group counts from
git ls-files: src/: 60 tracked backend/crawler source files.tests/: 114 tracked backend test files.frontend/src/: 80 tracked frontend source/assets files.- frontend tests/specs: 41 tracked files.
docs/: 211 tracked documentation files.scripts/: 86 tracked scripts..github/: 11 tracked CI/config files.- deployment/infrastructure/container surfaces: 1 directly counted top-level container/deploy match, plus many VPS/operator scripts under
scripts/. - dependency/tooling config matches: 17 tracked files.
Coverage tier: large repo. The tracked file count is above the v5 prompt-pack large-repo threshold, and the repo includes several operational subsystems. Coverage was therefore risk-based rather than every-file manual review.
Manual review focused on:
- entrypoint instructions, README, docs policy, architecture, live testing, docs nav, and report surfaces;
- package manifests, lockfiles, Makefile targets, pre-commit, CI workflows, frontend package scripts, and Node/Python version declarations;
- config/env/auth/CORS/security header code in
src/ha_backend/config.py,src/ha_backend/api/__init__.py, andsrc/ha_backend/api/deps.py; - public/admin route surfaces in
src/ha_backend/api/routes_public.pyandsrc/ha_backend/api/routes_admin.py; - compare-to-live SSRF and diff-rendering paths in
src/ha_backend/live_compare.py,src/ha_backend/diffing.py,frontend/src/app/[locale]/compare-live/page.tsx, andfrontend/src/app/[locale]/compare/page.tsx; - data/session/model/migration surfaces in
src/ha_backend/db.py,src/ha_backend/models.py,alembic/env.py, andalembic/versions/**; - worker/crawler/indexing paths in
src/ha_backend/jobs.py,src/ha_backend/worker/main.py,src/ha_backend/indexing/pipeline.py,src/ha_backend/indexing/deduplication.py, and related tests; - representative frontend architecture and public-only API boundary from
frontend/AGENTS.md,frontend/package.json,frontend/src/lib/api.ts, and route/component search results.
Excluded from manual review except for presence/consistency checks:
- generated/cached artifacts:
__pycache__/,.pytest_cache/,.mypy_cache/,.ruff_cache/,.next/, coverage outputs,site/, generateddocs/openapi.json, generateddocs/llms.txt; - dependency trees:
.venv/,frontend/node_modules/; - generated contract/lockfile internals except consistency checks:
frontend/src/lib/api-contract.generated.ts,frontend/package-lock.json; - large binary/source assets except repo placement and public presentation:
.xcf,.png,.webpfrontend assets; - ignored private/local material such as
.env,private/, and private symlinks/copies.
Command And Search Ledger
| Command or probe | Purpose | Result |
|---|---|---|
pwd | Confirm target working directory | Confirmed current repository root; exact local path omitted. |
git rev-parse --show-toplevel | Resolve target repo root | Passed |
git status --short --branch | Preserve current worktree state | Passed; dirty worktree documented |
rg --files <prompt-pack-root> | Inventory prompt-pack files | Passed |
sed reads of the prompt-pack entrypoint, safety contract, evidence gates, and all pass files | Required v5 workflow read order | Passed |
git ls-files, wc, awk inventory | Determine coverage tier and file groups | Passed; large repo tier |
sed reads of AGENTS, README, Makefile, pyproject, frontend package, docs, workflows | Repo-native workflow/context discovery | Passed |
rg "TODO|FIXME|HACK|XXX|debugger|console.log|print(" | Code-quality/stale-marker probe | Found expected CLI/script prints and planning TODO prose; no safe code cleanup candidate |
git grep private-key/token-shape regex | Secret-shape probe | No committed private-key/API-token shapes found |
git grep auth/secret/token/password/cors/csrf/session terms | Security/auth surface discovery | Found expected auth docs, workflow secrets references, admin auth, and public/private boundary docs |
git grep dangerous-operation terms | Injection/path/subprocess/HTML-rendering probe | Found expected subprocess calls, path handling, URL redirects, and controlled diff HTML rendering |
git grep HEALTHARCHIVE_* env vars | Env/config coverage probe | Found many backend/ops vars; .env.example remains intentionally local-focused |
rg "/home/|/mnt/|/srv/|/etc/|/opt/|api.healtharchive.ca|docs.healtharchive.ca" | Public/private path and URL boundary probe | Found public API/docs URLs plus many tracked VPS helper defaults; documented as owner-scope follow-up |
rg "pageSize|limit|yield_per|StreamingResponse|FileResponse" | Pagination/export/performance probe | Public/admin list/export paths are bounded; deduplication optimization remains follow-up |
find .github/workflows, find scripts | CI/scripts inventory | Passed |
focused pytest for tests/test_api_compare_live.py | Verify added compare-live URL guard tests | Passed: 19 tests, 1 existing warning |
Baseline And Verification Commands
Commands already run during this maintenance campaign before the v5 continuation:
make cipassed.make frontend-ciinitially failed becausenpmwas not on PATH in the non-interactive shell, then passed after loading nvm.make docs-checkpassed.make check-fullpassed after Ruff import-order cleanup.make ci-api-health-localpassed.- Focused config regression test passed after the archive-root fix.
- Targeted pre-commit on report/docs files passed.
git diff --checkpassed.
Additional v5 continuation verification:
TMPDIR="$(pwd)/.tmp/backend-tmp" TMP="$(pwd)/.tmp/backend-tmp" TEMP="$(pwd)/.tmp/backend-tmp" .venv/bin/pytest tests/test_api_compare_live.py -qpassed: 19 tests, 1 existing Starlette/httpx TestClient deprecation warning.make cipassed: Ruff format check, Ruff lint, mypy, and the fast backend pytest subset.make check-fullpassed: Ruff, mypy, full pytest, critical coverage, pre-commit, Bandit, pip-audit, and docs checks.make frontend-cipassed after loading nvm for Nodev20.19.0and npm10.8.2.make ci-api-health-localpassed with a temporary SQLite DB and ephemeral API server.make contract-checkpassed; regenerated OpenAPI/frontend contract outputs had no tracked diff.- Final
make docs-checkpassed after the report update.
Pass Completion Table
| Pass | Status | Evidence summary |
|---|---|---|
| 01 general code quality/docs/maintenance | Fixed changes | Fixed repo-local archive-root fallback, stale docs/auth wording, stale CI wording, and .env.example phase labels; added report evidence. |
| 04 test coverage/regression | Fixed changes | Added config fallback regression test and compare-live URL guard regression tests; targeted compare-live tests passed. |
| 03 security/privacy/secrets | Fixed changes plus deferred owner items | No secret-shape hits; fail-closed admin docs aligned; compare-live URL blockers now directly tested; tracked VPS helper defaults remain owner-scope. |
| 05 CI/automation/workflow | Fixed docs/no-op automation | Existing Makefile/CI commands are strong; README CI wording updated; no new workflow imposed. |
| 06 dependency/package hygiene | No-op with evidence | Manifests/lockfiles/audit config inspected; no dependency changes; audits previously passed. |
| 07 documentation/onboarding | Fixed changes | README, architecture, live-testing, CLI reference, docs index, and report updated to current behavior. |
| 02 architecture/maintainability | No-op with evidence | Architecture boundaries mapped; no safe broad refactor; production-shaped worker/script defaults deferred. |
| 09 performance/scalability | No-op with evidence plus follow-up | Bounded public/admin list/export paths inspected; potential deduplication query scaling follow-up documented. |
| 10 database/data/migrations | No-op with evidence | SQLAlchemy models, Alembic env/versions, session lifecycle, migration guard, local migrations verified through prior full/API smoke checks; no migration created. |
| 08 release readiness | No release action | Checks/build/docs/audit passed in prior campaign; no tag/version/publish/deploy; readiness improved via docs/config/tests. |
| 11 public repo presentation | Fixed docs plus deferred owner items | Public docs clarified; no marketing claims or new process; public/private script split remains owner decision. |
| 12 code-adjacent writing professionalization | Fixed small wording issues plus deferred owner items | Neutralized public docs role labels, removed a chatty API-guide closing, aligned quickstart operator copy with the public/private boundary, cleaned low-value source comments, and left historical review artifacts unchanged pending owner decision. |
Pass 01: General Code Quality, Documentation, And Maintenance
Scope decision:
- Applies to backend config, docs, tests, public/admin API surfaces, scripts, and frontend workflow docs.
- Broad source rewrites, public API changes, workflow replacements, and production-operator script redesigns were out of scope.
Surfaces inspected:
src/ha_backend/config.py,tests/test_config.py,.env.example.README.md,docs/architecture.md,docs/development/live-testing.md,docs/reference/cli-commands.md,docs/README.md.Makefile,pyproject.toml,frontend/package.json, CI workflows.- representative core modules:
api/deps.py,api/__init__.py,api/routes_public.py,api/routes_admin.py,jobs.py,worker/main.py,indexing/pipeline.py,live_compare.py.
Probes/searches:
- stale marker/debug probe;
- large-file/hotspot inventory;
- env-var probe;
- dangerous-operation/path/rendering probe;
- current diff/stat review.
Candidate findings considered:
- Fixed: machine-specific
DEFAULT_ARCHIVE_ROOTfallback in code. - Fixed: docs claiming tokenless local admin access was the default.
- Fixed: stale README CI wording about full backend workflow scheduling.
- Fixed:
.env.examplecomments still using implementation phase labels. - Not an issue: many
print()hits are CLI/script user output. - Deferred:
src/ha_backend/cli.py,worker/main.py, andscripts/vps-*contain production-shaped paths/defaults. These are intertwined with tested operator tooling and private runbooks, so changing them without owner context would be risky.
Change-gate decisions:
- Archive-root fallback was evidence-backed, local, testable, and public-safe.
- Docs updates were repo-native and aligned public docs with current tested behavior.
- VPS helper defaults were documented but not changed because they may be active operator contracts.
Verification:
- Config regression test was added and passed during prior verification.
make ci,make check-full,make frontend-ci,make ci-api-health-local, andmake contract-checkpassed in this v5 continuation.
Pass 04: Test Coverage And Regression Confidence
Scope decision:
- Applies strongly: repo has extensive pytest and Vitest suites and Makefile CI targets.
- New tests should be close to changed behavior and deterministic.
Surfaces inspected:
tests/test_config.py.tests/test_api_compare_live.py.tests/test_api_admin_and_metrics.py,tests/test_security_headers.py,tests/test_api_search_and_snapshot.py,tests/test_worker.py,tests/test_worker_tiering.pyby search/manual sampling.- frontend
package.jsontest scripts andfrontend/src/**test inventory.
Probes/searches:
- test inventory by
find testsand frontend test file counts; - mapping source hotspots to tests with
rgaround compare-live, admin auth, search, worker, and migration guard; - targeted pytest for compare-live after adding tests.
Candidate findings considered:
- Fixed: archive-root fallback lacked a regression test.
- Fixed: compare-to-live URL guardrails were security-sensitive but only indirectly tested through API error mapping.
- Deferred: existing TestClient/httpx deprecation warning is cross-suite maintenance and should be handled separately.
- Deferred: Python
datetime.utcnow(), SQLAlchemyQuery.get(), and SQLite ResourceWarnings from prior full verification remain focused cleanup work.
Tests added/updated:
tests/test_config.py::test_archive_tool_default_root_is_repo_local_and_public_safe.tests/test_api_compare_live.py::test_compare_live_normalize_url_blocks_unsafe_targets.tests/test_api_compare_live.py::test_compare_live_normalize_url_strips_fragment_without_network.
Verification:
- Focused compare-live file passed: 19 tests, 1 existing deprecation warning.
- Final full backend test/CI target is rerun below.
Pass 03: Security, Privacy, And Secrets Hygiene
Scope decision:
- Applies strongly: public API, admin/metrics auth, issue reports, compare-to-live network fetches, raw snapshot replay, CORS/security headers, public/private docs boundary, and CI secrets usage are in scope.
- Credential rotation, production policies, deploy scripts, and private runbook changes are owner-scope and not changed.
Surfaces inspected:
src/ha_backend/api/deps.py(require_admin).src/ha_backend/api/__init__.py(CORS, request size limits, CSP/HSTS and security headers).src/ha_backend/live_compare.py(URL normalization, private-IP blocking, redirect revalidation, byte limits).src/ha_backend/diffing.pyand frontend compare pages (escaped generated diff HTML rendered withdangerouslySetInnerHTML)..env.example,SECURITY.md,.github/workflows/*.yml,docs/documentation-guidelines.md, public/private boundary stubs.- Tracked scripts with production-shaped defaults, by path/secret probes.
Probes/searches:
- private-key/API-token-shape regex: no matches.
- broad auth/secret/password/token/cors/csrf/session grep.
- dangerous-operation grep for subprocess, raw SQL/text SQL, URL redirects,
dangerouslySetInnerHTML, file/path operations, and deserialization. - public/private path and URL grep.
Candidate findings considered:
- Fixed: code fallback no longer embeds a machine-specific archive root.
- Fixed: admin-auth docs now match fail-closed behavior.
- Fixed: compare-to-live private/loopback/link-local/credential/port blocking now has direct tests.
- Already satisfied:
require_adminfails closed unless a local/dev/test env also setsHEALTHARCHIVE_ALLOW_DEV_ADMIN_NO_TOKEN=true; admin routes and/metricsdepend on it. - Already satisfied: raw snapshot responses get restrictive CSP and non-raw API responses get restrictive defaults.
- Already satisfied: compare-live revalidates redirect targets and enforces content-type, redirect, timeout, port, and byte limits.
- Deferred: tracked VPS helper scripts include exact
/srv,/etc,/optdefaults and public operational paths. Existing docs/roadmap already identify splitting private operational assets as owner work; changing scripts here could break operator tooling.
Verification:
- Focused compare-live tests passed.
- Bandit and pip-audit passed in prior
make check-full. - Final verification is rerun below.
Pass 05: CI, Automation, And Developer Workflow
Scope decision:
- Applies strongly: repo uses Makefile targets, GitHub Actions, pre-commit, docs checks, frontend npm scripts, contract generation, and API smoke checks.
- No new CI provider, hooks, package manager, or release automation was added.
Surfaces inspected:
Makefile..github/workflows/backend-ci.yml,backend-ci-full.yml,frontend-ci.yml,docs.yml,workflow-lint.yml,platform-ops-integration.yml,production-smoke.yml..pre-commit-config.yaml,.nvmrc,frontend/package.json.- README workflow sections and live-testing commands.
Probes/searches:
- Makefile target review.
- workflow inventory and manual reads.
- frontend script review.
- prior command execution of backend, frontend, docs, full, and API smoke targets.
Candidate findings considered:
- Fixed: README CI description understated backend CI audit and API smoke, and incorrectly described the full workflow as nightly/manual rather than manual.
- Already satisfied: local Makefile targets mirror CI enough for backend, frontend, docs, API smoke, contract, and full checks.
- Deferred: non-interactive shells may not have
npmon PATH until nvm is loaded. This was documented as an environment note, not a Makefile change, because CI sets Node explicitly and local shell managers vary.
Verification:
make ci,make check-full,make frontend-ci,make ci-api-health-local, andmake contract-checkpassed in this v5 continuation.
Pass 06: Dependency And Package Hygiene
Scope decision:
- Applies to Python package metadata, dev/docs extras, requirements mirror, frontend npm dependencies, lockfile presence, Node version, pre-commit, Dependabot/audit config, and scripts.
- Broad upgrades/removals were out of scope without a specific failing audit or safe compatibility evidence.
Surfaces inspected:
pyproject.toml,requirements.txt,.pre-commit-config.yaml,.github/dependabot.yml.frontend/package.json,frontend/package-lock.json,.nvmrc.- Makefile
audit/audit-ci,security,frontend-install, andfrontend-citargets.
Probes/searches:
- dependency manifest reads;
- prior pip-audit through
make check-full; - frontend parity through
make frontend-ci; - lockfile/diff review showing no dependency files changed.
Candidate findings considered:
- No dependency changes: no high-confidence unused dependency removal or safe upgrade was identified.
- Already satisfied: Python audit target exists with a documented Pygments CVE ignore; frontend lockfile is present and used by CI.
- Deferred: package version upgrades should be scheduled separately if maintainers want freshness beyond security/audit-driven updates.
Dependency-change table:
| Package/config | Change | Reason | Risk | Verification |
|---|---|---|---|---|
| Python dependencies | None | No clear safe upgrade/removal needed | None | Prior make check-full and pip-audit passed |
| Frontend dependencies | None | No clear safe upgrade/removal needed | None | Prior make frontend-ci passed |
.env.example | Comment/doc-only updates | Local config clarity | Low | Pre-commit/docs/final checks |
Pass 07: Documentation And Onboarding
Scope decision:
- Applies strongly: repo has a public README, MkDocs portal, local development docs, architecture guide, CLI reference, frontend docs bridge, and docs coverage/reference checks.
- New docs should fit existing docs index/nav policy without inventing a new docs system.
Surfaces inspected:
README.md,.env.example,docs/README.md,docs/architecture.md,docs/development/live-testing.md,docs/reference/cli-commands.md,mkdocs.yml,docs/documentation-guidelines.md.
Probes/searches:
- documented admin behavior vs
api/deps.py; - documented archive-root default vs
config.py; - README CI description vs
.github/workflows/backend-ci.ymlandbackend-ci-full.yml; - docs generated-output policy vs
.gitignore; - docs/reference/coverage/build checks through
make check-full.
Candidate findings considered:
- Fixed: README/setup docs now state repo-local
.dev-archive-rootfallback and explicit staging/production override requirement. - Fixed: live-testing and architecture docs now describe fail-closed admin auth and explicit local tokenless override.
- Fixed: docs index links this maintenance report.
- Deferred: full public env-var reference would be useful, but
docs/deployment/environments-and-configuration.mdis intentionally a public boundary stub; expanding it needs owner docs-policy direction.
Documentation inventory:
- Docs site source:
docs/**with MkDocs nav inmkdocs.yml. - Generated docs outputs:
docs/openapi.jsonanddocs/llms.txt, ignored and refreshed through docs targets. - Frontend canonical docs:
frontend/docs/**, bridged throughdocs/frontend. - Report artifact:
docs/maintenance-audit.md.
Pass 02: Architecture And Maintainability
Scope decision:
- Applies through module boundaries, large files, API/data/worker/crawler responsibilities, frontend/backend split, and docs architecture map.
- Broad refactors, public API changes, source moves, and production-ops script redesign were out of scope.
Surfaces inspected:
docs/architecture.md, README layout,AGENTS.md.src/ha_backend/api/**,src/ha_backend/models.py,src/ha_backend/jobs.py,src/ha_backend/worker/main.py,src/ha_backend/indexing/**.src/archive_tool/main.py,src/archive_tool/docs/documentation.md.frontend/AGENTS.md,frontend/package.json,frontend/src/lib/api.ts,frontend/src/app/[locale]/**representative routes.
Architecture map:
- Backend package owns config, database/session helpers, ORM models, API routes and schemas, search/change logic, indexing, worker orchestration, CLI, runtime metrics, and public/admin auth.
archive_toolis an in-tree CLI package invoked by backend jobs and tested as a first-class component.- Worker flow: queued/retryable jobs ->
run_persistent_job()-> completed/failed/retryable ->index_job()-> indexed/index_failed. - Indexing flow: WARC discovery -> optional stable consolidation -> WARC verification -> HTML extraction/mapping -> snapshots/outlinks/pages -> page signals.
- Frontend consumes public API only, uses generated API contract types, and is intentionally forbidden from admin/metrics endpoints.
- Documentation portal remains MkDocs-based with
mkdocs.ymlas nav source of truth.
Candidate findings considered:
- No safe structural refactor: large files such as
cli.py,routes_public.py,archive_tool/main.py, and VPS helper scripts are real hotspots but not suitable for drive-by splitting. - Deferred: worker auto-tiering and VPS defaults deserve a focused private operator-context pass if maintainers want to make public/private boundaries cleaner in code.
- Already satisfied: public/admin boundary is explicit in route modules and frontend instructions.
Verification:
- Existing tests and full checks cover these boundaries; no architecture code changes were made in this continuation beyond added tests.
Pass 09: Performance And Scalability
Scope decision:
- Applies to public search/change/export APIs, indexing/WARC processing, worker loops, frontend build/render path, and database query patterns.
- New caching, indexes, queues, profiling stacks, or migrations were out of scope without stronger evidence.
Surfaces inspected:
src/ha_backend/api/routes_public.pyforpageSize,limit,StreamingResponse, export row limits, and direct WARC scans.src/ha_backend/api/routes_admin.pyfor admin pagination.src/ha_backend/indexing/pipeline.py,warc_reader.py,warc_verify.py,changes.py,indexing/deduplication.py.- frontend compare pages and
CompareLiveDiffPanelrendering path by search.
Probes/searches:
- pagination/limit/yield_per/streaming probe.
- dangerous-operation/path probe.
- large file/hotspot inventory.
Candidate findings considered:
- Already satisfied: public search/changes/timeline/admin endpoints use bounded
pageSize/limitparameters. - Already satisfied: exports cap rows with configured limits and materialize bounded rows before streaming to avoid holding DB transactions for slow clients.
- Already satisfied: WARC indexing flushes every 500 snapshots and uses streaming readers.
- Deferred:
find_same_day_duplicates()does group discovery then per-group duplicate queries. It is correct and tested, but could become a scaling issue on very large duplicate sets; a focused optimization with tests would be appropriate if it appears in production profiles. - Deferred: direct raw snapshot API scans are size-bounded by
_RAW_SNAPSHOT_MAX_DIRECT_WARC_BYTES; larger deployments should continue to prefer pywb replay indexes.
Verification:
- No performance code change was made. Existing/final tests protect behavior.
Pass 10: Database, Data, And Migration Health
Scope decision:
- Applies strongly: SQLAlchemy models, Alembic migrations, SQLite local tests, Postgres production-like docs, session lifecycle, and migration guard.
- Production DBs, remote data, destructive migration changes, and schema decisions were out of scope.
Surfaces inspected:
src/ha_backend/models.py,src/ha_backend/db.py,src/ha_backend/api/deps.py,src/ha_backend/api/__init__.py.alembic/env.pyand Alembic revisions 0001 through 0015.Makefilemigration and API-health targets.- tests around DB/session/migration/search/API by inventory and prior full run.
Probes/searches:
- Alembic file inventory.
- SQL/raw-text/dangerous-operation grep.
- pagination/query probes.
- prior
make ci-api-health-localmigration smoke andmake check-fullmigration guard/tests.
Candidate findings considered:
- No schema drift fix needed: no model changes were made.
- No migration needed: code change only altered default local archive root and tests/docs.
- Already satisfied: request-scoped DB sessions are closed immediately after route execution, before streaming bodies.
- Deferred: SQLite ResourceWarnings seen during prior coverage indicate a future test/session cleanup pass, not a correctness fix in this campaign.
Data safety notes:
- No production database or remote service was accessed.
- No data was modified outside local test SQLite databases.
- No migration was created or run against production.
- Local migration smoke passed earlier through
make ci-api-health-local.
Pass 08: Release Readiness
Scope decision:
- Applies as app deployment/internal handoff readiness, not package publication.
- No version bump, release branch, tag, publish, deploy, artifact upload, or license decision was authorized or performed.
Surfaces inspected:
pyproject.toml,frontend/package.json,LICENSE,SECURITY.md,CITATION.cff,README.md,.github/workflows/**, Makefile release-adjacent checks, docs build and frontend build behavior.
Candidate findings considered:
- Fixed: docs/config inaccuracies that would confuse release/deploy review.
- Already satisfied: backend, frontend, docs, audit, API smoke, and full checks exist and passed during the campaign.
- Owner decision required: production deployment/public release promotion stays outside unattended assistant scope and public docs intentionally defer environment-specific procedures.
Release-readiness checklist:
| Area | Status | Notes |
|---|---|---|
| Backend checks | Ready for review | make ci and make check-full passed |
| Frontend checks/build | Ready for review | make frontend-ci passed after nvm load |
| Docs build | Ready for review | make check-full included strict docs checks and passed |
| Security/dependency audit | Ready for review | Bandit and pip-audit passed through make check-full |
| Version/tag/publish/deploy | Not applicable | Not requested or performed |
| Production rollout | Owner decision | Public docs defer exact environment procedures |
Pass 11: Public Repo Presentation
Scope decision:
- Applies because this is a public repository for a public app/docs site.
- The goal is factual clarity and public-safety, not marketing, badges, screenshots, public maturity claims, or new contribution/governance process.
Surfaces inspected:
README.md,docs/README.md,docs/api-consumer-guide.mdby path probe, public docs policy,.env.example, package descriptions, security policy, docs nav, public/private boundary stubs, tracked VPS script defaults.
Probes/searches:
- public/private URL/path grep;
- README/docs command and behavior cross-checks;
- secret-shape grep;
- docs build/reference checks through
make check-full.
Candidate findings considered:
- Fixed: public docs now avoid a machine-specific default archive root and stale open-admin local behavior.
- Fixed: maintenance audit linked from docs index.
- Already satisfied: README states public/private deployment boundary and datasets repo separation.
- Deferred: tracked VPS helper scripts and some historical docs still expose exact operational path defaults. Existing roadmap item 34c tracks splitting private operational assets from public repo surface; this requires owner context and careful script migration.
Pass 12: Code-Adjacent Writing Professionalization
Scope decision:
- Applies to repository text that affects professional sharing, onboarding, review, handoff, or publication: public docs, agent instructions, source comments, CLI/script wording surfaced by searches, and dated frontend review artifacts.
- The goal was neutral, durable wording. The pass did not add a new docs program, governance artifact, style guide, marketing copy, license change, or maturity claim.
- The external prompt-pack files referenced by the pasted request were not available in the shell for this run:
PROMPT_PACK_ROOTwas unset, and constrained searches under the repo, Codex attachments, and.codexpaths did not find the shared safety contract or deep-evidence gate files. This pass followed the safety/evidence requirements embedded in the pasted request text and this repo'sAGENTS.md.
Audience and maturity assessment:
- HealthArchive is a public monorepo for a real public-interest archive, with active backend, crawler, frontend, docs, CI, and operations-adjacent scripts.
- The appropriate voice is factual, modest, public-safe, and precise: it should be clear about current limitations and the public/private operations boundary without sounding promotional or informal.
Surfaces inspected:
- Canonical context:
README.md,docs/architecture.md,docs/documentation-guidelines.md,docs/development/live-testing.md,src/archive_tool/docs/documentation.md,docs/planning/roadmap.md,frontend/docs/development/bilingual-dev-guide.md, andmkdocs.yml. - Public/docs entry points and code-adjacent docs:
docs/README.md,docs/quickstart.md,docs/api-consumer-guide.md,docs/project.md,docs/deployment/README.md,docs/operations/README.md,AGENTS.md, andfrontend/AGENTS.md. - Representative source comments and user-facing script/help wording under
src/ha_backend/,src/archive_tool/,scripts/,tests/, andfrontend/src/. - Tracked frontend review artifacts:
frontend/*_ANALYSIS.mdandfrontend/*_IMPROVEMENT_PLAN.md. - Metadata and public project files by probe:
pyproject.toml,CITATION.cff,CONTRIBUTING.md,SECURITY.md,.github/**,Makefile, andplatform-ops-contract.example.yaml.
Searches/probes run:
| Probe | Purpose | Result |
|---|---|---|
rg --files plus find docs, find frontend/docs, find .github | Inventory reviewable writing surfaces | Confirmed public docs, frontend docs, review artifacts, workflows, and scripts. |
rg -n -i "vibe|quick.?and.?dirty|hack|kludge|lol|wtf|stupid|dumb|ugly|gross|yolo|magic|whatever|good enough|don't judge|sorry|short on time|temporary|for now|fix later|ask the user|the user|medical school|med school|sleep|weekend" ... | Find unprofessional, temporary, or overly casual wording | Mostly legitimate technical uses; several safe comment/doc wording candidates fixed. |
rg -n -i "ChatGPT|Codex|Claude|AI generated|generated by AI|assistant|prompt|LLM|agent" ... | Find assistant/session residue and AI attribution | Expected agent instructions and generated llms.txt context references; no improper AI authorship added. |
rg -n "TODO|FIXME|HACK|XXX|NOTE|BUG" ... | Find vague stale markers and code comments | Mostly expected debug/log-level strings, shell notes, and template TODO labels; one obvious source-comment cleanup fixed. |
rg -n -i "Jeremy|author|\bI\b|\bmy\b|personal|private" ... | Find personal/private context and public/private boundary text | Mostly intentional boundary/provenance references; public quickstart copy and personal phrasing in AGENTS.md fixed. |
rg -n "[✅⚠️🔧💻📊📚👤]" docs frontend/*.md frontend/docs README.md AGENTS.md CONTRIBUTING.md SECURITY.md | Find symbol-heavy public docs and review artifacts | Public docs role labels and chatty closing fixed; dated frontend review artifacts left unchanged as owner-scope historical review material. |
rg -n "HOMEPAGE_ANALYSIS|ARCHIVE_ANALYSIS|CONTENT_PAGES_ANALYSIS|FEATURE_PAGES_ANALYSIS|SNAPSHOT_ANALYSIS|UTILITY_PAGES_ANALYSIS|IMPROVEMENT_PLAN" ... | Check references to tracked review artifacts | No broad references found beyond one paired analysis/plan link, so no autonomous relocation was needed. |
Candidate wording ledger:
| Candidate | Classification | Decision |
|---|---|---|
AGENTS.md virtualenv instruction with first-person workflow wording | Fixed | Reworded to neutral setup guidance. |
README.md database setup phrasing using "whatever" | Fixed | Reworded to refer to the configured database URL. |
docs/README.md role navigation with emoji labels | Fixed | Replaced with text-only role labels matching the repo's restrained docs style. |
docs/quickstart.md role headings, five-minute claim, and quick deploy snippet | Fixed | Replaced with neutral starting-point copy and public/private operator-boundary guidance. |
docs/api-consumer-guide.md chatty closing line with emoji | Fixed | Removed the closing line; the preceding next-step links already provide a durable close. |
frontend/src/app/globals.css comment using subjective colour criticism | Fixed | Reworded as a contrast rationale. |
src/ha_backend/config.py, src/ha_backend/cli.py, src/ha_backend/job_registry.py, src/archive_tool/main.py, src/archive_tool/utils.py comments with "for now" uncertainty or obvious narration | Fixed | Reworded comments around current intent, evidence-based tuning, and failure ownership; removed obvious section-marker comments. |
PHAC temporary exclusions in src/ha_backend/job_registry.py | Intentionally unchanged | The comment is dated, specific, and explains a real crawl-cost guardrail. |
"temporary crawl artifacts", time.sleep, and MagicMock search hits | Not an issue | Technical terms, API names, or test-library names. |
| Public/private operations wording throughout docs | Not an issue | Intentional public repository boundary language required by AGENTS.md. |
scripts/generate_llms_txt.py and docs/llms.txt references | Not an issue | Deliberate public-safe developer-assistant context generator, not AI authorship residue. |
pyproject.toml, CITATION.cff, and license author/provenance fields | Not an issue | Legitimate human authorship and citation metadata. |
Tracked frontend *_ANALYSIS.md and *_IMPROVEMENT_PLAN.md files | Owner decision | They are dated design-review artifacts with useful historical context. A mass rewrite would create high churn, and deletion/relocation would be a project-organization decision. |
| Generated/vendor-like files such as lockfiles | Skipped | Not useful to edit for prose style and often externally generated. |
Rewrite patterns applied:
- Personal or workflow-specific wording became neutral repo instructions.
- Chatty closings and decorative role icons were removed from public docs.
- Public quickstart deployment content was changed into public-safe boundary guidance rather than publishing live operator procedure.
- Comments were changed from temporary uncertainty to current technical intent.
- Obvious code narration was removed where it added no durable "why".
Verification for this pass:
git diff --checkpassed.make cipassed: Ruff format check, Ruff lint, mypy, and 385 backend tests passed with 1 existing Starlette/httpx deprecation warning.make docs-checkinitially failed because the audit report used inline-code formatting for unavailable external prompt-pack filenames; the docs reference checker treated those as missing repo-local paths. The report wording was corrected, andmake docs-checkthen passed.make frontend-ciinitially failed becausenpmwas not onPATHin the non-interactive shell. Rerunning with the bundled Node/npm directory prepended toPATHpassed Prettier, ESLint, TypeScript, Vitest, and the Next production build.
Maintenance closeout for this pass:
- Re-read
AGENTS.md,frontend/AGENTS.md,docs/documentation-guidelines.md,docs/development/testing-guidelines.md, anddocs/roadmap-process.mdbefore closeout. - Confirmed the current branch was
main, trackingorigin/main; local HEAD matchedorigin/mainbefore the closeout commit. - Confirmed
CLAUDE.md,GEMINI.md,frontend/CLAUDE.md, andfrontend/GEMINI.mdare relative symlinks to their respectiveAGENTS.mdfiles. - Inspected GitHub state with explicit-repo
ghcommands because the local shell PATH did not includeghand repo-relativeghcommands hit a safe-directory warning. Open PRs: none. Remote branches:mainandgh-pagesonly. Latestmainworkflow runs before this closeout were green for Backend CI, Frontend CI, documentation deployment, and platform integration. - Removed generated local artifacts from verification runs: MkDocs build output, temporary test directories, generated docs reference outputs, frontend build output, Python caches, frontend TypeScript build info, and local tool caches.
- Preserved intentional ignored local assets: local env files, virtualenv, frontend dependencies, private notes/copies, and local lock/cache files whose ownership was not part of this pass.
- No
.gitignoreupdate was needed; the generated artifacts removed during cleanup were already ignored. - Roadmap/planning action: no active implementation plan corresponds to this professionalization pass, and the remaining frontend review-artifact cleanup is documented as an owner decision rather than silently added to the active hot-path investigation plan.
Changes Made
Code/test:
src/ha_backend/config.py: default archive root fallback is now the repo-local, git-ignored.dev-archive-rootinstead of a machine-specific host path.tests/test_config.py: added regression coverage for the repo-local fallback.tests/test_api_compare_live.py: added deterministic URL-normalization guard tests for compare-to-live unsafe targets and fragment stripping.
Documentation/config:
.env.example: documented local-only tokenless admin override and removed stale phase labels.AGENTS.md: replaced first-person setup phrasing with neutral repository guidance.README.md: corrected archive-root default, admin auth behavior, and backend CI wording; this writing pass also neutralized database setup phrasing.docs/architecture.md: corrected archive-root/admin-auth behavior.docs/development/live-testing.md: updated admin local-vs-prod examples.docs/reference/cli-commands.md: corrected archive-root default/example.docs/README.md: linked this maintenance audit and replaced decorative role icons with text labels.docs/quickstart.md: replaced casual role headings and public deploy commands with public-safe starting-point guidance.docs/api-consumer-guide.md: removed a chatty closing line.docs/maintenance-audit.md: replaced the previous v4 report with this v5 deep-evidence report and added the code-adjacent writing pass ledger.
Comment-only wording:
frontend/src/app/globals.css,src/ha_backend/config.py,src/ha_backend/cli.py,src/ha_backend/job_registry.py,src/archive_tool/main.py, andsrc/archive_tool/utils.py: reworded or removed low-value comments without changing runtime behavior.
Security And Privacy Notes
- No committed private-key/API-token shaped values were found by regex probe.
- The default archive root no longer embeds a machine-specific path in code.
- Admin and metrics docs now match fail-closed runtime behavior.
- Compare-to-live URL safety blockers are now directly covered by tests.
- Reports and final output should not include actual secrets; none were found.
- Ignored private/local files were not inspected or modified.
- Production-shaped tracked VPS scripts remain a public/private boundary follow-up, not an autonomous edit.
Remaining Recommendations
- Decide whether tracked VPS helper scripts should remain public as active operator scripts or be split into public-safe templates plus private implementations.
- Investigate SQLite ResourceWarnings in full coverage runs and close any session/TestClient lifecycle gaps.
- Schedule a Python test hygiene pass for
datetime.utcnow(), SQLAlchemyQuery.get(), and Starlette/httpx TestClient deprecations. - Consider a focused optimization pass for same-day deduplication query shape if large duplicate groups show up in real datasets.
- Consider a focused docs-policy pass for a public-safe env-var reference, keeping deployment/host-specific details in private operations docs.
Final Verification
| Area | Status | Evidence |
|---|---|---|
| Formatting | Passed | Ruff format via make ci/make check-full; frontend Prettier via make frontend-ci; ruff format reported changed Python tests unchanged. |
| Linting | Passed | Ruff via make ci/make check-full; ESLint via make frontend-ci; pre-commit Ruff hooks via make check-full. |
| Type checking | Passed | Backend mypy via make ci/make check-full; frontend tsc --noEmit via make frontend-ci. |
| Backend tests | Passed | Fast subset: 385 passed, 1 warning. Full pytest: 843 passed, 21 warnings. Focused compare-live file: 19 passed, 1 warning. |
| Critical coverage | Passed | make check-full critical coverage: 81.63% total, above the configured 75% threshold; 843 tests passed with 488 warnings under coverage. |
| Frontend tests/build | Passed | Vitest: 38 files and 107 tests passed; Next production build completed successfully. |
| Docs | Passed | make check-full and a final standalone make docs-check ran OpenAPI export, docs/llms.txt generation, docs references, strict docs coverage, and strict MkDocs build successfully. MkDocs reported existing nav omissions, including this report, as INFO. |
| Security/dependencies | Passed | Bandit passed; pip-audit found no known vulnerabilities, with the configured Pygments ignore and local editable package skip. |
| API smoke | Passed | make ci-api-health-local passed public health, stats, sources, exports, search, usage, changes, and empty-index-tolerant RSS checks. |
| API contract | Passed | make contract-check regenerated OpenAPI and frontend types with no tracked diff. |
Remaining warnings:
- Starlette/httpx TestClient deprecation warning.
- Python
datetime.utcnow()deprecation warnings in test fixtures. - SQLAlchemy
Query.get()deprecation warnings in tests. - SQLite
ResourceWarningmessages during coverage.
These were not introduced by this pass and are recorded as focused follow-up work rather than weakened or suppressed.
Final Diff Summary
Primary maintenance commit diff summary before the closeout report amendment:
.env.example | 7 +++++--
README.md | 32 +++++++++++++++++---------------
docs/README.md | 1 +
docs/architecture.md | 22 ++++++++++++----------
docs/development/live-testing.md | 26 +++++++++++++++++++-------
docs/maintenance-audit.md | 768 +++++++++++++++++++++++++++++++++++++++
docs/reference/cli-commands.md | 4 ++--
src/ha_backend/config.py | 10 ++++++----
tests/test_api_compare_live.py | 34 +++++++++++++++++++++++++++++++++-
tests/test_config.py | 16 ++++++++++++++++
10 files changed, 879 insertions(+), 41 deletions(-)
docs/maintenance-audit.md is the combined report artifact for this run.
Risks, Assumptions, And Intentionally Unchanged Areas
- Real staging/production deployments are assumed to set
HEALTHARCHIVE_ARCHIVE_ROOTexplicitly, as now documented. - No database schema changed, so no migration was created.
- No dependencies were upgraded or removed.
- No frontend behavior was changed.
- No deploy, release, or production publish action was performed.
- Production/private operations decisions remain with the maintainer.