# Project Context

## What This File Is
This is the persistent context file for Claude Code. Keep it concise and useful.

## CLAUDE.md Maintenance Rules
- Keep this file under 160 lines
- Only record decisions, not explanations of common knowledge
- When adding something, check if anything existing is now outdated and remove it
- Use short bullet points, not paragraphs
- No boilerplate or filler text
- If a section grows past 10 items, consolidate or prune the least relevant ones
- Format: what we chose + why in one line (e.g., "vanilla JS over React — embeddable, no build step")

## Stack
- Python 3 (stdlib only) — fetch, parse, analyze, build_members, build_app, enrich_roster
- pytest — classifier unit tests + KPI parity gate against legacy/
- Vanilla JS (IIFE, no framework, no bundler) — frontend must drop into 3rd-party hosts
- Chart.js 4.4.0 + SortableJS 1.15.2 — vendored locally under `template/vendor/`, no CDN
- Data source: clerk.house.gov XML + senate.gov XML (no API key); Congress.gov v3 API for roster enrichment (key in `.env`)

## Architecture Decisions
- One unified vote-record schema for both chambers (`votes.jsonl`) — analyze.py is chamber-agnostic
- analyze.py is pure (no I/O); consumed by `build_members.py` workers
- `build_members.py` uses `multiprocessing.Pool` with initializer-shared records (no per-task serialization)
- Per-member JSON written via tmp + os.replace (atomic); members/ dir wiped on each build to avoid stale files
- `manifest.json` is the picker index AND carries per-member KPI dict (`k`) so ranking.html needs only one fetch
- `build_app.py` inlines the full manifest as `<script type="application/json" id="polisci-manifest">` so file:// works for the picker
- Per-member JSON still requires HTTP (78 MB total — too large to inline)
- Chart canvases wrapped in `.chart-canvas-wrap` (position:relative, flex:1 1 auto, min-height:0) inside fixed-height `.chart-frame` (340px / 280px mobile) — fixes Chart.js infinite-growth feedback loop
- All CSS scoped under `#polisci-root` — required by inline-embed contract (CSS audit gates this at 0 violations)
- `data-base="./data/"` on `#polisci-root` lets host pages relocate the data dir
- Senate vote XMLs use LIS member IDs (e.g. `S270`); Congress.gov uses bioguide — `lis_to_bioguide.json` crosswalk built by name+state+party match (Congress.gov v3 does not expose LIS reliably)
- Per-Congress term data (startYear/endYear/district, death_year, current_member) only on individual `/v3/member/{bg}` endpoint — bulk listing is too sparse, so enrich_roster runs detail lookups for every replacement-chain member
- Territorial House delegates report state="XX" in vote XMLs — parse.py overrides with the directory's real USPS code (AS/DC/GU/MP/PR/VI)
- Roster merge: vote-derived roster gets enriched with `full_name`, `district`, `served_from/to`, `photo_url`, `served_partial`, `is_delegate`, `congress_term`, `death_year`, `current_member`, `replaces`, `replaced_by` from `members_directory.json`
- ID regex hard limit: `^[A-Z]\d{6}$|^S\d{3,4}$` validated on every URL read AND against `manifestById`
- All upstream strings rendered via `textContent` / `createElement` — zero `innerHTML` (security-audited)
- localStorage namespaced `polisci:v119:*`; only `lastMember` persisted in MVP

## Pipeline
```
fetch.py → parse.py → enrich_roster.py → parse.py → pytest → build_members.py → build_app.py
```
- `fetch.py` — idempotent network fetch into `data/<C>/{house,senate}/cache/`
- `parse.py` — XML → `votes.jsonl` + `roster.json`; rejects upstream strings with `<`, `>`, control chars; merges directory fields (delegate flag, congress_term, death_year, current_member, replaces/replaced_by) and overrides "XX" state for delegates
- `enrich_roster.py` — three passes: (1) bulk Congress.gov → `members_directory.json` + LIS crosswalk; (2) rescue individual `/v3/member/{bg}` for vote-derived House bioguides missing from bulk (e.g. Gaetz); (3) replacement-linking by (state, district) within Congress window → `replaces`/`replaced_by`, then detail-enrich every chain member for term/death/current fields. 350ms throttle; `User-Agent: polisci-pipeline/1.0`
- `build_members.py` — parallel pool; passes full chamber records to `analyze.aggregate` (NOT pre-filtered by member) so absences count as N/A rows
- `build_app.py` — wipes + recreates `results/<C>/`; injects manifest into HTML heads; writes README with CSP + iframe snippet
- `build_all.py` — one-command wrapper; runs `parse.py` twice (before and after enrich) so directory data merges in

## Data Layout
- `data/<C>/{house,senate}/{cache/, votes.jsonl, roster.json}` — raw + parsed per-chamber
- `data/<C>/members/<id>.json` — per-member metrics (~80 KB each)
- `data/<C>/manifest.json` — picker index; per-member carries `k` (KPI dict for Rankings), `dl` (delegate), `un` (unseated), `rs`/`rb` (replacement refs), `sy`/`ey` (term years), `dy` (death year)
- `data/<C>/members_directory.json` — Congress.gov roster (~551 members)
- `data/<C>/lis_to_bioguide.json` — Senate ID crosswalk
- `data/<C>/api_cache/` — cached Congress.gov responses
- `data/<C>/build_report.json` — per-build success/failure log
- `results/<C>/` — embeddable artifact (the shipping output)

## Frontend Pages
- `app.html` — single-member dashboard: sidebar filters (chamber/party/state), typeahead, 8 KPI tiles (incl. Voted With GOP / Voted With Dem), 5 charts, sortable vote table
- `compare.html` — overlay up to 6 members across 5 comparison charts; per-member color-coded pills
- `ranking.html` — rank House or Senate members by any of 14 metrics; House/Senate radio + party checkboxes + metric dropdown + order radio; URL state; row click opens member dashboard
- Structural banners on app.html via `renderNote()` priority: delegate > unseated > died > replaced_by > replaces > served_partial; successor/predecessor rendered as in-app links via `state.membersById`
- Nav between pages: `Member · Compare · Rankings` in `<header>`
- Member IDs in URL only (`?id=`, `?ids=`, `?c=&m=&o=&p=`) — deep-linkable, validated against manifest

## Embedding Modes
- Standalone: open `app.html` directly (over HTTP)
- Iframe: `sandbox="allow-scripts allow-same-origin" referrerpolicy="no-referrer"`
- Inline: copy `<link>` + `<script>` + `<div id="polisci-root" data-base="...">` into host page
- Recommended CSP documented in `results/<C>/README.md`

## Verified Properties
- KPI parity vs `legacy/`: 8/8 PASS (script: `tests/parity_check.py`; report: `research/PHASE3_PARITY.md`)
- analyze.py classifier: 22 pytest tests against frozen XML fixtures
- CSS namespace audit: 0 violations (report: `research/PHASE6_CSS_AUDIT.md`)
- HTTP smoke (all assets + manifest + sample member + 404 path): 10/10
- No external network at runtime; vendor scripts loaded locally
- `build_members.py` for 552 members: ~5 s on local CPU

## Security
- API key in `.env` (gitignored); rotated via Congress.gov sign-up flow; redacted from DOCUMENTATION.md §2
- `.gitignore`: `.env`, `__pycache__/`, `*.pyc`, `data/*/cache/`, `data/*/api_cache/`
- Upstream-string validation in `parse.py` rejects `<`, `>`, control chars at XML parse time
- All DOM mutations via `textContent` / `createElement` / `replaceChildren` — `grep innerHTML` returns empty across template/ and results/
- No `eval`, no `Function()`, no inline event handlers

## Conventions
- One JS file per page (`app.js`, `compare.js`, `ranking.js`); IIFE wrappers
- ~200 lines of filter/typeahead logic duplicated between `app.js` and `compare.js` — accepted for v1 to avoid coupling
- Python scripts: CLI via argparse, `--congress N` (default 119), shebang + executable
- No comments in code unless WHY is non-obvious (e.g. security validation, LIS fallback rationale)
- Tests live in `tests/` (not the test-engineer sandbox) per project requirement

## Known Issues / Deferred (see NOTES.md for full list)
- Editorial label wording ("Helped Republicans", "Blocked Dem-Backed") — compliance flagged; user kept current copy for MVP, must revisit before publication
- compare.html "Voted against own party" uses `monthly['Helped Neither']` as proxy — analyze.py doesn't emit a true monthly own-party-defection series
- Visible "How to read this" caveats panel deferred to v1.1 (limitations live in DOCUMENTATION.md §10)
- localStorage only persists `lastMember`; filter state lives in URL only
- Cross-browser manual smoke not run (no headless browser in build env); `_iframe_test.html` + `_embed_test.html` available in `results/119/`
- 120th-Congress dry run impossible until 120th data exists

## Active Tasks
- Task 7.1 — Update `DOCUMENTATION.md` §8/§9/§11 for new file layout + regeneration commands + change-log
- Task 7.3 — Delete `legacy/` after user confirmation