What SongForgeAI is building.
Three honest columns: shipped, building, next. Updated with every meaningful deploy.
Currently on Build 4404. Every item here ties to a public commit on GitHub.
The Engineering Punch List · Tier A
Fifteen foundational discipline items, each ratcheting a real CI gate. Tier A = this quarter. Lives at the engineering punch list on GitHub — all 50 items + Tier B/C/D plans.
#1Kill explicit-any allowlist
Build 1178#2jsdom + first real renderHook test
Build 1174#3Design tokens module
Build 1175#4Bundle-size CI gate
Build 1176#5Golden-evals CI gate
Build 1177#6Kill the unchecked-index allowlist *(B1597 CLOSEOUT: cumulative 166 → 0 (100%). B1595 cleaned components/CoverArt.tsx (15 \u2014 hexToRgb signature widened to absorb 12 + RGBA pixel-stride asserts for 3); B1596 cleaned lib/genre-profile.ts (15 \u2014 ! on every GENRE_PROFILES static-key lookup since each key is guaranteed in the record); B1597 cleaned admin/lineage/page.tsx (30 \u2014 narrow-once locals in the Fruchterman-Reingold force-layout loops). Allowlist now empty. Tier-A item closed.)*
#7Kill the strict-tsc allowlist *(B1600 CLOSEOUT: cumulative 37 → 0 (100%). B1598 cleaned dashboard/page (4 dead-code orphans deleted: EvalDeleteButton + handleDeleteEval + upgrades pipeline). B1599 cleaned dashboard/SongDetail (10 violations: unused destructured hook returns + handleSunoExport orphan). B1600 cleaned forge/page (56 violations \u2014 ~40 dead imports from EXTRACT-1 factory pattern + 9 dead destructured hook fields + 2 orphan closures + cascading dead-setter cleanup). Allowlist now empty. Tier-A item closed.)*
#8Kill the console-log allowlist *(B1586 CLOSEOUT: 10 → 0. Cumulative since B1037: 251 → 0. Tier-A target hit. Final 8 calls migrated to logger.info: forge/criticalSend (2 in create-stream-plumbing), mycelial-pathways, data-intelligence/orchestrator, manhattan/creative-debt, manhattan/skill-frontier, side-effects/cover-art, side-effects/focus-group. Two legitimate exclusions documented in the script: observability/logger.ts (the canonical sink) and developer/page.tsx (a JS code snippet inside a multi-line backtick template). The exclusions are commented inline in scripts/check-console-log.ts.)*
#9Visual regression with Playwright toHaveScreenshot() (every page, every breakpoint) *(B1185: e2e/visual.spec.ts snapshots 8 public surfaces with dynamic-content masks (SongCounter, ShippingThisWeek, leaderboard, hero canvas). Per-platform baseline path (Linux is the CI source of truth). Tightened threshold 0.20 → 0.15. CI auto-uploads new baselines + diff PNGs as artifact for PR review. Single-breakpoint (1280x800) for now; mobile + tablet breakpoints land as follow-up when the desktop baseline is stable.)*
#10A11y audit with axe-core in CI, block on serious violations
Build 1179#11Storybook for every primitive (Button, Card, Badge, Input, ScoreBadge, etc.) *(B1602 deferred; **B2358 stand-up + B2372 closeout**. B2358 installed Storybook 8.6 + the first 3 stories (Button 12, Chip 10, ScoreBadge 9). B2372 shipped the remaining 7 (Disclosure, Surface, MetadataRow, NextActionPair, TrustBlock, FooterLadder, CrucibleVoiceIcons) in a single Audit 2026-05-14 A16 batch — every design-system primitive now has a story file with realistic args + variant coverage. Item flipped to closed at B2421 (the punch list lagged the actual shipment by ~50 builds; B2421 caught the drift during a punch-list audit).)*
#12Real component library (replace ad-hoc Tailwind with <Button intent="forge" size="md">) *(B1602 kickoff: Button primitive shipped \u2014 typed intent="forge|ai|outline" + size="sm|md|lg", polymorphic <button> / <Link> via href prop, loading state with spinner, leadingIcon/trailingIcon slots. First migration: dashboard BillingTab upgrade CTA (next/link \u2192 Button href=...). 11 contract tests assert intent/size class composition + canon-compliance (no off-palette utilities, no rounded-3xl, no inline gradients). Closes when 30+ btn-* call sites are migrated to Button + the design-canon gate adds a forbid-rule for raw <button className="btn-*">.)*
#13Type-generated Supabase schema (database.types.ts regenerated on migration; no hand-typed table shapes) *(B1186 prep: workflow + npm script + placeholder + freshness check shipped. CLOSEOUT B1761: four-build chain landed the criteria — B1756 typed factory createTypedServiceRoleClient() scaffold + 6 contract tests; B1757 operator ran npm run db:gen-types + types regenerated against live schema (32 tables, 4 RPCs, schema 14.4); B1758 wave 1 migration (admin/regression-check + admin/costs, launderer ceiling 13 → 11); B1761 wave 2 migration (admin/forge-metrics + admin/cost-per-song, ceiling 11 → 8, including FULL/FALLBACK dynamic-select dead-code removal). Five admin routes now use the typed client; row types narrow against the live schema; the placeholder is gone; the freshness check would fail loudly if a migration drifted the types. Item closed.)*
#14Zod runtime validation on every SSE event (no trusted casts) *(B1180: PipelineEventSchema + CrucibleEventSchema; runtime validation in withSSEStream, forge create-stream-plumbing, and crucible route. 34 schema tests covering valid variants + drift catalog. Catches typo'd type discriminators, missing required fields, mixed envelope drift (text vs message). Critical events that fail validation get replaced with a generic error envelope so the client never sees a corrupt payload.)*
#15Discriminated-union state for ForgeResult (Draft | Evaluated | Gauntleted, each with required fields) *(B1181: ForgeSongState union + deriveSongState() pure derive function in src/app/forge/song-state.ts. useSongState() hook combines useForgeSessionState + useGauntletState through deriveSongState. 20 unit tests cover every transition path including the partial-Gauntleted guard. Source-of-truth state stays in the existing hooks; consumers migrate to the union as files are touched.)*
Recent commits (auto-derived from git)
- B4404the D2b build-half enters the record — the experiment run is the open remainder.07-21
- B4403the cross-model critic panel enters the lab — one draft, two wound sources, a blind verdict per cell.07-21
- B4402the gate arming enters the record — and this deploy activates SF_GAUNTLET_GATE_AB=1.07-21
- B4401the draft-score gauntlet gate ships dark — SF_GAUNTLET_GATE_AB arms the first live A/B in the experiment registry.07-21
- B4400the cost program's probe items close against B4399 and the production gauntlet-gate rollout files as MIXED.07-21
- B4399the sizing lands and both recommended probes execute — the writer was 12% of the bill, and the program found its halving anyway.07-21
- B4398the per-task cost decomposition lands on /admin/cost-per-song — the pipeline-level split the draft-swap prize is sized from.07-21
- B4397repair-closure PROVEN — chain output is draft-source-invariant (kimi paired diff literally +0.00 vs the all-Sonnet chain), and the mechanism is selection, not erasure.07-21
- B4396D2b files — the operator's cross-model critic panel — and the gauntlet-chain build row closes against B4395.07-21
- B4395the gauntlet-chain cell lands — cheap draft + wound-fed house repair, the D2 repair-closure engine — and the Probe B verdict enters the record.07-21
- B4394working notes can never be scored as a song again — the scaffolding vocabulary learns the new generation, the lab gains a sectionless-output backstop, and the fallback flag detects the parse path instead of the preamble.07-21
- B4393prompt mode is now surfaced at the point of action — run header + start log announce PRODUCTION PROMPT vs neutral.07-21
- B4392the SF_FORGE_PROVIDER seam row flips [~] — B4390+B4391 recorded as the shipped build half, routing promotion held open for Rung 5.07-21
- B4391production-prompt cell mode lands — Probe B is now a clickable experiment.07-20
- B4390the writer-override seam lands in the forge core — dark, attribution-safe, stamp-true — and DeepSeek V4 Pro clears the full screening rule.07-20
- B4389DeepSeek V4 Pro outscores the control at 1/24th the cost — the new savings frontrunner — Terra is a cost-parity quality candidate, Luna is eliminated.07-20
- B4388KIMI misses strict promotion by half a point — CI lower bound -1.54 vs the pre-registered -1 gate — while GPT confirms a third time and GLM's parity dies under replication.07-20
- B4387the candidate-arm row flips [~] — B4386's three arms recorded as the built half, the screening run as the honest remainder.07-20
- B4386three candidate arms land — DeepSeek V4 Pro, GPT Terra, GPT Luna — registry 8 -> 11, every price verified live, zero new keys.07-20
- B4385the harness gains the replicate-variance decomposition and the Rung-2 promotion rule — the 288-cell verdict becomes one command.07-20
Auto-pulled from git log. Regenerated on every push (the snapshot is committed). Never goes stale.
Highlighted shipped (curated, last 90 days)
banned words re-enter the cross-song loop + the lexicon-free recurrence signal — Phase…
Build 4356banned words re-enter the cross-song loop + the lexicon-free recurrence signal — Phase…
the lexeme-convergence instrument + gate #92 — the tell class is now measurable.
Build 4355the lexeme-convergence instrument + gate #92 — the tell class is now measurable.
the prompt layer stops teaching jaw/sternum — 6 anatomy teaching sites stripped.
Build 4354the prompt layer stops teaching jaw/sternum — 6 anatomy teaching sites stripped.
the B4352 stamp-gap finding is FILED; B4351/52 notes reconciled.
Build 4353the B4352 stamp-gap finding is FILED; B4351/52 notes reconciled.
the B4351 test file's import path rides in — the amend staged the pre-move content.
Build 4352the B4351 test file's import path rides in — the amend staged the pre-move content.
forge-helpers.ts goes cast-free — 8 -> 0, ceiling 1013 -> 1005.
Build 4351forge-helpers.ts goes cast-free — 8 -> 0, ceiling 1013 -> 1005.
stream-phase.ts goes cast-free — 9 -> 0, ceiling 1022 -> 1013, launderer class 4 -> 3.
Build 4350stream-phase.ts goes cast-free — 9 -> 0, ceiling 1022 -> 1013, launderer class 4 -> 3.
the admin lyrics-export row boundary goes cast-free — 8 -> 0, ceiling 1030 -> 1022.
Build 4349the admin lyrics-export row boundary goes cast-free — 8 -> 0, ceiling 1030 -> 1022.
SuperBoost's winning version now persists — the saved song matches the announced best.
Build 4348SuperBoost's winning version now persists — the saved song matches the announced best.
B4346 PROGRESS-noted on the standing as-cast walk.
Build 4347B4346 PROGRESS-noted on the standing as-cast walk.
the eval route goes cast-free — 13 -> 0 on the scoring-truth path, ceiling 1043 -> 1030.
Build 4346the eval route goes cast-free — 13 -> 0 on the scoring-truth path, ceiling 1043 -> 1030.
B4344 booked with honest partials.
Build 4345B4344 booked with honest partials.
honest sitemap dates + the category-head guides + IndexNow-on-deploy + the GSC nightly…
Build 4344honest sitemap dates + the category-head guides + IndexNow-on-deploy + the GSC nightly…
B4341-B4342 booked — all 11 P1 quick-wins closed.
Build 4343B4341-B4342 booked — all 11 P1 quick-wins closed.
Actively building
DATA: Make the taste-test the homepage secondary CTA. Move from `/leaderboard` footer…
Make the taste-test the homepage secondary CTA. Move from `/leaderboard` footer to homepage hero slot.
STANDARDS: Ship `npm install @songforgeai/scoring-rubric` — pure data + helper functions,…
Ship `npm install @songforgeai/scoring-rubric` — pure data + helper functions, MIT-licensed. Every install is a citation.
DISTRIBUTION: Free tier: 100 calls/month, no card required. Get the SDK in every developer's…
Free tier: 100 calls/month, no card required. Get the SDK in every developer's hands.
VELOCITY: Continue ratchet stack expansion: cyclomatic complexity ceiling,…
Continue ratchet stack expansion: cyclomatic complexity ceiling, dependency-cycle ceiling, untested-public-API ceiling, prompt-token-budget ceiling.
Dated commitments
Five public commitments with target dates. Slipping a date isn't fatal — but it's logged, and the next Bet Review explains why. No vapor.
First real testimonial on the homepage
DistributionDue 2026-05-14
68d overdue
One named, attributed quote from a real user — not staff, not company-authored. Lands in the TestimonialsSection registry (lib/testimonials.ts); the section currently renders an honest "we don't ship fabricated testimonials" disclosure card while the registry is empty (B2025).
Founder Loom walkthrough (10 min)
DistributionDue 2026-05-09
73d overdue
Solo-operator + AI-pair tour through the forge → score → refine loop. Posted on the homepage above the fold + the about page.
Inngest Phase 2B — audio generation off SSE
VelocityDue 2026-05-31
51d overdue
Multi-step Suno API pattern routed through the queue so the user-facing forge stream is no longer blocked on audio. Phase 1 (cover-art) and 2A (focus group) already shipped in B1429 + B1550.
Track status →First external academic citation of the Lyric Scoring Standard
StandardsDue 2026-06-30
21d overdue
A music school, ISMIR paper, or third-party tool that imports @songforgeai/scoring-rubric and credits the standard by name + version. The npm package shipped at v1.1.0 in B1759.
Track status →100 npm installs of @songforgeai/scoring-rubric
DistributionDue 2026-07-31
10d out
Adoption-counter milestone. Today: 0 installs. The badge on /scoring/standard updates live; this is the first crossing of the threshold where the standard has measurable real-world reach.
Track status →
Next up
DATA: Recruit 10 paid songwriter raters at $500/quarter each. (45-day milestone.)
Recruit 10 paid songwriter raters at $500/quarter each. (45-day milestone.)
DATA: Recruit 50 paid contributors active by month 12.
Recruit 50 paid contributors active by month 12.
DATA: Hum Test corpus (B1303): once 6 months of data lands, publish first…
Hum Test corpus (B1303): once 6 months of data lands, publish first longitudinal-recall analysis.
DATA: Quarterly calibration report cadence: "Rubric v1.X was calibrated against N…
Quarterly calibration report cadence: "Rubric v1.X was calibrated against N human preferences with Y% agreement at 13+pt score gaps."
STANDARDS: Quarterly RFC cycle, public. RFC-0002 (Anti-Platitude) shipped privately at…
Quarterly RFC cycle, public. RFC-0002 (Anti-Platitude) shipped privately at B1255. Future RFCs go through 7-day public comment via `/rfc`.
STANDARDS: Academic paper route: submit to ICCC 2027 or ISMIR. The rubric + anti-inflation…
Academic paper route: submit to ICCC 2027 or ISMIR. The rubric + anti-inflation discipline + calibration corpus = a real paper.
STANDARDS: Lyric Scoring Foundation prep: when corpus reaches 10K+ entries and 3+…
Lyric Scoring Foundation prep: when corpus reaches 10K+ entries and 3+ third-party citations, file the non-profit paperwork.
DISTRIBUTION: Co-branded proposal to Suno + Udio: API tier, integration docs, OG-cards on…
Co-branded proposal to Suno + Udio: API tier, integration docs, OG-cards on shared songs that show our seal.
DISTRIBUTION: "Powered by Lyric Scoring Standard" badge: SVG + HTML snippet at…
"Powered by Lyric Scoring Standard" badge: SVG + HTML snippet at `/developer/embed`.
DISTRIBUTION: B2B sales motion: music supervisors, sync licensors, music-rights orgs.…
B2B sales motion: music supervisors, sync licensors, music-rights orgs. Reproducibility seal IS the legal-grade evidence they need.
DISTRIBUTION: Zapier/Make integration on top of the existing webhook outbound (B1274).
Zapier/Make integration on top of the existing webhook outbound (B1274).
VELOCITY: `/engineering/punch-list` public page that updates from `docs/PUNCH-LIST.md`.…
`/engineering/punch-list` public page that updates from `docs/PUNCH-LIST.md`. Recruiting tool.
VELOCITY: Codify the Deep Pass Protocol cadence: 4x/year, one major subsystem per quarter.
Codify the Deep Pass Protocol cadence: 4x/year, one major subsystem per quarter.
VELOCITY: Open-source `@songforgeai/quality-tools`: Forbidden Archive parser, audit…
Open-source `@songforgeai/quality-tools`: Forbidden Archive parser, audit scripts, ratchet checkers.
VELOCITY: Quarterly "X builds shipped, Y ratchets tightened, Z punch list items closed"…
Quarterly "X builds shipped, Y ratchets tightened, Z punch list items closed" public report.
Under consideration
Mobile-first forge UX
Current flow is responsive but desktop-first.
Songwriter-craft certification program
Pass X metrics → get a badge. B2B-adjacent and community-building.
Cluster-pillar SEO reorganization
33 songwriting guides regrouped into hub-and-spoke clusters. Current structure is flat.
Why this page exists
Most SaaS roadmaps are marketing artifacts — aspirational lists that never get updated. This one is the opposite. If something is on the shipped list, there is a commit behind it. If it moves between sections, the commit message says so.
A publicly-tracked roadmap is accountability, not a promise. If you want to lobby for something near the top of “Next up,” email me.