Skip to content
All RFCs
RFC-0008acceptedopened 2026-04-26

Open Scoring Corpus contribution policy

Pins the policy for how third parties contribute scored lyrics to the open Lyric Scoring corpus. Punch List #33 entry point. Eventual goal: 1,000+ human-scored lyrics, public, used to version + calibrate the rubric across the industry.

Motivation

The Lyric Scoring Standard ships under CC BY 4.0. Anyone can re-implement against the rubric. But the corpus the rubric was calibrated against — the human-scored examples that distinguish "50 = generic" from "85 = standout" — is not yet public.

Without the corpus, two implementers of the rubric reach different scores for the same lyrics because they have different mental models of "85." With the corpus, the rubric is genuinely reproducible: the canonical anchors live at known URLs.

This RFC pins the policy for how the corpus opens.

Punch List #33 has been on the roadmap since the rubric published. This RFC is the entry point.

Proposal

What gets contributed

A corpus contribution is a tuple: ``` { lyrics: string, // full lyric, including section markers genre: string, // canonical genre slug vocalGender: 'male' | 'female' | 'neutral', scores: { // 12 metric scores 0-100 prosody: number, specificity: number, truth: number, voice: number, imagery: number, originality: number, arc: number, transcendence: number, memorability: number, craft: number, 'genre-fit': number, wholeness: number, }, composite: number, // weighted composite (computed) contributorAttribution: string, // public attribution name rubricVersion: string, // which rubric version was used scoringNotes?: string, // optional craft notes } ```

Who can contribute

Three contributor classes:

1. **Open contributors** — anyone via a /contribute form (CC BY 4.0; attribution required; no payment). Score must self-attest AND replicate within ±3pts when re-scored against the locked rubric on the service.

2. **Verified contributors** — songwriters with a public track record (commercial release, ASCAP/BMI affiliation, etc.) who have applied + been confirmed. Their scores enter the corpus immediately with verified: true.

3. **Paid corpus contributors** — songwriter-raters paid $500/quarter (45-day target from THREE-YEAR-PLAN.md).

What "in the corpus" means

  • Public + queryable at /scoring/corpus
  • Downloadable as scoring-corpus.json from @songforgeai/scoring-rubric package
  • Cited in the quarterly calibration report's reproducibility audit (RFC-0007)

Anti-gaming

  • Rubric-version pinning per entry
  • Statistical verification (replicate ±3pts per RFC-0007)
  • Source-IP rate limit: 5 contributions / IP / day for open
  • Public review window: 7 days; verified contributors can challenge with evidence

Cost containment

Open contributions cost ~$0.05 per re-score verification. Acceptance criteria assume ≥40% rejection rate.

Out of scope

  • Multi-language contributions (Punch #42; future RFC)
  • Voice-fingerprint corpus (Punch #43; different artifact)
  • Ground-truth listener-engagement scores (RFC-0006 territory)

Comment window

This RFC is open for comment until 2026-05-03. Email support@songforgeai.com with the subject `RFC-0008`.

Resolution

(Pending — will be filled in after 2026-05-03 with comments summary + accepted text. The /contribute form ships in a follow-on build after acceptance.)