Daily

Case study

A safer guard. A worse score.

A count guard refused mismatched output. Recorded recall fell and unwanted results rose.

01

Shifted verdicts

August 31 scorecard traces

  • Giants' 53-man roster to include Odell Beckham

    “The article discusses a music EP, which is irrelevant to the user's interests.”

    fixture Ray · dropped at blended

    Inspect story trace
More recorded examples

  • Mirziyoyev signs decree abolishing exit visa-style registration

    “The article discusses NFL team rosters, which is not relevant to the user's interests.”

    fixture Daniel · dropped at blended · planted needle

    Inspect story trace

  • World mostly shrugs off Bessent's 'D-Day' Iran sanctions threat

    “The article discusses China's manufacturing activity…”

    fixture Frank · dropped at rank

    Inspect story trace
02

Article IDs missing

Order was the only link

The scorer requests ordered verdicts without article IDs. The parser continues assigning them after a count mismatch.

One recorded Lab case has 40 inputs and 254 verdicts.

Inspect the audited parser revision ↗

Schematic · merge point illustrative

ARTICLE SENTVERDICT APPLIED01about article 0102about article 0203about article 0304about article 0405about article 0506about article 0607about article 0708about article 0809about article 0910about article 1011about article 1112about article 12NOTHING MERGED
What the code assumes. No article identifier is sent and the model is never asked to echo an index, so this assumption is the only thing holding the pairing together.
03

Count-guard results

September 21 recorded experiment

The guard discards mismatched batches and retries. Reordered verdicts can still pass when the count matches.

Capped recall@12Higher is better
22.1%Original scorecard

Original, 22.07 percent, on a scale from zero to one hundred percent.

Unwanted rateLower is better
26.9%Original scorecard

Original, 26.94 percent, on a scale from zero to one hundred percent.

Verdicts assigned by position.

The scorer continued after count mismatches, leaving article associations unverified.

6

failed checks

Historical working-tree experiment with provisional labels; the baseline was retained.

Experiment provenance

Corpus 2026-09-02, 10 fixtures, k=12. Reported cache misses: 0 original, 0 guard.

Base revision 3b11a3c.

Inspect summary and source hashes ↗
04

Parser tests

Independent grading

Run a parser in Sandbox and inspect each failed case. Both criteria generations and their verdicts remain available. A changed scoring request needs new model recordings.

Run a parser