Daily

Daily makes
news personal.

Daily builds news editions around a reader's interests. Its parser couldn't reliably match AI scores to articles. Daily Lab tests the fixes.

Explore the Lab Read an edition

Recorded parser: count-guard-v1. Suite: 64 cases, including 22 fault-injected cases.

01 / Inside Daily Lab

Follow a parser run.

Run the default parser
01 / Parser

Start with a parser.

Daily's parser matches AI scores to articles. This count guard checks the response length.

0.00s

scope checked

01 / Parser

Start with a parser.

Daily's parser matches AI scores to articles. This count guard checks the response length.

candidate.pycount-guard-v1

02 / Sandbox

Give it a boundary.

A fresh microVM. Networking denied. The grader stays outside.

Vercel SandboxNetworking denied

03 / Tests

Put it through the cases.

Recorded responses and injected failures exercise the parser. Four probes check isolation.

64 cases22 fault-injected

  • DNS lookup fails: passed
  • HTTPS request fails: passed
  • Grader not on disk: passed
  • No credentials in env: passed
04 / Verdict

Inspect the result.

The microVM stops. An independent grader checks the parser output.

Independent grader Outside the microVM

4failed casesRejected
  • 48 correct
  • 4 failed
  • 12 not applicable
One injected case

Reordered verdicts passed the count check.

syn-positional-reordered
Try your parser
Inspect run details Events, criteria and source
  1. 0.00
    scope checked: only candidate.py is written
  2. 0.84
    creating microVM
  3. 1.07
    uploading 5 files
  4. 1.20
    running the harness
  5. 1.74
    probing isolation
    • DNS lookup fails: passed
    • HTTPS request fails: passed
    • Grader not on disk: passed
    • No credentials in env: passed
  6. 2.61
    stopping the microVM
  7. 6.77
    microVM stopped
  8. graded outside the microVM

Run date:

Criteria generation 2
f027762ab4d08b35

Times are recorded elapsed seconds.

Inspect the source record

02 / Guard experiment

A safer guard. A worse score.

Capped recall@12Higher is better
22.1%Original18.7%With guard
Unwanted rateLower is better
26.9%Original42.5%With guard
Read the investigation Inspect source hashes

Test your parser in the Lab.

Run a parser