Daily

Daily Lab · recorded replay · run agent-k2-generous-out-05 · sandbox

agent-k2-generous-out-05

Failed

the investigation ended without proposing a candidateUnder the current criteria, generation 2. This run faced generation 1 when it executed. Both verdicts are below.

unknown

01

Verdict

2 criteria generations

Criteria generation

spec f027762ab4d08b35

Failed

the investigation ended without proposing a candidate

Acceptance criteria under generation 2, how many cases each applied to, and whether it was satisfied
CriterionSatisfiedResult

Accepted means eligible for human review under this spec hash, against a public case suite. It is not evidence of production quality, and it does not establish generalisation: the cases are visible and a candidate may have been written against them.

03

The patch

against positional_v0.py

This run is the baseline itself, so there is nothing to diff against.

Reproduce this run

cd backend
EVAL_OFFLINE=1 venv/bin/python -m lab.orchestrate \
  --candidate agent-k2-generous-out-05 --tag clean
cd ../web && npm run export:lab -- --check

Source under test

unknownsha256 unknown… · unknown bytestranscribed from unknown

04

Timeline

0 attempts

    Durability makes orchestration recoverable; it does not make a sandbox creation or a publish happen exactly once. An attempt the orchestrator never saw finish is recorded as unknown-outcome rather than assumed to have failed.

    05

    Cases

    0 scored, 0 not applicable
    Recorded cases
    0real batches, replayed
    Fault-injected
    0labelled synthetic
    Correct
    0of the scored cases
    Wrong
    0see the table

    Every scored case was correct. That is what the verdict above is asserting, and nothing more.

    07

    Provenance

    What can and cannot be established
    Executed at revision
    1c97bc0483f43673fc9f31820d8187d53659a0ff+dirtyrecorded when the harness ran, not re-derived at export
    Inputs sha256
    693cacced41778accases, records, candidates and event logs
    Evaluator sha256
    277ec81521ba14d8
    Spec hash
    f027762ab4d08b35
    Execution mode
    offline-replayThe harness reads committed responses from disk and makes no network call. The candidate imports nothing beyond the standard library.
    Python
    unknown
    Model calls
    0
    Spend for this run
    $0Grading replayed committed recordings and made no inference call; `replay_spend_usd` is that, and is $0. The investigation that authored the candidate did call a model: 0 call(s) to moonshotai/kimi-k2 through the AI Gateway, 0 tokens, an amount the gateway did not fully report as the gateway reported it per call. That is in `provider_reported`.
    Recording cost
    unknown
    Sandbox limits
    python3.13, network disabled120s wall clock, none secrets. This candidate matched a committed implementation, so it ran locally and the boundary was not exercised here.

    Case suites

    backend/lab/cases/observed.json42 cases · sha256 3d7f4b4143d85ab3…backend/evals/.cache/llm via offline replay

    backend/lab/cases/synthetic.json22 cases · sha256 21c92da332a9837e…lab/build_synthetic.py — fault injection, ground truth by construction

    • Note. Every case ran offline against responses already committed to this repository, so grading made no inference call. The investigation that wrote the candidate did, and its spend is recorded under usage.backend/lab/runs/agent-k2-generous-out-05-sandbox.trace.json
    • Caution. The case suite is public. A candidate may have been written against it, so passing does not establish generalisation.backend/lab/cases/
    The full artifact, as published

    Validated against the schema in web/lib/lab/artifact.ts before it was written. Download the JSON.