Skip to main content

Experiments & Runs API (Phase 5 E3)

All endpoints verified working via curl against a real dev server (real Postgres, a real BullMQ worker process — apps/worker — consuming a real local Redis, and a real OpenAI account behind the gateway connection — no mocked fetch, no mocked queue). Document updated only after curl confirmation, per CLAUDE.md's API-reference rule — except the prompt-mismatch-warning field's shape, documented from source; see "Prompt-mismatch warning" below.

An experiment ties a dataset (examples to evaluate against) to a (prompt-version × model) grid to sweep. Starting a run freezes the dataset's examples into exampleSnapshot, resolves the full grid — including an automatic baseline cell per model (the prompt's production alias if one exists, otherwise its latest committed version), added whenever the experiment names a promptId and the resolved baseline version isn't already an explicit version_ids entry — and enqueues a BullMQ Flow (one finalize job with one cell job per grid × example pair). Each cell job renders the prompt version against the example's input variables and calls the gateway in-process (GatewayService.complete), so budgets, rate limits, and the cost ledger apply exactly like live traffic. The finalize job (a BullMQ Flow parent) only runs once every child cell has settled, and marks the run succeeded or failed.

Auth: requireAnyAuth (session cookie or personal/team API key) on every route, no role restriction. Team-scoped throughout.

Verified with apps/api running via npm run dev and a real, separately booted apps/worker process (node dist/index.js), both pointed at the same Redis instance so the API's enqueued jobs are actually picked up by the worker — not an in-process/inline stub.

POST /api/v1/experiments​

Creates an experiment: a dataset to evaluate, an optional prompt under test (enables the automatic baseline cell), and the explicit (version × model) grid to sweep. version_ids and models must each have at least one entry.

Two more fields exist alongside those, both optional:

  • alias (request) — which alias's version the automatic baseline cell (added when prompt_id is set and that version isn't already among the explicit version_ids) points to. Omitted, the baseline resolves to the prompt's production alias if one exists, otherwise its latest committed version. Has no effect when prompt_id is omitted, since no baseline cell is ever injected. See "Alias-based baseline" below.
  • promptMismatchWarning (response only) — a non-blocking notice returned when the dataset's examples were sourced from a prompt other than prompt_id. See "Prompt-mismatch warning" below.
curl -X POST $ACRUXCORE_BASE_URL/experiments \
  -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dataset_id": "4595c340-d7fa-4d76-977e-76b60ebefb62", "prompt_id": "173ad5cd-969c-4a04-8896-fbf625bbc2e5", "name": "E3 Task7 greeting sweep", "version_ids": ["b5327b3e-3c45-4858-ac62-1e5852279624"], "models": ["gpt-4o-mini-e3t7"]}'

Response (status 201) — runs is empty right after creation:

{
  "id": "9462c9b7-62a7-4f01-80fe-71f142cdd02b",
  "teamId": "c2e28d00-c7ed-4c27-85d9-434927282f67",
  "datasetId": "4595c340-d7fa-4d76-977e-76b60ebefb62",
  "promptId": "173ad5cd-969c-4a04-8896-fbf625bbc2e5",
  "name": "E3 Task7 greeting sweep",
  "config": { "models": ["gpt-4o-mini-e3t7"], "versionIds": ["b5327b3e-3c45-4858-ac62-1e5852279624"] },
  "createdBy": "6937e9cb-6754-4f45-aac8-a780d2e3ef57",
  "createdAt": "2026-07-07T14:26:50.940Z",
  "runs": []
}

Error responses​

Missing version_ids (status 400):

curl -X POST $ACRUXCORE_BASE_URL/experiments \
  -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dataset_id": "4595c340-d7fa-4d76-977e-76b60ebefb62", "models": ["gpt-4o-mini-e3t7"]}'
{ "error": { "code": "VALIDATION_ERROR", "message": "Required" } }

Alias-based baseline (alias field)​

By default (no alias given), the automatic baseline cell (when prompt_id is set) resolves to the prompt's production alias if one exists, otherwise its latest committed version. Passing alias points it at a different named alias's version instead — verified here against a prompt whose production alias is pinned to v1 while a staging alias was separately promoted to v2, requesting v2 explicitly plus alias: "staging":

curl -X POST $ACRUXCORE_BASE_URL/experiments \
  -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"dataset_id": "c4629544-565c-4e34-9d76-6074f525c596", "prompt_id": "e7ec9bcd-e5c1-4d28-bd48-1da7ad3c5027", "name": "Task7 alias baseline demo", "version_ids": ["06c027e3-7f6e-4b1d-8696-6868bae24f7d"], "models": ["gpt-4o-mini"], "alias": "staging"}'

Response (status 201) — alias is stored verbatim on config, alongside models/versionIds:

{
  "id": "b819389b-6422-4bce-87a4-e7901d2052e2",
  "teamId": "13a4f9a6-cd2a-47bf-b937-4e18675d6b48",
  "datasetId": "c4629544-565c-4e34-9d76-6074f525c596",
  "promptId": "e7ec9bcd-e5c1-4d28-bd48-1da7ad3c5027",
  "name": "Task7 alias baseline demo",
  "config": { "alias": "staging", "models": ["gpt-4o-mini"], "versionIds": ["06c027e3-7f6e-4b1d-8696-6868bae24f7d"] },
  "createdBy": "39fcb676-6296-4960-9388-a02cab78d9f9",
  "createdAt": "2026-08-04T09:28:05.299Z",
  "runs": []
}

alias is only consulted when POST /experiments/:id/runs resolves the automatic baseline cell (AliasesService.resolveBaselineVersion) — it has no effect on the explicit version_ids grid itself. An alias that doesn't exist on this prompt fails the request synchronously: RunsService .startRun awaits resolveGrid (which calls resolveBaselineVersion) before creating the run row, so the NotFoundError it throws propagates straight out of POST /experiments/:id/runs as a synchronous 404 — no run is ever created, and there is nothing to look up afterwards on GET /runs/:id.

Prompt-mismatch warning (promptMismatchWarning, response only)​

Not curl-verified in this environment: reproducing it needs a dataset whose examples were built via POST /datasets/from-feedback, which requires a real POST /gateway/chat/completions call against a configured LLM provider to populate a trace's sourcePromptVersionId — no provider key (OPENAI_TEST_KEY/ANTHROPIC_TEST_KEY) is configured in this environment. Documented from source instead (experiments.service.ts/experiments.types.ts on this branch): when the dataset's examples were sourced from a prompt other than prompt_id, the 201 response carries an extra top-level promptMismatchWarning object (camelCase, unlike the optimize endpoint's snake_case prompt_mismatch_warning — this endpoint's whole response is camelCase):

{
  "id": "9462c9b7-62a7-4f01-80fe-71f142cdd02b",
  "teamId": "c2e28d00-c7ed-4c27-85d9-434927282f67",
  "datasetId": "4595c340-d7fa-4d76-977e-76b60ebefb62",
  "promptId": "173ad5cd-969c-4a04-8896-fbf625bbc2e5",
  "name": "E3 Task7 greeting sweep",
  "config": { "models": ["gpt-4o-mini-e3t7"], "versionIds": ["b5327b3e-3c45-4858-ac62-1e5852279624"] },
  "createdBy": "6937e9cb-6754-4f45-aac8-a780d2e3ef57",
  "createdAt": "2026-07-07T14:26:50.940Z",
  "runs": [],
  "promptMismatchWarning": {
    "mismatchedPrompts": [
      { "promptId": "1e5e7cd3-7570-402c-97ff-e5b0c4eadfb3", "name": "support-reply", "exampleCount": 2 }
    ]
  }
}

mismatchedPrompts groups by the other prompt each mismatched example was sourced from, with exampleCount counting only that prompt's examples in this dataset. It's informational only — the experiment is created either way — and the field is omitted entirely (not null) when every example either matches prompt_id or carries no resolvable lineage (manually-added examples, or a since-deleted source version). It is only computed when prompt_id is given, since without a target prompt there is nothing to compare an example's lineage against.

POST /api/v1/experiments/:id/runs​

Starts a run: freezes the dataset's current examples, resolves the grid (here: the explicit v2 cell + an automatic production-baseline cell, since production still pointed at v1 — not among the explicit version_ids), and enqueues a BullMQ Flow. Returns immediately — the actual gateway calls happen asynchronously in the worker process.

curl -X POST $ACRUXCORE_BASE_URL/experiments/9462c9b7-62a7-4f01-80fe-71f142cdd02b/runs \
  -H "Authorization: Bearer $ACRUXCORE_API_KEY"

Response (status 202):

{ "run_id": "891710e4-1ac4-4bb7-b618-0f81e31aa291", "status": "queued" }

Error responses​

Nonexistent experiment id (status 404):

curl -X POST $ACRUXCORE_BASE_URL/experiments/00000000-0000-0000-0000-000000000000/runs \
  -H "Authorization: Bearer $ACRUXCORE_API_KEY"
{ "error": { "code": "NOT_FOUND", "message": "Experiment not found." } }

GET /api/v1/runs​

Lists the team's runs, newest first — the run history. Each row carries the dataset and prompt the run evaluated, the shape of its frozen grid, and the scores it produced, so you can find the run worth opening without fetching each one's report.

kind is "optimize" when the run swept optimizer-drafted candidates (POST /prompts/:id/optimize) and "evaluation" otherwise. avgScore, passRate and topVariantLabel are null — never 0 — until the run has at least one judge-scored result, so an unscored run is never mistaken for one that scored badly. avgScore is the mean across every scored result in the run, weighted by example (a 1-example variant does not count the same as a 100-example one), and topVariantLabel names the highest-mean variant.

Query params (all optional):

ParamTypeDefaultMeaning
statusqueued | running | succeeded | failed—Lifecycle filter
dataset_iduuid—Only runs whose experiment used this dataset
prompt_iduuid—Only runs whose experiment named this prompt
pageinteger ≥ 111-based page number
limitinteger 1–10020Rows per page
curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs

Response (status 200) — one optimize run and one evaluation run, both finished:

{
  "data": [
    {
      "id": "df727319-f64d-4ebd-8e3d-fa1b43d0f377",
      "status": "succeeded",
      "kind": "optimize",
      "experimentId": "fc90fd51-4f00-400f-b547-8a17017434dd",
      "experimentName": "optimize",
      "datasetId": "d95a4068-3415-48fe-959a-ba7cfd3d96f5",
      "datasetName": "support replies",
      "promptId": "1e5e7cd3-7570-402c-97ff-e5b0c4eadfb3",
      "promptName": "support-reply",
      "variantCount": 3,
      "modelCount": 1,
      "exampleCount": 3,
      "results": { "total": 9, "succeeded": 9, "errored": 0, "scored": 9 },
      "avgScore": 83.9,
      "passRate": 1,
      "topVariantLabel": "candidate-A",
      "startedBy": { "id": "035f4fca-ec77-47e2-b6de-755b0111c798", "name": "Run History", "email": "runhist@example.com" },
      "createdAt": "2026-08-04T03:32:57.141Z",
      "startedAt": "2026-08-04T03:33:00.093Z",
      "endedAt": "2026-08-04T03:33:08.434Z",
      "durationMs": 8341
    },
    {
      "id": "54a06825-dce7-4c53-9c4a-7115ebd91ace",
      "status": "succeeded",
      "kind": "evaluation",
      "experimentId": "01009014-6f50-41a8-9089-ddcc8aca75e9",
      "experimentName": "support reply sweep",
      "datasetId": "d95a4068-3415-48fe-959a-ba7cfd3d96f5",
      "datasetName": "support replies",
      "promptId": "1e5e7cd3-7570-402c-97ff-e5b0c4eadfb3",
      "promptName": "support-reply",
      "variantCount": 2,
      "modelCount": 1,
      "exampleCount": 3,
      "results": { "total": 6, "succeeded": 6, "errored": 0, "scored": 6 },
      "avgScore": 81.7,
      "passRate": 1,
      "topVariantLabel": "production",
      "startedBy": { "id": "035f4fca-ec77-47e2-b6de-755b0111c798", "name": "Run History", "email": "runhist@example.com" },
      "createdAt": "2026-08-04T03:32:35.362Z",
      "startedAt": "2026-08-04T03:32:35.398Z",
      "endedAt": "2026-08-04T03:32:43.461Z",
      "durationMs": 8063
    }
  ],
  "total": 2,
  "page": 1,
  "limit": 20
}

A run still in flight lists the partial numbers it has produced so far rather than nothing — polled while the run above was mid-flight, 4 of its 6 results were already scored:

{ "status": "running", "results": { "total": 6, "succeeded": 6, "errored": 0, "scored": 4 }, "avgScore": 82.5, "durationMs": null }

Paging (total is the full filtered count, not the page's length):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs?limit=1&page=2"
{ "total": 2, "page": 2, "limit": 1 }

A filter that matches nothing returns an empty page, not a 404:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs?status=failed"
{ "data": [], "total": 0, "page": 1, "limit": 20 }

Error responses​

A status outside the four lifecycle values (status 400):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs?status=cancelled"
{ "error": { "code": "VALIDATION_ERROR", "message": "Invalid enum value. Expected 'queued' | 'running' | 'succeeded' | 'failed', received 'cancelled'" } }

A limit above the 100 ceiling (status 400):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs?limit=500"
{ "error": { "code": "VALIDATION_ERROR", "message": "Number must be less than or equal to 100" } }

GET /api/v1/runs/:id​

Polled immediately after starting the run above — the real worker had not yet picked up any of the 4 cell jobs (2 grid cells × 2 dataset examples):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs/891710e4-1ac4-4bb7-b618-0f81e31aa291

Response (status 200) — in-flight, queued, zero results so far:

{
  "id": "891710e4-1ac4-4bb7-b618-0f81e31aa291",
  "experimentId": "9462c9b7-62a7-4f01-80fe-71f142cdd02b",
  "status": "queued",
  "startedAt": null,
  "endedAt": null,
  "error": null,
  "createdAt": "2026-07-07T14:27:16.920Z",
  "grid": [
    { "model": "gpt-4o-mini-e3t7", "cellKey": "v2|gpt-4o-mini-e3t7", "variantKind": "version", "variantLabel": "v2", "promptVersionId": "b5327b3e-3c45-4858-ac62-1e5852279624", "isProductionBaseline": false },
    { "model": "gpt-4o-mini-e3t7", "cellKey": "production|gpt-4o-mini-e3t7", "variantKind": "version", "variantLabel": "production", "promptVersionId": "8250ce7b-cc5f-463b-83be-3a497f3f6f0a", "isProductionBaseline": true }
  ],
  "exampleCount": 2,
  "results": { "total": 0, "succeeded": 0, "errored": 0 }
}

Polled again ~2 seconds later, same run id — the real worker process had by then dequeued and processed all 4 cell jobs (2 grid cells x 2 examples) against the real OpenAI account, and the finalize job marked it succeeded:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs/891710e4-1ac4-4bb7-b618-0f81e31aa291

Response (status 200) — succeeded, all 4 results in:

{
  "id": "891710e4-1ac4-4bb7-b618-0f81e31aa291",
  "experimentId": "9462c9b7-62a7-4f01-80fe-71f142cdd02b",
  "status": "succeeded",
  "startedAt": null,
  "endedAt": "2026-07-07T14:27:18.799Z",
  "error": null,
  "createdAt": "2026-07-07T14:27:16.920Z",
  "grid": [
    { "model": "gpt-4o-mini-e3t7", "cellKey": "v2|gpt-4o-mini-e3t7", "variantKind": "version", "variantLabel": "v2", "promptVersionId": "b5327b3e-3c45-4858-ac62-1e5852279624", "isProductionBaseline": false },
    { "model": "gpt-4o-mini-e3t7", "cellKey": "production|gpt-4o-mini-e3t7", "variantKind": "version", "variantLabel": "production", "promptVersionId": "8250ce7b-cc5f-463b-83be-3a497f3f6f0a", "isProductionBaseline": true }
  ],
  "exampleCount": 2,
  "results": { "total": 4, "succeeded": 4, "errored": 0 }
}

Note: startedAt stayed null through this observed run — the run engine does not appear to stamp it on transition to succeeded in this build; only endedAt is populated. Documented as observed, not as a defect judgment.

Drift note (re-verified 2026-07-12): isProductionBaseline on each grid entry above was missing from this doc's original example even though the live server includes it for runs started via POST /experiments/:id/runs — added back here to match observed behavior. By contrast, a run started via the optimize loop (POST /prompts/:promptId/optimize, documented in optimize.mdx) does NOT include isProductionBaseline on its grid entries (only GET /runs/:id/report's variants/cells carry that field for an optimize-originated run) — a real, observed difference between the two run-creation code paths, not a doc error.

Each of the 4 result rows landed in eval_results with real, distinct model output and its own real gateway trace_id (verified via a read-only SELECT, not part of the HTTP surface but confirming the worker really called the gateway rather than stubbing it):

v2 / Alice          -> "Hey Alice! 🌟 It's a fantastic day and I'm so excited to see you shine! 🎉"
v2 / Bob            -> "Hey there, Bob! 🎉 Get ready to seize the day and make it amazing!"
production / Alice  -> "Hello, Alice! It's wonderful to see you!"
production / Bob    -> "Hello, Bob! It's great to see you!"

Error responses​

Nonexistent run id (status 404):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs/00000000-0000-0000-0000-000000000000
{ "error": { "code": "NOT_FOUND", "message": "Run not found." } }

GET /api/v1/runs/:id/report​

Comparison report (Phase 5 E5): the full (variant x model) matrix with per-cell averages, each non-baseline cell's regression delta vs. the same-model production-baseline cell, a leaderboard, and an advisory winner. Computed on read from the run's eval_result rows — no dedicated report table. Verified against a fresh experiment (prompt "support-reply", v1 promoted to production with a vague, unconstrained system message; v2 an explicit "reply in exactly two sentences, no exclamation marks" instruction), run against 2 dataset examples whose criteria both demand the same strict two-sentence format — arranged specifically so v2 and production diverge sharply on the judge's score, rather than landing on a flat/unknown delta that wouldn't demonstrate the feature.

Note: the judge model itself is a fixed gpt-4o-mini public name (JUDGE_MODEL in judge.service.ts, overridable via EVAL_JUDGE_MODEL) — distinct from whatever public model name(s) the experiment's grid sweeps. It must be registered separately via POST /gateway/models (same as any other model) or every judge call fails with Model 'gpt-4o-mini' is not registered. (observed firsthand on a first attempt before registering it; see the cell drill-down section below for that raw error text).

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs/6e8bcaaf-8b5b-40f5-9b59-6d83eec71596/report

Response (status 200) — v2 clearly beats the production baseline: avgScore 85 vs 20, passRate 1 vs 0, deltaVsBaseline.label "improved" (+65 score points), and the winner is v2:

{
  "runId": "6e8bcaaf-8b5b-40f5-9b59-6d83eec71596",
  "status": "succeeded",
  "models": ["gpt-4o-mini-e5t3"],
  "variants": [
    { "variantKind": "version", "promptVersionId": "3bbf603b-e99e-475a-adf6-ab9edcb0ecc0", "variantLabel": "v2", "isProductionBaseline": false },
    { "variantKind": "version", "promptVersionId": "2bd8f192-d4cf-4123-9816-4d8eba9158bb", "variantLabel": "production", "isProductionBaseline": true }
  ],
  "cells": [
    {
      "cellKey": "v2|gpt-4o-mini-e5t3",
      "variantLabel": "v2",
      "model": "gpt-4o-mini-e5t3",
      "isProductionBaseline": false,
      "avgScore": 85,
      "passRate": 1,
      "exampleCount": 2,
      "scoredCount": 2,
      "unscoredCount": 0,
      "deltaVsBaseline": { "score": 65, "passRate": 1, "label": "improved" }
    },
    {
      "cellKey": "production|gpt-4o-mini-e5t3",
      "variantLabel": "production",
      "model": "gpt-4o-mini-e5t3",
      "isProductionBaseline": true,
      "avgScore": 20,
      "passRate": 0,
      "exampleCount": 2,
      "scoredCount": 2,
      "unscoredCount": 0,
      "deltaVsBaseline": null
    }
  ],
  "leaderboard": ["v2|gpt-4o-mini-e5t3", "production|gpt-4o-mini-e5t3"],
  "winner": { "cellKey": "v2|gpt-4o-mini-e5t3", "variantLabel": "v2", "model": "gpt-4o-mini-e5t3", "avgScore": 85 }
}

Error responses​

Nonexistent run id (status 404) — same message/code as GET /runs/:id:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/runs/00000000-0000-0000-0000-000000000000/report
{ "error": { "code": "NOT_FOUND", "message": "Run not found." } }

GET /api/v1/runs/:id/cells/:cellKey​

Drill-down (Phase 5 E5): one grid cell's per-example outputs, judge reasoning, and traces. cellKey is ${variantLabel}|${model} (as minted into the run's grid) and must be URL-encoded (| -> %7C) — decoded again by Express before it reaches the controller. Each example in the response also carries history: the same array frozen onto the dataset example at run-start time (see history on GET /api/v1/datasets/:id in the datasets reference), or null for a single-turn example — this is the exact conversation that was replayed ahead of input for that cell's completion, not the live (possibly since-edited) dataset value. Not separately re-curled below: it's the same frozen JSON already curl-verified on the dataset endpoint, and the run/judge wiring is covered by the real-Postgres integration test apps/api/src/evaluations/runs/run-engine.test.ts. Curled against the same run as the report above, both of its two cells:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs/6e8bcaaf-8b5b-40f5-9b59-6d83eec71596/cells/v2%7Cgpt-4o-mini-e5t3"

Response (status 200) — v2's two examples: one perfect (score 100), one partial-credit (score 70, missing the concrete next step) — both passed:

{
  "cellKey": "v2|gpt-4o-mini-e5t3",
  "variantLabel": "v2",
  "model": "gpt-4o-mini-e5t3",
  "examples": [
    {
      "exampleId": "95f4e926-c036-4fa6-881a-4453b7c3d3fd",
      "input": { "complaint": "My package arrived three days late and the box was crushed." },
      "criteria": "Response must be EXACTLY two sentences: first sentence apologizes and names the specific issue (late delivery / crushed box), second sentence states one concrete next step (e.g. replacement or refund). No exclamation marks. No greeting or sign-off.",
      "output": "I apologize for the delay and the condition of your package. Please provide your order number so we can initiate a replacement or refund for you.",
      "score": 100,
      "passed": true,
      "reason": "The output consists of exactly two sentences: the first sentence apologizes for the delay and mentions the condition of the package, while the second sentence clearly states the next step of providing the order number for a replacement or refund.",
      "traceId": "04e58fce-3fd0-4488-907e-d0a79a7cc47d",
      "judgeTraceId": "2fd53824-36bd-48f9-9550-b9ba43ae1663"
    },
    {
      "exampleId": "e4db8a4f-f530-49ac-87e7-f4f208b2c19b",
      "input": { "complaint": "I was charged twice for the same order and nobody has responded to my emails." },
      "criteria": "Response must be EXACTLY two sentences: first sentence apologizes and names the specific issue (double charge / no response to emails), second sentence states one concrete next step (e.g. refund of duplicate charge). No exclamation marks. No greeting or sign-off.",
      "output": "I apologize for the inconvenience caused by the double charge and the lack of response to your emails. Please provide your order number so that I can investigate this issue further.",
      "score": 70,
      "passed": true,
      "reason": "The output contains an apology for the double charge and the lack of response to emails, fulfilling the first sentence requirement. However, it does not provide a concrete next step, which is necessary to fully meet the criteria.",
      "traceId": "2b8607f6-d7e8-43bb-92ff-3c105efb7ed4",
      "judgeTraceId": "db82535f-3c88-478f-8ff2-489a8c2e5bb8"
    }
  ]
}

The production-baseline cell, for contrast — both examples score 20, passed: false, and the judge's reason names the exact violation (too many sentences, greeting/sign-off present):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs/6e8bcaaf-8b5b-40f5-9b59-6d83eec71596/cells/production%7Cgpt-4o-mini-e5t3"

Response (status 200) — excerpted to one example (the other is the same shape):

{
  "cellKey": "production|gpt-4o-mini-e5t3",
  "variantLabel": "production",
  "model": "gpt-4o-mini-e5t3",
  "examples": [
    {
      "exampleId": "e4db8a4f-f530-49ac-87e7-f4f208b2c19b",
      "input": { "complaint": "I was charged twice for the same order and nobody has responded to my emails." },
      "criteria": "Response must be EXACTLY two sentences: first sentence apologizes and names the specific issue (double charge / no response to emails), second sentence states one concrete next step (e.g. refund of duplicate charge). No exclamation marks. No greeting or sign-off.",
      "output": "I'm really sorry to hear about the double charge and the delay in getting a response. We certainly want to resolve this issue for you as quickly as possible. \n\nTo assist you better, could you please provide me with the following details? \n\n1. The order number.\n2. The date of the order.\n3. Any reference numbers or transaction details for both charges.\n\nOnce I have that information, I'll be able to look into it right away and help you sort this out. Thank you for your patience, and I appreciate your understanding!",
      "score": 20,
      "passed": false,
      "reason": "The output does not meet the requirement of being exactly two sentences; it contains multiple sentences and includes a greeting and sign-off, which are not allowed.",
      "traceId": "c43822d7-5562-405e-8fa0-02a7acb3995a",
      "judgeTraceId": "3e396879-160e-4f3d-9f05-95dde37ece79"
    }
  ]
}

Error responses​

Nonexistent run id (status 404) — same message/code as GET /runs/:id/report:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs/00000000-0000-0000-0000-000000000000/cells/v2%7Cgpt-4o-mini-e5t3"
{ "error": { "code": "NOT_FOUND", "message": "Run not found." } }

A cellKey that does not match any of the run's grid cells (status 200, not an error) — the run itself is real, the cell key is simply not one of its grid entries, so it returns gracefully with an empty examples array:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" "$ACRUXCORE_BASE_URL/runs/6e8bcaaf-8b5b-40f5-9b59-6d83eec71596/cells/nonexistent%7Cmodel-x"
{ "cellKey": "nonexistent|model-x", "variantLabel": "nonexistent", "model": "model-x", "examples": [] }

GET /api/v1/experiments​

Lists the team's experiments. Note: unlike the JSDoc on ExperimentDto (which says runs is "omitted (empty array) on list"), the list endpoint as observed actually DOES populate runs per experiment (newest first) — this doc reflects the real observed response, not the comment.

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/experiments

Response (status 200) — abbreviated to one experiment with its 2 runs:

{
  "data": [
    {
      "id": "9462c9b7-62a7-4f01-80fe-71f142cdd02b",
      "teamId": "c2e28d00-c7ed-4c27-85d9-434927282f67",
      "datasetId": "4595c340-d7fa-4d76-977e-76b60ebefb62",
      "promptId": "173ad5cd-969c-4a04-8896-fbf625bbc2e5",
      "name": "E3 Task7 greeting sweep",
      "config": { "models": ["gpt-4o-mini-e3t7"], "versionIds": ["b5327b3e-3c45-4858-ac62-1e5852279624"] },
      "createdBy": "6937e9cb-6754-4f45-aac8-a780d2e3ef57",
      "createdAt": "2026-07-07T14:26:50.940Z",
      "runs": [
        { "id": "891710e4-1ac4-4bb7-b618-0f81e31aa291", "experimentId": "9462c9b7-62a7-4f01-80fe-71f142cdd02b", "status": "succeeded", "startedAt": null, "endedAt": "2026-07-07T14:27:18.799Z", "error": null, "createdAt": "2026-07-07T14:27:16.920Z" },
        { "id": "0cb95fe0-5459-4287-a7b3-f8929ce3d0e1", "experimentId": "9462c9b7-62a7-4f01-80fe-71f142cdd02b", "status": "succeeded", "startedAt": null, "endedAt": "2026-07-07T14:26:59.929Z", "error": null, "createdAt": "2026-07-07T14:26:56.763Z" }
      ]
    }
  ]
}

GET /api/v1/experiments/:id​

Fetches one experiment with its runs (newest first).

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/experiments/9462c9b7-62a7-4f01-80fe-71f142cdd02b

Response (status 200) — same shape as the list entry above for this experiment.

Error responses​

Nonexistent experiment id (status 404):

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" $ACRUXCORE_BASE_URL/experiments/00000000-0000-0000-0000-000000000000
{ "error": { "code": "NOT_FOUND", "message": "Experiment not found." } }

DELETE /api/v1/experiments/:id​

Deletes an experiment and every run under it, including each run's cells and reports. The dataset it swept and any prompt version promoted out of one of its runs are separate resources and are untouched; so is any optimizer candidate the runs drafted (the candidate's run link is simply cleared).

Refused while any of the experiment's runs is still queued or running — the worker is mid-flight and would keep writing results for a row that no longer exists. Wait for the run to settle, or delete the settled runs individually with DELETE /api/v1/runs/:id.

curl -X DELETE -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  $ACRUXCORE_BASE_URL/experiments/ff661325-588f-4984-8825-6102acfe3161

Response (status 200):

{ "success": true }

Error responses​

A run is still in flight (status 409):

curl -X DELETE -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  $ACRUXCORE_BASE_URL/experiments/608693aa-e344-4c34-a5c0-12d9d18cbbb5
{
  "error": {
    "code": "RUN_IN_FLIGHT",
    "message": "This experiment has 1 run still queued or running. Wait for it to finish, then delete."
  }
}

Already deleted, nonexistent, or another team's id (status 404 — never confirms whether it exists elsewhere):

curl -X DELETE -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  $ACRUXCORE_BASE_URL/experiments/ff661325-588f-4984-8825-6102acfe3161
{ "error": { "code": "NOT_FOUND", "message": "Experiment not found." } }

DELETE /api/v1/runs/:id​

Deletes one run and every cell it produced, so its report goes with it. The parent experiment stays — a run is one execution of it — and so does any optimizer candidate the run drafted, which matters because a candidate may already have been promoted to a real prompt version.

Refused while the run is still queued or running, for the same reason as the experiment delete above.

curl -X DELETE -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  $ACRUXCORE_BASE_URL/runs/f595cade-4145-49a9-b228-aa344dad15c8

Response (status 200):

{ "success": true }

A subsequent read of the same run 404s, while its experiment still resolves:

curl -H "Authorization: Bearer $ACRUXCORE_API_KEY" \
  $ACRUXCORE_BASE_URL/runs/f595cade-4145-49a9-b228-aa344dad15c8
{ "error": { "code": "NOT_FOUND", "message": "Run not found." } }

Error responses​

The run has not finished (status 409):

{
  "error": {
    "code": "RUN_IN_FLIGHT",
    "message": "This run is still queued. Wait for it to finish, then delete it."
  }
}