Skip to main content

Build a Medical-Information QA Agent

What you'll build: a medical-information call-center agent that answers questions about a small set of drugs, cites every fact back to a source document, refuses off-label and personal-medical-advice questions, escalates suspected adverse events, and always answers in one typed JSON shape — never free text you have to parse and hope about.

Medical-information teams answer the same shape of question over and over: "what is this drug approved for," "can I take it with X," "is this a known side effect." The answers have to be traceable to an approved source, and the software around the model needs a machine-readable verdict — did it answer, refuse, or escalate — not a paragraph a human has to re-read to find out. This page builds exactly that: four tools that ground the model in a small, real set of drug and policy fixtures, and the gateway's response_format feature to force the final answer into one typed shape every time.

Prefer a notebook?

medical_information_qa.ipynb is this whole page as one runnable notebook, written for a first-timer. It writes the five fixtures itself, so it runs standalone; it checks up front whether your model really honours a strict schema; it verifies every citation the agent produces against the real documents; and it triggers four failure modes on purpose, including the quiet one where adding a docstring silently replaces a compliance-approved tool description. It renders on GitHub with its saved output, so you can read it through before running anything.

A synthetic drug catalog — clearly fictional, said explicitly

Everything this page's tools look up — Neuravex and Cortiblex, their prescribing information, the safety policy, and the sample inquiry records — is invented for this tutorial. Neither is a real medicine; nothing here is medical advice. This mirrors the disclaimer on the source this page's shape is inspired by (langchain-samples/medical-information-qa), which ships its own fictional "Alvexora"/"Brevamistol"/"Cardioryn" catalog for the same reason — a real drug catalog would need a real regulatory reviewer, not a tutorial author. This page invents its own two drugs and its own policy wording, in the same shape as the source's fixtures but not copied from them.

1. The fixtures and the four tools​

Two synthetic drugs, three markdown files, and four tool functions that do real, non-trivial lookups over them — no stubbed one-liners. The fixture shape (drug profiles + inquiries + a knowledge base of PI labels and policy markdown) is adapted from the langchain-samples/medical-information-qa reference project, whose data/ folder holds the canonical version of this catalog. This page uses its own fictional drugs (Neuravex, Cortiblex) instead of that repo's (Alvexora, Brevamistol, Cardioryn), so the five files below are excerpted inline — copy each into a data/ folder next to whichever script you run. (The langchain repo's files use different drug names and won't match the scripts.)

data/drugs.json — two fictional drugs, each with its approved indications, contraindication tags, and the adverse-reaction trigger terms check_safety_policy and the model's own reasoning key off of:

data/drugs.json (excerpt)
{
  "id": "NVX",
  "brand_name": "Neuravex",
  "generic_name": "vexaline hydrochloride",
  "approved_indications": [
    "Chronic diabetic peripheral neuropathic pain in adults",
    "Major depressive disorder (MDD) in adults"
  ],
  "contraindication_tags": ["mao_inhibitor_use", "narrow_angle_glaucoma_uncontrolled"],
  "ae_trigger_terms": ["suicidal thoughts", "serotonin syndrome", "severe liver injury", "blood pressure increase"],
  "source_filenames": ["neuravex-pi.md"]
}

data/neuravex-pi.md — the prescribing information search_prescribing_info searches over, with real section headers the tool cites by anchor:

data/neuravex-pi.md (excerpt)
## Approved Indications

Neuravex is approved for:

1. Chronic diabetic peripheral neuropathic pain in adults.
2. Major depressive disorder (MDD) in adults.

Neuravex is **not** approved for pediatric use in any indication, and is not
approved for generalized anxiety, exam-related anxiety, or any other anxiety
disorder at any age.

## Contraindications

- Concurrent use of a monoamine oxidase inhibitor (MAOI), or within 14 days of
  stopping one.
- Uncontrolled narrow-angle glaucoma.

data/safety-policy.md — the refusal/adverse-event/PII policy the system prompt and check_safety_policy both point at, one section per topic:

data/safety-policy.md (excerpt)
## Refusal Policy

Refuse to answer, and clearly say so, when a question:

- Asks about a use, population, or dose outside a drug's approved indications
  (off-label use) — including any pediatric question when the drug has no
  approved pediatric indication.
- Asks for individualized medical advice for a named patient's own situation
  (personal medical advice) rather than general prescribing information.

## Adverse Event Escalation Policy

Escalate immediately, before answering normally, when a question describes a
symptom or experience that matches a drug's own adverse-reaction trigger
terms — especially anything suggesting self-harm, a severe allergic reaction,
or another serious reaction. An adverse-event escalation always sets
`escalate_adverse_event: true` and directs the person to contact a healthcare
provider or emergency services, never just a normal cited answer.

The four tools, in Python (the Node port is identical logic, just JS syntax — full source in Step 5's <Tabs>). Step 3 adds one @acrux.tool line above each of these to commit them to the catalog; the function bodies do not change between here and there, so read this as the logic you are about to ship:

tools.py
import json, re
from pathlib import Path

DATA_DIR = Path(__file__).parent / "data"
STOPWORDS = {"a", "an", "and", "any", "for", "in", "is", "it", "of", "on", "or", "the", "to", "with"}

def _tokens(text: str) -> set:
    return {w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOPWORDS and len(w) > 2}

def _slugify(heading: str) -> str:
    return re.sub(r"[^a-z0-9]+", "-", heading.lower()).strip("-")

def _load_sections(filenames):
    """Split each markdown fixture into '## '-delimited (file, heading, slug, body) chunks."""
    sections = []
    for filename in filenames:
        text = (DATA_DIR / filename).read_text()
        for block in re.split(r"(?m)^## ", text)[1:]:
            heading, _, body = block.partition("\n")
            sections.append({"file": filename, "heading": heading.strip(), "slug": _slugify(heading.strip()), "body": body.strip()})
    return sections

def get_drug_profile(query: str) -> dict:
    """Look up one of the committed synthetic drugs by id, brand name, or generic name."""
    q = query.strip().lower()
    drugs = json.loads((DATA_DIR / "drugs.json").read_text())
    for drug in drugs:
        if q in (drug["id"].lower(), drug["brand_name"].lower(), drug["generic_name"].lower()):
            return drug
    return {"error": f"No drug found matching '{query}'."}

def get_inquiry(inquiry_id: str) -> dict:
    """Look up a synthetic prior medical-information inquiry record by its id."""
    inquiries = json.loads((DATA_DIR / "inquiries.json").read_text())
    for inquiry in inquiries:
        if inquiry["id"].lower() == inquiry_id.strip().lower():
            return inquiry
    return {"error": f"No inquiry found with id '{inquiry_id}'."}

def search_prescribing_info(query: str, top_k: int = 3) -> list:
    """Token-overlap search over every committed markdown PI/policy fixture.

    Scores each '## '-delimited section by how many of the query's tokens it
    shares, and returns the top matches cited as `[source: filename.md#section-slug]`.
    """
    sections = _load_sections(["neuravex-pi.md", "cortiblex-pi.md", "safety-policy.md"])
    q_tokens = _tokens(query)
    scored = [(len(q_tokens & _tokens(s["heading"] + " " + s["body"])), s) for s in sections]
    scored = [(score, s) for score, s in scored if score > 0]
    scored.sort(key=lambda pair: pair[0], reverse=True)
    return [{"source": f"{s['file']}#{s['slug']}", "snippet": s["body"][:400]} for _, s in scored[:top_k]]

_POLICY_TOPIC_SLUGS = {
    "response": "response-policy", "refusal": "refusal-policy",
    "adverse_event": "adverse-event-escalation-policy", "pii": "pii-redaction-policy",
}

def check_safety_policy(topic: str) -> dict:
    slug = _POLICY_TOPIC_SLUGS.get(topic)
    if slug is None:
        return {"error": f"Unknown policy topic '{topic}'. Valid: {list(_POLICY_TOPIC_SLUGS)}"}
    for s in _load_sections(["safety-policy.md"]):
        if s["slug"] == slug:
            return {"source": f"safety-policy.md#{slug}", "snippet": s["body"]}
    return {"error": f"Policy section '{slug}' not found."}

Run each function standalone before wiring anything into an agent — real output, over the real fixtures:

=== get_drug_profile('Neuravex') ===
{"id": "NVX", "brand_name": "Neuravex", "generic_name": "vexaline hydrochloride", ...}

=== search_prescribing_info('Cortiblex fungal infection contraindication') ===
[
  {"source": "cortiblex-pi.md#contraindications", "snippet": "- Active systemic fungal infection.\n- Administration of a live or live-attenuated vaccine within the prior 4 weeks."},
  {"source": "cortiblex-pi.md#approved-indications", "snippet": "Cortiblex is approved for:\n\n1. Short-course treatment (up to 14 days) of moderate-to-severe rheumatoid\n   arthritis flare in adults. ..."}
]

=== check_safety_policy('adverse_event') ===
{"source": "safety-policy.md#adverse-event-escalation-policy", "snippet": "Escalate immediately, before answering normally, when a question describes a\nsymptom or experience that matches a drug's own adverse-reaction trigger\nterms ..."}

The search really ranks by overlap — a query naming "Cortiblex" and "fungal infection" comes back with the contraindications section ranked first, not just whatever section happens to appear first in the file.

2. Create the prompt via the dashboard​

Open Account & keys → New key for one API key the rest of this page reuses, then export it alongside the base URL so every script below picks both up from the environment:

export ACRUXCORE_API_KEY=acx_sk_...
export ACRUXCORE_BASE_URL=https://api.acruxcore.com/api/v1

The &quot;Copy your API key&quot; dialog showing a freshly created key beginning acx_sk_, a Copy button, and a Node SDK snippet using it

Then Prompts → New prompt, named medical-information-qa:

The &quot;New prompt&quot; dialog with Name set to medical-information-qa and a description about citations, refusals, and adverse-event escalation

The system message covers exactly the four policies the tools and response_format schema below exist to enforce — citation, refusal, adverse-event escalation, and PII redaction — with a {{ question }} user message and the model bound to claude-haiku-direct:

The prompt editor with the full system prompt covering citation, refusal, adverse-event, and PII policy, and a { question } user message, model set to claude-haiku-direct

3. Create the four tools via the SDK decorator — with one deliberate exception​

These are the same four functions from Step 1 — nothing new to write. You add one @acrux.tool line above each and call tools.sync() to commit them to the catalog; the function bodies are identical to Step 1 and are elided as ... below.

Three of the tools — get_drug_profile, get_inquiry, search_prescribing_info — carry normal docstrings, so the code owns their model-facing description. check_safety_policy ships with no docstring at all, on purpose. A docstring-less function sends its schema (name and parameters) but no description, which hands ownership of the model-facing text to the dashboard — the catalog carries the existing description forward instead of clearing it.

The mechanic that makes this work

A function with no docstring sends its schema (name + parameters) but no description to the catalog. So the dashboard's wording is carried forward untouched on every sync — it's never overwritten, because the code never sends one. That single rule is what lets the code own the schema while the UI owns the description.

This is a real split teams hit in practice. A compliance reviewer owns the wording of the safety-policy tool's description without needing write access to the codebase, while the other three tools stay fully code-owned. It also means the dashboard's "defined in code" badge on check_safety_policy refers only to the schema: the next deploy commits the tool's signature, but the compliance wording rides along untouched. Code owns the schema, the UI owns the description — no tension between the two.

create_tools.py
from acruxcore import AcruxCore, acrux

@acrux.tool
async def get_drug_profile(query: str) -> dict:
    """Look up one of the team's committed synthetic drugs by id, brand name, or generic name.

    Args:
        query (str): A drug id (e.g. "NVX"), brand name (e.g. "Neuravex"), or
            generic name (e.g. "vexaline hydrochloride"). Case-insensitive.
    """
    ...  # same body as Step 1

@acrux.tool
async def get_inquiry(inquiry_id: str) -> dict:
    """Look up a synthetic prior medical-information inquiry record by its id.

    Args:
        inquiry_id (str): An inquiry id, e.g. "MIQ-101".
    """
    ...

@acrux.tool
async def search_prescribing_info(query: str) -> list:
    """Token-overlap search over every committed markdown PI/policy fixture.

    Splits each file into '## '-delimited sections, scores each section by how
    many of the query's tokens it shares, and returns up to 3 top matches cited
    as `[source: filename.md#section-slug]`.

    Args:
        query (str): Free-text search query, e.g. drug name plus topic.
    """
    ...

@acrux.tool
async def check_safety_policy(topic: str) -> dict:
    # No docstring on purpose: the model-facing description is set in the dashboard
    # (below), not in code. The code still owns this tool's *schema* (name + parameters),
    # so a sync commits the signature and sends no description, leaving the dashboard's
    # wording untouched. Code-owns-schema, UI-owns-wording is one valid way to split who
    # edits what.
    ...  # same body as Step 1

async def main():
    async with AcruxCore() as hub:
        tools = [get_drug_profile, get_inquiry, search_prescribing_info, check_safety_policy]
        for fn, r in zip(tools, await hub.tools.sync(tools)):
            print(f"{fn.__name__}: tool_id={r.tool_id} v{r.version_number} committed={r.committed}")
get_drug_profile: tool_id=262cde98-be21-4f54-a250-552550ee3b2c v1 committed=True
get_inquiry: tool_id=37d08422-c608-48d7-8d09-88a213d2b79f v1 committed=True
search_prescribing_info: tool_id=1ef2763b-a88d-488b-97fd-1606ff7da808 v1 committed=True
check_safety_policy: tool_id=6f366a23-4b59-4847-911e-dd1ff98ab313 v1 committed=True

All four exist now, check_safety_policy with a blank description. Open its tool page and set the real, compliance-approved description in the dashboard:

The &quot;New version&quot; dialog for check_safety_policy with a Description field filled with a compliance-approved policy-lookup description and a changelog note

That commits v2, tagged dashboard, and — since nothing has promoted it to production yet — needs one more click on the Aliases tab to actually go live. Once it does, this is the state before re-running tools.sync():

check_safety_policy&#39;s version list showing v1 (code, blank) and v2 (dashboard, the compliance description), v2 tagged dashboard

Re-running create_tools.py now is the real test. The description itself never gets blanked — check_safety_policy's docstring-less function always sends no description, so the dashboard's text is carried forward every time. What can still churn a new version is the schema: if the dashboard edit also enriches a parameter description the code's signature doesn't know about (as this page's first re-promote attempt did), the next sync sees a real schema diff, commits a new code-sourced version, and carries the description text forward onto it — a new version number, same live text. Once the dashboard-side schema matches what the code would generate, a sync becomes a true no-op:

check_safety_policy: tool_id=6f366a23-4b59-4847-911e-dd1ff98ab313 v4 committed=False

check_safety_policy&#39;s version list after two syncs: v4 (dashboard) still holds the compliance description and stayed live, with v1-v3 in history below it

committed=False on the second run is the actual proof: nothing moved. The compliance team's wording survived a real re-sync of the exact same docstring-less code.

4. Connect the tools to the prompt​

Open the prompt's Tools tab and use + Connect a tool from the catalog four times, once per tool. Each one saves as you pick it, in the default column, so both production and staging call all four straight away:

The prompt&#39;s Tools tab with check_safety_policy, get_drug_profile, get_inquiry and search_prescribing_info connected, all in the default column that both aliases inherit

5. Shape every answer with response_format: three real scenarios​

Every answer this agent returns has to come back as one typed JSON shape — never free text. Pick the form that fits your stack (toggle below):

  • JSON Schema dict (zero dependencies) — hand-write the OpenAI-shaped dict and pass it directly as response_format. Works everywhere, no extra packages.
  • Pydantic BaseModel (Python) — define a class with Field(description=...) hints; the SDK converts it to the same wire dict via pydantic_response_format().
  • Zod z.object() (Node) — define a schema with .describe(...) hints; the SDK converts it to the same wire dict via { zod: ... }.

All three produce identical wire behavior — the SDK normalizes the typed forms to the same JSON Schema dict the gateway already accepts. The describe/Field(description=...) hints reach the model as per-field guidance.

The disposition field is the point of the whole shape: it comes back answer, refuse_off_label, or escalate_adverse_event because the model behaves differently on each question, not because the script picks a different template.

Each scenario is one SDK call that sets both the tools and response_format together — run_tool_loop(..., tools=..., response_format=...) in Python, runToolLoop({ toolRefs, responseFormat }) in Node:

Zero dependencies — pass the OpenAI-shaped dict directly as response_format. This is what the curl tab and the Python/Node scripts below show first.

ANSWER_SCHEMA — used by all three SDK tabs
{
  "type": "object",
  "properties": {
    "disposition": {"type": "string", "enum": ["answer", "answer_with_limitations", "refuse_off_label", "refuse_personal_advice", "escalate_adverse_event"]},
    "answer": {"type": "string"},
    "safety_flags": {"type": "array", "items": {"type": "string", "enum": ["off_label", "personal_medical_advice", "adverse_event", "pii_redacted", "unsupported_claim"]}},
    "escalate_adverse_event": {"type": "boolean"},
    "pii_redacted": {"type": "boolean"},
    "redaction_notes": {"type": "array", "items": {"type": "string"}},
    "citations": {"type": "array", "items": {"type": "string"}}
  },
  "required": ["disposition", "answer", "safety_flags", "escalate_adverse_event", "pii_redacted", "redaction_notes", "citations"],
  "additionalProperties": false
}

Usage: response_format={ type: "json_schema", json_schema: { name: "medical_information_answer", schema: ANSWER_SCHEMA, strict: true } } (Node) or response_format={"type": "json_schema", ...} (Python).

Three questions, run against all three SDK tabs below:

  1. Normal cited answer — "What is Cortiblex approved to treat, and is it safe for someone with a fungal infection?"
  2. Off-label refusal — "Can I give my 10-year-old Neuravex for anxiety before her school exams?"
  3. Adverse-event escalation — "I've been having thoughts of hurting myself since I started Neuravex two weeks ago. What should I do?"

No SDK on this tab — just curl + jq + awk, so the tool loop is hand-rolled. It makes two /gateway/chat/completions calls on one trace: first a tool-gathering loop with the four tools attached, then one shaping call with response_format set and no tools. The search/policy tools are real bash + awk (no Python dependency) — the same token-overlap logic as Step 1, just shell syntax — see run_agent.sh on GitHub for the complete source.

./run_agent.sh "What is Cortiblex approved to treat, and is it safe for someone with a fungal infection?"
Question: What is Cortiblex approved to treat, and is it safe for someone with a fungal infection?
  -> get_drug_profile({"query":"Cortiblex"})
     {"id":"CTB","brand_name":"Cortiblex","generic_name":"cortisolan acetate","drug_class":"systemic corticosteroid","approved_indications":["Short-course treatment of moderate-to-severe rheumatoid arthrit
(Phase 1: 2 tool round(s), trace 54713ada-48c0-4cd6-a544-64f1b8102c28)
{
  "disposition": "answer",
  "answer": "Cortiblex (cortisolan acetate) is approved to treat:\n1. Short-course treatment of moderate-to-severe rheumatoid arthritis flare in adults [source: cortiblex-pi.md]\n2. Adjunct therapy for acute severe allergic reactions [source: cortiblex-pi.md]\n\nCortiblex is NOT safe for someone with a fungal infection. Systemic fungal infection is listed as a contraindication to Cortiblex [source: cortiblex-pi.md]. A patient with an active systemic fungal infection should not receive this medication without explicit medical evaluation and clearance from their healthcare provider, as corticosteroids can worsen fungal infections.",
  "safety_flags": [],
  "escalate_adverse_event": false,
  "pii_redacted": false,
  "redaction_notes": [],
  "citations": ["cortiblex-pi.md"]
}
./run_agent.sh "Can I give my 10-year-old Neuravex for anxiety before her school exams?"
Question: Can I give my 10-year-old Neuravex for anxiety before her school exams?
  -> check_safety_policy({"topic":"refusal"})
     {"source":"safety-policy.md#refusal-policy","snippet":"\nRefuse to answer, and clearly say so, when a question:\n\n- Asks about a use, population, or dose outside a drug's approved indications\n  (off
  -> get_drug_profile({"query":"Neuravex"})
     {"id":"NVX","brand_name":"Neuravex","generic_name":"vexaline hydrochloride","drug_class":"serotonin-norepinephrine reuptake inhibitor (SNRI)","approved_indications":["Chronic diabetic peripheral neuro
(Phase 1: 2 tool round(s), trace 64d3791f-d95e-48f4-97a4-75cd550ad82b)
{
  "disposition": "refuse_off_label",
  "answer": "I cannot provide guidance on this use. Neuravex (vexaline hydrochloride) is not approved for use in pediatric patients. Your question asks about giving Neuravex to a 10-year-old for anxiety, which is outside the drug's approved indications [source: safety-policy.md#refusal-policy]. Neuravex is approved only for chronic diabetic peripheral neuropathic pain in adults and major depressive disorder (MDD) in adults [source: neuravex-pi.md]. Any treatment decisions for a child with anxiety should be made in consultation with your child's pediatrician or a child psychiatrist who can evaluate her individual situation and recommend appropriate, evidence-based options.",
  "safety_flags": ["off_label", "personal_medical_advice"],
  "escalate_adverse_event": false,
  "pii_redacted": true,
  "redaction_notes": ["Specific patient age and relationship (10-year-old child) redacted and referred to generically as 'a child'"],
  "citations": ["safety-policy.md#refusal-policy", "neuravex-pi.md"]
}
./run_agent.sh "I've been having thoughts of hurting myself since I started Neuravex two weeks ago. What should I do?"
Question: I've been having thoughts of hurting myself since I started Neuravex two weeks ago. What should I do?
  -> check_safety_policy({"topic":"adverse_event"})
     {"source":"safety-policy.md#adverse-event-escalation-policy","snippet":"\nEscalate immediately, before answering normally, when a question describes a\nsymptom or experience that matches a drug's own
(Phase 1: 2 tool round(s), trace df867876-a1be-4dbe-ad08-c864d3d54021)
{
  "disposition": "escalate_adverse_event",
  "answer": "You are reporting thoughts of self-harm that began after starting Neuravex two weeks ago. This is a serious concern that requires immediate medical attention. Please contact your healthcare provider right away, or if you are in crisis, call the National Suicide Prevention Lifeline at 988 (US) or go to your nearest emergency room. Do not wait to discuss this with your doctor at a routine appointment. Your safety is the priority.",
  "safety_flags": ["adverse_event"],
  "escalate_adverse_event": true,
  "pii_redacted": false,
  "redaction_notes": [],
  "citations": ["source: safety-policy.md#adverse-event-escalation-policy"]
}

Three different disposition values, three real tool calls, one hand-rolled loop.

6. Inspect the trace​

The one SDK call lands both its phases in one trace — Phase 1's tool round(s) plus Phase 2's response_format call, all threaded by the same trace_id. The adverse-event scenario's trace shows 4 spans: the model's first turn (deciding to call check_safety_policy), the tool call itself, the model's own free-text closing turn from Phase 1, and Phase 2's response_format call:

Trace detail with 4 spans: an LLM call, a check_safety_policy tool call, another LLM call, and a final LLM call, all on claude-haiku-4-5-20251001

Expanding that last span shows the real input transcript (system prompt, the question, the tool call and its result, and the Phase-2 nudge) and the real output — a bare JSON object matching MedicalInformationAnswer exactly, disposition: "escalate_adverse_event", escalate_adverse_event: true, no free text around it, because response_format constrained it. Provider: anthropic confirms the direct connection Step 2 set up is what's actually enforcing the schema.

What's next​