Skip to main content

Chat, stream, and collect feedback with the SDK

What you'll build: the same gateway call you'd otherwise make with curl, but through the SDK's hub.gateway.chat() — including a streamed response, a tool-calling loop that traces itself, and reading + rating a trace afterwards. Everything here is plain Node/TypeScript or Python; nothing requires the dashboard.

npm install @acruxcoreai/sdk
import AcruxCore from '@acruxcoreai/sdk';

const hub = new AcruxCore({
  apiKey: process.env.ACRUXCORE_API_KEY,
  baseUrl: process.env.ACRUXCORE_BASE_URL,
});

1. A plain completion with hub.gateway.chat()​

hub.gateway.chat() wraps POST /gateway/chat/completions — one request, one response, no looping. It returns the assistant's text plus the gateway's own metadata for that call (which provider handled it, what it cost, whether it was a cache hit):

const result = await hub.gateway.chat({
  model: 'support-model',
  messages: [{ role: 'user', content: 'Say hi in one word.' }],
});

console.log(result.content);      // 'Hello!'
console.log(result.usage);        // { promptTokens, completionTokens, totalTokens }
console.log(result.gateway);      // { requestId, provider, model, costUsd, cache }

The gateway already records a trace for every completion it serves, so hub.gateway.chat() doesn't write a second one — the gateway metadata carries the request id if you want to correlate the call with the trace it produced.

2. Stream the response​

Call hub.gateway.stream() (Node) or hub.gateway.stream() (Python) and iterate the result instead of awaiting it — useful for piping tokens to a UI as they arrive:

for await (const chunk of await hub.gateway.stream({
  model: 'support-model',
  messages: [{ role: 'user', content: 'Count to three.' }],
})) {
  process.stdout.write(chunk.delta.content ?? '');
}

Each chunk mirrors one SSE frame from the gateway; the loop ends when the gateway sends its closing [DONE] frame.

3. Let the model call tools — with the loop traced for you​

If your prompt has tools attached, rendering it returns them ready to pass straight into the tool loop, which drives the call → run → respond round-trip until the model is done:

const rendered = await hub.prompts.render('support-agent', 'production', {
  ticket: 'Where is my order #4471?',
});

const lookupOrder = async () => ({ status: 'shipped', eta: '2 days' });

const result = await hub.gateway.runPromptWithTools(rendered, {
  clientTools: { lookup_order: lookupOrder },
});

console.log(result.content);   // final assistant text
console.log(result.traceId);   // one trace covering every round-trip + tool call

clientTools is keyed by catalog tool name and holds only the tools your process has to run — a bound tool with an http executor runs on the platform and needs no entry. Everything else the loop needs, including the model and the prompt version id for trace lineage, comes from the render.

Two other inputs exist for tools the catalog does not hold: tools for tools you declared with acrux.tool, which carry their own body — see Build and attach a tool — and toolDefs plus dispatch for raw OpenAI-shaped definitions you assembled yourself.

If the model asks for several tools in one turn, the loop runs all of them in parallel — independent tools don't wait on each other, and their results are appended in call order. So write a tool function to be safe to run concurrently.

Unlike a standalone hub.gateway.chat() call, the tool loop does auto-report a trace — by default. Each model round-trip becomes an llm span, and each tool call becomes a tool span, timed for real. The gateway can only see its own requests; it has no way to see lookup_order running on your machine, so this is the one place the SDK adds tracing curl genuinely can't. Open Observability → Traces and the returned trace id shows up with the whole chain nested under it — turn it off with trace: false (Node) / trace=False (Python) if you're already tracing this yourself.

4. Read the trace back and leave feedback​

Once you have a trace id — from the tool loop, from hub.traces.ingest(), or copied out of the dashboard — you can read it back and attach feedback without leaving your app.

One line matters here: flushing. The spans are written in the background (see section 5), so reading the trace immediately after the call can arrive before its own spans do.

await hub.gateway.flush(); // wait for the loop's spans to be written

const { trace, spans } = await hub.traces.get(result.traceId!);
console.log(trace.status, trace.totalCostUsd, trace.totalTokens);

const feedback = await hub.traces.submitFeedback({
  traceId: result.traceId!,
  rating: 1,
  label: 'resolved-in-one-turn',
});

// Correct it later — only the original author's feedback can be edited:
await hub.traces.updateFeedback({ traceId: result.traceId!, feedbackId: feedback.id, rating: -1 });

hub.traces.list() covers the list view — e.g. pulling every trace for a session:

const { data } = await hub.traces.list({ sessionId: 'tokyo-trip-plan-01', limit: 20 });

5. When traces are sent​

hub.gateway.chat() and the tool loop hand back the model's answer straight away and report their spans in the background, so tracing costs you nothing on the call itself. Traces aren't delayed to do it — an idle client sends each span as soon as it records it, and spans only group into one request while another is already in flight.

Two situations need one extra line:

// Reading traces back straight after a call
const result = await hub.gateway.chat({ model: 'gpt-4o-mini', messages });
await hub.gateway.flush(); // wait for the spans to be written
const { trace } = await hub.traces.get(result.gateway.traceId!);

// A long-running server
process.on('SIGTERM', async () => {
  await hub.gateway.close(); // flush, then stop accepting spans
  server.close();
});

A script that finishes and exits needs neither — the SDK flushes as the process winds down. It installs no signal handlers of its own, because that would stop Ctrl-C working in your application, so a process killed by a signal drops whatever was still buffered.

What's next​