Skip to main content

Chat, stream, and collect feedback with the SDK

What you'll build: the same gateway call you'd otherwise make with curl, but through @acruxcoreai/sdk's chat() — including a streamed response, a tool-calling loop that traces itself, and reading + rating a trace afterwards. Everything here is plain Node/TypeScript; nothing requires the dashboard.

npm install @acruxcoreai/sdk
import AcruxCore from '@acruxcoreai/sdk';

const hub = new AcruxCore({
  apiKey: process.env.ACRUXCORE_API_KEY,
  baseUrl: process.env.ACRUXCORE_BASE_URL,
});

1. A plain completion with chat()​

chat() wraps POST /gateway/chat/completions — one request, one response, no looping. It returns the assistant's text plus the gateway's own metadata for that call (which provider handled it, what it cost, whether it was a cache hit):

const result = await hub.chat({
  model: 'support-model',
  messages: [{ role: 'user', content: 'Say hi in one word.' }],
});

console.log(result.content);      // 'Hello!'
console.log(result.usage);        // { promptTokens, completionTokens, totalTokens }
console.log(result.gateway);      // { requestId, provider, model, costUsd, cache }

The gateway already records a trace for every completion it serves, so chat() doesn't write a second one — result.gateway.requestId is there if you want to correlate the call with the trace it produced.

2. Stream the response​

Pass stream: true and iterate the result instead of awaiting it — useful for piping tokens to a UI as they arrive:

for await (const chunk of await hub.chat({
  model: 'support-model',
  messages: [{ role: 'user', content: 'Count to three.' }],
  stream: true,
})) {
  process.stdout.write(chunk.delta.content ?? '');
}

Each chunk mirrors one SSE frame from the gateway; the loop ends when the gateway sends its closing [DONE] frame.

3. Let the model call tools — with the loop traced for you​

If your prompt has tools attached, renderPrompt returns them ready to pass straight into runToolLoop, which drives the call → dispatch → respond round-trip until the model is done:

const { messages, tools } = await hub.renderPrompt('support-agent', 'production', {
  ticket: 'Where is my order #4471?',
});

const dispatch = async (name: string, args: Record<string, unknown>) => {
  if (name === 'lookup_order') return { status: 'shipped', eta: '2 days' };
  throw new Error(`Unknown tool: ${name}`);
};

const result = await hub.runToolLoop({ model: 'support-model', messages, tools, dispatch });

console.log(result.content);   // final assistant text
console.log(result.traceId);   // one trace covering every round-trip + tool call

Unlike a standalone chat() call, runToolLoop does auto-report a trace — by default. Each model round-trip becomes an llm span, and each dispatch call becomes a tool span, timed for real. The gateway can only see its own requests; it has no way to see lookup_order running on your machine, so this is the one place the SDK adds tracing curl genuinely can't. Open Observability → Traces and result.traceId shows up with the whole chain nested under it — turn it off with trace: false if you're already tracing this yourself.

4. Read the trace back and leave feedback​

Once you have a traceId — from runToolLoop, from trace(), or copied out of the dashboard — you can read it back and attach feedback without leaving Node:

const { trace, spans } = await hub.getTrace(result.traceId!);
console.log(trace.status, trace.totalCostUsd, trace.totalTokens);

const feedback = await hub.submitFeedback({
  traceId: result.traceId!,
  rating: 1,
  label: 'resolved-in-one-turn',
});

// Correct it later — only the original author's feedback can be edited:
await hub.updateFeedback({ traceId: result.traceId!, feedbackId: feedback.id, rating: -1 });

listTraces() covers the list view — e.g. pulling every trace for a session:

const { data } = await hub.listTraces({ sessionId: 'tokyo-trip-plan-01', limit: 20 });

What's next​