The AcruxCore Team

Laminar (lmnr) is a Y Combinator S24 project that calls itself observability purpose-built for AI agents — a Rust ingest server, ClickHouse for spans, Quickwit for search, and an OpenTelemetry SDK that instruments fifteen-plus agent frameworks from one line of code — Vercel AI SDK, Claude Agent SDK, OpenAI Agents SDK, LangChain DeepAgents, Mastra, Pydantic AI, Browser Use, Stagehand, Playwright, LiteLLM, OpenCode and OpenHands among them. It is the most differently shaped tool we have compared AcruxCore against. Its project sidebar has dashboards, traces, evaluations, datasets, labeling, a SQL editor, playgrounds, a debugger and settings — and no prompt registry anywhere. That is not an oversight; it is the product being about agent runs rather than about the calls inside them. Both projects are Apache 2.0, so this is a comparison of shape, not of who is open.

MLflow is the open-source ML/GenAI platform originally built at Databricks, and by far the largest, oldest project we've compared AcruxCore against — 27,000+ GitHub stars, 444 contributors, shipping since 2018. Its GenAI side is newer: a Prompt Registry, a tracing store, LLM-as-judge evaluation, and — the real surprise of this comparison — an AI Gateway that sits in the request path, the same design AcruxCore uses. We built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call through a gateway, inspect the trace, build a dataset, and call it from an SDK script.

Every latency number we'd published before this measured one thing: the completion call. But a real request to any of these platforms is usually two round trips — fetch or resolve a stored prompt, then complete it — and most of them were benchmarked against OpenRouter, not the provider whose name is on the model.

So this one measures the full cycle, on real OpenAI billing, across all six platforms plus both of AcruxCore's calling modes — eight paths in one interleaved run. We also ran it four independent times, because a single 100-round run against a live third-party API turned out to not be enough to trust a single number, and that's worth showing rather than hiding.

Helicone is a proxy-first LLM observability tool — you point your API base URL at it and it sits in the request path, which makes it architecturally closer to AcruxCore's own gateway than most competitors in this series. So we built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on self-hosted Helicone, then tried to run the identical sequence we've run on every platform in this series: create the prompt, send a live call, inspect the trace, build a dataset, and call it from a script. On this self-hosted build, three of those steps hit real, reproducible errors on Helicone's side before we ever got a trace. We're publishing those errors, not routing around them — that's the most useful thing we found.

Opik is Comet's open-source LLM-ops platform, and unlike some competitors we've covered it ships with genuinely no login wall when self-hosted — you docker compose up and you're working inside a real workspace immediately. We built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.

Arize Phoenix is the OTel-native tracing-and-eval project a lot of teams reach for first, so it deserves a real answer, not a table copied from two docs sites. We built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.

Langfuse is the LLM-ops platform we get compared to most often, and it deserves a real answer, not a table copied from two docs sites. So we built the same prompt — vip-support-triage, a support agent that changes tone for VIP customers and lists their open tickets — on both platforms, then ran the identical sequence on each: create the prompt, version it, send a live call, inspect the trace, build a dataset, run an experiment, and call it from an SDK script.

A gateway virtual key can be given a cache window: set cacheTtlSeconds and an exact-match repeat of a call — same model, same messages, same temperature/max_tokens/top_p/stop — is served from Postgres instead of the provider. We ran the same fixed set of prompts through it repeatedly to see what a hit is actually worth, in milliseconds and in dollars.

Your app sends one POST /gateway/chat/completions request with an OpenAI-shaped body; changing providers is a one-field edit to model. What that field hides is more interesting: underneath, Anthropic, OpenAI, and Gemini each expect a genuinely different request — a different auth header, different sampling-param nesting, and a different key for the same tool definition. We took one fixed request — a system message, a user message, and one tool definition — and traced it through all three adapters that ship today to show exactly what each one rewrites and why the gateway needs to do it at all.

Most tool comparisons are written from docs and marketing pages. We didn't do that here. We ran nine LLM-ops platforms ourselves — LangSmith, Langfuse, PromptLayer, Arize Phoenix, Opik, MLflow, Helicone, and Laminar, plus our own AcruxCore as the baseline — and did the same thing on each one: create a prompt, version it, run it live with a real model key, inspect the resulting trace, and try to build an eval. Then we wrote a small script against each platform's own SDK and ran that too, and timed every one of them against a direct call to the provider.

Tracing and monitoring — what most people mean by "LLM observability tools" — is only one of nine angles below; prompt management, evals, guardrails, and tool-calling get the same hands-on treatment.