The Acrux Core Team

Putting a gateway in front of your model providers buys you a lot: one endpoint for every provider, cost accounting, caching, virtual keys, and budgets. But it raises an obvious worry — am I paying for that with latency? Every request now takes an extra hop, and for a user-facing app, milliseconds matter.

So we measured it. Same model, same prompt, the same call sent three ways, and the answer is clearer than we expected: the gateway's own software adds about 26 milliseconds. Everything else you might see is network distance — and that's in your hands, not ours.

If you're building on LLMs, you eventually need four things: prompts you can change without redeploying, a reliable path to model providers, visibility into what each call did, and a way to tell whether a change helped. LangSmith and Acrux Core both address this space from different starting points. This is an honest look at where they overlap and where they differ.