Observe every trace.
Improve every answer.
Helios gives AI teams a complete view into LLM calls, agent workflows, RAG retrieval, prompt versions, evaluations, latency, and cost, so production AI systems can be debugged, measured, and improved.
Everything your AI system does,
captured in one place.
Helios is built for engineers running agents, RAG, and LLM pipelines in production. One observability console for traces, evaluations, prompts, retrieval quality, cost, and latency.
Capture every LLM call, tool call, retrieval step, and generated answer as a nested span tree with timings and metadata.
Version prompts as first-class artifacts. Diff outputs, scores, latency, and cost across model providers.
Score model output against fixed datasets with deterministic, LLM-as-judge, and code-based evaluators.
Measure retrieval hit rate, citation coverage, and missing-source analysis on production traffic.
Aggregate token usage and spend by model, prompt, environment, and customer. Set per-project budgets.
Replay agent runs from the first user input to the final response with deterministic seeds and tool inputs.
An engineering pipeline, not a dashboard.
Helios sits between your application and your model provider. Spans flow in, evaluations run continuously, regressions surface as diffs you can act on.
@helios/sdk for Python, TS, Go. One line wraps your LLM client.
OpenTelemetry-compatible spans for calls, tools, retrievers.
Append-only event log with replayable inputs and outputs.
Score against datasets: code, regex, LLM-judge, custom.
Cluster by tag, prompt version, model, customer, cost band.
Promote prompts and retrieval configs, then re-run regressions.
Every span. Every input. Every output.
Helios records the full request graph: spans, inputs, outputs, retrieved chunks, model settings, tool calls, errors, and cost breakdowns, and links them back to the prompt version that produced them.
- Nested span tree with timings
- OpenTelemetry-compatible exporter
- Full input/output payloads, redactable
- Trace-level tags, sessions, and users
Compare prompts and models against the same dataset.
Run reproducible eval suites with deterministic, LLM-as-judge, and code-based scorers. Promote the winning prompt with a single click and keep a full regression history.
Find what your retriever is missing.
Measure retrieval quality through citation coverage and missing-source analysis. Surface the top failing queries and the chunks your retriever is consistently skipping.
- what is the refund window for annual plans?0 src
- how do I rotate API keys without downtime?0 src
- is there a SOC2 type II report available?0 src
- can I export traces to datadog?0 src
One line of SDK.
A complete view of your AI system.
pip install helios-sdk import helios helios.init(api_key="hel_•••••••••") helios.trace(openai.chat.completions.create)