Langfuse

AI evaluation and observability
EvalsBuild MCPAPI/CLIFile import

Independent review. IonBin has no partnership with Langfuse and was not paid for this page. Our evaluation is our own, based on public documentation and our research. Product names are trademarks of their owners and are used for identification only. How disclosure works.

A tracing and evaluation platform for inspecting AI application behavior and testing changes on datasets. Useful when production failures should become reviewable test cases. Documentation checked; not hands-on tested.

Best for
  • Tracing an instrumented AI application
  • Comparing experiments and collecting human review scores
Not for
  • Automatic proof that a source or answer is true

Top use cases

  • Inspect application traces and observations source ↗
  • Create datasets and compare experiments source ↗
  • Import input and expected-output pairs from CSV source ↗

In depth

Evaluation loop

Add reviewed production examples to a dataset, compare changes and use human scores to calibrate automated grading. Instrumentation and dataset quality determine what you can evaluate. source ↗

MCP access

The project-scoped MCP server exposes read and write operations. Its documentation gives Claude Code, Codex and Cursor configurations. Limit write tools in the client when the task only needs inspection. source ↗

Plan choice

The cloud pricing page offers a free Hobby plan with usage and history limits, alongside paid plans. Check retention, hosting and evaluator API costs for the intended workload; this record does not promise unlimited free production use. source ↗

How to use it with AI

How Langfuse connects to each AI platform: native connector or MCP, API through code the agent writes, browser control, or file upload.

PlatformMCP / connectorAPI (agent-written code)BrowserUpload / export
Claude (incl. Claude Code) Not documented Claude Code setup is documented. A named plugin, skill or CLI does not by itself establish an MCP/native connector pairing. Verify the route and client setup. Langfuse named integration docs ↗Claude (incl. Claude Code) MCP docs ↗ Not documented Claude (incl. Claude Code): Claude Code edits files and runs commands Langfuse publishes an API, but this pairing still needs network, authentication and execution-scope checks. Langfuse API ↗Claude (incl. Claude Code) docs ↗ Inferred Claude Code with the Claude in Chrome extension (Chrome, Edge and other Chromium browsers). Langfuse is a web app; the pairing itself is not documented. Claude (incl. Claude Code) browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Langfuse's recorded upload/import status is yes; Claude (incl. Claude Code)'s file status is yes. These flags do not establish a file-transfer route. Langfuse file docs ↗Claude (incl. Claude Code) file docs ↗
OpenAI Codex Not documented Codex setup is documented. A named plugin, skill or CLI does not by itself establish an MCP/native connector pairing. Verify the route and client setup. Langfuse named integration docs ↗OpenAI Codex MCP docs ↗ Not documented OpenAI Codex: Edits files and runs commands locally; cloud tasks run in isolated workspaces Langfuse publishes an API, but this pairing still needs network, authentication and execution-scope checks. Langfuse API ↗OpenAI Codex docs ↗ Inferred Only in the ChatGPT desktop app; not in the Codex CLI or IDE extension. Langfuse is a web app; the pairing itself is not documented. OpenAI Codex browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Langfuse's recorded upload/import status is yes; OpenAI Codex's file status is yes. These flags do not establish a file-transfer route. Langfuse file docs ↗OpenAI Codex file docs ↗
Google Antigravity Not documented Langfuse has an official MCP server and Google Antigravity supports MCP, but Langfuse's docs don't name this pairing. Auth setup may differ. Langfuse MCP docs ↗Google Antigravity MCP docs ↗ Not documented Google Antigravity: Agents edit code and run tasks Langfuse publishes an API, but this pairing still needs network, authentication and execution-scope checks. Langfuse API ↗Google Antigravity docs ↗ Inferred The /browser guide documents App and CLI support through Chrome DevTools MCP; the overview comparison conflicts on CLI availability. IDE browser features are separate. Langfuse is a web app; the pairing itself is not documented. Google Antigravity browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Langfuse's recorded upload/import status is yes; Google Antigravity's file status is yes. These flags do not establish a file-transfer route. Langfuse file docs ↗Google Antigravity file docs ↗

Confirmed is reserved for evidence of the specific connection route. The current named integrations include plugins, skills and CLIs, so a named platform alone does not confirm MCP or a native connector. Inferred means each side documents a relevant capability, but no document describes the pairing. Code execution still needs network and credential access; file handling still needs a compatible format and transfer direction. Not documented means the records do not establish the route. IonBin has not tested these pairings. Recorded Oct 18, 2026.

Agent compatibility

MCPAPI/CLIFile import
LevelStatusDetail / source
Native connector / MCPYes Official docs ↗
API / CLIYes Official docs ↗
Browser / manualNo Browser access is recorded alongside a documented programmatic route.
File upload / importYes Official docs ↗

Recorded Oct 18, 2026. Setup and plan limits vary; check the vendor's docs. How levels are assigned.

Appears in these proposed stacks

Connection capabilities to compare

An official MCP server is documented for this tool. These platforms separately document MCP client support: ChatGPT, Claude, Cursor, Gemini (Google), Google Antigravity, Grok, Grok Bot, Lindy, LM Studio, Mistral Vibe, OpenAI Codex. This inventory does not establish compatible transport, authentication or actions for a pairing. Check the connection matrix and linked sources.

Automation hubs to assess for a custom API connection: Gumloop, Make, n8n, Zapier. Verify credentials, supported requests, limits and error handling; no configured integration is implied.

Workflows using this tool

Explained in the wiki

Review

No long-form review yet. Verdict is based on research, not hands-on testing.

Alternatives

promptfoo

AI evaluation and observability

A CLI and library for comparing prompts and models against your own cases. Useful for repeatable regression checks in CI; the rubric and test coverage still need human review. Documentation checked; not hands-on tested.

MCPAPI/CLIFile import
Evals