Braintrust

AI evaluation and observability
Evals MCPAPI/CLIFile import

Independent review. IonBin has no partnership with Braintrust and was not paid for this page. Our evaluation is our own, based on public documentation and our research. Product names are trademarks of their owners and are used for identification only. How disclosure works.

Evaluation datasets, experiment runs and traces in one platform. Useful when a team needs to compare changes against reviewed examples. Documentation checked; not hands-on tested.

Reference entry: listed because workflows on IonBin use it. IonBin has no affiliate or paid relationship with Braintrust.

Best for
  • Versioned evaluation datasets and repeated experiments
  • Connecting an assistant to evaluation records
Not for
  • Assuming an automated score is a human acceptance decision

Top use cases

In depth

Control agent writes

MCP includes dataset edits and evaluation execution. A run can invoke code or paid providers; row deletion can be permanent. Give an agent only the access its task needs. source ↗

Review expected answers

When copying logs, an absent expected value can be populated from model output. Review that value before treating the row as a reference answer. source ↗

How to use it with AI

How Braintrust connects to each AI platform: native connector or MCP, API through code the agent writes, browser control, or file upload.

PlatformMCP / connectorAPI (agent-written code)BrowserUpload / export
Claude (incl. Claude Code) Not documented Braintrust has an official MCP server and Claude (incl. Claude Code) supports MCP, but Braintrust's docs don't name this pairing. Auth setup may differ. Braintrust MCP docs ↗Claude (incl. Claude Code) MCP docs ↗ Not documented Claude (incl. Claude Code): Claude Code edits files and runs commands Braintrust publishes an API, but this pairing still needs network, authentication and execution-scope checks. Braintrust API ↗Claude (incl. Claude Code) docs ↗ Inferred Claude Code with the Claude in Chrome extension (Chrome, Edge and other Chromium browsers). Braintrust is a web app; the pairing itself is not documented. Claude (incl. Claude Code) browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Braintrust's recorded upload/import status is yes; Claude (incl. Claude Code)'s file status is yes. These flags do not establish a file-transfer route. Braintrust file docs ↗Claude (incl. Claude Code) file docs ↗
OpenAI Codex Not documented Braintrust has an official MCP server and OpenAI Codex supports MCP, but Braintrust's docs don't name this pairing. Auth setup may differ. Braintrust MCP docs ↗OpenAI Codex MCP docs ↗ Not documented OpenAI Codex: Edits files and runs commands locally; cloud tasks run in isolated workspaces Braintrust publishes an API, but this pairing still needs network, authentication and execution-scope checks. Braintrust API ↗OpenAI Codex docs ↗ Inferred Only in the ChatGPT desktop app; not in the Codex CLI or IDE extension. Braintrust is a web app; the pairing itself is not documented. OpenAI Codex browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Braintrust's recorded upload/import status is yes; OpenAI Codex's file status is yes. These flags do not establish a file-transfer route. Braintrust file docs ↗OpenAI Codex file docs ↗
Google Antigravity Not documented Braintrust has an official MCP server and Google Antigravity supports MCP, but Braintrust's docs don't name this pairing. Auth setup may differ. Braintrust MCP docs ↗Google Antigravity MCP docs ↗ Not documented Google Antigravity: Agents edit code and run tasks Braintrust publishes an API, but this pairing still needs network, authentication and execution-scope checks. Braintrust API ↗Google Antigravity docs ↗ Inferred The /browser guide documents App and CLI support through Chrome DevTools MCP; the overview comparison conflicts on CLI availability. IDE browser features are separate. Braintrust is a web app; the pairing itself is not documented. Google Antigravity browser docs ↗ Not documented Verify which side exports and which imports, supported formats and permissions. Braintrust's recorded upload/import status is yes; Google Antigravity's file status is yes. These flags do not establish a file-transfer route. Braintrust file docs ↗Google Antigravity file docs ↗

Confirmed is reserved for evidence of the specific connection route. The current named integrations include plugins, skills and CLIs, so a named platform alone does not confirm MCP or a native connector. Inferred means each side documents a relevant capability, but no document describes the pairing. Code execution still needs network and credential access; file handling still needs a compatible format and transfer direction. Not documented means the records do not establish the route. IonBin has not tested these pairings. Recorded Oct 18, 2026.

Agent compatibility

MCPAPI/CLIFile import
LevelStatusDetail / source
Native connector / MCPYes Official docs ↗
API / CLIYes Official docs ↗
Browser / manualNo Browser access is recorded alongside a documented programmatic route.
File upload / importYes Official docs ↗

Recorded Oct 18, 2026. Setup and plan limits vary; check the vendor's docs. How levels are assigned.

Appears in these proposed stacks

Connection capabilities to compare

An official MCP server is documented for this tool. These platforms separately document MCP client support: ChatGPT, Claude, Cursor, Gemini (Google), Google Antigravity, Grok, Grok Bot, Lindy, LM Studio, Mistral Vibe, OpenAI Codex. This inventory does not establish compatible transport, authentication or actions for a pairing. Check the connection matrix and linked sources.

Automation hubs to assess for a custom API connection: Gumloop, Make, n8n, Zapier. Verify credentials, supported requests, limits and error handling; no configured integration is implied.

Workflows using this tool

Explained in the wiki

Review

No long-form review yet. Verdict is based on research, not hands-on testing.

Alternatives

Langfuse

AI evaluation and observability

A tracing and evaluation platform for inspecting AI application behavior and testing changes on datasets. Useful when production failures should become reviewable test cases. Documentation checked; not hands-on tested.

MCPAPI/CLIFile import
Evals

promptfoo

AI evaluation and observability

A CLI and library for comparing prompts and models against your own cases. Useful for repeatable regression checks in CI; the rubric and test coverage still need human review. Documentation checked; not hands-on tested.

MCPAPI/CLIFile import
Evals