Independent review. IonBin has no partnership with promptfoo and was not paid for this page. Our evaluation is our own, based on public documentation and our research. Product names are trademarks of their owners and are used for identification only. How disclosure works.
A CLI and library for comparing prompts and models against your own cases. Useful for repeatable regression checks in CI; the rubric and test coverage still need human review. Documentation checked; not hands-on tested.
Use a stable dataset and save the prompt, provider configuration and rubric with each run. Model-graded assertions are useful signals; spot-check them against human judgments. source ↗
The runner can execute locally while calling the model providers you configure. Its web viewer displays results. The MCP server can run evaluations and create public sharing links; review what is shared. source ↗
Community is free; the pricing page lists custom Enterprise and On-Premise plans. Model API calls and hosting can add costs. Checked October 18, 2026. source ↗
How promptfoo connects to each AI platform: native connector or MCP, API through code the agent writes, browser control, or file upload.
| Platform | MCP / connector | API (agent-written code) | Browser | Upload / export |
|---|---|---|---|---|
| Claude (incl. Claude Code) | Not documented Claude Desktop setup is documented. A named plugin, skill or CLI does not by itself establish an MCP/native connector pairing. Verify the route and client setup. promptfoo named integration docs ↗Claude (incl. Claude Code) MCP docs ↗ | Not documented Claude (incl. Claude Code): Claude Code edits files and runs commands promptfoo publishes an API, but this pairing still needs network, authentication and execution-scope checks. promptfoo API ↗Claude (incl. Claude Code) docs ↗ | Inferred Claude Code with the Claude in Chrome extension (Chrome, Edge and other Chromium browsers). promptfoo is a web app; the pairing itself is not documented. Claude (incl. Claude Code) browser docs ↗ | Not documented Verify which side exports and which imports, supported formats and permissions. promptfoo's recorded upload/import status is yes; Claude (incl. Claude Code)'s file status is yes. These flags do not establish a file-transfer route. promptfoo file docs ↗Claude (incl. Claude Code) file docs ↗ |
| OpenAI Codex | Not documented promptfoo has an official MCP server and OpenAI Codex supports MCP, but promptfoo's docs don't name this pairing. Auth setup may differ. promptfoo MCP docs ↗OpenAI Codex MCP docs ↗ | Not documented OpenAI Codex: Edits files and runs commands locally; cloud tasks run in isolated workspaces promptfoo publishes an API, but this pairing still needs network, authentication and execution-scope checks. promptfoo API ↗OpenAI Codex docs ↗ | Inferred Only in the ChatGPT desktop app; not in the Codex CLI or IDE extension. promptfoo is a web app; the pairing itself is not documented. OpenAI Codex browser docs ↗ | Not documented Verify which side exports and which imports, supported formats and permissions. promptfoo's recorded upload/import status is yes; OpenAI Codex's file status is yes. These flags do not establish a file-transfer route. promptfoo file docs ↗OpenAI Codex file docs ↗ |
| Google Antigravity | Not documented promptfoo has an official MCP server and Google Antigravity supports MCP, but promptfoo's docs don't name this pairing. Auth setup may differ. promptfoo MCP docs ↗Google Antigravity MCP docs ↗ | Not documented Google Antigravity: Agents edit code and run tasks promptfoo publishes an API, but this pairing still needs network, authentication and execution-scope checks. promptfoo API ↗Google Antigravity docs ↗ | Inferred The /browser guide documents App and CLI support through Chrome DevTools MCP; the overview comparison conflicts on CLI availability. IDE browser features are separate. promptfoo is a web app; the pairing itself is not documented. Google Antigravity browser docs ↗ | Not documented Verify which side exports and which imports, supported formats and permissions. promptfoo's recorded upload/import status is yes; Google Antigravity's file status is yes. These flags do not establish a file-transfer route. promptfoo file docs ↗Google Antigravity file docs ↗ |
Confirmed is reserved for evidence of the specific connection route. The current named integrations include plugins, skills and CLIs, so a named platform alone does not confirm MCP or a native connector. Inferred means each side documents a relevant capability, but no document describes the pairing. Code execution still needs network and credential access; file handling still needs a compatible format and transfer direction. Not documented means the records do not establish the route. IonBin has not tested these pairings. Recorded Oct 18, 2026.
| Level | Status | Detail / source |
|---|---|---|
| Native connector / MCP | Yes | Official docs ↗ |
| API / CLI | Yes | Official docs ↗ |
| Browser / manual | No | Browser access is recorded alongside a documented programmatic route. |
| File upload / import | Yes | Official docs ↗ |
Recorded Oct 18, 2026. Setup and plan limits vary; check the vendor's docs. How levels are assigned.
An official MCP server is documented for this tool. These platforms separately document MCP client support: ChatGPT, Claude, Cursor, Gemini (Google), Google Antigravity, Grok, Grok Bot, Lindy, LM Studio, Mistral Vibe, OpenAI Codex. This inventory does not establish compatible transport, authentication or actions for a pairing. Check the connection matrix and linked sources.
Automation hubs to assess for a custom API connection: Gumloop, Make, n8n, Zapier. Verify credentials, supported requests, limits and error handling; no configured integration is implied.
No long-form review yet. Verdict is based on research, not hands-on testing.
A tracing and evaluation platform for inspecting AI application behavior and testing changes on datasets. Useful when production failures should become reviewable test cases. Documentation checked; not hands-on tested.
With your permission, Google Analytics uses cookies to measure page visits and help us improve IonBin. It stays off until you choose. We do not enable advertising tracking.
You can change this choice anytime below. Turning analytics off reloads open IonBin tabs where analytics is running. Save unfinished form text first. Privacy details.