Stack 10 / Evals

AI evaluation & quality control

Check an AI task against reviewed examples before changing the prompt, model or workflow.

Independent reviews. IonBin has no partnerships with the tools on this page and was not paid for these listings. Our evaluations are our own, based on public documentation and our research. Product names are trademarks of their owners and are used for identification only. How disclosure works.

Short answer

Start with real examples and a written standard for acceptable answers. Compare the current system with a candidate on the same cases, inspect the failures and keep a separate held-out set for final checks. Production errors can become new test cases after review. A passing score supports a decision; it does not guarantee correctness.

The stack · editorial pick

3 tools, one job

Paid placement never changes a stack.

01Test runner

promptfoo

Runs configured checks against candidate prompts and models.

MCPAPI/CLIFile import
02Cases and scores

Langfuse

Holds datasets and experiments; human review helps explain failures.

MCPAPI/CLIFile import
03Version history

GitHub

Keeps configuration and review decisions together through commits and pull requests.

MCPAPI/CLIFile import
How this proposed stack fits together

This is a proposed stack, not a tested integration. Start with file-based cases; add application tracing when there is a live system to observe. Keep expected answers, model outputs and reviewer decisions separate. People set the pass bar and decide which failures block a release.

Connection methods are proposed from recorded capabilities, not evidence that IonBin tested the complete workflow. Check supported actions, account access and file direction before setup. "Manual" means no documented direct route is recorded here.

Proposed data flow
  1. Langfuse→promptfoo Map approved dataset items into runner-compatible cases
    manual
  2. promptfoo→GitHub Save configuration and access-controlled result artifacts
    manual
  3. GitHub→Langfuse Review changes and add approved failure cases
    manual
IonBin’s Brightest

From IonBin

House ad · unsold slot

Put this stack to work this week

Step-by-step workflows with a starter prompt, a watch-out line and a human approval step. Browse workflows · Recent additions

This is IonBin’s own promotion. The IonBin’s Brightest slot is a paid, labeled placement; see options.

Workflows

Put this stack to work

Documentation-reviewed recipes with sources and implementation checks. No affiliate links.

Score AI drafts with two reviewersBuild an evaluation set from support ticketsRe-run your evaluation set when a model changes
Wiki

Learn the pieces

Needs, questions, models and concepts behind this stack.

AI evaluation and observability tools · ServiceData labelling and annotation · ServiceEvals (AI evaluations) · ConceptGemma 4 · ModelGolden set: reviewed evaluation examples · ConceptHow do I know if AI output is good enough to use? · QuestionRubric for scoring AI output · ConceptShould we fine-tune a model or just write better prompts? · Question
Swap-ins

Also works for this job

Alternatives for one role in the stack.

Gemini (Google)

AI assistant

Google's AI assistant with Deep Research, Canvas, Gems and file analysis. Personal paid plans raise limits and add features in Google apps. Eligible US personal accounts can connect custom MCP apps; organizational accounts have separate access rules.

Agent platform API/CLIFile import
Used by IonBin’s founder SOPs

Gmail, Drive, Docs, Sheets, Calendar and Meet on your own domain. Google's remote MCP servers (Developer Preview) let AI agents search mail and files, draft emails, upload files and schedule meetings with the user's own permissions.

MCPAPI/CLIFile import
Used by IonBin’s founder Data

ChatGPT

AI assistant

OpenAI's general-purpose AI assistant for drafting, analysis and research, with file uploads, deep research and images. Projects organize ongoing work; scheduled-task access depends on account eligibility. Plugins connect tools, while Work and Codex support longer tasks.

Agent platform API/CLIFile import
Content

Braintrust

AI evaluation and observability

Evaluation datasets, experiment runs and traces in one platform. Useful when a team needs to compare changes against reviewed examples. Documentation checked; not hands-on tested.

MCPAPI/CLIFile import
Evals

Claude

AI assistant

Anthropic's AI assistant on web, desktop and mobile, with file analysis, Projects and app connectors. Paid plans add usage and Claude Code. Every plan can add custom remote MCP connectors; Free is limited to one.

Agent platform MCPAPI/CLIFile import
SOPs

Label Studio

Data labelling

Annotation software for turning examples into reviewed labels. Match the edition and task configuration to the review process. Documentation checked; not hands-on tested.

MCPAPI/CLIFile import
Evals

OpenAI Codex

AI coding agent

OpenAI's coding agent across desktop, CLI, IDE extension and cloud. It can inspect, edit and review code; cloud tasks use their own workspaces. MCP is supported as a client, while the old Codex MCP-server mode has been removed.

Agent platform API/CLIFile import
Build

Compare at a glance

RoleToolAgent compatibilityPricingUsed by IonBin’s founder
Test runnerpromptfoo MCPAPI/CLIFile import freemium · Free Community edition; Enterprise and On-Premise by quote (checked Oct 18, 2026)—
Cases and scoresLangfuse MCPAPI/CLIFile import freemium · Free Hobby cloud plan; paid cloud plans and self-hosting options (checked Oct 18, 2026)—
Version historyGitHub MCPAPI/CLIFile import freemium · Free $0; Team $4 and Enterprise from $21 USD/user/month advertised for the first 12 months; confirm renewal terms (checked Oct 18, 2026)Yes