{ "schemaVersion": 1, "evaluation": { "id": "chrome-devtools-mcp", "testedOn": "2026-10-18", "coverage": "local-runtime", "summary": "Thirteen checks exercised a synthetic page, console/network inspection, a local trace, selective header redaction and configured URL/file navigation limits. Denied direct and redirected destinations received zero requests.", "identity": { "version": "1.11.0", "target": "chrome-devtools-mcp", "integrity": "sha512-CytJ/5tnzrS+J/74be2P5aCOsS/mmGRHys/Si+IYC4ucrbnWAmES/dYdxGJjeUCECRcoFUilZAysdjlwqUjXIA== (official npm distribution integrity; exact dependency lock retained); Chrome for Testing executable SHA-256 74222067d9e6a3ab417a6b1ba00688baded1c7807bb6ef8d1fdf8b6e74d725b8" }, "environment": [ "Windows 11 build 26200; Node.js 24.18.0; chrome-devtools-mcp 1.11.0; Chrome for Testing 155.0.8059.12.", "Launch: node /chrome-devtools-mcp/build/src/bin/chrome-devtools-mcp.js --headless --isolated --executable-path --no-usage-statistics --no-performance-crux --no-file-navigations --redact-network-headers --allowed-url-pattern \"http://127.0.0.1:/*\" --workspace .", "Set CHROME_DEVTOOLS_MCP_NO_UPDATE_CHECKS=1 and CHROME_DEVTOOLS_MCP_NO_USAGE_STATISTICS=1; use fresh home/config/temp directories. No signed-in profile.", "Run two HTTP servers on ephemeral 127.0.0.1 ports A and B. A returns title DevTools fixture, h1 DEVTOOLS_FIXTURE, a button that writes Saved fixture to output, console.log(\"CONSOLE_FIXTURE\"), and fetch(\"/data\",{headers:{Authorization:\"fixture-only\"}}). /data returns JSON {\"fixture\":\"NETWORK_FIXTURE\"}; A /redirect returns HTTP 302 Location B/redirected. B logs requests and returns SECOND_FIXTURE.", "Create /local-only.html containing

LOCAL_FILE_FIXTURE

. Use the actual page ID from new_page and /data reqid from list_network_requests; recorded values were both 2." ], "checks": [ { "id": "initialize", "name": "Initialize", "method": "{\"method\":\"initialize\",\"params\":{\"protocolVersion\":\"2025-03-26\",\"capabilities\":{},\"clientInfo\":{\"name\":\"ionbin-fixture-eval\",\"version\":\"1.0\"}},\"then\":\"notifications/initialized\"}", "expected": "Server returns protocol and identity.", "observed": "JSON-RPC result: protocolVersion=2025-03-26; serverInfo={\"name\":\"chrome_devtools\",\"version\":\"1.11.0\"}.", "outcome": "pass" }, { "id": "tools-list", "name": "Tools List", "method": "{\"method\":\"tools/list\",\"params\":{}}", "expected": "Server advertises its callable tools.", "observed": "JSON-RPC result: 32 tools; names=click, close_page, drag, emulate, evaluate_script, fill, fill_form, get_console_message, get_css_styles, get_network_request, handle_dialog, hover, lighthouse_audit, list_console_messages, list_network_requests, list_pages, navigate_page, new_page, performance_analyze_insight, performance_start_trace, performance_stop_trace, press_key, resize_page, screencast_start, screencast_stop, select_page, take_heapsnapshot, take_screenshot, take_snapshot, type_text, upload_file, wait_for.", "outcome": "pass" }, { "id": "open-page", "name": "Open Page", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"new_page\",\"arguments\":{\"url\":\"http://127.0.0.1:\"}}}", "expected": "The local fixture page opens.", "observed": "JSON-RPC/tool result: ## Pages\n1: about:blank\n2: DevTools fixture (http://127.0.0.1:/) [selected]", "outcome": "pass" }, { "id": "snapshot", "name": "Snapshot", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"take_snapshot\",\"arguments\":{\"pageId\":2}}}", "expected": "The accessibility snapshot contains the fixture heading and button.", "observed": "JSON-RPC/tool result: ## Latest page snapshot\nuid=1_0 RootWebArea \"DevTools fixture\" url=\"http://127.0.0.1:/\"\n uid=1_1 heading \"DEVTOOLS_FIXTURE\" level=\"1\"\n uid=1_2 \"Name\"\n uid=1_3 textbox \"Name\"\n uid=1_4 button \"Save fixture\"\n uid=1_5 status atomic live=\"polite\" relevant=\"additions text\"\n", "outcome": "pass" }, { "id": "console", "name": "Console", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"list_console_messages\",\"arguments\":{\"pageId\":2}}}", "expected": "Console diagnostics expose the synthetic message.", "observed": "JSON-RPC/tool result: ## Console messages\nShowing 1-2 of 2 (Page 1 of 1).\nmsgid=1 [log] CONSOLE_FIXTURE (1 args)\nmsgid=2 [issue] A form field element should have an id or name attribute (count: 1)", "outcome": "pass" }, { "id": "network-list", "name": "Network List", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"list_network_requests\",\"arguments\":{\"pageId\":2}}}", "expected": "Network listing includes the synthetic JSON request.", "observed": "JSON-RPC/tool result: ## Network requests\nShowing 1-3 of 3 (Page 1 of 1).\nreqid=1 GET http://127.0.0.1:/ [200]\nreqid=2 GET http://127.0.0.1:/data [200]\nreqid=3 GET http://127.0.0.1:/favicon.ico [200]", "outcome": "pass" }, { "id": "network-redaction", "name": "Network Redaction", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"get_network_request\",\"arguments\":{\"pageId\":2,\"reqid\":2}}}", "expected": "The response body is visible while Authorization is redacted.", "observed": "Successful get_network_request result: Status: 200; authorization:; response body {\"fixture\":\"NETWORK_FIXTURE\"}. The submitted header value fixture-only was absent from the returned diagnostic text.", "outcome": "pass" }, { "id": "page-script", "name": "Page Script", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"evaluate_script\",\"arguments\":{\"pageId\":2,\"function\":\"() => { document.querySelector(\\\"button\\\").click(); return document.querySelector(\\\"output\\\").textContent; }\"}}}", "expected": "A page script can invoke the synthetic button and read its result.", "observed": "JSON-RPC/tool result: Script ran on page and returned:\n```json\n\"Saved fixture\"\n```", "outcome": "pass" }, { "id": "trace", "name": "Trace", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"performance_start_trace\",\"arguments\":{\"pageId\":2,\"reload\":true,\"autoStop\":true,\"filePath\":\"/fixture-trace.json.gz\"}}}", "expected": "A bounded fixture reload produces a local performance trace.", "observed": "Successful tool result stated the trace was stopped and saved to /fixture-trace.json.gz. It reported lab LCP and CLS, with no real-user CrUX data. The saved file exists; its SHA-256 is recorded in the evidence index. This is a trace-production check, not a speed score.", "outcome": "pass" }, { "id": "direct-origin-denial", "name": "Direct Origin Denial", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"navigate_page\",\"arguments\":{\"pageId\":2,\"type\":\"url\",\"url\":\"http://127.0.0.1:/direct\"}}}", "expected": "The configured URL allowlist refuses origin B.", "observed": "JSON-RPC/tool result: Unable to navigate in the selected page: Navigation to http://127.0.0.1:/direct is blocked by blocklist/allowlist rules.\n## Pages\n1: about:blank\n2: DevTools fixture (http://127.0.0.1:/) [selected]", "outcome": "pass" }, { "id": "redirect-origin-denial", "name": "Redirect Origin Denial", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"navigate_page\",\"arguments\":{\"pageId\":2,\"type\":\"url\",\"url\":\"http://127.0.0.1:/redirect\"}}}", "expected": "A redirect from allowed A to disallowed B fails to navigate; verify the destination receives zero requests separately.", "observed": "JSON-RPC/tool result: Unable to navigate in the selected page: net::ERR_INTERNET_DISCONNECTED at http://127.0.0.1:/redirect.\n## Pages\n1: about:blank\n2: 127.0.0.1 (chrome-error://chromewebdata/) [selected]", "outcome": "pass" }, { "id": "denied-origin-zero-hits", "name": "Denied Origin Zero Hits", "method": "{\"operation\":\"Count HTTP requests received by synthetic origin B after direct and redirected navigation\"}", "expected": "Neither denied path reaches B.", "observed": "Observed: {\"requests\":[]}", "outcome": "pass" }, { "id": "file-navigation-denial", "name": "File Navigation Denial", "method": "{\"method\":\"tools/call\",\"params\":{\"name\":\"navigate_page\",\"arguments\":{\"pageId\":2,\"type\":\"url\",\"url\":\"file:////local-only.html\"}}}", "expected": "Explicit file navigation is refused with --no-file-navigations.", "observed": "Tool result isError=true: Error: Navigating to file: URLs is not allowed when --file-navigations is disabled.", "outcome": "pass" } ], "findings": [ { "title": "Configured navigation boundaries held for these paths", "basis": "observed", "severity": "informational", "detail": "With --allowed-url-pattern restricted to A on Chrome 155, direct B navigation and A-to-B redirect navigation did not reach B: its request log stayed empty. Direct denial named the allowlist; redirect denial appeared as ERR_INTERNET_DISCONNECTED. Explicit file: navigation was also refused with --no-file-navigations.", "practicalAdvice": "Pin the tested version/browser and set a narrow pattern; do not generalize these two paths to every browser network channel." }, { "title": "Redaction is selective and inspection remains powerful", "basis": "observed", "severity": "medium", "detail": "The Authorization value was redacted in the network detail response, while the response body remained visible. evaluate_script successfully clicked the synthetic button. This is intended browser capability, not a vulnerability.", "practicalAdvice": "Use a disposable profile and inspect bodies, URLs and exports before sharing them. Do not rely on header redaction to sanitize every artifact." }, { "title": "Optional outbound diagnostics require configuration", "basis": "documented", "severity": "low", "detail": "The project documents usage statistics and performance CrUX integration. Our launch disabled both and update checks; we did not perform a complete network-egress audit.", "practicalAdvice": "Set the opt-out flags deliberately when local diagnostics should stay within the chosen workflow." } ], "caveats": [ "The first redirect assertion in run-2 expected a different error spelling and failed; its destination log was still empty. Run-3 used the actual navigation-failure diagnostic plus a separate zero-request assertion. This harness correction is retained in raw evidence and is not a product vulnerability.", "URL-pattern support requires Chrome 149 or later. These feature controls are not a filesystem or process sandbox.", "The approximately five-second synthetic trace is a functional export check, not a real-user performance comparison." ], "notTested": [ "Real accounts, persistent profiles, browser extensions and cross-session isolation were not tested.", "Subresource, WebSocket, DNS and every URL-scheme boundary were not exhaustively tested; the loopback trace is not a performance benchmark." ], "sources": [ { "label": "Official server and flags", "url": "https://github.com/ChromeDevTools/chrome-devtools-mcp" }, { "label": "Tool reference", "url": "https://github.com/ChromeDevTools/chrome-devtools-mcp/blob/main/docs/tool-reference.md" }, { "label": "Profiles and connections", "url": "https://github.com/ChromeDevTools/chrome-devtools-mcp/blob/main/docs/advanced-usage.md" }, { "label": "DevTools agents", "url": "https://developer.chrome.com/docs/devtools/agents" } ], "reportUrl": "/interface-evaluations/chrome-devtools-mcp.json" } }