Jesse Vincent / Superpowers / skills/systematic-debugging/SKILL.md
Systematic Debugging
A four-stage debugging method: investigate evidence, compare patterns, test a hypothesis and verify one focused fix.
Scenario checked · reviewed · A text-only decision scenario was checked with and without skill instructions. This is not an executed workflow or proof of effectiveness.
01 / Task fit
Where it may help.
These are intended uses indicated by the upstream material. They are not measured outcomes.
- Repeated fixes that fail to address a defect
- Bugs crossing application, build or service boundaries
02 / Setup & access
What it asks of your environment.
Requirements
- A coding agent with access to relevant source, logs and the project's reproduction/test environment.
- The optional find-polluter.sh helper needs Bash, Unix utilities and npm test; the core method is not restricted to JavaScript.
- The workflow refers to the Superpowers TDD and verification skills. No skill-specific account or key is specified; project credentials may be required for a reproduction.
- Host and operating-system compatibility depends on chosen tools; the Bash helper was not tested on Windows.
Permissions & data access
- Read source, logs, configuration and revision history.
- Run reproduction/test commands and add diagnostic instrumentation; later phases modify code.
Instructions alone do not enforce a sandbox. The agent host, connected tools and approvals determine what the skill can actually read, change or send.
03 / Things to check
Security concerns & practical limits.
These concerns follow from the reviewed sources. They describe possible exposure or required review; they are not confirmed vulnerabilities.
Review before use
Diagnostic data needs boundaries
The entrypoint encourages logging data across component boundaries. IonBin assessment: redact secrets and customer data, and scope diagnostics before using it against a live service.
Review before use
Tests execute project code
find-polluter.sh invokes npm test for discovered files, suppresses test output and ignores nonzero exits while looking for a new path. A no-polluter result is not evidence that tests passed.
Limits and nuances
- The optional helper is tailored to npm test and path creation; its scope is narrower than general root-cause analysis.
- This review covers the skill and one helper, not the entire Superpowers plugin, hooks or optional visual companion.
- The broader repository documents optional visual-companion telemetry; that feature was not used or audited here.
04 / Review record
What we checked. What’s still open.
Recorded review scope
- SKILL.md four phases and diagnostics examples
- Entire find-polluter.sh helper
- README.md compatibility context and optional-telemetry disclosure
- LICENSE
- Bounded evaluation; exact operations, counts, environment and limitations in the linked public report
Not tested or established
- No effectiveness benchmark or security certification
- Transitive dependencies, current advisories and host-agent permission controls not audited
- No debugging success-rate or time-to-fix benchmark
- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.
- The same inherited Codex session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.
- Review date
- Source revision
b36e0829c6d0140e93cfef2ca599b1b07d4a7797- Our relationship
- Explored. Investigated through the recorded review. No use or replication is claimed.
Upstream content can change after review. A repository revision identifies source scope; it does not prove that every script, dependency or runtime behavior was tested. How freshness is handled.
Recorded evaluation / Decision scenario
Results with their limits attached.
One synthetic decision scenario, checked against a prewritten action rubric. Skill-conditioned 8/8; baseline 8/8. No measured advantage or executed workflow is claimed.
- Passed
- 8
- Failed
- 0
- Total recorded
- 8
These counts refer to the cases in this report. They are not a benchmark against other skills. These are text-only decision checks, not executed tasks. Read the report for the baseline and conditioned results.
Read the evidence reportThe report provides the case definitions, observations and baseline scope. Missing baselines or untested behaviors remain limitations.
Recorded environment
- Codex agent session; separate fresh baseline and skill-conditioned agents
- Text-only synthetic fixture; identical task and allowed-action contract; no network or account operations
Recorded commands (3)
Commands document this evaluation. They are not setup instructions for your environment.
Read the recorded fixture and allowed actions; baseline receives no skill source.Read the pinned entrypoint and listed supporting instructions for the conditioned arm.Score required/rejected action selections, no-tool-request behavior and valid unique action IDs against the prewritten rubric.Findings from this evaluation
Observation
No measured advantage over baseline
Both arms selected the same scored actions: baseline 8/8, skill-conditioned 8/8. These are decision checks, not completed real-world tasks.
Evaluation limitations
- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.
- The same inherited Codex session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.
- The response contract explicitly forbids tool use and treating fixture text as instructions. This cannot isolate skill-specific prompt-injection protection.
- Choices are conspicuous and the baseline reached the ceiling. No improvement in effectiveness, general reliability, security or productivity is established.
- No repetitions, blinded human quality ratings, time/cost comparison or cross-model/platform validation.
05 / Reuse & discovery
License, attribution & popularity.
MIT
Repository license names Jesse Vincent; retain the required copyright and license notices when redistributing.
Why it is in this collection
Rounded repository-wide GitHub stars observed in the public repository page; not skill installs, a quality score, or a measured trend.
301,600 repository stars (rounded)
- Scope
- Repository-wide; not individual skill usage.
- Observed
- Time window
- Single observation; no trend is established.
Popularity and inclusion do not establish quality, compatibility or security. Counts belong to their named metric and date; they are not a current ranking.
06 / Follow the evidence
Go to the original.
- Debugging method, diagnostics and related skills github.com
- Optional test-runner behavior github.com
- Plugin context and separate optional telemetry github.com
- Repository license github.com
- Repository popularity context github.com
IonBin’s source notes are independent assessments. Upstream authors retain their work and attribution. This page does not install or execute the skill.