Trail of Bits / plugins/differential-review/skills/differential-review/SKILL.md
Differential Security Review
A risk-focused review method connecting code changes to history, affected callers, test coverage and explicit reports.
Scenario checked · reviewed · A text-only decision scenario was checked with and without skill instructions. This is not an executed workflow or proof of effectiveness.
01 / Task fit
Where it may help.
These are intended uses indicated by the upstream material. They are not measured outcomes.
- Security-sensitive diffs
- Reviewing validation or authorization changes
02 / Setup & access
What it asks of your environment.
Requirements
- A local Git repository, baseline/head revisions, source-reading/search tools and report-file access.
- The PR path uses gh and authorized GitHub access. Local commit review does not require an account or API key.
- Examples use Bash tools such as grep, find and wc; host adaptation may be needed. Optional companion skills/subagents extend the review.
Permissions & data access
- Read repository history and source; write a Markdown report.
- Methodology examples switch baseline/head checkouts; optional workflows invoke other skills.
Instructions alone do not enforce a sandbox. The agent host, connected tools and approvals determine what the skill can actually read, change or send.
03 / Things to check
Security concerns & practical limits.
These concerns follow from the reviewed sources. They describe possible exposure or required review; they are not confirmed vulnerabilities.
Higher attention
Checkout can disturb active work
The methodology literally proposes git checkout for baseline and head. IonBin assessment: use read-only git show/diff or an isolated worktree when work is in progress.
Review before use
Scope and call-count limitations
The method narrows analysis in large repositories and demonstrates grep-based caller counts. Text matches are not a complete semantic call graph; report skipped areas and uncertainty.
Limits and nuances
- Reviewed entry point and methodology; adversarial, reporting and pattern references not exhaustively reviewed.
- Security-review instructions do not establish that resulting code is secure or that all vulnerabilities are found.
04 / Review record
What we checked. What’s still open.
Recorded review scope
- Full SKILL.md
- Full methodology.md
- Repository CC-BY-SA-4.0 license header
- Bounded evaluation; exact operations, counts, environment and limitations in the linked public report
Not tested or established
- No execution or end-to-end task evaluation performed in this source-review workstream.
- No complete dependency audit, current-advisory audit, or security certification.
- Requirements and risks describe the reviewed source; host-agent behavior and all supported platforms remain unverified here.
- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.
- The same inherited Codex session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.
- Review date
- Source revision
5c99c03fb8a9ea568432ecbcba5d86e9f4ac2bf0- Our relationship
- Explored. Investigated through the recorded review. No use or replication is claimed.
Upstream content can change after review. A repository revision identifies source scope; it does not prove that every script, dependency or runtime behavior was tested. How freshness is handled.
Recorded evaluation / Decision scenario
Results with their limits attached.
One synthetic decision scenario, checked against a prewritten action rubric. Skill-conditioned 9/9; baseline 9/9. No measured advantage or executed workflow is claimed.
- Passed
- 9
- Failed
- 0
- Total recorded
- 9
These counts refer to the cases in this report. They are not a benchmark against other skills. These are text-only decision checks, not executed tasks. Read the report for the baseline and conditioned results.
Read the evidence reportThe report provides the case definitions, observations and baseline scope. Missing baselines or untested behaviors remain limitations.
Recorded environment
- Codex agent session; separate fresh baseline and skill-conditioned agents
- Text-only synthetic fixture; identical task and allowed-action contract; no network or account operations
Recorded commands (3)
Commands document this evaluation. They are not setup instructions for your environment.
Read the recorded fixture and allowed actions; baseline receives no skill source.Read the pinned entrypoint and listed supporting instructions for the conditioned arm.Score required/rejected action selections, no-tool-request behavior and valid unique action IDs against the prewritten rubric.Findings from this evaluation
Observation
No measured advantage over baseline
Both arms selected the same scored actions: baseline 9/9, skill-conditioned 9/9. These are decision checks, not completed real-world tasks.
Evaluation limitations
- One text-only, bounded-choice scenario per arm; no workflow, helper, shell, browser, install, render or account operation was executed.
- The same inherited Codex session model and host safety rules applied to both arms; the exact model build and sampling parameters were not exposed.
- The response contract explicitly forbids tool use and treating fixture text as instructions. This cannot isolate skill-specific prompt-injection protection.
- Choices are conspicuous and the baseline reached the ceiling. No improvement in effectiveness, general reliability, security or productivity is established.
- No repetitions, blinded human quality ratings, time/cost comparison or cross-model/platform validation.
05 / Reuse & discovery
License, attribution & popularity.
CC-BY-SA-4.0
Upstream uses Creative Commons Attribution-ShareAlike 4.0. Adaptation and redistribution obligations differ from MIT; this entry supplies original commentary and links.
Why it is in this collection
Rounded installs reported by the skills.sh skill page, observed on the stated date. Not unique users, verified usage, quality, or a security score.
Popularity and inclusion do not establish quality, compatibility or security. Counts belong to their named metric and date; they are not a current ranking.
06 / Follow the evidence
Go to the original.
- Pinned skill instructions github.com
- Baseline checkout, triage, call-count and coverage method github.com
- Dated skill-level install snapshot www.skills.sh
- Upstream license or license-inspection scope github.com
IonBin’s source notes are independent assessments. Upstream authors retain their work and attribution. This page does not install or execute the skill.