How accurate is the reviewer?
A reviewer that cries wolf gets ignored. We used six well-made third-party skills (ibelick/ui-skills (MIT) as a false-positive corpus) as a false-positive corpus: skills that should grade well. Before we fixed the reviewer's rules they graded D, C, F, C, D, A; after, A, A, B, A, B, A.
| Skill | Grade before fixes | Grade after fixes |
|---|---|---|
| baseline-ui | D | A |
| fixing-accessibility | C | A |
| fixing-metadata | F | B |
| fixing-motion-performance | C | A |
| improve-ui | D | B |
| ui-skills-root | A | A |
Method and limits
Grades come from skillhawk review (A to F with line-level findings), run on the same six skills before and after fixing the reviewer's false positives. This tests false positives only. It does not measure how many real problems the reviewer misses, and six skills from one author is a small corpus. The reviewer uses deterministic heuristics, no model calls.