skillhawk

Model watch

Re-run every skill's evals on a model you name and an older one, report the drops and draft fixes.

Improve step · Free and open source. Runs on your machine.

The command

npx skillhawk watch run skills/ --model <new> --allow-model

Output

Runs when you start it; needs --allow-model and an API key, so no sample output here.

All tools · See it in the app · Source on GitHub