Matt Pocock’s skills vs plain Claude Code on 3 small tasks. /grill-me was the useful one

Matt Pocock’s skills (mattpocock/skills, MIT) are 27 Claude Code skills built around planning, testing and debugging. They were run against plain Claude Code on a tiny store project with a cart and checkout. Claude Code 2.1.312, Sonnet, skills version 1.3.2, one run per task.

bug fix (coupon discounting more on every refresh): plain $0.08 / 20 sec, /diagnosing-bugs $0.12 / 28 sec. Both fixed it and added a regression test
feature (coupon rules with expiry, min spend, per-customer limit): plain $0.08 / 20 sec, /tdd $0.32 / 90 sec. Both passed all 10 hidden tests
plan (deliberately vague coupon request): plain $0.08 / 29 sec, /grill-me $0.23 / 83 sec. 9 decisions listed vs 20 questions

So on this small project the skills didn’t make the code more correct, and they took longer and cost more. /tdd was about 4x the cost for the same result.

/diagnosing-bugs did give a slightly cleaner fix. Plain Claude worked around the bug in one function, the skill fixed the function that was actually changing the cart. But it was a 19-line file, and the skill itself said it skipped one of its steps because the bug was easy.

/grill-me was the most useful. 20 numbered questions over three rounds, each with a recommended answer, no code until you confirm. It went deeper than plain Claude on things like whether minimum spend applies before or after the discount. Plain Claude still caught a lot of it though, its one reply listed 9 decisions and flagged the same double-discount bug.

The warning: when every recommendation was accepted, a small coupon feature grew into SQLite storage, 15-minute checkout holds and a reservation system. In another run it recommended simple in-memory storage instead. So you have to actually answer its questions.

Install cost is small, about 400 extra tokens at startup with all 27 in the project.

Not tested: /improve-codebase-architecture, /to-spec, /to-tickets, /implement, the code-review skill, or anything on a big codebase, which is where the project says they matter most.

https://aiparabellum.com/matt-pocock-skills-claude-code-install-test/

JJE1992 I mean with small projects like this, quite a few of Matt Pocock's skills are overkill, which shows here. Grilling is always useful, but many other of his skills shine more in larger projects, so I don't think this is the best way to test them. 6h ago 4 likes

shinytoyrobots Yes.

The principle of testing skills more rigorously against the non-skill equivalent (and, these days, probably against equivalent purpose skills too) is a good one.

20 seconds and one run isn't going to give you any robust data though. 3h ago 1 like

Hellrazor21198 Can I ask you a question about Large projects. What do you consider large projects and when would you use the bigger skills such as wayfinder and etc. 2h ago 1 like

Working_Account4563 1% difference after many iterations and generated code over the days/months/years is the difference between a great solution and a huge headache.

Naturally, use each option wisely. But if it's something robust you are building, every improvement counts. When the big bugs are gone, it is the small bugs that slowly kill the product. 2h ago 1 like