How I test a model before I recommend it

Every tool on this site gets the same five tests, in the same order, with the same inputs. That is the only way a score out of 5 means anything across a free local model and a paid cloud one.

The five tests

1. Long document summary. A 40-page document with facts planted in the middle. I check whether the summary keeps the planted facts, drops detail evenly, and finishes in a reasonable time.
2. A real coding task. Not a puzzle. One of my actual jobs: a build script change, a multi-file edit, a bug that lives between two systems. I record whether it finished, how many attempts it took, and whether it touched files it was not asked about.
3. A hard reasoning question. Multi-step, with a correct answer I already know.
4. Image understanding. A screenshot of a real interface with a question about what is on it.
5. Cost for a week. Not a benchmark number. The price per million tokens, plus what one real week of my own use would have cost at that rate, in dollars.

How the score works

Out of 5, and the findings table on every review shows the raw result behind it:

• 5 does everything well, including the things I did not test for.
• 4 is good at what it is for, with a clear limit I can name.
• 3 works, but I have to stay involved.
• 2 works sometimes, and I would not build a habit on it.
• 1 does not do the job.

Half scores are for tools that sit between two of those. If a review has a half score, the findings table says which test pushed it up or down.

What I do not do

I do not score marketing claims, I do not test a tool once and call it done, and I do not take money, affiliate links or free accounts from anyone. Every review says when it was tested. If a tool changes enough to matter, the review is rerun and the date changes.