Visual recognition · the Antilibrary

The best case: how well AI reads a tidy shelf

Prior tests used hard photos. This one hands the models the easy case — neat shelves, sharp pictures — and asks whether the tiers still separate.

Earlier in this arc we learned that on hard inputs the photograph matters more than the model, and that an OCR pre-step is dead weight for a capable reader. So we removed the difficulty. These six frames are well-organized and shot straight-on. The question: on the best case, how well do the best models read a shelf — and do the tiers converge? The hypothesis, stated up front: they cluster near-perfect, and the only real signal left is the small residual — a mis-bound author, a rare invented book, a title dropped off a flat stack.

01How the test was kept honest

This test grades Claude models among others, so the ground truth could not be model-produced. Bharat verified the books on each physical shelf by hand; the answer keys are human. The running assistant only standardized the photos, called the APIs, and scored the outputs — it never identified a book itself. Every model call was stateless, temperature 0, prompt caching off, one call per condition per image, same minimal prompt throughout. Scoring is a deterministic fuzzy string match — no model judges another model. Photos were standardized to 2000 px long-edge before any run.

02Six reading conditions

Only the model and the path vary. Four read the photo directly; two read PaddleOCR text instead, to isolate the OCR effect on a strong reader and a weak one.

03Read it yourself — every engine on every shelf

Rows are the six shelves (with the human-validated book count); columns are the six engines. Each cell shows books found of the total, tinted by how well it did. Gray −N = books missed; +N = a book fabricated (a real-sounding title not on the shelf); ?N = a spine the model honestly flagged as unreadable, kept separate from invention. Click any cell to see the exact titles it missed or made up; click a shelf to jump to its photo.

04The six shelves

Presentation varies on purpose — upright spines read sideways, flat stacks read left-to-right, and one frame spans two shelves. Some frames overlap (two shots of one shelf; one frame containing another), noted in each caption so books aren't double-counted.

05What the numbers say

Recall = of the real books, how many were found. Precision = of the books it actually claimed, how many were real — an honest “couldn’t-read” flag is an abstention, not a claim, so it no longer counts against precision. Garble = a distorted version of a real title; Made up = an invented book unlike anything on the shelf — the trust signal. Couldn’t read = a spine the model flagged as unreadable rather than guess — split out from made-up, because honest uncertainty is not fabrication.

06What we found

1. The tiers converge. The four capable direct readers all land at 93–98% recall and 97–99% precision. On a clean shelf the gap between the best closed model and an open-weight one is a handful of books, not a tier. The hypothesis holds.
2. Opus wins, narrowly; Sonnet is the value pick among closed models. Opus: 98% recall, 99% precision, best author accuracy, zero garbles, and no true invention (its lone made-up title is the subtitle of a book that is on the shelf). Sonnet matches it within a book or two at under half the cost and the lowest latency.
3. The open-weight model is the cost story. Qwen3-VL-235B reads at 93% recall / 97% precision with only 3 invented books across all six shelves — for $0.011 total, a fraction of the closed models.
4. The residual is dropped books, and they cluster — not random inventions. With recall saturated, the errors that remain are misses, and the same few recur across engines (see §06). Genuine inventions are rare and idiosyncratic — no book was made up by more than one capable model, so fabrication is not a shared, systematic failure. Once honest “can’t-read” reports are split out (§07), the true-invention counts fall further: Opus and Sonnet one each — and Opus’s is a subtitle of a book actually on the shelf.
5. GPT-5.6 reads as well but trusts itself more and is slow. Same recall band, but more garbles + inventions than Opus (incl. genuine ones like Tehran), ~7× Sonnet's latency, and the priciest direct arm.
6. OCR is worse than dead weight — it makes a good model fabricate. Feeding OCR text to Sonnet dropped recall (97→91%) and multiplied its genuine fabrications 1→6 (the raw made-up count jumps 4→18, but most of that is the model honestly refusing to name garbled OCR text) at ~12× the cost and 13× the latency of just looking at the photo. And it didn't rescue the weak reader — OCR→Qwen-7B collapsed to 23% recall.

07Patterns in the errors

With so few mistakes left, the interesting question is whether they are random or structured. Two clear patterns:

08The two-shelf frame: where the noise came in

One frame, k2-k3, accounts for most of the residual error — and it turns out to be a natural controlled experiment. It is a single photograph spanning two shelves, so its 53 books are the same physical books also shot as tight, single-shelf frames (k2 and k3). Same books, same models — only the framing differs.

On the tight single-shelf shots the capable models were near-perfect (Opus 29/29 on k2, 24/24 on k3). On the wide combined frame Opus dropped to 50/53 — and the books it lost, Reykjavik, Hungry, and Steve Kerr, are exactly the ones it had read correctly moments earlier on the tighter shots. It isn't the books; it's the frame.

The practical rule: one shelf per photo. Cramming two shelves into a frame to save shots is a false economy — it silently drops the hardest-to-read spines, and nothing in the output tells you which. Shoot tight; the models are near-perfect when you do.

09Caveats

• Six shelves, one library. Directions, not decimals. A larger, more varied set would tighten every number here.
• One prompt, one temperature. A different prompt could move any single model; the prompt was held identical so the comparison is fair, not tuned.
• Fuzzy scoring has edges. After a manual audit two artifacts were fixed (author-in-title matching; crediting a model that reads a spine the human marked unreadable). The garble-vs-invention split is still a deterministic threshold, not a human read of every miss. The made-up column is now hand-split into genuine fabrications and honest “unreadable” flags; precision is now computed against genuine claims only — an honest “unreadable” flag is an abstention, not a false positive. That lifts OCR→Sonnet 90→96% and Opus/Sonnet to 99%; the never-hedging engines (Qwen3-VL, OCR→Qwen-7B) are unchanged. Recall and garble counts are untouched.
• Prices and models drift. The cost ordering is the durable finding; the exact dollars are a snapshot.