Antilibrary · Sangrah · spine-OCR test
Reading a bookshelf into an antilibrary
One shelf, thirteen books, two architectures. Can a machine turn a photo of book spines into a clean, identified list — and what is the cheapest honest way to do it?
01Perception is easy; assembly is the hard part
A pure local OCR model (PaddleOCR) read the spines almost perfectly — titles and authors at ~0.99 confidence. But it returned 82 scrambled fragments, not 13 books: the frame auto-rotated, fragments from adjacent spines interleaved, and a barcode and publisher logos came along for the ride. Reading the characters is trivial. Regrouping them into books is the real work — and that is the job we handed to a language model.
02Does the assembly step need a frontier model?
We fed the identical OCR fragments to a size ladder of ten models and scored each against the answer key. Green = book correctly identified. Watch the bottom two rows together: recall (books found) is meaningless without precision (share of outputs that were real).
| Book (13) | Claude Sonnet-5cloud reference100%reference | Mistral Small24B100% | Qwen330B · a3b MoE76%best local | Qwen314B76% | Gemma 312B76% | Llama 3.211B53% | Qwen38B38% | Qwen 2.57B30% | Gemma 34B15% | Llama 3.23B7% |
|---|---|---|---|---|---|---|---|---|---|---|
| Inside the Outbreaks | ✓ | ✓ | ✓ | · | · | ✓ | · | · | · | · |
| Happy Accidents | ✓ | ✓ | ✓ | ✓ | ✓ | · | ✓ | · | · | · |
| Apollo's Arrow | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | · | ✓ | · |
| The Song of the Cell | ✓ | ✓ | ✓ | ✓ | ✓ | · | ✓ | · | · | · |
| The Gene | ✓ | ✓ | · | · | · | · | · | · | · | · |
| The Emperor of All Maladies | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | · |
| Other Minds | ✓ | ✓ | · | · | ✓ | · | · | · | · | · |
| Mind in Motion | ✓ | ✓ | ✓ | ✓ | ✓ | · | · | · | · | · |
| The Ghost Map | ✓ | ✓ | · | ✓ | · | · | · | ✓ | · | · |
| Behave | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | ✓ | · | · |
| Why Zebras Don't Get Ulcers | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | · | · |
| Smallpox: The Death of a Disease | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | ✓ | · | · |
| The Viral Storm | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | · | ✓ | ✓ |
| Hallucinations | 0 | 35 | 2 | 3 | 10 | 9 | 5 | 3 | 2 | 2 |
| Precision | 100% | 27% | 83% | 77% | 50% | 44% | 50% | 57% | 50% | 33% |
| Cost / shelf | 4.75¢ | 0.04¢ | 0.09¢ | 0.19¢ | 0.03¢ | 0.12¢ | 0.21¢ | 0.02¢ | 0.03¢ | 0.04¢ |
| Latency | 39s | 76s | 6s | 171s | 27s | 10s | 77s | 8s | 61s | 10s |
Best lower-end open model: Qwen3-30B-A3B. Same 76% recall as a 12B dense model but 5× cleaner (2 hallucinations vs 10) and 5× faster — a Mixture-of-Experts running at ~3B active parameters. Still not production-clean: it misses 3 books and mis-binds authors.
Mistral-24B’s 100% is a trap. It hit every book only by
wrapping every OCR fragment in “The” (The 8160, a barcode; The
本, CJK noise) — 48 outputs, 35 invented. Recall alone is gameable; precision
exposes it. Small models fail by pattern-matching fragments instead of reasoning about books.
03Does the OCR stage earn its place at all?
The control: hand the raw photo directly to the same Sonnet model, no OCR.
| Metric | OCR → Sonnet | Image → Sonnet (direct) |
|---|---|---|
| Accuracy (recall) | 13 / 13 | 13 / 13 |
| Hallucinations | 0 | 1 (honest ‘unknown’ for a partial spine) |
| Author correct | 13 / 13 | 12 / 13 |
| Shelf order & direction | lost | preserved, correct |
| Prompt tokens | 3,417 | 2,674 |
| Completion tokens | 4,071 | 538 |
| Cost | $0.0475 | $0.0107 |
| Latency | 39s | 7.91s |
| Steps / dependencies | 2 (PaddleOCR + Sonnet) | 1 |
Same accuracy, but going direct is 4.4× cheaper, ~5× faster, one step instead of two, and it preserves shelf order — which the OCR pipeline threw away. For a capable multimodal model, the OCR stage is dead weight.
When OCR → LLM makes sense
- You cannot use a cloud VLM (privacy, offline, cost at extreme scale)
- You want an inspectable, cacheable text intermediate
- You have a strong model for the assembly step
Why direct image → VLM wins here
- Same accuracy, 4× cheaper, 5× faster, one dependency
- Preserves spatial order and direction for free
- Flags unreadable spines honestly instead of inventing
04Where this leaves the antilibrary
For turning shelves into a book list, go straight to a capable multimodal model. At roughly a cent per shelf, identifying a ~1,000-book library costs a few dollars once. The dream of a free local model for the assembly step does not hold: local text models fail at it, and the OCR stage that would feed them is itself unnecessary when the model can just see the photo. The one live question is whether a local vision model can close the gap — the only path that keeps it free and offline.
Method: single shelf (n=1), ground truth hand-verified, fuzzy title matching with
±1–2 scoring noise; treat as a smoke test, not a benchmark. OCR: PaddleOCR 3.7 (local).
Models via OpenRouter. Full statistics, per-model outputs, and code in the experiment folder
(experiments/spine-ocr/). Generated from run data by build_report.py.