Old Ideas Are Underpriced

There’s a line about reading in Vivek’s essay How to Be Good at Research worth pinning above your desk: if your information diet is the arXiv trending page plus the group chat, you’ll reach the same conclusions as everyone else, at the same time — “which makes those conclusions worth approximately nothing.”

The fix it proposes is contrarian: old material is criminally underpriced. The field reruns its own past on a delay — mixture of experts (1991), LSTMs (1997), backprop going mainstream (1986) — so the archive is full of ideas about to come back. Rich Sutton’s Bitter Lesson is about a thousand words from 2019 and forecasts the shape of the field better than surveys ten times its length. Claude Shannon’s 1952 talk on creative thinking still hands you a usable move: shrink a problem until it’s nearly trivial, solve that version, then add the difficulty back one piece at a time.

Two more habits from the same section. Range beats depth past a point — interpretability borrows from neuroscience, eval design is really mechanism design, and knowing how a GPU moves memory tells you which architecture papers are doomed. And read the paper, not the thread: the appendix is where the bodies are buried, and the limitations section is usually the most honest paragraph in the whole thing.

The trending page tells you what everyone already knows. The 1997 paper doesn’t.

References

• Vivek (@itsreallyvivek). How to Be Good at Research. X — for the “upgrade your inputs” argument and its examples.