There’s a line about reading in Vivek’s essay How to Be Good at Research worth pinning above your desk: if your information diet is the arXiv trending page plus the group chat, you’ll reach the sam...

Scaling laws come with an asterisk that’s easy to skip past. The clean version — loss falls as a predictable power law in model size, data, and compute, so you can fit a few cheap runs and extrapol...

Make an attention map of a large vision transformer — light up the patches it attends to — and you expect it to trace the objects in the scene. Often it does. But scattered across the image, usuall...

Richard Hamming’s line, quoted in Vivek’s essay How to Be Good at Research, is that knowledge and productivity compound like interest. It’s a small claim with a sharp edge: the daily edges of a goo...

A diffusion model makes pictures by denoising: start from pure static and, step by step, remove noise until an image emerges. There’s a design choice buried in that loop that most people treat as a...

Every time your heart beats, blood floods the tiny vessels under your skin and shifts its color by an amount far too small for the eye to notice. A camera notices. Point an ordinary webcam at a fac...

Almost every model you’ve heard of lately — the chatbots, the image encoders, the code assistants — descends from one 2017 paper with an unusually confident title: Attention Is All You Need (Vaswan...

When a language model “thinks out loud” — writing a chain of thought before its answer — it is tempting to read that trace as a window into how it decided. It might not be. As surveyed in Lilian We...

How do you teach a model what images mean when you have no labels to tell it? SimCLR (Chen, Kornblith, Norouzi, and Hinton, 2020) gave an answer so clean it became the textbook example: teach it by...

A recurring theme in Vivek’s essay How to Be Good at Research is unglamorous and easy to skip: write everything down. Not for posterity — for yourself, right now. The case has three strands, each ...