Your Camera Can See Your Pulse
Every time your heart beats, blood floods the tiny vessels under your skin and shifts its color by an amount far too small for the eye to notice. A camera notices. Point an ordinary webcam at a face, watch how the green channel flickers over a few seconds, and you can read a heart rate off the pixels — no cuff, no clip, nothing touching the body. The field calls this remote photoplethysmography (rPPG), and Daniel McDuff’s survey Camera Measurement of Physiological Vital Signs (McDuff, 2023) is a good map of how far it has come.
Where the signal comes from
The physics is a reflection model. Light hitting skin splits into a mirror-like specular part and a diffuse part that scatters through tissue before coming back out. The diffuse part is where the pulse lives: hemoglobin absorbs light, so as blood volume rises and falls with each beat, the reflected color is modulated. The effect is strongest around 570 nm — green — which is why the humble green channel is a surprisingly good pulse detector. It is buried under motion, lighting, and sensor noise, but it is there.
If you want to see it rather than just measure it, Eulerian Video Magnification (Wu et al., SIGGRAPH 2012) amplifies exactly this band of change, turning the invisible flush of each heartbeat into something obvious on screen.
Getting a number out
The classic recipe is pure signal processing: average the pixels over a patch of skin, then project the RGB time series onto a direction that isolates the pulse from lighting changes. Hand-designed projections like CHROM and POS, or blind source separation like ICA, do this without any learning at all. Later, supervised deep networks (DeepPhys and its descendants) learned the mapping end to end.
The prettiest recent result skips the labels entirely. Millisecond-synchronized ground-truth waveforms are painful to collect, so Yue, Shi, and Ding’s self-supervised method (TPAMI, 2023) leans on the one thing you always know about a pulse: it is periodic. It builds positive examples that preserve the beat frequency and negatives that deliberately shift it, then contrasts them in the frequency domain. With no recorded labels, it matches or beats supervised state of the art across five benchmarks — a frequency-domain cousin of the contrastive learning idea from SimCLR (Chen et al., 2020).
Two cautions worth keeping
It would be easy to file this under “deep learning wins again.” Two findings push back.
First, it is not always deep learning that wins. A 2025 respiration benchmark by Boccignone et al. measured breathing rate from video and found that classical optical-flow tracking of chest motion beat the fashionable end-to-end deep models, which generalized poorly across datasets. (The nuance: the best motion estimator was itself a deep optical-flow network used as a prior — so the lesson is not “deep is bad,” but “don’t hand the whole problem to a black box when the signal is scarce.”) It is a quiet counterexample to the bitter lesson (Sutton, 2019), and the authors blame data scarcity and video compression rather than any flaw in learning itself.
Second, and more seriously: the method is not equally accurate for everyone. The pulse signal rides on skin color, and on darker skin the darker pixels suppress that color change, so the signal-to-noise ratio drops. Both McDuff’s survey and Yue’s paper document the gap — Yue reports the worst heart-rate error on African subjects, and camera-based blood-oxygen estimates tend to overstate saturation on darker skin, the same bias that plagues contact pulse oximeters. Both treat this as a first-class problem rooted in hardware, data, and models together, not a bug to be patched later.
A camera that can read your heartbeat is a genuinely lovely piece of physics. It is worth remembering that “works on average” and “works on you” are different claims.
References
- McDuff, D. (2023). Camera Measurement of Physiological Vital Signs. ACM Computing Surveys.
- Wu, H., Rubinstein, M., Shih, E., Guttag, J., Durand, F., & Freeman, W. (2012). Eulerian Video Magnification for Revealing Subtle Changes in the World. SIGGRAPH.
- Yue, Z., Shi, M., & Ding, S. (2023). Facial Video-Based Remote Physiological Measurement via Self-Supervised Learning. IEEE TPAMI.
- Boccignone, G., Cuculo, V., D’Amelio, A., Grossi, G., Lanzarotti, R., & Patania, S. (2025). Remote Respiration Measurement with RGB Cameras: A Review and Benchmark. ACM Computing Surveys.