Recording 'the mouse turned left' captures the decision. But the mouse was also twitching its whiskers, moving its paws, dilating its pupils — and all of that movement influences brain activity too. Modern neuroscience increasingly tracks everything.

The problem with video data: a behavioral video has thousands of pixels per frame and hundreds of frames per second. That's millions of numbers — a very high-dimensional dataset. It's hard to analyze or relate to neural activity in that raw form.

Solution 1: Body part tracking with DeepLabCut
Neuroscientists developed DeepLabCut — a software tool that uses deep neural networks to automatically track body parts:
1. You manually label a few frames (where are the fingers? the nose? the tail?)
2. A deep network trains on your labels
3. The network tracks those body parts automatically in all remaining frames

Result: instead of raw pixels, you get x/y coordinates of each body part over time — much easier to analyze. Works on mice, flies, fish, monkeys, humans.

Solution 2: Dimensionality reduction
Even beyond body parts, there are micro-movements (throat movements, tiny nose twitches) that DeepLabCut doesn't easily capture. A complementary approach: compress the entire video into a low-dimensional representation.

In the Niesol/Churchland study, each video frame (thousands of pixels) was compressed into ~200 numbers that captured most of the movement information. This makes statistical analysis tractable.

🔑 Key finding from the Churchland lab: cortex-wide neural activity was dominated by movement — especially uninstructed movements (whisking, fidgeting) not related to the task. This shows why capturing ALL behaviors matters: they're not noise, they're driving brain activity.

Naturalistic behavior: behavioral syllables
Bob Datta's lab (Harvard) used a depth camera to record mice freely exploring an open arena. Using dimensionality reduction + Hidden Markov Models, they automatically segmented behavior into discrete 'syllables': walking, pausing, rearing, grooming. This allows quantitative analysis of natural behavior without any task — just a mouse being a mouse.