Neuroscience: Decoding the LGN
Now the payoff. Two populations of LGN neurons read out the retina through weight matrices: population p uses matrix P, population q uses matrix Q. Same retinal input — very different fate.
Population p (matrix P — full rank). Its two neurons carry two independent readouts of the retina. Range = all of 2-D, rank 2, null space = just the origin. No information is lost — you can perfectly decode the retinal activity from p's response. np.linalg.inv(P) hands the retina straight back.
Population q (matrix Q — rank 1). Its two neurons compute essentially the same thing, so together they carry only one number. Range collapses to a line (rank 1), and a whole 1-D line of retinal patterns (its null space) produces identical q-responses. Those patterns are a genuine blind spot — q literally cannot tell them apart. You can never fully decode the retina from q.
So the responses of population p have intrinsic dimensionality 2 (you need 2 numbers to describe them), while q's have intrinsic dimensionality 1 (1 number is enough). Rank, decodability, and information loss are all the same story seen from different angles.
Drag on Q one more time and feel the blind spot directly: slide r along the red line and watch q's response refuse to change. Everything on that line looks identical to population q.
The red dashed line is the null space: every input on it lands on the same output. Drag r along it — the output g doesn't move! The output only ever lands on the green range line (rank 1). So given g, you can't tell which r it came from → not invertible, information lost.
🧠 Real brains do this constantly. Every relay from one population to the next can be full-rank (lossless) or rank-deficient (compressing, discarding). Measuring the rank of neural activity tells you how many independent signals a population truly carries — one of the most-used analyses in all of systems neuroscience, and the doorway to dimensionality reduction (PCA) coming up.