+10 XP

The Chain Rule

Sometimes you want how one thing changes with another, but they're only connected through time. Example: a neuron's gain a drifts over time, and its firing rate r depends on a. How does r change as a changes? The chain rule.

dr/da = (dr/dt) · (dt/da) = (dr/dt) ÷ (da/dt)

Slope of r with respect to a = (how fast r changes in time) divided by (how fast a changes in time).

In plain words: you can't measure r-vs-a directly, but you can watch both r and a change over time. The chain rule says: divide how fast r is moving by how fast a is moving, and you get how much r moves per unit of a.

It's like working out your speed-per-step from your speed-per-second and your steps-per-second.

A subtle trap (and a great gotcha): what if, at some moment, a isn't changing at all (its line is flat, da/dt = 0) — but r is changing?

Then dr/da = (something) ÷ 0 → undefined. If a is frozen, you simply can't talk about "how r changes as a changes" at that instant — a isn't changing! You can still talk about dr/dt, just not dr/da.

The chain rule also cracks hard derivatives. Take r(a) = e^(a⁴ + 1). There's no table entry for that. But set the inside (a⁴ + 1) as one chunk: the derivative of e^(stuff) is e^(stuff) times the derivative of stuff. Derivative of a⁴+1 is 4a³, so:

dr/da = e^(a⁴+1) · 4a³

That's the chain rule: outer derivative × inner derivative.