+10 XP

The Power of Powers

Why does the weight matrix decide everything? Because applying it over and over is the same as raising it to a power. Let's unroll the loop.

Start at a₀ and step forward, substituting each result into the next:

a₁ = W·a₀
a₂ = W·a₁ = W·(W·a₀) = W²·a₀
a₃ = W·a₂ = W³·a₀

The pattern is unmistakable:

aᵢ = Wⁱ · a₀

The activity at step i is the weight matrix raised to the i-th power, times the starting activity.

So the circuit's entire future is controlled by what happens to W when you raise it to higher and higher powers. And here's the magic: raising a matrix to a power is easy to understand through its eigenvalues — because along an eigenvector, a matrix is just a number (λ).

That's the bridge. Wⁱ looks scary, but along each eigenvector it becomes plain λⁱ — a single number raised to a power. And you already know how λⁱ behaves: it explodes if λ > 1 and vanishes if λ < 1.