Matrix Inversion Lemma

This equation is used everywhere.


Matrix Inversion Lemma: $$\mathbf{(A^{-1} + B^{-1})^{-1} = A - A(A+B)^{-1} A = B-B(A+B)^{-1}B}$$
I saw this and went WTF? But let's not panic. Let's take this in the one dimensional case and see if it holds.
Working backwards ... $$ \begin{align*} a-a(a+b)^{-1} a &= a-\frac{a^2}{a+b} \\ &= \frac{a(a+b)}{a+b} - \frac{a^2}{a+b} \\ &= \frac{a^2 + ab}{a+b} - \frac{a^2}{a+b} \\ &= \frac{ab}{a+b} \end{align*} $$
Oh hey that's not too bad. The last equation is easily recognizable as $$\frac{1}{\frac{1}{a} + \frac{1}{b}}$$ which when written as $$(a^{-1} + b^{-1})^{-1}$$ is our original equation as expected.
OK so just repeat it with matrices! $$ \begin{align*} \mathbf{(A^{-1} + B^{-1})^{-1}} &= \mathbf{ (A^{-1} (BB^{-1}) + (A^{-1}A)B^{-1})^{-1} } \\ &= \mathbf{(A^{-1} (A+B) B^{-1})^{-1}} \\ &= \mathbf{ (B (A+B)^{-1} A ) } \\ &= \mathbf{ ((A+B - A) (A+B)^{-1} A ) } \\ &= \mathbf{(A+B)(A+B)^{-1}A - A (A+B)^{-1}A} \\ &= \mathbf{A - A(A+B)^{-1}A} \end{align*} $$ Note: This is also known as the Woodbury matrix identity.