From 75d8ca5fb747ae591ee3fcf641801d5b7803ab3f Mon Sep 17 00:00:00 2001 From: Glen Whitney Date: Wed, 19 Feb 2025 20:27:20 +0000 Subject: [PATCH] Update Gram matrix parameterization --- Gram-matrix-parameterization.md | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/Gram-matrix-parameterization.md b/Gram-matrix-parameterization.md index e6c4927..bdaeeea 100644 --- a/Gram-matrix-parameterization.md +++ b/Gram-matrix-parameterization.md @@ -39,7 +39,12 @@ on $\operatorname{Hom}(\mathbb{R}^n, V)$. Finding a global minimum of the *loss #### The first derivative of the loss function Write the loss function as -\[ \begin{align*} f & = \|\Delta\|^2 \\ & = \operatorname{tr}(\Delta^\top \Delta), \end{align*} \] +```math +\begin{align*} + f & = \|\Delta\|^2 \\ + & = \operatorname{tr}(\Delta^\top \Delta), +\end{align*} +``` where $\Delta = G - \mathcal{P}(A^\top Q A)$. Differentiate both sides and simplify the result using the transpose-invariance of the trace: \[ \begin{align*} df & = \operatorname{tr}(d\Delta^\top \Delta) + \operatorname{tr}(\Delta^\top d\Delta) \\ & = 2\operatorname{tr}(\Delta^\top d\Delta). \end{align*} \] To compute $d\Delta$, it will be helpful to write the projection operator $\mathcal{P}$ more explicitly. Let $\mathcal{C}$ be the set of indices where the Gram matrix is unconstrained. We can express $C$ as the span of the standard basis matrices $\{E_{ij}\}_{(i, j) \in \mathcal{C}}$. Observing that $E_{ij} X^\top E_{ij} = X_{ij} E_{ij}$ for any matrix $X$, we can do orthogonal projection onto $C$ using standard basis matrices: