Use a more meaningful metric on the configuration space #131

New issue

Open

opened 2025-11-07 20:32:29 +00:00 by Vectornaut · 0 comments

Vectornaut commented

2025-11-07 20:32:29 +00:00

Member

To do uniform regularization of Newton's method, we need to choose a metric on the search space. As of pull request #118, we choose a computational basis that identifies the element space W with \mathbb{R}^5, and then we use the Frobenius metric on the configuration space \operatorname{Hom}(\mathbb{R}^n, W) that comes from the standard inner product on \mathbb{R}^5. This inner product is mostly unrelated to the Lorentz form on W, which is what appears in the loss function and encodes the conformal symmetries of our constraint problems. Our current choice of configuration space metric therefore seems poorly adapted to the task at hand. Choosing a metric that's more meaningfully related to our problems would make it easier to conceptualize the effects of regularization, and it might also improve realization performance.

Implementation idea

The value of the loss function at a configuration A \colon \mathbb{R}^n \to W depends mostly on the Gram matrix G = A^\dagger A, where \dagger is the adjoint with respect to the inner product on \mathbb{R}^n and the Lorentz form on W. The Gram matrix belongs to the space of symmetric operators on \mathbb{R}^n, which comes with a natural Frobenius metric. This metric, pulled back along the map A \mapsto A^\dagger A, provides a partial metric on the configuration space, which I'll call the Gram partial metric.

At a configuration whose elements span W, the null space of the Gram partial metric should be the tangent subspace comprising the infinitesimal conformal symmetries of the configuration. This suggests that if the loss function is invariant under conformal symmetries, the derivative of the loss function might always have a representation in terms of the Gram partial metric.

To do [uniform regularization](https://code.studioinfinity.org/StudioInfinity/dyna3/wiki/Numerical-optimization#uniform-regularization) of Newton's method, we need to choose a metric on the search space. As of pull request #118, we choose a computational basis that identifies the [element space](https://code.studioinfinity.org/StudioInfinity/dyna3/wiki/Gram-matrix-parameterization) $W$ with $\mathbb{R}^5$, and then we use the Frobenius metric on the configuration space $\operatorname{Hom}(\mathbb{R}^n, W)$ that comes from the standard inner product on $\mathbb{R}^5$. This inner product is mostly unrelated to the Lorentz form on $W$, which is what appears in the loss function and encodes the conformal symmetries of our constraint problems. Our current choice of configuration space metric therefore seems poorly adapted to the task at hand. Choosing a metric that's more meaningfully related to our problems would make it easier to conceptualize the effects of regularization, and it might also improve realization performance. ### Implementation idea The value of the loss function at a configuration $A \colon \mathbb{R}^n \to W$ depends mostly on the [Gram matrix](https://code.studioinfinity.org/StudioInfinity/dyna3/wiki/Gram-matrix-parameterization#constraints-as-gram-matrix-entries) $G = A^\dagger A$, where $\dagger$ is the adjoint with respect to the inner product on $\mathbb{R}^n$ and the Lorentz form on $W$. The Gram matrix belongs to the space of symmetric operators on $\mathbb{R}^n$, which comes with a natural Frobenius metric. This metric, pulled back along the map $A \mapsto A^\dagger A$, provides a partial metric on the configuration space, which I'll call the _Gram partial metric_. At a configuration whose elements span $W$, the null space of the Gram partial metric should be the tangent subspace comprising the infinitesimal conformal symmetries of the configuration. This suggests that if the loss function is invariant under conformal symmetries, the derivative of the loss function might always have a representation in terms of the Gram partial metric.