We investigate the recovery of a partially observed high-rank matrix whose columns obey a nonlinear structure. The structures covered are unions of clusters and algebraic varieties, which include unions of subspaces. Using a nonlinear lifting to a space of features, as proposed by Ongie et al. in 2017, we reduce the problem to a constrained nonconvex optimization formulation, which we solve using Riemannian optimization methods. We give theoretical convergence and complexity guarantees and provide encouraging numerical results for low dimensional varieties and clusters.
Discrete gradient methods are popular numerical methods from geometric integration for solving systems of ODEs. They are known for preserving structures of the continuous system, e.g. energy dissipation, making them interesting for optimisation problems. We consider a derivative-free discrete gradient applied to dissipative ODEs such as gradient flow, thereby obtaining optimisation schemes that are implementable in a black-box setting and retain favourable properties of gradient flow. We give a theoretical analysis in the nonsmooth, nonconvex setting, and conclude with numerical results.
We present and analyze a novel approach to perform first-order optimization with orthogonal and unitary constraints. This approach is based on a parametrization stemming from Lie group theory through the exponential map. The parametrization transforms the constrained optimization problem into an unconstrained one over a Euclidean space, for which common first-order optimization methods can be used. The theoretical results presented are general enough to cover the special orthogonal group, the unitary group and, in general, any connected compact Lie group.