joint work with Insoon Yang
To solve dynamic programs with continuous state and action spaces, we first approximate the output of the Bellman operator as a convex program with uniform convergence. This convex program is then solved using stochastic subgradient descent. To avoid the full projection onto the high-dimensional feasible set, we develop an algorithm that samples, in a coordinated fashion, a mini-batch for a subgradient and another for projection. We show several properties of this algorithm, including convergence, and a reduction in the feasibility error and in the variance of the stochastic subgradient.
joint work with Jukka Liukkonen
We present a new approach to foundations of quantum mechanics via stochastic optimization. It is shown that the Schrödinger equation can be interpreted as an optimal dynamic for the value function for a coordinate-invariant linear-quadratic stochastic control problem. The optimal path for the quantum particle obeys a stochastic gradient descent scheme, which minimizes relative entropy or the free energy. This in turn allows for a new interpretation for quantum mechanics through the tools of non-equilibrium statistical mechanics and information theory vis-a-vis the path integral formulation.
joint work with Andre Carlon, Ben Dia, Rafael Lopez, Raul Tempone
We focus on the Optimal Experimental Design OED problem of finding the setup that maximizes the expected information gain. We augment the stochastic gradient descent with the Nesterov's method and the restart technique, as O'Donoghue and Candes originally proposed for deterministic optimization. We couple the stochastic optimization with three estimators for OED: the double-loop Monte Carlo estimator DLMC, the Monte Carlo estimator using the Laplace approximation for the posterior distribution MCLA and the double-loop Monte Carlo estimator with Laplace-based importance sampling DLMCIS.