joint work with Mikael Johansson
Many matrix update formulas in quasi-Newton methods can be derived as solutions to nearest-matrix problems. In this talk, a similar idea is proposed for developing limited memory quasi-Newton methods. We characterize a class of symmetric matrices that can be stored with limited memory. We then derive an efficient way to find the closest member of this class to any symmetric matrix and propose a trust region method that makes use of it. Numerical experiments show that our algorithm has superior performance on large logistic regression problems, compared to a regular L-BFGS trust region method.
joint work with Joshua Griffin, Riadh Omheni
This talk will focus on solving bound constrained nonconvex optimization problems with expensive-to-evaluate functions. The motivation for this research was from collaborations with members from the statistics community seeking better handling of their optimization problems. The method first solves a QP using the projected gradient method to improve the active set estimate. The method next uses a modified conjugate gradient method applied to the corresponding trust region subproblem to leverage second order information. The numerical results will be provided.
We propose inexact first-order methods for solving constrained nonlinear optimization problems. By controlling the inexactness of the subproblem solution, we can significantly reduce the computational cost needed for each iteration. A penalty parameter updating strategy during the subproblem solve enables the algorithm to automatically detect infeasibility. Global convergence for both feasible and infeasible cases are proved. Complexity analysis for the KKT residual is also derived under loose assumptions. Performance of proposed methods is demonstrated by numerical experiments.