joint work with Peipei Tang, Defeng Sun, Kim-Chuan Toh
In this paper, we consider high-dimensional nonconvex square-root-loss regression problems. We shall introduce a proximal majorization-minimization (PMM) algorithm for these problems. Our key idea for making the proposed PMM to be efficient is to develop a sparse semismooth Newton method to solve the corresponding subproblems. By using the Kurdyka-Lojasiewicz property exhibited in the underlining problems, we prove that the PMM algorithm converges to a d-stationarity point. Extensive numerical experiments are presented to demonstrate the high efficiency of the proposed PMM algorithm.
joint work with Dunbiao Niu, Chengjing Wang, Qingsong Wang, Enbin Song
In this talk, we propose a highly efficient semismooth Newton based augmented Lagrangian method for solving a large-scale convex quadratic programming problem generated by the dual of the SVM with constraints consisting of one linear equality constraint and a simple box constraint. By leveraging on available error bound results to realize the asymptotic superlinear convergence property of the augmented Lagrangian method, and by exploiting the second-order sparsity of the problem through the semismooth Newton method, the algorithm we propose can efficiently solve these difficult problems.
joint work with Juan Yin
SVM is an important and fundamental technique in machine learning. In this talk, we apply a semismooth Newton method to solve two typical SVM models: the L2-loss SVC model and the $\epsilon$-L2-loss SVR model. A common belief on the semismooth Newton method is its fast convergence rate as well as high computational complexity. Our contribution is that by exploring the sparse structure of the models, we significantly reduce the computational complexity, meanwhile keeping the quadratic convergence rate. Extensive numerical experiments demonstrate the outstanding performance of the method.