We will present a very general framework for unconstrained stochastic optimization which encompasses standard frameworks such as line search and trust region using random models. In particular this framework retains the desirable practical features such step acceptance criterion, trust region adjustment and ability to utilize of second order models. The framework is based on bounding the expected stopping time of a stochastic process, which satisfies certain assumptions. Then the convergence rates are derived for each method by ensuring that the stochastic processes generated by the method satisfies these assumptions. The methods include a version of a stochastic trust-region method and a stochastic line-search methods and provide strong convergence analysis under weaker conditions than alternative approaches in the literature.
You are here
Katya Scheinberg, Lehigh UniversityTitle: New Framework for Convergence Analysis of Stochastic Optimization MethodsAbstract:Bio: Katya Scheinberg is the Harvey E. Wagner Endowed Chair Professor at the Industrial and Systems Engineering Department at Lehigh University. She also serves as a director on the Lehigh Institute for Data, Intelligent Systems and Computation. She attended Moscow University for her undergraduate studies in applied mathematics and then moved to New York and received her PhD degree in operations research from Columbia University. After receiving her doctoral degree she has worked at the IBM T.J. Watson Research Center as a research staff member for over a decade before joining Lehigh in 2010. In 2016-2017 Katya spent her sabbatical leave visiting Google Research in NY and University of Oxford. Katya’s main research areas are related to developing practical algorithms (and their theoretical analysis) for various problems in continuous optimization, such as convex optimization, derivative free optimization, machine learning, quadratic programming, etc. She has been focusing on large-scale optimization method for Big Data applications and Machine Learning since 2000. In 2015, jointly with Andy Conn and Luis Vicente, she received the Lagrange Prize awarded jointly by SIAM and MOS. Katya is the editor-in-chief of SIAM-MOS Optimization book series, a co-editor of Mathematical Programming and and associate editor of SIAM Journal on Optimization and SIAM Journal on Mathematics of Data Science.