Hybrid globalization of convergence of the Levenberg-Marquardt method for equality-constrained optimization problems
- Authors: Izmailov A.F.1, Uskov E.I.2
-
Affiliations:
- Lomonosov Moscow State University
- Derzhavin Tambov State University
- Issue: Vol 30, No 149 (2025)
- Pages: 41-55
- Section: Articles
- URL: https://journal-vniispk.ru/2686-9667/article/view/304173
- ID: 304173
Cite item
Full Text
Abstract
The Levenberg-Marquardt method possesses local superlinear convergence for general systems of nonlinear equations under weal assumptions allowing for nonisolated solutions. This justifies its application to first-order optimality systems of constrained optimization problems with possibly violated constraint qualifications, the latter giving rise to nonuniqueness of Lagrange multipliers. However, the existing strategies for globalization of convergence of the Levenberg-Marquardt method are not optimization-oriented by nature, i.e., when applied to optimization problems, they are intended not for finding solutions, but rather any stationary points of such problems. In this work, we propose optimization-oriented globalization strategies for the Levenberg-Marquardt method applied to optimization problems with equality constraints. The proposed strategies are hybrid by their character, i.e., they combine a globally convergent optimization outer phase method with asymptotic switching to the Levenberg-Marquardt method. Global convergence properties and superlinear rate of convergence are established. Numerical results are provided, demonstrating that the proposed hybrid algorithms are workable.
About the authors
Alexey F. Izmailov
Lomonosov Moscow State University
Author for correspondence.
Email: izmaf@cs.msu.ru
ORCID iD: 0000-0001-9851-0524
Doctor of Physical and Mathematical Sciences, Professor of the Operations Research Department
Russian Federation, 1 Leninskiye Gory, Moscow 119991, Russian FederationEvgeniy I. Uskov
Derzhavin Tambov State University
Email: euskov@cs.msu.ru
ORCID iD: 0000-0002-3639-0317
Candidate of Physical and Mathematical Sciences, Researcher at the Scientific and Educational Center “Fundamental Mathematical Research”
Russian Federation, 33 Internatsionalnaya St., Tambov 392000, Russian FederationReferences
- A.F. Izmailov, M.V. Solodov, Chislenniye Metody Optimizatzii, 2nd edition, revised and enlarged, Fizmatlit, Moscow, 2008 (In Russian).
- A.F. Izmailov, M.V. Solodov, Newton-Type Methods for Optimization and Variational Problems, Springer Series in Operations Research and Financial Engineering, Springer, Cham, 2014.
- A.F. Izmailov, M.V. Solodov, "Critical Lagrange multipliers: what we currently know about them, how they spoil our lives, and what we can do about it", TOP, 23:1 (2015), 1-26.
- S.J. Wright, "Superlinear convergence of a stabilized SQP method to a degenerate solution", Computational Optimization and Applications, 11:3 (1998), 253-275.
- W.W. Hager, "Stabilized sequential quadratic programming", Computational Optimization and Applications, 12:1-3 (1999), 253-273.
- S.J. Wright, "Modifying SQP for degenerate problems", SIAM J. on Optimization, 13:2 (2002), 470-497.
- D. Fernández, M.V. Solodov, "Stabilized sequential quadratic programming for optimization and a stabilized Newton-type method for variational problems", Mathematical Programming, 125:1 (2010), 47-73.
- A.F. Izmailov, M.V. Solodov, "Stabilized SQP revisited", Mathematical Programming, 133:1-2 (2012), 93-120.
- A.F. Izmailov, E.I. Uskov, "Subspace-stabilized sequential quadratic programming", Computational Optimization and Applications, 67:1 (2017), 129-154.
- A.F. Izmailov, M.V. Solodov, E.I. Uskov, "A globally convergent Levenberg-Marquard tmethod for equality-constrained optimization", Computational Optimization and Applications, 72:1 (2019), 215-239.
- A. Fischer, A.F. Izmailov, M.V. Solodov, "The Levenberg-Marquardt method: an overview of modern convergence theories and more", Computational Optimization and Applications, 89:1 (2024), 33-67.
- A.F. Izmailov, A.M. Krylova, E.I. Uskov, "Hybrid globalization of the stabilized method of sequential quadratic programming", Theoretical and Applied Problems of Nonlinear Analysis, CC RAS, Moscow, 2011, 47-66 (In Russian).
- E.D. Dolan, J.J. More, "Benchmarking optimization software with performance profiles", Mathematical Programming, 91:2 (2002), 201-213.
- A.F. Izmailov, M.V. Solodov, E.I. Uskov, "Combining stabilized SQP with the augmented Lagrangian algorithm", Computational Optimization and Applications, 62:2 (2015), 405-429.
- A.F. Izmailov, E.I. Uskov, Yan Zhibai, "The piecewise Levenberg-Marquardt method for generalized Nash equilibrium problems", Advances in System Sciences and Applications, 24:2 (2024), 19-31 (to appear).
Supplementary files
