As known, finding an effective restart procedure for the conjugate gradient methods has been considered as an open problem. Here, we aim to study the problem for the Dai–Liao conjugate gradient method. In this context, based on a singular value analysis conducted on the Dai–Liao search direction matrix, it is shown that when the gradient approximately lies in the direction of the maximum magnification by the matrix, the method may get into some computational errors as well as it may converge hardly. In such situation, ignoring the Dai–Liao search direction in the sense of performing a restart may enhance the numerical stability as well as may accelerate the convergence. Numerical results are reported; they demonstrate effectiveness of the suggested restart procedure in the sense of the Dolan–Moré performance profile.
Mots-clés : Nonlinear programming, unconstrained optimization, conjugate gradient method, restart strategy, maximum magnification
@article{RO_2020__54_4_981_0,
author = {Aminifard, Zohre and Babaie-Kafaki, Saman},
title = {A restart scheme for the {Dai{\textendash}Liao} conjugate gradient method by ignoring a direction of maximum magnification by the search direction matrix},
journal = {RAIRO - Operations Research - Recherche Op\'erationnelle},
pages = {981--991},
publisher = {EDP-Sciences},
volume = {54},
number = {4},
year = {2020},
doi = {10.1051/ro/2019045},
mrnumber = {4091856},
language = {en},
url = {http://archive.numdam.org/articles/10.1051/ro/2019045/}
}
TY - JOUR AU - Aminifard, Zohre AU - Babaie-Kafaki, Saman TI - A restart scheme for the Dai–Liao conjugate gradient method by ignoring a direction of maximum magnification by the search direction matrix JO - RAIRO - Operations Research - Recherche Opérationnelle PY - 2020 SP - 981 EP - 991 VL - 54 IS - 4 PB - EDP-Sciences UR - http://archive.numdam.org/articles/10.1051/ro/2019045/ DO - 10.1051/ro/2019045 LA - en ID - RO_2020__54_4_981_0 ER -
%0 Journal Article %A Aminifard, Zohre %A Babaie-Kafaki, Saman %T A restart scheme for the Dai–Liao conjugate gradient method by ignoring a direction of maximum magnification by the search direction matrix %J RAIRO - Operations Research - Recherche Opérationnelle %D 2020 %P 981-991 %V 54 %N 4 %I EDP-Sciences %U http://archive.numdam.org/articles/10.1051/ro/2019045/ %R 10.1051/ro/2019045 %G en %F RO_2020__54_4_981_0
Aminifard, Zohre; Babaie-Kafaki, Saman. A restart scheme for the Dai–Liao conjugate gradient method by ignoring a direction of maximum magnification by the search direction matrix. RAIRO - Operations Research - Recherche Opérationnelle, Tome 54 (2020) no. 4, pp. 981-991. doi : 10.1051/ro/2019045. http://archive.numdam.org/articles/10.1051/ro/2019045/
[1] , Numerical comparison of conjugate gradient algorithms for unconstrained optimization. Stud. Inform. Control 16 (2007) 333–352.
[2] , Open problems in conjugate gradient algorithms for unconstrained optimization. B. Malays. Math. Sci. Soc. 34 (2011) 319–330. | MR | Zbl
[3] , On the sufficient descent condition of the Hager–Zhang conjugate gradient methods. 4OR 12 (2014) 285–292. | DOI | MR | Zbl
[4] and , The Dai-Liao nonlinear conjugate gradient method with optimal parameter choices. Eur. J. Oper. Res. 234 (2014) 625–630. | DOI | MR | Zbl
[5] and , A descent family of Dai-Liao conjugate gradient methods. Optim. Methods Softw. 29 (2014) 583–591. | DOI | MR | Zbl
[6] and , Two optimal Dai-Liao conjugate gradient methods. Optimization 64 (2015) 2277–2287. | DOI | MR
[7] and , Descent symmetrization of the Dai-Liao conjugate gradient method. Asia-Pac. J. Oper. Res. 33 (2016) 1650008. | DOI | MR
[8] and , Two–point stepsize gradient methods. IMA J. Numer. Anal. 8 (1988) 141–148. | DOI | MR | Zbl
[9] , A derivation of conjugate gradients In: Numerical Methods for Nonlinear Optimization, edited by . Academic Press, New York, NY (1972) 39–43. | MR | Zbl
[10] and , Linear convergence of the conjugate gradient method. IBM J. Res. Dev. 16 (1972) 431–433. | DOI | MR | Zbl
[11] and , A nonlinear conjugate gradient algorithm with an optimal property and an improved Wolfe line search. SIAM J. Optim. 23 (2013) 296–320. | DOI | MR | Zbl
[12] and , New conjugacy conditions and related nonlinear conjugate gradient methods. Appl. Math. Optim. 43 (2001) 87–101. | DOI | MR | Zbl
[13] , , , , and , Convergence properties of nonlinear conjugate gradient methods. SIAM J. Optim. 10 (1999) 348–358. | Zbl
[14] , and , On restart procedures for the conjugate gradient method. Numer. Algorithms 35 (2004) 249–260. | DOI | MR | Zbl
[15] and , Benchmarking optimization software with performance profiles. Math. Program. 91 (2002) 201–213. | DOI | MR | Zbl
[16] and , Function minimization by conjugate gradients. Comput. J. 7 (1964) 149–154. | DOI | MR | Zbl
[17] and , Global convergence properties of conjugate gradient methods for optimization. SIAM J. Optim. 2 (1992) 21–42. | DOI | MR | Zbl
[18] , and , CUTEr and SifDec: a constrained and unconstrained testing environment, revisited. ACM Trans. Math. Softw. 29 (2003) 373–394. | DOI | Zbl
[19] and , A survey of nonlinear conjugate gradient methods. Pac. J. Optim. 2 (2006) 35–58. | MR | Zbl
[20] and , Methods of conjugate gradients for solving linear systems. J. Res. Nat. Bur. Stand. 49 (1952) 409–436. | DOI | MR | Zbl
[21] , Accuracy and Stability of Numerical Algorithms. SIAM, Philadelphia, PA (2002). | DOI | MR | Zbl
[22] and , Evaluating a restart procedure for conjugate gradients. Report RC–4382. IBM Research Center, Yorktown Heights (1973).
[23] and , Numerical Optimization. Springer, New York, NY (2006). | MR | Zbl
[24] , A modified conjugate gradient algorithm. Oper. Res. 26 (1978) 1073–1078. | DOI | MR | Zbl
[25] , Restart procedures for the conjugate gradient method. Math. Program. 12 (1977) 241–254. | DOI | MR | Zbl
[26] and , Optimization Theory and Methods: Nonlinear Programming. Springer, New York, NY (2006). | MR | Zbl
[27] , Fundamentals of Matrix Computations. John Wiley and Sons, New York, NY (2002). | DOI | MR | Zbl
[28] and , A survey of quasi-Newton equations and quasi-Newton methods for optimization. Ann. Oper. Res. 103 (2001) 213–234. | DOI | MR | Zbl
[29] , Nonlinear programming computational methods. In: Integer and Nonlinear Programming, edited by . North-Holland, Amsterdam (1970) 37–86. | MR | Zbl
Cité par Sources :
