Convergence analysis of adaptive trust region methods
RAIRO - Operations Research - Recherche Opérationnelle, Tome 41 (2007) no. 1, pp. 105-121.

In this paper, we propose a new class of adaptive trust region methods for unconstrained optimization problems and develop some convergence properties. In the new algorithms, we use the current iterative information to define a suitable initial trust region radius at each iteration. The initial trust region radius is more reasonable in the sense that the trust region model and the objective function are more consistent at the current iterate. The global convergence, super-linear and quadratic convergence rate are analyzed under some mild conditions. Numerical results show that some special adaptive trust region methods are available and efficient in practical computation.

DOI : 10.1051/ro:2007007
Classification : 90C30, 49M37, 65K05
Mots-clés : adaptive trust region method, unconstrained optimization, global convergence, super-linear convergence 
@article{RO_2007__41_1_105_0,
     author = {Shi, Zhen-Jun and Zhang, Xiang-Sun and Shen, Jie},
     title = {Convergence analysis of adaptive trust region methods},
     journal = {RAIRO - Operations Research - Recherche Op\'erationnelle},
     pages = {105--121},
     publisher = {EDP-Sciences},
     volume = {41},
     number = {1},
     year = {2007},
     doi = {10.1051/ro:2007007},
     mrnumber = {2310543},
     language = {en},
     url = {http://archive.numdam.org/articles/10.1051/ro:2007007/}
}
TY  - JOUR
AU  - Shi, Zhen-Jun
AU  - Zhang, Xiang-Sun
AU  - Shen, Jie
TI  - Convergence analysis of adaptive trust region methods
JO  - RAIRO - Operations Research - Recherche Opérationnelle
PY  - 2007
SP  - 105
EP  - 121
VL  - 41
IS  - 1
PB  - EDP-Sciences
UR  - http://archive.numdam.org/articles/10.1051/ro:2007007/
DO  - 10.1051/ro:2007007
LA  - en
ID  - RO_2007__41_1_105_0
ER  - 
%0 Journal Article
%A Shi, Zhen-Jun
%A Zhang, Xiang-Sun
%A Shen, Jie
%T Convergence analysis of adaptive trust region methods
%J RAIRO - Operations Research - Recherche Opérationnelle
%D 2007
%P 105-121
%V 41
%N 1
%I EDP-Sciences
%U http://archive.numdam.org/articles/10.1051/ro:2007007/
%R 10.1051/ro:2007007
%G en
%F RO_2007__41_1_105_0
Shi, Zhen-Jun; Zhang, Xiang-Sun; Shen, Jie. Convergence analysis of adaptive trust region methods. RAIRO - Operations Research - Recherche Opérationnelle, Tome 41 (2007) no. 1, pp. 105-121. doi : 10.1051/ro:2007007. http://archive.numdam.org/articles/10.1051/ro:2007007/

[1] R.H. Byrd, J. Nocedal and Y.X. Yuan, Global convergence of a class of quasi-Newton methods on convex problems. SIAM J. Numer. Anal. 24 (1987) 1171-1190. | Zbl

[2] A.R. Conn, N.I.M. Gould and P.L. Toint, Trust-Region Methods, Society for Industrial and Applied Mathematics (SIAM), Philadelphia PA (2000). | MR | Zbl

[3] G. Corradi, A trust region algorithm for unconstrained optimization. Int. J. Comput. Math. 65 (1997) 109-119.

[4] Y.H. Dai, D.C. Xu, A new family of trust region algorithms for unconstrained optimization. J. Comput. Math. 21 (2003) 221-228. | Zbl

[5] R. Fletcher, Practical Method of Optimization, Unconstrained Optimization, Vol. 1, John Wiley, New York (1980). | MR

[6] J.Y. Fan, W.B. Ai and Q.Y. Zhang, A line search and trust region a lgorithm with trust region radius converging to zero. J. Comput. Math. 22 (2004) 865-872. | Zbl

[7] J.H. Fu and W.Y. Sun, Nonmonotone adaptive trust-region method for unconstrained optimization problems. Appl. Math. Comput. 163 (2005) 489-504. | Zbl

[8] E.M. Gertz, A quasi-Newton trust-region method. Math. Program. Ser. A 100 (2004) 447-470. | Zbl

[9] N.I.M. Gould, D. Orban, A. Sartenaer and P.L. Toint, Sensitivity of trust-region algorithms to their parameters. 4OR 3 (2005) 227-241. | Zbl

[10] N.I.M. Gould, D. Orban and P.L. Toint, Numerical methods for large-scale nonlinear optimization. Acta Numer. 14 (2005) 299-361. | Zbl

[11] L. Hei, A self-adaptive trust region algorithm. J. Comput. Math. 21 (2003) 229-236. | Zbl

[12] J.J. Moré, Recent developments in algorithms and software for trust region methods, in Mathematical Programming: The State of the Art, edited by A. Bachem, M. Grotchel and B. Korte, Springer- Verlag, Berlin (1983) 258-287. | Zbl

[13] J.J. Moré, B.S. Grabow and K.E. Hillstrom, Testing Unconstrained Optimization software. ACM Trans. Math. Software 7 (1981) 17-41. | Zbl

[14] D. Mauricio and N. Maculan, A trust region method for zero-one nonlinear programming. RAIRO Rech. Opér. 31 (1997) 331-341. | Numdam | Zbl

[15] J. Nocedal and S.J. Wright, Numerical Optimization. Springer-Verlag, New York (1999). | MR | Zbl

[16] Q. Ni, A globally convergent method of moving asymptotes with trust region technique. Optim. Methods Softw. 18 (2003) 283-297. | Zbl

[17] M.J.D. Powell, Convergence properties of a class minimization algorithms, in Nonlinear Programming 2, edited by O.L. Mangasarian, R.R. Meyer and S.M. Robinson, Academic Press, New York (1975) 1-27. | Zbl

[18] M.J.D. Powell, On the global convergence of trust region algorithms for unconstrained optimization. Math. Prog. 29 (1984) 297-303. | Zbl

[19] A. Sartenaer, Automatic determination of an initial trust region in nonlinear programming. SIAM J. Sci. Comput. 18 (1997) 1788-1803. | Zbl

[20] G.A. Shultz, R.B. Schnabel and R.H. Byrd, A family of trust-region-based algorithms for unconstrained minimization with strong global convergence. SIAM J. Numer. Anal. 22 (1985) 47-67. | Zbl

[21] W.Y. Sun, Nonmonotone trust region method for solving optimization problems. Appl. Math. Comput. 156 (2004) 159-174. | Zbl

[22] J. Sun, On piecewise quadratic Newton and trust region problems. Math. Programming, Ser. B 76 (1997) 451-467. | Zbl

[23] Z.J. Shi and H.Q. Wang, A new self-adaptive trust region method for unconstrained optimization. Technical Report, College of Operations Research and Management, Qufu Normal University (2004).

[24] J.Z. Zhang and C.X. Xu, Trust region dogleg path algorithms for unconstrained minimization, Management science in China (Kowloon, 1996). Ann. Oper. Res. 87 (1999) 407-418. | Zbl

[25] X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica 12 (1996) 1-10.

[26] X.S. Zhang, Trust region method in neural network. Acta Math. Appl. Sinica (English Ser.) 13 (1997) 342-352. | Zbl

[27] X.S. Zhang, J.L. Zhang and L.Z. Liao, An adaptive trust region method and its convergence. Science in China 45 (2002) 620-631. | MR | Zbl

[28] X.S. Zhang, Z.W. Chen and J.L. Zhang, A self-adaptive trust region method for unconstrained optimization. OR Transactions 5 (2001) 53-62.

[29] Y.X. Yuan, A review of trust region algorithms for optimization, in ICIAM 99 (Edinburgh), Oxford Univ. Press, Oxford (2000) 271-282. | Zbl

[30] Y.X. Yuan, Trust region algorithms for nonlinear equations. Information 1 (1998) 7-20. | Zbl

Cité par Sources :