New Three – Term CG-Method for Unconstrained Optimization

Ban A. Mitras dr.banah.mitras@gmail.com Nada F. Hassan nada.fajer10@gmail.com College of Computer Sciences and Mathematics University of Mosul, Mosul, Iraq Received on: 29/06/2011 Accepted on: 14/12/2011 ABSTRACT  In this paper, we proposed a new three-term nonlinear Conjugate Gradient (CG) method for solving unconstrained optimization problems. The new three-term method generates decent direction with an inexact line search under Wolfe conditions and the descent property of the new method is proved. Numerical results on some well-known test function with various dimensions showed that the new method is an efficient .


Introduction.
In this paper, we deal with conjugate gradient methods for solving the following unconstrained optimization problem: Minimize f(x) …(1) where f:R n →R is smooth and its gradient g(x)=f(x) is available. Conjugate gradient methods are very efficient for solving large-scale unconstrained optimization problems (1). For solving this problem, starting from initial guess x0 R n , a nonlinear conjugate gradient methods generates a sequence {xk}as: where step size k  is positive, which is computed by carrying out some line search, and the direction dk is generated as: In (3) βk is known as conjugate gradient parameter. The search direction ,assumed to be a descent one which is play the main role in these methods. On the other hand, the step size αk guarantees the global convergence in some cases and is crucial in efficiency. Plenty of conjugate gradient methods are known, and an excellent survey of these methods, with special attention on their convergence, is given by Hager and Zhang [5]. Different conjugate gradient algorithms correspond to different choices for the scalar βk. The line search in the conjugate gradient algorithms often is based on standard Wolf conditions. The standard Wolfe conditions [9,10] , …(5) where dk is a decent direction and 0 < δ < σ < 1. For some conjugate gradient algorithms, stronger Wolfe conditions defined by: are needed to ensure convergence and to enhance stability. It has been shown [9] that for FR scheme, the strong Wolfe conditions may not yield a direction of decent unless σ  1/2. In typical implementations of the Wolfe conditions, it is most efficient to choose σ close to one. It is known that choices of βk affect numerical performance of the method, and hence many researchers studied choices of βk. Well-known formulas for βk are the Hestenes-Stiefel (HS) [6], Fletcher-Reeves (FR) [4],Conjugate-Decent(CD) [3]and Dai-Yuan (DY) [2] formulas, which are respectively given by Where ║.║means the Euclidean norm and, yk-1= gk -gk-1.
Note that these formulas for βk are equivalent each other if the objective function is a strictly convex quadratic function and αk is the one dimensional minmizer.

Three-Term CG-Methods:
The first three-term nonlinear CG-method was presented by Nazareth [8], in which the search direction is determined by: The main property of dk is that, for convex quadratic function, it remains conjugate even without ELS.
Zhang et al. [11] proposed the modified FR method (ZFR) which is defined by: , . Since this search direction satisfies all k, It can be written by the three-term form: , .
They also proposed the modified PR method(ZPR) [12] and the modified HS method(ZHS) [13],which are respectively given by: , These three-term conjugate gradient methods which always satisfy the sufficient descent condition: for a positive constant c, Independently of line searches.

New Three-term CG-Method.
Mitras and Hassan [7] proposed a seven -parameter family defined by: parameters.(i.e. λk , δk , k  , φk and ψk are impossible to be equal to one at the same time; the same thing is also correct for μk and ωk).
The seven-parameter family contains already existing twelve well-known formulas for βk, so there were 27=128 cases,12 cases were succeeded,116 cases were failed.
In the present work, we derived a new three term conjugate gradient method from this family. We choose one of the failed cases that is when ( ) are equal to (0,1,0,0,1,0,1) respectively ,so β seven yield to: We call eq. (16) a new three-term conjugate gradient method. So we can write the direction of the new three-term method as follows:

New Algorithm
Step 2. if k g <0 ,then stop.

Descent Property of the New Algorithm
The search directions generated by (17) are descent for all k if the step size satisfies Wolfe conditions. Proof: Let d1=-g1, for k 1  assume dk gk <0, then for k=k+1 we have:

Numerical Result
Tables (1) , (2), (3) and (4) are comparing between new algorithm and Zhang, Zhou and Li three-term conjugate gradient methods. The comparison involves some wellknown test function with different dimensions(500,1000,5000,10000). The program is written in double precision using Fortran (2000) .The comparative Performance of the algorithm is evaluated by considering both the total number of function evaluations which is normally assumed to be the most costly factor in each iteration and the total number of iterations. The actual convergence criterion was 6 10 − 