A New hybrid generalized CG- method for non-linear functions

In this paper a new extended generalized conjugate gradient algorithm is proposed for unconstrained optimization, which is considered as anew inverse hyperbolic model .In order to improve the rate of convergence of the new technique, a new hybrid technique between the standard F/R CGmethod and Sloboda CG-method using quadratic and non-quadratic models is proposed by using exact and inexact line searches. This method is more efficient and robust when applied on number of well-known nonlinear test function.


Introduction
The conjugate gradient method is particularly useful for minimizing function of mult variables because it does not require the storage of any matrices. However, the rate of convergence of the algorithm is only linear unless the iterative procedure is "restarted" occasionally .At present it is usual to restart every n or (n+1) iterations, where n is the number of variables, but it is known that frequency of restarts should depend on the objective function. . In general, the method has the following form The successive directions are conjugate vectors for the successive gradients obtained as the method progresses.

Non Quadratic Models:
The conjugate gradient methods so far discussed is a local quadratic representation of the objective function .In problems when the quadratic representation is not good, or when we are remote from such a region, quadratic function F(q(x)) ,where F is monotonic increasing ,may be better to represent the objective and thus it gives an advantage to algorithm based on this model. In order to obtain better global rate of convergence for minimization algorithms when applied to more general functions than the quadratic. In this paper several new algorithms, which are invariant to nonlinear scaling of quadratic functions are proposed. There is some precedent for this approach, if ) (x q is quadratic function then a function f is defined as nonlinear scaling of ) (x q if the following condition holds The following properties are immediately derived from the above conditions. i -every contour line of q(x) is a contour line of f ; ii -if x* is minimizer of q(x) then it is a minimizer of f ; iii -that x* is global minimum of q(x) does not necessarily mean that it is a global minimum of f. For more details see Boland et al., (1979). Many authors have proposed special models as follows: Al-Assady and Huda (1997) …(12) Al-Assady and Al-Ta ' ai (2002) …(13)

New Extended Generalized Conjugate Gradient Method by using Hyperbolic Inverse Model:
Let Since k  is a parameter, which is defined as

Sloboda CG-Method:
The rate of convergence of a variety of CG-algorithm has been investigated by many authors: the most general results have been given by Baptist and Storey (1977) where it was also shown that the algorithms with ELS (exact line searches) have the property of n-step quadratic convergence (Store, 1977). In order to improve the rate of convergence of CG-algorithm it is necessary to construct special algorithms for more general function than the quadratic .In series of papers:

Outlines of Sloboda CG-Method:
Step1: Step 2: Step 4: Step 5: test for convergence: if achieved stop, if not continue Step 6: if k=n or (any equivalent restarting criterion) go to step 3 else continue.
Step 7: compute where Step 9: set k=k+1 and go to Step 3.

Hybrid Conjugate Gradient Methods:
Despite the numerical superiority of Polak-Riebiere (P/R) algorithm over Fletcher-Reeves (F/R) algorithm, the later has better theoretical properties than the former. Under certain conditions F/R-method can be shown to have global convergence with exact line search (Powell, 1986) and also with inexact line search satisfying the strong Wolfe-Powell condition. (see Al-Baali, 1985).
Normally this leads to speculation on the best way to choose k  .
Step3: If Otherwise set Here  ,  and ˆ are user supplied parameters. This hybrid was shown to be globally convergent under both exact and inexact line searches and to be quite competitive with P/R-algorithm and F/R-algorithm. See Hu and Storey ( 1991 ) .
Touati and Storey suggested also the following algorithm to compute the conjugancy coefficient k  : , return to main program. Otherwise go to step2.
Step2: If ; return to main program. Otherwise, go to step3.
Step3: If ( ; return to main program. Otherwise, set ; return to main program.

New Suggestion for Hybrid CG-Methods:
In this section we are going to study develop a new CG-method based on quadratic and non-quadratic models; taking the idea of exact and inexact line searches .The new technique use new hybrid idea between the standard F/R CG-algorithm and Sloboda (1980) CG-method.

Outlines of the New Suggested Algorithm:
Step1:Set Step 4:Set Step 5:Check for convergence i.e ,if  ,then stop else Step 6: Find is satisfied go to step(8) otherwise go to step (13) Step 8: Step 9:Find Step 13: Step 14: compute Step 19:if n k = or (any equivalent restarting critenion)go to step (2), else Step 20: set, 1 + = k k and go to step (3)

Numerical Results:
In order to assess the performance of the new proposed algorithm (Hybrid model). Three minimization algorithms are tested over (10) nonlinear unconstrained test functions with different dimensions see (Appendix).
All the results are obtained using (Pentium computer). All programs are written in FORTRAN language and for all cases the stopping criterion taken to be In Table (1) we represent comparison between new algorithm with Standard F/R CG-algorithm and Sloboda CG-algorithm. Our numerical results, which are presented in Table (2) confirm that the Hybrid model algorithm is superior to both Standard CG-algorithm and Sloboda CG-algorithm with respect to the total number of NOF and NOI.