A Combined Cubic and Novel Line Search CG-Algorithm

13 A Combined Cubic and Novel Line Search CG-Algorithm Abbas Y. Al-Bayati profabbasalbayati@yahoo.com Hamsa TH. Chilmeran hamsathrot@uomosul.edu.iq College of Computer Sciences and Mathematics University of Mosul, Iraq Received on: 10/12/2006 Accepted on: 15/08/2007 ABSTRACT In this paper a new line search technique is investigated. It uses (cubic and novel) line searches in the standard CG-algorithm for unconstrained optimization. Applying our new modified version on CG-method shows that, it is too effective when compared with other established algorithms, in this paper, to solve standard unconstrained optimization problems.


Introduction.
The Conjugate Gradient (CG) method is particularly useful for minimizing function of many variables because it does not require the storage of any matrices. However, the rate of convergence of the algorithm is only linear unless the iterative procedure is "restarted" occasionally. At present it is usual to restart the CG-algorithm every n or (n+1) iterations, where n is number of variables, but it is known that frequency of restarts should depend on the objective function. CG-algorithm is useful to find minimum of a function R R f n → : , In general, the method has the following form where k g denotes the gradient )  is a step-length obtained by a line search, and k  is chosen so that it becomes the k-th conjugate direction when the function is quadratic and the line search is exact (Sun, et al, 2003), a Well Known formula for k  is given by: The successive direction are conjugate vectors for successive gradients obtained as the method progresses. At step k one evaluates the current negative gradient and adds it to a linear combination of the pervious direction vectors to obtain a new conjugate direction along which to move (Cristain, 2003).

Conjugate Gradient Algorithm with a Novel Line Search:
The problem of finding a minimizer of f is often investigated by a CGalgorithm , where k  is the optimal step size, computed by a novel line search technique (Sun, et al, 2003).

The One -Dimensional Novel Line Search Procedure:
The following steps describe novel line search technique: be any unimodal function. Set ) ( , 1 1 1 and a is assumed to be available; 0 . 0 = a is normally taken.
Step 2: If and go to step (1) else go to the next step (3).
Step 3: a) Set 1 f~ be its corresponding function value. Take adjacent point of B and set them as Ã and C , such that: and their respective function values . Again choose three appropriate points and set them as 3 1 , y y and 5 y such that Step 6: If the inequality for most the usually encountered problems (Rao and Chandra, 1983).

Cubic Interpolation Technique:
In the proposed cubic polynomial method , a gradient and a function evaluation are made at every iteration at k .At each iteration an update is performed when a new point 1 k x + that satisfies the condition ) in many of the n-dimensional routines, and the values of the first derivative are calculated. It may be desirable to use this information. An efficient algorithm based on fitting a cubic equation through the data at two points is often using this algorithm.
To start the procedure, point (a) is selected. The derivative is evaluated and a step h taken in the direction of decreasing f The derivative is chosen as the new origin. The step size h is doubled and the step repeated.
is zero or has changed sign, then a cubic is fitted to cubic and may be written as They may be found by solving a set of four equations, for convenience, the origin will be taken at the point a At the required solution: The solution of above equation are And , of the former of these , only the positive singe satisfies equation (9) Let us define: Then the most readily obtainable solution is obtained from: And the minimum is given by This solution is not acceptable, however, as when 0 = A , it becomes indeterminate. Davidon (1959) therefore replaces it by: In an n-dimensional problem. It is usual to accept the point x predicted by equation (18) as the minimum provided To fit the cubic we will use the function and its gradient at two points as (Sun, J., etc, 2003(Sun, J., etc, , 2005.

A new combined Novel and Cubic Line Searches:
This new technique combines between the cubic interpolation & Novel line searches to find the minimum of multi-dimension function subject to necessary condition 0 ) ( =  x f and sufficient condition )

1 Outline of the Combined Technique are as follows:
Step 1: Let 0 x be the initial point; set Step 2: Find the function of Step 3 : Step 5 : Set Step 6 : Find the function of Step 7: Check if and go to step (5); otherwise go to step (8).

Flowchart of CG algorithm with New Combined Novel and Cubic Line
Searches:

Numerical Results:
In this section, we discuss computational results of H/S-CG method using cubic and novel line searches techniques for finding the minimum of nonlinear objective functions. The results are obtained using Pentium IV Computer. The programs were written in FORTRAN Language. The comparative performance of the algorithms is evaluated by considering NOF , NOI , and TW is the best measure of actual work done. Table (1) includes the numerical results of (H/S) CG-algorithm by using cubic line search and Novel line search for 1000 4   n . Table (2) includes the numerical results of (F/R)-CG method by using cubic line search and employ new combined cubic-Novel line research for test function of 1000 4   n .