New Hybrid (Oren- Al-Bayati) Self-Scaling Algorithm With Armijo Line-Search

Received on: 7/11/2006 Accepted on: 24/12/2006 ABSTRACT In this paper the development, and numerical testing of a class of self-scaling Quasi-Newton update with adaptive step size are presented . In our work a new combined (Oren-Al-Bayati) self-scaling algorithm is presented with a modified Armijo line search procedure. This algorithm has implemented both theoretically and numerically and tested on some well-known test cases. Numerical experiments indicate that this new algorithm is effective and superior to the standard, with respect to the number of functions evaluations (NOF) and number of iterations (NOI).


Introduction
The self-scaling update proposed by Oren [7] has some good characteristics. With a self-scaling parameter  , this class of updates can be written as where k R is defined by (3). And k  is defined by (2). Equation (4)

Proof:
The update (6) can be written as : , quoted earlier which holds for line searches. The vector * g can be substituted for * g H k by using property We also know that * BFGS d and * CG d are identical (See Nazareth [5]) and * new d is identical to * BFGS d with exact line searches. Hence equation (7) becomes Hence the proof . #

Original Armijo Line Search rule [9]:
Given o s  and   , are parameters Obviously, Armijo's rule is easy to implement and useful in practice. The most important advantage of the Armijo line search rule is that it enable use to estimate an initial test step-size s. Good estimation for s can make us cut down the function evaluation at each iteration. How to choosing the parameters ( such as   , , s ) is very important for practical problems. Several choosing techniques have been appearing in many literatures. In this paper, we propose a new inexact line search rule for Armijo for Quasi-Newton method and establish some global convergent results of this method. Theses results are useful in designing new Quasi-Newton methods with the new line search rule.

Inexact Line Search rule
We first assume that ) has a lower bound on the level set In what follows, we first describe the Quasi-Newton Method.
Step (3): and k  is determined by , modify k B as 1 + k B by using BFGS or DFP formula or other Quasi-Newton formulae.
Oren algorithm (III) processes the following properties for a quadratic function: for all k, then the vector k d are mutually conjugate (with respect to G) and hence the solution is obtained in at most n iterations.
The condition number of the matrix The proofs of these properties can be found in [2] and [6].

A New Hybrid (Oren-Al-Bayati) Self-Scaling VM Algorithm
Our objective is to propose a new class of two parameter updates which will combine the relative merits of different types of QN updates. This class of modified QN updates can be expressed as follows : This formula is invariant under linear transformation provided that k  and k  are constructed from invariant scalars. Other properties, such as termination and conjugate gradients, are also preserved. Likewise positive definite matrices are preserved for 0  k  and sufficiently large k  .

5.New Modified Armijo Line Search
We modify original Armijo algorithm as follows for a smooth differential function f and for an starting point 0 x with parameters   , 0 follow these steps by considering an stopping criterion  : Step (1): Step (2): and go to the next step, otherwise, go to step (8) Step (4): and go to the next step.
Step (8): The above algorithm provides an effective and very useful step-size adaptation procedure for various applications, for more details see [8].

The Out lines of New Preconditional CG Algorithm With New Armijo Line Search Procedure
Step (1): Step(2): For n k ,..., 2 , 1 = , set where k  is optimal step-size obtained from Armijo line search procedure.

Numerical Results
The comparative test involves eleven well-known standard test functions(given in the appendix) with different dimensions. The line search routine is a new modified Armijo line search which uses only function values.
The results are given in the Table (1A) is specifically quoting the number of function evaluations (NOF) an the number of iterations (NOI). All programs are written in FORTRAN 90 language and for all cases the stopping criterion is taken to be 5 1 10

Appendix :
All the test functions used in this paper are from general literature: