A Sufficient Descent Property for a Different Parameter to Enhance Three-Term Method

Abstract


INTRODUCTION
Consider the following unconstrained nonlinear optimization problem: with being soft and ( ) ( ). One method for obtaining the smallest amount (1) [1] is the nonlinear CG method, which does not demand any matrices. This is how iterative CG methods look.
, (2) such that denotes a positive step size and denotes the new search direction, which is typically calculated as follows: The scalar parameter is typically selected so that (2) - (3) can be diminished to the linear CG method [2].
Similarly, if r(x) If the exact line search yields a purely convex quadratic function, then (ELS). [3 -9] define six pioneering forms of .
However, for large-scale problems, an exact line search is usually not possible, so any value of that meets certain properties known as SWC is accepted.
,and that is a path to the minimum must be descent [10].

MOTIVATION AND FORMULA
Many scholars have recently proposed plenty of three-term CG (TTCG) methods for unconstrained optimization problems as modifications of the classical CG algorithms Using the Armijo line search, Zhang et al. [11] suggested a three-term MPRP procedure and demonstrated its global such that ,the TTHS method was created by Zhang et al. [12] in a similar content. This is written as: The TTHS method has the steepest descent capability; when an ELS would be used, it is reduced to the classic HS method. Furthermore, to ensure the global convergence properties of the search direction specified in (7), an MTTHS algorithm on the search direction is used: Offered that MTTHS were presented in (7) to indicate the search direction's global convergence properties, one could guess why (7) is not used to prove the search direction's global convergence properties. Rather than disregarding (7), it should be made efficient and globally convergent. As a result, (7) can be tweaked to meet the global convergence requirements. This modification is expected to outperform the MTTCG algorithm in the sense of numerical effectiveness. Zhang et al. [13] proposed a new Dai-Liaobased TTCG method motivated by this appealing descent property: Such that and . Once again, regardless of the line search method used, the sufficient descent process occurs. i.e. for this method, ‖ ‖ for all k. Al-Bayati and Sharif [14] developed a specialization of the TTCG given by (9) that evaluates the search direction as We propose the following three conjugate gradient terms: Many scholars have studied the values of and which are critical for the algorithm's global convergence and numerical efficiency (see [15][16][17][18][19]).
When investigating the sufficient descent condition, the step-size is critical as well as global convergence properties We start with a motivation explanation before delving into the specifics of our approach. Mandara and colleagues [20] posited an RMIL+ based new CG formula.
They keep the RMIL+ denominator but change the sign of the numerator from negative to positive.
Create a globally convergent method in the face of an imprecise line search (ILS). In this part, we will implement a novel N3T method by modifying the as follows: If ELS used, then (15) reduced to (14). The N3T method is defined as follows: .

|| ||
We get after some algebraic operations with , we obtain We will notice that equation (19)  , proposed by Al-Bayati [14].
Step 2: If ‖ ‖ then end; or else, go on to the next step.
Step 6: If ‖ ‖ , stop; Otherwise, proceed to the next step.
Step 7: If k=n or | | (‖ ‖ ) is satisfied, go to step 1; Or else, proceed to the following step.

N3T 's Global Convergence property
This part investigates the convergence of the suggested method. The sufficient descent condition is one of the requirements for an algorithm to converge. The proof of the sufficient descent condition of the proposed method with ILS is shown below.
When we multiply both sides of (16) by , we get

|| ||
Mandara and colleagues [20] proved that We obtain by applying the second condition of the SWC To establish the global convergence property of an algorithm, the following basic presumptions on the objective function must be made.
ii-The function r is continuously differentiable, and its gradient is Lipschitz continuous in a specific neighborhood of , with the notable exception that there is a constant such that: Since * ( )+ is decreasing, it is clear that the sequence * + generated by N3T Algorithm is contained in .
Furthermore, using the N3T Algorithm, we can deduce from Presumption (I) that there is a positive constant, resulting in ‖ ‖ [24].
As a result of the inconsistency with the Zountendijk theorem [25], ‖ ‖

EXPERIMENTS WITH NUMBERS
The main task of this section is to report on the performance of the N3T algorithm on a set of test problems. Fortran77 and double precision arithmetic were used to write the codes.
Our experiments were carried out on a set of 28 nonlinear unconstrained problems, and we compared the proposed algorithm, which contains four types of parameter that are dependent on the parameter , (NEW-A1, NEW-A2, NEW-A3, and NEW-A4), and we tried on small dimensions (n=100) and large dimensions (n=1000) of the variables with SWC with and , respectively. The same test functions were used to compare our algorithms' reliability to the well-known routines Dx [26], LS [27], MMAU [28], RMIL [29], and MMWA [20].
When the following stopping criteria are met ‖ ‖ , all of these methods terminate. These routines are also forced to stop if the number of iterations exceeds 600.
The tests were based on number of iterations (No. I) and the number of function evaluations (No. F). As shown in the figures below, the four suggested algorithms outperform the other algorithms. At the same time, when we compare the four suggested algorithms, the NEW-A4 is the best, followed by the NEW-A3, the NEW-A2, and finally the NEW-A1.
The Dolan and More' method [30] was used to plot these results for better comparison, and the results are shown in the figures below: