Using "Filter" Approach to Solve the Constrained Optimization Problems

In this paper, the solution of constrained nonlinear programming problems by a Sequential Quadratic Programming (SQP) is considered. The aim of the present work is to promote global convergence without the need to use a penalty and Barrier functions in the mixed interior-exterior point method. Instead, a new concept of a “filter” that aims to minimize the objective function and its approach that allows appoint to be accepted if reduces the objective function and satisfies the constraint violation function. If that point is rejected a new point is tested. Keyword: Filter approach, constrained Optimization, SQP method. ةديقملا ةيلثملأا لئاسم لحل حشرملا بولسا مادختسا سارتم دمحا ناب احلا مولع ةيلك بوس تايضايرلاو لصوملا ةعماج ثحبلا ملاتسا خيرات : 1/3/2009 :ثحبلا لوبق خيرات 24/6/2009


Introduction:
The ideas that are introduced in the filter approach are such that it is not obvious how to apply standard techniques for proving global convergence of penalty function methods (that is, convergence to a stationary point from an arbitrary initial estimate). At present the question of global convergence is open, although it has provoked some interesting discussions with other colleagues. In the context of our algorithm, heuristics are readily suggested that exclude obvious situations in which the algorithm might fail to converge. However we observe in practice that we can dispense with these heuristics and yet solve a substantial proportion of standard test problems using SQP algorithm without penalty function.
Consider the most general form of the problem Subject to the general (possibly nonlinear) inequality constraints ... (2) and (possibly nonlinear) equality constraints with the simple bounds , …(4) where f and cj are all assumed to be twice continuously differentiable and any of the bounded in eq.(4) may be infinite. [4]

The Exterior-Point Method (Penalty Function):
The exterior-point method is suitable for equality and in equality constraints. The new objective function where k  is a positive scalar and the remainder of the second term is the penalty function.

The Interior-Point Method (Barrier Function):
Interior-point method is suitable for inequality constraints. The new objective function  is a positive scalar and the reminder of the second term is the Barrier function [6].
Although both exterior and interior-point methods have many points of similarly, they represent two different points of view. In an exterior-point procedure, we start from an infeasible point and gradually approach feasibility. While doing so, we move away from the unconstrained optimum of the objective function. In an interior-point procedure we start at a feasible point and gradually improve our objective function, while maintaining feasibility. The requirement that we begin at a feasible point and remain within the interior of the feasible inequality constrained region is the chief difficulty with interior-point methods. In many problems we have no easy way to determine a feasible starting point, and a separate initial computation may be needed. Also, if equality constraints are present, we do not have a feasible inequality constrained region in which to maneuver freely. Thus interior-point methods cannot handle equalities.
We many readily handle equalities by using a "mixed" method in which we use interior-point penalty functions for inequality constraints only. Thus, if the first m constraints are inequalities and constraints (m+1) to n are equalities, our problem becomes: The function is then minimized for a sequence of monotonically decreasing In this paper, a modification of mixed exterior-interior point method is proposed based on the use of filter approach.

An Nonlinear Programming Filter: [5]
This section describes the basic concepts that underpin the new approach towards inducing global convergence in SQP. There are two competing aims in nonlinear programming. The first is the minimization of the objective function f and the second is the satisfaction of the constraints. Conceptually, these two conflicting aims can be written as Minimize h(c(x)), …(9) where The problem of satisfiability has now been written as a minimization problem.
A penalty function combines eq.(8) and eq.(9) into a single minimization problem. Instead we prefer to view eq.(8), eq.(9) as separate aims, akin to a multi-objective optimization problem. However our situation is somewhat different from multi-objective optimization in that it is essential to find a point for which h(c(x))=0 if at all possible. In this sense the minimization of h(c(x)) has priority. Nevertheless, it is convenient to make use of the concept of domination from multi-objective optimization. Let (fk,hk) denote values of f(x) and h(c(x)) evaluated at xk. In the context of nonlinear programming, this means that xk is at least as good as xl with respect to eq.(8) and eq.(9). With this concept it is now possible to define a filter, which will be used in a trust-region type algorithm as a criterion for accepting or rejecting a trail step.

Definition 2:
A filter is a list of pairs (fl,hl) such that no pair dominates any other. A point (fk,hk) is said to be acceptable for inclusion in the filter if it is not dominated by any point in the filter.
We stress that only two scalars are stored for each entry in the filter. No vectors such as xl are stored and this has negative implications for the use of backtracking in the resulting algorithm. Each point in the filter generates a block of non-acceptable points and the union of these points represents the set of points that are not acceptable to the filter.
The idea is to use the filter as a criterion for accepting or rejecting a step in an SQP method. Starting with xk, the solution of the quadratic problem produces a trial step dk. Set xk+1=xk+dk, the new trial point xk+1 is acceptable by the filter if the corresponding pair (fk+1,hk+1) is not dominated by any point in the filter [5].

Mixed Exterior-Interior Point Methods:
We can solve the constrained problem given in eq.(1) to eq.(3) construct a new objective function which is defined in eq.(7). Now our aim is to minimize the function  where k  is a constant equal to 10 and the search direction dk in this case can be defined …(11) where H is a positive definite symmetric approximation matrix to the inverse Hessian matrix G -1 and g is the gradient vector of the function The next iteration is set to further point …(12) where  is a scalar chosen in such that We thus test ci(xk+1) to see that it is positive for all i. We find a feasible xk+1 and we can then proceed with the interpolation. Then the matrix Hk is updated by a correction matrix to get: where vk and yk are difference vector between two successive points and gradients respectively and  is any scalar.
The initial matrix H0 chosen to be identity matrix I. Hk is updated through the formula of BFGS update [1] [7].

Combined Barrier-Penalty Algorithm:
Step (1): Find an initial approximation x0 in the interior of the feasible region for the inequality constraints i.e. gi(x0)<0.
Step (2): Set k=1 and 1 0 =  is the initial value of k  .

5.New Nonlinear Programming without Penalty and Barrier Functions:
Here, we described the concept of filter to nonlinear programming without a Penalty and Barrier Functions such that:

Minimize f(x), …(14) and
Minimize h1(c(x)), …(15) Minimize h2(c(x)), …(16) where is the L1, L2 norm of the constraint violations. Here A penalty and Barrier functions combine eq.(14); eq.(15) and eq.(16) into a single minimization problem. Instead we prefer to view eq.(14); eq.(15) and eq.(16) as separate aims, akin to a multi-objective optimization problem. However our situation is somewhat different from multi-objective optimization in that it is essential to find a point for which h1(c(x))=0 and h2(c(x))=0 if at all possible. In this sense the minimization of h1(c(x))and h2(c(x)) have priority. Then, we have two new definitions:

Definition 4:
A filter is a list of pairs (fl,h1,l) and (fl,h2,l) such that no pairs dominates any other. A points (fk,h1,k) and (fk,h2,k) are said to be acceptable for inclusion in the filter if it is not dominated by any point in the filter.

New Filter Mixed Exterior-Interior point Algorithm:
Step (1): Find an initial approximation x0 in the interior of the feasible region for the inequality constraints i.e. gi(x0)<0.

Results and Calculation:
Several test functions were tested with different dimensions. Our programs were written in FORTRAN 90.
In order to test the effectiveness of the new algorithm that have been used to Barrier-Penalty function method, the comparative tests involving several well-known test function (see Appendix) have been chosen and solved numerically by utilizing the new and established method. So the new algorithm has been compared with some established algorithm.
In  (2) it is clear that, taking the standard Barrier-Penalty algorithm as 100%, and the new algorithm has 61.9%, 48.7% improvements on the standard Barrier-Penalty algorithm in about number of iterations NOI and number of function evaluations NOF respectively.