Iterative Image Interpolation vs. Traditional Interpolation Methods

Image interpolation is an important image processing operation applied in diverse areas ranging from computer graphics, rendering, editing, medical image reconstruction, and online image viewing. In this paper, image is interpolated from a lower resolution (LR) to a higher resolution (HR) based on the combination of two different procedures. First, an adaptive algorithm interpolating locally image pixel values along the direction is applied, where second order image derivative is low. Then interpolated values are modified using an iterative refinement to minimize the differences in second order image derivatives, maximize the second order derivative values and smooth the curves. The first algorithm itself provides edge-preserving images that are measurable better than those obtained with conventional methods presented in this literature. Objective and subjective tests on a series of natural images show the advantages of the used technique over existing approaches.


Introduction
Image interpolation is a technology for image resolution conversion that generates a high-resolution image from its low-resolution image counterpart [5]. In the digital age, interpolation has become increasingly important and, with the introduction of digital imaging, the subject of image interpolation has become increasingly popular [1]. It has been widely applied to various image/video capture/display devices such as digital still cameras, digital video cameras, printers, and multiple-function peripherals [5]. In literature, image interpolation techniques are referred to many terminologies such as image resizing, image resampling, digital zooming, and image magnification or enhancement [6].
Image interpolation algorithms can be grouped in two categories: adaptive and non-adaptive [6]. The adaptive methods exploit geometric structure or regularity of image features, such as step-edge, ridge and valleys. The orientation of such features is first extracted from the image, and then the interpolation is locally adapted to perform interpolation along the features, rather than across them [1]. In non-adaptive methods, certain computations are performed indiscriminately to the whole image for interpolation regardless of its contents. The computational logic of an adaptive image interpolation technique is mostly dependent upon the intrinsic image features and contents of the input image whereas computational logic on non-adaptive image interpolation technique is fixed irrespective of the input image features [6].
Besides the traditional interpolation techniques such as nearest-neighbor, bilinear, and bicubic interpolation, several algorithms have been used to enhance the performance of interpolating. In this paper, the applied Iterative Curve Based Interpolation (ICBI) method uses the local second order information to adapt the interpolation and an iterative refinement used to remove artifacts while preserving relevant image features and natural texture. The objective and subjective tests that are performed to evaluate the performance of the applied method against other traditional interpolation methods show that the ICBI method is clearly an improvement. The tests show that the ICBI method provides a reasonable reconstruction of the missing information and requires a considerably reasonable computational power.
The rest of this paper is organized as follows: Section 2 presents the concept of interpolation and reviews the conventional methods for image interpolation. In section 3, introduce the Iterative Curve Based Interpolation (ICBI), which considers the effects of the curvature continuity and enhancement. Section 4 presents some experimental results in terms of the image quality using the ICBI and some of traditional interpolation methods. Section 5 draws some conclusions.

Interpolation
Interpolation is a process used to estimate an image value at a location in between image pixels. Interpolation methods determine the value for an interpolated pixel by finding the point in the input image that corresponds to a pixel in the output image and then calculate the value of the output pixel by computing a weighted average of some set of pixels in the vicinity of the point. The weightings are based on the distance for each pixel from the point [7] The fastest scheme for calculating interpolated pixel is nearest-neighbor assignment (sometimes called zero-order interpolation) [4]. In this scheme, the output pixel is assigned the value of the pixel that the point fills within. No other pixels are considered [7]. The only advantage of nearest neighbor interpolation is its simplicity (no calculations are required to derive the output pixel value) and low computation [6]. However, it provides the worst results in terms of smoothness; and it does not generate any new data points [3].
A smoother interpolated image is obtained with bilinear (first order) interpolation. This algorithm uses the four input pixels surrounding the point (x,y) to estimate the output pixel (Fig. 1). Bilinear interpolating is usually implemented by first convolving the input image along its rows, creating new interpolated columns of pixels, and then along the new columns to create new interpolated rows of pixels, with a triangle weighting function in both directions [4]. The intensity value I(x,y) at the interpolated point P at (x,y) in the image can be considered as where I(x1,y1), I(x1,y2), I(x2,y1), and I(x2,y2) are the intensity values of the four neighboring pixels [6]. Bilinear interpolation uses more memory than the nearest neighbor method, and requires slightly more execution time [3]. Although the bilinear interpolation is smooth, there remains a jagged pattern along the curved road at the top. This is aliasing that originated in the digitization of the photograph [4].
The smoothing incurred with bilinear interpolation may be avoided with bicubic (second-order) interpolation, at the expense of more computation [4]. The output pixel value is a weighted average of pixels in the nearest 4-by-4 neighborhood [7]. The general form for a bicubic interpolation is given by [ where ai,j is 4x4 array containing the values of the 16 surrounding points.
x: horizontal distance between the point and the four points left of it, between (0-1) y: vertical distance between the point and the four points below it, between (0-1) The number of pixels considered affects the complexity of computation, therefore, the bicubic method takes longer time than bilinear interpolation. However, the greater the number considered, the more accurate effect is, so there is a tradeoff between processing time and quality [7].

Iterative Curve Based Interpolation
Iterative curve based interpolation (ICBI) method is based on the composition of two different procedures. The first is a simple rule-based hole-filling method (filling gabs in the enlarge grid at locations (2x+1,2y+1)) which computes new samples by interpolating along the direction where the second order image derivative is lower. The second refines the values of the interpolated pixels through an iterative algorithm trying to force second order derivative continuity with some additional constraints for edge preservation [2]. First, the energy term is introduced as where I11, I22 are local approximations of second order derivatives, computed as The energy term in equation (3) sums local directional changes of second order derivatives. Weights wi are set to 1 when the first order derivative in the corresponding direction is not larger than a fixed threshold and to 0 otherwise. In this way smoothing is avoided because there is a strong discontinuity. The minimization of this term roughly corresponds to force continuity in the local curvatures of the continuous surface ideally describing the image. The only problem of this method can be an excessive smoothing, which can be reduced by adding an energy term to enhance the absolute value of the second order derivative [2].
This energy alone is not able to produce very good and natural interpolated images, therefore it is necessary to add a derived potential to the curvature continuity and curvature enhancement term to improve the quality of interpolation results in edge regions. The potential is defined as The ICBI method considers the effects of the curvature continuity, and curvature enhancement. By proper weighting between these effects, the ICBI method produces perceptually pleasant image. However, the performance depends on the chosen parameters [8].

Experimental Results
The experiment presented on four images "Img1", "img2", "Img3" & "img4" which are used as benchmark images for testing to demonstrate the generalization ability of the used approaches. The ICBI method has been compared with two traditional interpolation methods namely bilinear and bicubic methods. The algorithms used in this paper implemented in matlab running on a PC with Intel(R) Pentinum(R), 1.70GHz CPU and 512MB Ram. The test images as shown in Figures (2a), (3a), (4a) & (5a) are resized by a factor of two, four and eight with the decimation approach respectively. Then the images in Figures (2 b-d), (3 b-d) (4 b-d) and (5 b-d) are interpolated to 256×256 by a factor of two, four and eight, respectively using the bilinear, bicubic, and ICBI interpolation methods. The interpolated images are compared with the original high-resolution images for subjective evaluation.
For subjective performance evaluation, the Peak Signal-to-Noise Ratio (PSNR) is used and defined as in equation (7)  where Lx,y and Yx,y are the pixels in the original and interpolated images at location (x,y), respectively. H & W are the height and width of the image, respectively [8]. The PSNR numbers indicate the overall image quality with errors between the original and interpolated results pixel-by-pixel [5]. The results of PSNR on the test images (Img, Img2, Img3 and Img4) are summarized in Table (1). The ICBI method produced clearly the bested results, but the most interesting fact is that the iterative method is relevantly slower than bilinear and bicubic methods. to 128×128 (c) "Img2" resized to 64×64 (d) "Img2" resized to 32×32. Table 1: PSNR values in dBs of interpolating "Img1", "Img2, "Img3", & "Img4" images with bilinear, bicubic, and ICBI interpolation methods.   In order to compare perceived image quality, the tested RGB images are taken and resized them by a factor of two, four and eight with three different algorithms (bilinear, bicubic, and ICBI). Then a group of 15 people are asked to compare them in order to select the method providing the best average "perceived quality". All the different possible couples of corresponding images are presented (in random order) to the subjects involved in the test, who are asked to choose the preferred image for each of them. The sum of the successful comparisons for each interpolation method is then taken as the quality score of the method itself.

Images
For visual quality comparisons, Figures (6), (7), (8) & (9) show the resolution enhanced portions of "Img1", "img2", "Img3" and "img4" images by using bilinear, bicubic, and ICBI methods. It can be noticed that the bilinear interpolation made some jaggedness along the edges and blurred the image. The bicubic method made less blurring than the bilinear method, but it made more observable jagged edges. The used ICBI method creates images that have natural aspect and present less artifacts than the images interpolated with bilinear and bicubic technique. So, images interpolated with ICBI are always rated as the best. The program steps of the performance the ICBI method is presented as followings:

Conclusions
In this paper, the implementation of the Iterative Curvature Based Interpolation (ICBI) method and the traditional (bilinear, bicubic) methods on "Img1" and "img2" images are presented. The results show that the traditional methods put the original pixels in an enlarged grid and fill holes averaging neighboring pixels in two steps whereas, the ICBI method is interpolated directionally in order to preserve locally second order derivatives. Thus the (ICBI) adds to the previous method an iterative refinement that try to reduce second order derivatives changes keeping original pixel values fixed and also preserving large gray level variations and smoothing.
The performance of the ICBI method has been verified with extensive simulation and comparison with bilinear and bicubic interpolation methods. Simulation results showed that the presented method has achieved outstanding perceptual performance with consistent subjective and objective performance. Finally, because of iteration in ICBI method and manual programming in matlab, the execution time will increase when using this technique.