Image Compression Technique Using a Hierarchical Neural Network

This paper present a Resilient Backpropagation (RBP) algorithm based on hierarchical neural network for image compression. The proposed technique includes steps to break down large images into smaller blocks for image compression/ decompression process. Furthermore, a Linear Backpropagation (LBP) algorithm is also used to train hierarchical neural network, and both training algorithms are compared. A number of experiments have been achieved, the results obtained, are the compression rate and Peak Signal to Noise Ratio PSNR of the compressed/ decompressed images which are presented in this paper.


Introduction
Image compression is a key technology in the development of various multimedia computer services and telecommunication applications, such as teleconferencing, digital broadcast codec and video technology, etc [5]. Traditional techniques that have already been identified for data compression include: predictive coding, transform coding and vector quantization [2].
Artificial Neural Networks ANNs have been applied in many problems; have demonstrated their superiority over traditional methods, when dealing with noisy data. One such application is for image compression. Neural networks seem to be well suited to this particular function, as they have the ability to preprocess input patterns to produce simpler patterns with fewer components [6]. This compressed information (stored in a hidden layer) preserves the full information obtained from the external environment. Not only can ANN based techniques provide sufficient compression rates of the data in question, but security is easily maintained. This occurs because the compressed data that is sent along a communication line is encoded and does not resembles its original form.
Several literatures were discussed the subject of applying ANNs to image compression in detail as in [2][3][4][5]. Many different training algorithms and architectures have been used. Some of those are: linear backpropagation [5], and sigmoidal back propagation [10], training two layer feed forward auto associative neural networks. Radial basis function RBF neural network [11], and self-organizing maps SOMs neural network, which after training acts as a code book [1].
The purpose of this paper is to implement a technique for compression of images using a Resilient Backpropagation RBP algorithm [8] based hierarchical feed forward auto associative neural network [5]. The method for training is faster than some of the more popular algorithms such as Backpropagation [9]. Results are presented for hierarchical neural network trained with both Resilient Backpropagation RBP algorithm, and Linear Back propagation LBP algorithm. The results convey information about the compression rate achieved, the Peak Signal to noise ratio PSNR, and a comparison of the images after decompression. This paper is organized as follows: Section 2, states the RBP training algorithm. Section 3, discusses the hierarchical neural network architecture for image compression. Results are presented in section 4. A discussion of the results is supplied in section 5, and conclusions are drawn in section 6.

Resilient Back propagation RBP Algorithm
The conventional Backpropagation BP learning algorithm proposed by Rumelhart et. al. [9] uses the partial derivatives of the error function (i.e. the gradient) to minimize the global error of the neural network by performing a gradient descent.
The choice of the learning rate parameter  , which scales the derivative, has an important effect on the time needed until convergence is reached. As it can be easily observed, the size and the sign of the derivative are considered when updating the synaptic weights ij w . Resilient Backpropagation RBP algorithm is a local adaptive learning scheme, much faster and stable than other usual variations of the backpropagation algorithm. The basic principle is to eliminate the harmful and un-foreseeable influence of the size of the partial derivative on the weight step. So, only the sign of the derivative is used to indicate the direction of the weight update. The size of the update is given by a weightspecific update value The update values are determined from the following equation: The RBP learning algorithm is based on learning by epoch; that means the weight update is performed only after the gradient information is completely available, after each training pattern has been presented and the gradient of the sum of pattern errors is known.
Some authors have noticed that when using standard back propagation [9], the weights in the hidden layer are updated with much smaller amounts than the weights in output layer, so they modify much slower. Another advantage of RBP is that all of the weights grow uniformly.

Hierarchical Neural Network for Image Compression [1]
The hierarchical neural network structure can be shown in Fig. (1), in which the three hidden layers are termed as the combiner layer, the compressor layer, and decompressor layer. The idea is to exploit correlation between pixels by inner hidden layer and to exploit correlation between blocks of pixels by outer hidden layers. From the input layer to the combiner layer and from the decombiner layer to the output layer, local connections are designed which have the same effect as M fully connected neural sub-networks. As seen in Fig. (1), all three hidden layers are fully connected. The basic idea is to divide an input image into M disjoint subscenes and each sub-scene is further partitioned into T pixel blocks of size  The total number of neurons for the decombiner layer and the output layer is the same as that of the combiner layer and the input layer, respectively. Training of such neural network can be explained as follows: Outer loop neural network (OLNN) training. By taking the input layer, the combiner layer and the output layer out of the network shown in Fig. (1), we can obtain M 64-8-64 outer loop neural networks where 64-8-64 represents the number of neurons for its input layer, hidden layer and output layer, respectively. As the image is divided into 8 sub-scenes and each sub-scenes has 128 pixels blocks each of which has the same size as that of the input layer, we have 8 training sets to train the 8 outer-loop neural networks independently. Each training set contains 128 training patterns (or pixel blocks). In this training process, the proposed RBP learning rule is directly applied in which the desired output is equal to the input.
(ii) Inner loop neural network (ILNN) training. By taking the three hidden layers in Fig. (1) into consideration, an inner loop neural network can be derived as shown in Fig. (2). As the related parameters are designed in such a way that 8, the inner loop neural network is also a 64-8-64 network. Corresponding to the 8 sub-scenes each of which has 128 training patterns (or pixel blocks), we also have 8 groups of hidden layer outputs from the operation of step 1, in which each hidden layer output is an 8dimensional vector and each group contains 128 such vectors. Therefore, for the inner loop network, the training set contains 128 training patterns each of them is a 64dimensional vector when the outputs of all the eight hidden layers inside the OLNN are directly used to train the ILNN. Again, the RBP learning rule is used in the training process. Throughout the two steps of training for both ILNN and OLNN, the linear transfer function (or activating function) is used.
Reconstruction of the over all neural network. From the previous two steps of training, we have four sets of coupling weights, two out of step 1 and two out of step 2. Hence, the over all neural network coupling weights can be assigned in such a way that the two sets of weights from step 1 are given to the outer layers in Fig. (1) involving the input layer connected to the combiner layer, and the decombiner layer connected to the output layer. Similarly, the two sets of coupling weights obtained from step 2 can be given to the inner layer in Fig. (2) involving the combiner layer connected to the compressor layer and the compressor layer connected to the decombiner layer. After training is completed, the neural network is ready for image compression in which half of the network acts as an encoder and the other half as a decoder. The neuron weights maintained the same throughout the image compression process.

Performance assessments
The performance of the hierarchical neural network trained by RBP and LBP learning rules can be assessed by considering the following measurements: 1.compression ratio [7], and can be defined for a narrow channel compression neural network as follows: where q is the number of bits used to quantize each neuron output of compressor layer. [4], in lossy compression, the peak signal to noise ratio PSNR is used as the measure of similarity or of dissimilarity, although it does not necessarily reflect visual quality. Assuming that the original and reconstructed images are represented by functions ) ,

Peak Signal to Noise Ratio PSNR
( y

Experimental Results and Discussions
The RBP based hierarchical neural network from section 2, along with the LBP algorithm were tested, and the results were compared for the task of image compression. We take Lenna as one of most widely used image for testing image compression algorithms. Tables (1 and 2) summaries the experimental results, and Figures (3 to 6) show the compression performance for the RBP and LBP based hierarchical neural network for different network parameters.  Tables (1 and 2) show the performance of RBP and LBP learning rules. The compression rate in the two tables reflected how accurately the images were decompressed. The two schemes showed high deterioration of the image quality for high compression rates and very little deterioration when using an appropriate number of neurons in the internal hidden layer (compressed layer) of the neural networks. As the number of neurons in the compressed layer was lowered, the greater deterioration in the decompressed images. However, the lower the number of compressed layer neurons, the better the compression ratio of the image.
As can be seen from figures (3 to 6) and figures (7 to 9 ,)which test image compression / decompression capabilities and performance of the RBP and LBP. All images suffered in quality with different degrees but no deterioration. A higher Peak Signal to noise ratio PSNR was achieved for RBP learning rule, and lower PSNR for LBP learning rule, for the same hierarchical neural network parameters. Overall, it was found that superior performance could be achieved with the proposed RBP based hierarchical neural network, while the LBP shows inferior performance.

Conclusions
This paper has successfully applied Resilient backpropagation RBP to a large, complex task. The results appear to be promising in image compression/ decompression problems. It is used to train a hierarchical neural network. We segmented, compressed, decompressed, and reconstructed various images using this method. A number of experiments have been conducted. Results showed that a superior PSNR could be achieved with proposed RBP compared with LBP.