ABSTRACT
Research attention has been focussed on the reduction of image data size (major problem) for its efficient compression, storage, and transmission. In this work, the developed brightness enhancement model was used to enhance the Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), Lifting Wavelet Transform (LWT), and Firefly Optimization Algorithm (FOA) compression, which were then used to compress the six video data. These video samples were obtained from the cameras of National Agricultural Extension Research Liaison Service (NAERLS),Ahmadu Bello University, Zaria (as NAERLS1.avi and NAERLS2.avi), Nigerian Television Authority (NTA), Abuja (as NTA1.avi and NTA2.avi), and the benchmark online media database (as Foreman.avi and Akiyo.avi). In the pre-processing stage, the video data were converted into frames of pictures for easy analysis. The hue and saturation were then extracted and the images were noised and filteredusing MATLAB R2014b simulation environment. Based on the analysis of random variation of pixel intensity and histogram distribution, the developed brightness enhancement technique was used to improve frame signal as a result of loss during compression. The performances of various enhanced compression techniqueswere evaluated through a number of MATLAB R2014b simulations using Peak signal to noise ratio(PSNR) as a performance metric. The results showed that the PSNR values for the grey level (black and white) imageswere improved by 31.95dB and 22.30dB for NAERLS1.avi and NAERLS2.aviwhen subjected to brightness enhancement technique.Also, PSNR improvements of 17.71dB and 23.31dB were obtained for the NTA1.avi and NTA2.avi, respectively, as well as15.06dB and 19.17dB improvements were obtained for the Foreman.avi and Akiyo.avi benchmark samples respectively. Similarly, improvement in terms of PSNR was also registered when coloured images were subjected to the developed brightness enhancement technique. The research implemented four video compression techniques DCT, DWT, LWT, and FOA compression,which were used as benchmarks for the developed modified FOA (mFOA)compression technique. Their respective outputswereimproved using the developed brightness enhancement model in order to account for the loss of signal quality which mighthave occurredduring compression. PSNR simulation results showed that themFOA compression technique performed better than DCT, DWT, LWT, and FOA compression techniques. For example,before enhancement,it was found that the mFOAPSNR result was better than the LWT by 73.64%, 80.04%, 80.03%, and 80.40%,respectively for NAERLS1.avi, NAERLS2.avi, NTA1.avi and NTA2.avi captured video frames and an improvement of 75.78% and 77.56% for Akiyo.avi and Forman.avi benchmark video frames.The mFOA was also discovered to outperform the FOA by 7.34%, 3.30%, 4.90%, and 5.75% for NAERLS1.avi, NAERLS2.avi, NTA1.avi and NTA2.avi captured video frames before enhancement and an improvement of 3.56% and 3.86% for Akiyo.avi and Forman.avi benchmark video frames. Similarly, the enhanced mFOA (E-mFOA) compression technique also producedPSNR improvement of 72.09%, 79.04%, 79.51% and 78.81% over enhanced LWT (E-LWT) for NAERLS1.avi, NAERLS2.avi, NTA1.avi and NTA2.avi capture video frames and an improvement of 74.67% and 76.08% for Akiyo.avi and Forman.avi benchmark video frames.The E-mFOA compression technique also produced a better PSNR improvement of 4.59%, 1.14%, 2.08%, and 1.17% over E-FOA for NAERLS1.avi, NAERLS2.avi, NTA1.avi and NTA2.avi captured video frames, except for the Akiyo.avi and Forman.avi benchmark video frames, where an insignificant improvement of 0.41% and -0.06%
viii
were registered. These might have been as a result of the low level of light present when the video clips were taken.
TABLE OF CONTENTS
Title page- – – – – – – – – – – i Declaration- – – – – – – – – – – ii Certification- – – – – – – – – – – iii Dedication- – – – – – – – – – – iv Acknowledgements- – – – – – – – – – v Abstract- – – – – – – – – – – vii Table of Contents- – – – – – – – – – ix List of Figures- – – – – – – – – – xiii List of Tables- – – – – – – – – – – xiv Abbreviations And Acronyms- – – – – – – – xvii Mathematical Symbols – – – – – – – – – xxi CHAPTER ONE: INTRODUCTION 1.1 Background- – – – – – – – – – 1 1.2 Aim and Objectives- – – – – – – – – 3 1.3 Statement of Problem- – – – – – – – 4 1.4 Scope and Limitations- – – – – – – – 5 1.5 Research Motivation- – – – – – – – – 5 1.6 Hypothesis- – – – – – – – – – 6 1.7 Research Methodology- – – – – – – – 6 1.8 Thesis Organization- – – – – – – – – 7 CHAPTER TWO: LITERATURE REVIEW 2.1 Introduction- – – – – – – – – – 9 2.2 Review of Fundamental Concept- – – – – – – 9 2.2.1 Different Types of Digital Image- – – – – – – 9 2.2.1.1 Image layout – – – – – – – – 10 2.2.1.2Image Colour – – – – – – – – 10
x
2.2.2 Digital Image Definitions- – – – – – – – 13 2.2.2.1 Common Values- – – – – – – – – 14 2.2.3 Resolution and Quantization – – – – – – – 16 2.2.4 Digital Video – – – – – – – – – 18
2.2.4.1 Digital Image Statistics – – – – – – – 18 2.2.5 Digital video Processing- – – – – – – – 22 2.2.5.1 Video Quality Measure- – – – – – – – 22 2.2.6 Pixel Operations – – – – – – – – 24 2.2.6.1. Intensity Scaling – – – – – – – – 24
2.2.6.2 Image Histogram Equalization – – – – – – – 25
2.2.6.3 Histogram Shaping – – – – – – – – 27
2.2.7 Arithmetic Operations between Images – – – – – 29 2.2.8 Image Noise – – – – – – – – – 30
2.2.8.1 Sources of Noise in Images- – – – – – – – 30
2.2.8.2 Different Noise Types- – – – – – – 31
2.2.9 Image De-noising – – – – – – – – 34 2.2.10 Linear Filters and Non Linear Filters – – – – – – 35 2.2.10.1 Different Type of Linear and Non-Linear Filters- – – – – 35 2.2.11 Fundamental Steps in Image Processing – – – – – 37
2.2.12 Methods of Noise Suppression – – – – – 39
2.2.13 Images Differences for Change Detection – – – – – 41 2.2.14 Frequency Domain Techniques – – – – – – 42
2.2.15 Video and Image Compression – – – – – – 45
2.2.15.1 Huffman Coding- – – – – – -46 2.2.15.2 Arithmetic Coding- – – – – – 46
xi
2.2.15.3 Substitution (Dictionary based) Coding- – – – 47
2.2.15.4 Sample/based Coding- – – – – – – 47
2.2.15.5 Transform Domain Coding- – – – – – 48
2.2.15.6 Wavelet-based Coding- – – – – – – 49
2.2.16 Discrete Cosine Transform – – – – – – – 51 2.2.17 Discrete Wavelet Transform – – – – – – – 54 2.2.18 Lifting Wavelet Transform – – – – – – – 55 2.2.19 Purpose of Image and Video Processing – – – – – 55 2.2.20 Application of Image Processing – – – – – – 56 2.2.21 Firefly Optimization Algorithm – – – – – – 56 2.2.22 Performance Metric – – – – – – – – 61 2.3 Review of Similar Works – – – – – – – 62 2.3.1 Review of Similar Works on Image Processing – – – – 62 CHAPTER THREE: MATERIALS AND METHODS 3.1 Introduction- – – – – – – – – – 84
3.2Materials used in this research work – – – – – 84
3.3Research Methodology- – – – – – – – 85 3.3.1 Pre-Processing- – – – – – – – – 85 3.3.2 Video Acquisition- – – – – – – – – 85 3.3.3Eliminate Hue and Saturation and Retain Luminance Intensity- – – 88 3.3.4 Noising and Filtering – – – – – – – – 89 3.3.5 Brightness Enhancement- – – – – – – – 89 3.4 Enhanced Discrete Wavelet Transform- – – – – – 94 3.5Enhanced Discrete Cosine Transform- – – – – – 98 3.6Enhanced Lifting Wavelet Transform- – – – – – 101 3.7Compression Using Fire Fly Algorithm- – – – – – 105
xii
CHAPTER FOUR: RESULTS AND DISCUSSION 4.1 Introduction- – – – – – – – – – 111 4.2 Results- – – – – – – – – 111 4.3 Performance Evaluation Using Peak Signal to Noise Ratio- – – – 121 4.4 Compression Analysis- – – – – – – – 124 4.4.1 Comparison of Techniques before Enhancement- – – – 129 4.4.2 Comparison of Techniques after Enhancement- – – – – 129 4.5 Minimization Plots – – – – – – – – 130 CHAPTER FIVE: SUMMARY, CONCLUSION AND RECOMMENDATIONS 5.1 Introduction- – – – – – – – – – 134 5.2 Summary of Findings – – – – – – – – 134 5.3 Conclusion- – – – – – – – – – 135
5.4 Significant Contributions- – – – – – – 137
5.5 Recommendations for Further Research- – – – – – 139 REFERENCES- – – – – – – – – 140 Appendix A M-file implementation- – – – – – – – 153 Appendix B M-File Implementation for Coloured Frames- 156 Appendix C M-file implementation of Discrete WaveletTransform – 157 Appendix D: M-File Implementation of Discrete Cosine- Transform 160 Appendix E M-File Implementation of Lifting WaveletTransform – 162 Appendix F M-Implementation of Fire Fly Algorithm 167 Appendix E2 M-file of objective function 171 Appendix F2
M-Implementation of Modified Fire Fly Algorithm (mFOA) 174
xiii
CHAPTER ONE
INTRODUCTION
1.1 Background
Digital image processing has rapidly evolved over the decade, with increasing applications in the field of engineering and science. The image processing is a visual task, which involves image acquisition, enhancement and processing at the final stage. The image processing mostly involves taking an array of picture elements (pixels) as input and producing an array as output pixels which usually represent an improvement on the original picture. Image enhancements are employed in order to increase the contrast of the image, thereby enabling the visualization of the distinct features of the image. This usually augments the efficiency of image classification and interpretation. The digital image is considered as a large array of discrete dots, each of which has a brightness associated with it. These dots are called picture elements or simply pixels (McAndrew, 2004). Usually, captured images are often not a true reflection of the real object(s) the image is representing. For an efficient image processing, it is important to address some of the unwanted image background information (which cause a random variation in pixel intensity) by way of pre-processing. The main challenge for the pre-processing systems is that the captured images are often associated with low resolution. Although cameras on most mobile devices are capable of taking higher resolution images, the computation cost is still an issue nowadays (Optical character recognition) (Yu et al 2012). Image pre-processing is the term for operations carried out on images at the lowest level of abstraction. These operations do not increase image information content. The aim of pre-processing is an improvement of the image data that suppresses undesired distortions or enhances some image features such as edges and lane, etc relevant for further processing and analysis task. Image pre-processing used the redundancy in images. Neighbouring pixels corresponding to one real object have the same or similar brightness value. If a distorted pixel
2
is picked out from the image, it is restored as an average value of neighbouring pixels (Olgao, 2009). Image pre-processing methods are classified into categories according to the size of the pixel neighbourhood that is used for the calculation of new pixel brightness. Video data usually contain large amount of bits which makes it difficult for transportation and file shearing. This has prompted researchers in image processing to seek for ways of reducing this large amount of bits in the form of compression without necessarily damaging the image quality.
Compression refers to the process of reducing the number of bits required to represent the image and video (Vishnuet al. 2013). The main objective of video compression is to overcome the cost of transmission and required bandwidth. The size of video is also a factor that affects its transmission (Catania, 2008). Digital image compression comes in two forms namely lossless and lossy. The lossless compression is a process to reduce image or video data for storage and transmission while retaining the quality of original image (that is, the decoded image quality is required to be identical to image quality prior to encoding) (Vishnuet al. 2013). In lossy compression, on the other hand, some information present in the original image or video is discarded so that the original raw representation of image or video can only be approximately reconstructed from the compressed representation with high compression ratio (Vishnu et al. 2013). In other words, compression is a reversible conversion (encoding) of data that contains fewer bits. This allows more efficient storage and transmission of the data. The inverse process is called decompression (decoding). Software and hardware that can encode and decode are called encoders and decoders, respectively. In video, compression becomes necessary because the correlation between one pixel and its neighbouring pixels is high the values of one pixel and its adjacent pixels are very similar. This is called the intra-frame correlation in video compression because it is the correlation in a single frame. Once the correlation between the pixels is reduced, the storage quantity is
3
equally reduced (Djordje, 2006). The image compression method is also applied to video compression. However, there exists also temporal correlation. The video is composed of a large number of still images that are taken at short time distance of which any two neighbouring images are similar. Therefore, it is known that there exists high correlation between the images or frames in the time direction (Djordje, 2006). The correlation in the time direction is called the intra-frame correlation. If the intra-frame correlation can efficiently be reduced, then video compression can be achieved (Wei, 2010). This research focuses on developing an enhancement technique for efficient video compression using Firefly Optimization Algorithm (FOA), Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), and Lifting Wavelet Transform (LWT).
Fireflies are amongst the most charismatic of all insects and their spectacular courtship displays have inspired poets and scientists alike. Fireflies are characterized by their flashing light produced by biochemical process known as bioluminescence. Such flashing light may serve as the primary courtship signals for mating. Besides attracting mating partners, the flashing light may also be used to serve as warning to potential predators (Fister et al., 2013). These interestingcharacteristics of natural Fireflies inspired the development of FOA(Yang, 2008). 1.2 Aim and Objectives The aim of this research work is to develop a video frame enhancement technique and modify Firefly Optimization Algorithm (mFOA) for improvedvideo compression. In order to achieve this stated aim, the following objectives were employed:
1. To acquire video data and apply pre-processing technique in order to reduce the effect of background noise, as well as to develop a video brightness (luminance) enhancement modeland modify FOA for improving the visual quality of the video data using pixel intensity and histogram distribution.
4
2. To implementthe enhanced compressed sampled videos using Discrete Cosine Transforms (DCT), Discrete Wavelet Transform (DWT), and Lifting Wavelet Transform (LWT), FOA, as well as themFOA to obtain their respective coefficients.
3. To enhance the transformed coefficients developed in item2using the developed enhancement technique in item 1.
4. To estimate the quality of the final processed sampled video data using peak signal to noise ratio (dB), compression ratio, and size reduction efficiency(bytes) which are the fundamental metrics for measuring the performance of these compression techniques.
1.3 Statement of Problem
The three main challenges today in image and video enhancement are size of video window, frame rate and very much importantly quality of image or video. Digital video files are large, which makes them difficult for easy transportation and storage. Various quantization techniques and algorithms have been proposed in order to reduce the size of an image by reducing the number of colour content in a digital image(s) while preserving the significant information. However, high amount of quantization may lead to signal distortion between image regions and may not be acceptable to human visual perception. Image signal distortions are largely interpreted as a random variation in pixel intensity which results in an uneven distribution of image histogram, which is a problem in image compression. A lot of researchers have proposed various techniques for reducing large amount of size associated with video and enhanced efficient transportation and storage. Most ofthese methods deliver high reduction of image size in the form of compression. Usually, higher compression ratio is associated with lower size and quality degradation, which is also another problem associated with compression. Furthermore, the video signal encoding and decoding requires a high amount of computational resources and it is difficult for a real time application with low bandwidth requirement to compress a video with a computational expensive algorithm which
5
may take too long to encode and decode video data. This is also another limitation in image processing. Thus, this thesis developed a video frame enhancement technique based on pixel intensity, and histogram distribution for an improved video compression.
1.4 Scope and Limitations
i. The scope of this thesis is to produce a good quality video by developing and implementing brightness enhancement technique and Firefly Optimization Algorithm to achieve better video resolution output quality for the true analysis of the improvement rendered by these developed techniques compared to the standard ones.
This research did not consider the following:
ii. The audio content of the video data since the focus is on enhancing the visual content and size reduction.
iii. The environmental changes and conditions of device used for capturing the video data.
iv. The practical hardware components required for implementation of the system. The limitations of this research are highlighted as follows:
v. The standard high resolution video cameras for capturing the video data are not available. Thus, the research made use ofthe cameras obtained from National Agricultural Extension Research Liaison Service (NAERLS) and Nigerian Television Authority (NTA).
vi. Due to lack of high speed computer and storage devices, the video data size was limited to less than 50MB.
1.5 Research Motivation
Video compression is considered one of the most important aspects of digital image processing. The analysis of video data is complicated due to its relative large size. This has
6
posed a challenge in video data transportation and loss of video data due to insufficient amount of storage devices available. Several research efforts have been directed toward reducing the large amount of size associated with these videos in the form of transformations and compression with a promising result. However, this approach is usually associated with loss of signal and reduction in quality of the video signal. These challenges necessitate the development of a luminance enhancement technique.
1.6 Hypothesis
The fundamental video processing problem is the relationship between the quality of the actual object in the video and the background object. The only explicitly known contents of the video are the hue, saturation and luminance. However, the intensity of the luminance is what determines the contrast level of the video. Thus, this thesis hypothesised that the contrast level or the brightness level of a video is dependent only on the luminance intensity of the video. Hence, an approach which focuses on improving the luminance intensity level of a video base pixel intensity and histogram distribution can present an efficient enhancement in video visual quality.
1.7 Research Methodology
The step by step procedures adopted in this research, which include the development of an image brightness enhancement and modifying FOA in order to achieve better image quality are highlighted as follows:
1. Pre-processing:
i. Acquisition of video sample data and the online benchmark data base for analysis.
ii. Elimination of hue and saturation in order to retainonly luminance intensityof the sample video data to pave way for luminance (grey level) enhancement.
7
iii. Noising and filtering of the video data such that its pixel is varied from its true value in order to separate the true picture from its background for easy of analysis.
iv. Applying the developed brightness enhancement techniqueto improve the luminance intensity of the video frames obtained from NEARLS and NTA to determine the improvement achieved in the image quality.
2. Video Compression:
i. Implementing the standard Discrete Cosine Transform (DCT), Discrete Wavelet Transform (DWT), Lifting Wavelet Transform (LWT), FOA, and mFOAcompression techniques on images obtained from NEARLS and NTA to know the improvement achieved by the modified FOA.
ii. Applying the developed enhancement technique on both the standards and mFOA compressed images.
iii. Carrying out analysis of the results obtained in item 3 and validation of these results to ascertain the improvement of the modified FOA compression technique.
1.8 Thesis Organisation
The general background information relevant for the course of this research work has been presented in chapter one. The brief overview of the remaining chapters‟ layout is as follows: Chapter two presents detail fundamental existing works that border on the areas of investigation of this research. Studies cover major components that are addressed in this research. Chapter two provides the theoretical basis and the body of knowledge surrounding the investigation in this work. Chapter three presents the in-depth approach and relevant mathematical models necessary for the successful implementation of this research work. Chapter four presents the analysis, performance, and discussion of the results obtained during
8
simulation. Chapter five gives the summary of the main findings, conclusions of this research, its contributions, and suggestions for further research.
9
IF YOU CAN'T FIND YOUR TOPIC, CLICK HERE TO HIRE A WRITER»
