ABSTRACT
This research developed a license plate and classification scheme using deep learning architecture which utilized transfer learning using pre-trained Convolutional Neural Network (CNN). The developed scheme used images obtained from Caltech dataset, Peking University VehicleID (PKU VehicleID) dataset and a developed dataset of vehicle licence plates from Ahmadu Bello University, Zaria called the ABU dataset. De-noising, downscaling operation and grayscale conversion was applied on the acquired image to reduce the cost inqured in using the original image. Sobel operation was performed to detect the edge of the pre-processed image. Edge density filtering and connected component analysis were used to extract and verify the region which constituted the licence plate number. AlexNet model pre-trained on ImageNet was used to extract features and classify the detected license plate candidate’s regions. The performance of the developed scheme was evaluated on 150 images of each dataset of PKU VehicleID, Caltech, and ABU test images taken under different conditions. With the Caltech dataset the scheme achieved a precision rate of 85.10%, recall rate of 98.50%, and recognition accuracy of 98.08% while the PKU VehicleID dataset gave precision rate of 97.91%, recall rate of 97.91%, and accuracy of 100%. For the ABU dataset, the method obtained a precision rate of 95.8%, recall rate of 100%, recognition accuracy of 99.82%. The results for the developed deep learning-based scheme showed some performance improvements of 3.96% and 14.75% in the precision and recall rate, respectively, and 8.15% improvement in recognition rate, when compared with the existing scheme which utilized the edged based approach with SVM and achieved detection rate of 98% on the PKU VehicleID dataset, 90% on the Caltech dataset, and 96% on the ABU dataset in the presence of complex backgrounds and highly variable license plate patterns.
TABLE OF CONTENTS
DECLARATION III
CERTIFICATION IV
DEDICATION V
ACKNOWLEDGEMENT VI
ABSTRACT VII
TABLE OF CONTENT VIII
LIST OF FIGURES XI
LIST OF TABLES XIII
LIST OF APPENDICES XIV
LIST OF ABBREVIATIONS XV
CHAPTER ONE
INTRODUCTION 1
1.1 Background of Research 1
1.2 Significance of the Research 2
1.3 Statement of Problem 3
1.4 Aim and Objectives 3
CHAPTER TWO
LITERATURE REVIEW 5
2.1 Introduction 5
2.2 Review of Fundamental Concepts 5
2.2.1 License plate detection and recognition 5
2.2.1.1 Image acquisition 6
ix
2.2.1.2 Pre-processing 7
2.2.1.3 Image downscaling 7
2.2.1.4 Grayscale conversion 8
2.2.1.5 Image de-noising 9
2.2.2 Candidate extraction 11
2.2.2.1 Edge detection 11
2.2.2.2 Image binarization 14
2.2.2.3 Edge density filter 16
2.2.2.4 Candidate extraction using geometric properties of the license plate. 19
2.2.3 Datasets 19
2.2.4 Deep learning algorithm 20
2.2.4.1 Deep convolutional neural network (dCNN) 20
2.2.4.2 Convolutional neural network architecture(ConvNet) 21
2.2.4.3 Alexnet deep convolutional neural network 22
2.2.5 Character segmentation 24
2.2.6 Character recognition 24
2.2.7 Performance Evaluation 25
2.3 Review of Similar Work 26
CHAPTER THREE
MATERIALS AND METHODS 34
3.1 Introduction 34
3.2 Materials 34
3.3 Methodology 34
3.4 Development of ABU Vehicle Image Dataset 35
3.4.1 Image acquisition 36
3.4.2 Pre-processing the acquired image 37
3.4.2.1 Down scaling of the image 38
3.4.2.2 Converting RGB to grayscale Images 38
3.4.2.3 Noise elimination 38
3.4.3 Collation of publicly available license plate dataset 39
3.4.3.1 Caltech vehicle dataset 39
3.4.3.2 Pekins university dataset (PKU) 40
3.5 Development of the Deep Learning Based License Plate Detection Scheme. 41
3.5.1 Edge detection 44
3.5.2 Candidate’s region localization 44
3.5.3 Verify the candidate region using connected component analysis (CCA). 45
x
3.5.4 Determination of the final License plate candidate region 46
3.5.5 Feature extraction and fine-tuning 47
3.5.6 Modification of the classification and output layer 47
3.5.7 Transfer learning and retraining of the model 47
3.5.8 Classification of the output 50
CHAPTER FOUR
RESULTS AND DISCUSSIONS 54
4.1 Introduction 54
4.2Grayscale Conversion of Images for Caltech, PKU VehicleID and ABU Images 54
4.3 Sobel Edge Detection on Caltech, PKU VehicleID and ABU Image 54
4.4 Candidate Result Obtained by the Edge Density Filter 55
4.5 Detection Result Obtained by the Developed LPD Approach on Caltech, PKU VehicleID and ABU Images 56
4.5.1 Detection result on the Caltech dataset 56
4.5.2 Detection result on the PKU VehicleID dataset 57
4.5.3 Detection result on the PKU dataset. 58
4.6 Extracted License Plate Number of Caltech, PKU VehicleID and ABU Images 59
4.7 Analysis of the Developed License Plate Detection 60
4.7.1 Detection performance on the dataset 60
4.7.2 Failed detection 61
4.7.4 Evaluating the precision, recall and classification accuracy of the scheme 64
4.8 Result of Detection on ABU Images Considering Various Environmental Conditions 65
CHAPTER FIVE
CONCLUSION AND RECOMMENDATIONS 67
5.1 Summary of Findings 67
5.2 Conclusion 67
5.4 Significant Contributions 68
5.5 Recommendations for Further Work 68
REFERENCES 70
CHAPTER ONE
INTRODUCTION
1.1 Background of Research
Advancement in intelligent transportation systems, has attracted considerable research interests in computer vision (Yuan et al., 2016). This is because of its application in areas like vehicle management, electronic payment system (toll collection in express ways and parking fees payment), access control for monitoring area with limited accessibility like embassies, factories, military barracks, etc. and for identifying lost or stolen vehicles, border control and road traffic monitoring etc. (Du et al., 2013).
The first licensed plate detection system was developed in 1976 at the police scientific development branch in the United Kingdom (UK) (Nguwi & Lim, 2015). At that time, the functions of license plate detection system were very limited. The essence of number plate detection is to apprehend unlicensed and auto thefts (Jenkins, 2007). In 2007, the automatic license plate recognition (ALPR) system was incorporated into the red-light camera in the United States of America (USA) to apprehend drivers whose vehicles drove past the red traffic lights. The offender’s car plate information is captured by the camera and processed by the automatic license plate recognition (ALPR) (Jenkins, 2007).
License plate recognition system is divided into two component parts, detection and recognition (Zhao et al., 2011). Detection is the ability to localize the license plate and generate a suitable bounding boxes that encompasses the detected license plate, while plate recognition aims to identify the characters depicted within the bounding boxes and classify it as a license plate. License plate detection and recognition are two separate processes, research on these two schemes are always been performed separately. Different algorithms
2
have been developed and applied to the two processes (Nguyen et al., 2015). License plate detection is the most important aspect of the license plate recognition system, because the accuracy of the recognition depends on the detection stage (Zhao et al., 2011). However, license plate recognition system is required for real time applications thus high detection rate is paramount in order to meet the requirements for such real time applications.
Although many algorithms have been proposed for license plate detection in the past two decades, some of which requires sophisticated camera to produce high quality images, demand vehicles to slowly pass a fixed access gate or even at a complete stop (Nguwi & Lim, 2015). All this conditions is to achieve a clearer view of the object. Despite all these, it is still a challenging task to detect license plates in an open and noisy environment. The problem becomes more complex, especially when the license plate number are not standardized, such as which may be faded, partially occluded by dirt and taken under different environmental conditions (Fomani & Shahbahrami 2017). Detecting these plates with the traditional methods may result in many false positives (Li & Shen, 2016). To tackle this problem, state of the art deep learning technique was explored.
{{
1.2 Significance of the Research
License plate detection system is required for real time applications such as access control, traffic management and electronic toll system. Hence such real time applications require a license plate detection scheme that will accurately detect the license plate at a faster rate irrespective of the environmental condition and produce a fast, cost effective and highly accurate recognition system. This can be achieved by using current best image processing and deep learning technology.
3
1.3 Statement of Problem
Detecting plate number from an image is a very difficult task as the image contains a lot of noise which reduces its quality. These noises are as a result of the image taken under different weather conditions and lighting conditions such as uneven illuminations and presence of background clutters. Conventional approaches which mainly rely on certain morphological operations have been largely used for detecting the license plate position but have limitations in real time applications due to their time-consuming nature. Some of the techniques such as texture and colour based techniques are also not robust to detect noisy or corrupted images.
For the purpose of effectively and accurately detecting the license plate from any image irrespective of the complexity of the background, there is a need to achieve at least a human level accuracy. This can be achieved by using a deep CNN framework that utilizes a deep feature extraction technique via transfer learning approach. This is necessary in order to improve the detection rate while reducing the rate of achieving false positive results.
1.4 Aim and Objectives
The aim of this research work is to develop a deep learning based license plate detection scheme. The objectives of the research are:
1. To develop a dataset of vehicles images in Ahmadu Bello University (ABU) called the ABU dataset
2. To develop a dCNN-based licence plate detection system
3. To test the functionality of the developed scheme on the ABU, Caltech and PKU Vehicle ID.
4
4. To evaluate the performance of the developed scheme by comparing with the work of Yuan et al., 2016
5
Do you need help? Talk to us right now: (+234) 08060082010, 08107932631 (Call/WhatsApp). Email: [email protected].
IF YOU CAN'T FIND YOUR TOPIC, CLICK HERE TO HIRE A WRITER»
