Abstract

 

In recent years Machine learning and deep learning application-based researchers have achieved interest and the most significant one is handwritten recognition. Because it has the tremendous application such as Bangla OCR. EkushNet is the first research which can recognize Bangla handwritten basic characters, digits, modifiers, and compound characters. Handwritten recognition is one of the most interesting issues in the present time due to its variant applications and helps to make the old form and information digitization and reliable.  In spite of, there is no single model which can classify all types of Bangla characters. One of the most common reasons conducting with handwritten scripts is big challenge because every person has a unique style to write and also has different shape and size. Therefore, EkushNet has proposed a model which help to recognize Bangla handwritten 50 basic characters, 10 digits, 10 modifiers, and 52 mostly used compound characters. The proposed model train and validate with Ekush dataset and cross-validated with CMATERdb dataset. The proposed method is shown satisfactory recognition accuracy 97.73% for Ekush dataset which is our proposed dataset, and 95.01% cross-validation accuracy on CMATERdb dataset, the best accuracy for Bangla character recognition.

The proposed dataset Ekush contains Bangla modifiers, vowels, consonants, compound letters and numerical digits that consists of 367,018 isolated handwritten characters written by 3086 unique writers which were collected within Bangladesh. This dataset is freely available for any kind of academic research work.

TABLE OF CONTENTS

CONTENS	PAGE
BOARD OF EXAMINERS	I
DECLARATION	II
ACKNOWLEDGEMENTS	III
ABSTRACT	IV
CHAPTER
CHAPTER 1:   INTRODUCTION	1-4
1.1 MOTIVATION	2
1.2 OBJECTIVES	3
1.3 EXPECTED OUTCOME	3
1.4 REPORT LAYOUT	4
CHAPTER 2:    LITERATURE REVIEW	5-7
2.1 RELATED WORK	5
2.3 RESEARCH SUMMARY	6
2.3 CHALLENGES	7
CHAPTER 3:     PROPOSED RESEARCH	8-27
3.1 EKUSH	8
3.2 A UNIVERSAL WAY TO PROCESS HANDWRITTEN DATA	9
3.2.1 FORM PREPARATION	11
3.2.2 FORM SCANNING	11
3.2.3 SKEW CORRECTION AND CROP	12
3.2.4 SEPARATE ALL THE CHARACTERS	13
3.2.5 IMAGE SMOOTHING AND NOISE REMOVAL	15
3.2.6 REMOVING UNNECESSARY INFORMATION	16
3.2.7 INVERT THE IMAGES	16
3.2.8 GRAPHICAL USER INTERFACE (GUI) FOR PROCESS	17
3.3       THE EKUSH DATASET	18
3.3.1. CONSTRUCTING EKUSH	18
3.3.2 VISUAL REPRESENTATION OF THE EKUSH DATASET	20
3.3.3. DATA LABELING	21
3.3.4. POSSIBLE USES OF EKUSH DATASET	22
3.4       EKUSHNET	23
3.4.1    EKUSHNET ARCHITECT	23
3.4.2    OPTIMIZER AND LEARNING RATE	25
3.4.3 DATA AUGMENTATION	26
3.4.4 TRAINING THE MODEL	26
3.5       IMPLEMENTATION REQUIREMENTS	26
CHAPTER 4:      RESULTS AND DISCUSSION	28-30
4.1 TRAIN, TEST AND VALIDATION SPLIT	28
4.2 MODEL PERFORMANCE	29
4.3 RESULT COMPRESSION	29
4.4 ERROR ANALYSIS	30
CHAPTER 5:       CONCLUSION AND FUTURE SCOPE	31-32
5.1 SUMMARY OF THE BODY	31
5.2 CONCLUSION	31
5.3 RECOMMENDATIONS	32
5.4 IMPLICATION FOR FURTHER STUDY	32
REFERENCES	33-34

LIST OF TABLES

TABLE	PAGE NO
TABLE 1: NUMBER OF IMAGES IN DIFFERENT DATASETS	7
TABLE 2: NUMBER OF CHARACTER IN EKUSH BY GENDER	19
TABLE 3. EKUSHNET MODEL ARCHITECT SUMMARY.	24
TABLE 4. COMPRESSION BETWEEN SOME PREVIOUS WORK.	29

LIST OF FIGURES

FIGURES	PAGE NO
FIGURE 1: A FLOW DIAGRAM OF “EKUSH” DATASET	8
FIGURE 2: DATA COLLECTION FORM	10
FIGURE 3: FORM WITH LABELED ON THE BOX	11
FIGURE 4: SOME SCANNED FORM ON “SIMPLE SCAN”	12
FIGURE 5 (A): SCANNED FORM	13
FIGURE 5 (B): CROPPED FORM	13
FIGURE 6: EXAMPLE OF ROW-WISE CROPPING	14
FIGURE 7: EXAMPLE OF SEPARATED CHARACTERS	15
FIGURE 8: SOME EXAMPLE OF SMOOTHING IMAGES	15
FIGURE 9: EXAMPLE OF UNNECESSARY INFORMATION IMAGES	16
FIGURE 10: EXAMPLE OF INVERTED IMAGE	17
FIGURE 11: GUI FOR AN AUTOMATIC PROCESS	17
FIGURE 12: DATA FROM DEFERENT AGES BASE ON GENDER	20
FIGURE 13 (A): BANGLA MODIFIER ON EKUSH DATASET	20
FIGURE 13 (B): BANGLA DIGIT ON EKUSH DATASET	20
FIGURE 13 (C): BANGLA BASIC CHARACTERS ON EKUSH	21
FIGURE 13 (D): BANGLA COMPOUND CHARACTERS ON EKUSH	21
FIGURE 14: ARCHITECTURE OF EKUSHNET	24
FIGURE 15 (A): TRAINING AND VALIDATION LOSS.	29
FIGURE 15 (B) TRAINING AND VALIDATION ACCURACY	29
FIGURE 16 (A): ERROR FOR VALIDATION SET (B) TEST SET	30
FIGURE 16 (B): ERROR FOR TEST SET	30

LIST OF PUBLICATION

PUBLICATION

A UNIVERSAL WAY TO COLLECT AND PROCESS HANDWRITTEN DATA FOR ANY LANGUAGE.

BORNONET: BANGLA HANDWRITTEN CHARACTERS RECOGNITION USING CONVOLUTIONAL NEURAL NETWORK.

EKUSHNET: USING CONVOLUTIONAL NEURAL NETWORK FOR BANGLA HANDWRITTEN RECOGNITION.

BANGLA HANDWRITTEN DIGIT RECOGNITION USING CONVOLUTIONAL NEURAL NETWORK.

EKUSH: A MULTIPURPOSE AND MULTITYPE COMPREHENSIVE DATABASE FOR ONLINE OFF-LINE BANGLA HANDWRITTEN CHARACTERS.

ONKOGAN: BANGLA HANDWRITTEN DIGIT GENERATION WITH DEEP CONVOLUTIONAL GENERATIVE ADVERSARIAL NETWORKS.

SHONKHANET: A DYNAMIC ROUTING FOR BANGLA HANDWRITTEN DIGIT RECOGNITION USING CAPSULE NETWORK.

CHAPTER 1

INTRODUCTION

Automatic handwritten recognition is one of the most important research fields in recent years for its different application such as OCR which helps to recognize the character from images. Over the last few years, information has transferred from handwritten hardcopy documents to digital file formats, which is more reliable. However, this system to handle new forms of document. But till now a large number of older documents are written by hand.  The challenge is attempting to convert them if follow tradition method to copy the document by manual typing. Because it takes a long time and needs a huge amount of manpower. But OCR process can help to digitize old information with less time and less manpower. For that reason, a robust model of handwritten character recognition plays an important role. In spite of this Bangla handwritten character recognition has no strong model that helps to build robust Bangla OCR. Because of yet there is no model which can classify all kind of characters (basic character, numeral, modifiers, compound character). Many works have been done but those are concentrated for digit [1] or basic characters [2] or compound characters [3]. Dealing with handwritten character is complicated because of different shape and style. And the arrangement of Bangla character is complex due to its alignment and many of them are similar apart from compound characters that complement other basic characters.

One language fundamental is different from other languages like Latin scripts are differ from Bangla because Bangla comes from Sanskrit scripts. In Bangla language written scripts has 50 basic characters, 10 numerical digits, more than 200 compound characters, and 10 modifiers.

1.  MOTIVATION

Bangla is 4th most popular language in the world. It is the first language of Bangladesh with a rich heritage. February 21st is announced as the International Mother Language Day by UNESCO to respect the language martyrs for the Bangla language in Bangladesh in 1952.  It is the second most popular language in the Indian subcontinent. So, overall About 300 million people use Bangla language as their writing and speaking purpose. Consider all that situation Bangla handwritten character recognition plays an important role to help those people in different purpose as Bangla traffic number plate recognition, automatic postal code identification, extracting data from hardcopy forms, automatic ID card reading, automatic reading of bank cheques and digitalization of documents etc.

• Although enormous people such as 205 million people speak in Bangla, it does not get proper admiration to other major languages such as Roman, Chinese and Arabic script.
• Bangla is often known as the second most beautiful language in the world after French.
• In the world of literature contribution of Bangla is no less than English or any other language. Rabindranath Tagore was awarded the Nobel Prize for Gitanjali, which was written in Bangla.
• International Mother Language day is based on Bangla language.
• Through, several studies have been conducted for Bangla handwriting recognition, a robust model for Bangla numerals & Characters classification is still due.

2.  OBJECTIVES

• To classify Bangla 10 digits, 10 modifiers, basic 50 characters, 50 compound characters.
• To collect more than 1500 Male data and 1500 female data and labeled them.
• To label them with age, gender, hometown, and educational status.
• To recognize Bangla Handwritten documents, which includes characters.
• To recognize Bangla handwritten from the photo.
• To improve Bangla Picture to text and picture to speech work.

Forensic analysis from handwriting.

3.  EXPECTED OUTCOME

• Ekush: A dataset for Bangla Handwritten characters.
• EkushNet: A computer vision algorithm that can recognize all types of Bangla Character.
• Research on Bangla Handwritten Characters in both quality and quantity.
• Our goal is that the Bangla language shows one step upward in front of the world.
• A universal way to process handwritten data for any language.

4.  REPORT LAYOUT

There are five chapters in this research paper. They are Introduction, Literature Review,

Proposed Research, Results and Discussion, Conclusion and Future.

Chapter one: Introduction; Objective, Motivation, Expected Outcome, Report layout.

Chapter two: Literature Review; Sentiment Analysis, Related works, challenges.

Chapter three: Proposed Research; Research Methodology, Data Collection, Data

Processing, Flow Model, Experimental layout.

Chapter four: Results and Discussion; Experimental Result, Discussion.

Chapter five: Conclusion and Future; Conclusion, Future Scope.

CHAPTER 2

LITERATURE REVIEW

In the present world, one of the most interesting topics is Handwritten Recognition due to its academic and commercial interest in different research fields. There are many types of research held on Bangla handwriting and other languages like English, Arabic, Hindi, Chinese etc. So, in this chapter giving an explanation of related work, summary of this research, the scope of the problem and lastly will show the challenges of this research.

1.  RELATED WORK

In past studies there are many works for recognition of handwritten character in a different language as Latin [4], Chines [5], Japanese [6] achieve great success. There are a few works are available for Bangla handwritten basic character, digit and compound character recognition, some literature has been made on Bangla characters recognition in the past years as “A complete printed Bangla OCR system” [7], “On the development of an optical character recognition (OCR) system for printed Bangla script” [8]. there are also few types of research on handwritten Bangla numeral recognition that reaches to the desired recognition accuracy. Pal et al. have conducted some exploring works for recognizing handwritten Bangla characters those are “Automatic recognition of unconstrained offline Bangla hand-written numerals” [9], “A system towards Indian postal automation” [10]. And “Touching numeral segmentation using water reservoir concept” [11]. The proposed schemes are mainly based on extracted features from a concept called water reservoir. Apart from there also present several Bangla Handwritten Character Recognition and had achieved pretty good success. Halima Begum et al., “Recognition of Handwritten Bangla Characters using Gabor Filter and Artificial Neural Network” [12] works with own dataset that was collected from 95 volunteers and their proposed model achieved without feature extraction and with feature extraction around 68.9% and 79:4% of recognition rate respectively. “Recognition of Handwritten Bangla Basic Character and Digit Using Convex Hall Basic Feature” [13] achieve accuracy for Bangla characters 76.86% and Bangla numerals 99.45%. “Bangla Handwritten Character Recognition using Convolutional Neural Network” achieved 85.36% test accuracy using their own dataset. In “Handwritten Bangla Basic and Compound character recognition using MLP and SVM classifier” [14], the handwritten Bangla basic and compound character recognition using MLP and SVM classifier has been proposed and they achieved around 79.73% and 80.9% of recognition rate, respectively.

For recognizing handwritten characters dataset plays a vital rule and there are three open access datasets available for Bangla characters, these are the BanglaLekha-Isolated [15], the CMATERdb [16], and the ISI [17]. But every dataset has some drawback. BanglaLekha–Isolated dataset consists the total of 166,105 squared images (while preserving the aspect ratio of the characters), each containing a sample of one of 84 different Bangla characters which has 3 categories such as 10 numeral digits, 50 basic characters, and 24 compound characters. Two others datasets CMATERdb has also 3 different categories for basic characters, numerals and compound characters and ISI dataset has two different datasets for basic characters and numerals.

2.  RESEARCH SUMMARY

Our goal to make a model that can recognize Bangla Handwritten digits using Convolutional Neural Network which is training by Ekush and other Bangla character dataset. The Convolutional Neural Network (CNN) reveal new opportunities in the field of pattern recognition for classification, which is helping numerous researchers to implement their state-of-the-art system in solving. The CNN structure was first proposed by Fukushima et al. in 1980 [18] but it was not widely used because the algorithm was complex. In the 1990s, LeCun et al. applied a gradient-based learning algorithm to CNN and achieved successful results [19]. After that, many researchers work on it and improved CNN and made good results in pattern recognition. A few years ago, Cirean et al. [20] applied multi-column CNN to recognize digits, alpha-numerals, traffic signs, and the other object class.

The Ekush dataset consists of 367,018 images that contain 122 classes and it became the largest dataset for Bangla characters yet. A comparison between Ekush and with that three popular sources of Bangla handwriting related datasets (BanglaLekha-Isolated dataset, CMATERdb, and the ISI Handwriting dataset) are given in Table 1.

Table 1: Number of images in different datasets

Dataset Name	Modifiers	Basic Characters	Compound Characters	Numeral	Total
ISI	None	30,966	None	23,299	34,256
CMATERdb	None	15,103	42,248	6,000	63,351
BanglaLekha-Isolated	None	98,950	47,407	19,748	166,105
Ekush	30,667	154,824	150,840	30,687	367,018

The proposed method is shown satisfactory recognition accuracy 97.73% for Ekush dataset, and 95.01% cross-validation accuracy on CMATERdb dataset, which is so far, the best accuracy for Bangla character recognition.

3.  CHALLENGES

• Tough to collect female data.:  As our aim to collect data from 3000 unique people which was divided into two group equally male and female. So it was tough to collect 1500 female data.
• Preprocessing Lots and lots of data: As each form containing 120 characters and having a total of 3000 form. So, processing raw data into image data and also labeling it was a tough job for us.
• Optimize the Hyperparameter of the deep convolutional neural network.
• High-performance machine for the bigger network.

CHAPTER 3

PROPOSED RESEARCH

1. EKUSH

Ekush is a dataset of Bangla handwritten characters which can be used as multipurpose way. Ekush dataset of isolated Bangla handwritten characters structured and organized data was collected from 3086 peoples covering university, school, college students, where approximately 50% 1510 male and 50% 1576 female. The handwritten characters were written in a form and then scanned it to get the data in JPEG format, we also focused on some of the issues and requirements when collected handwritten data, such as creating a form, data collection methodology, process, software, and relevant tools. Figure 1 showing a flowchart of creating the Ekush dataset.

Figure 1: A flow diagram of “Ekush” dataset

2. A UNIVERSAL WAY TO PROCESS DATA

In recent years researches based on Machine learning and Deep learning have achieved much interest and one of its handwritten recognition. Handwritten recognition is very difficult due to its lack of dataset and also for collecting data from people. This research introduces a fast and comprehensive way to collect and process handwritten data to develop a way of Handwritten Recognition (HWR) algorithm for any languages. In this research handwritten characters wrote on a paper and then scanned to get the data into a JPEG format. We also focused on some of the other issues and requirements while collecting handwritten data, creating a form, data collection methodology, process, using software and relevant tools. We described these issues in the context of our own effort to create a handwritten database for the Bangla language. Our designed Graphical User Interface (GUI) is also able to process 100 scanned images per minute where each scanned image contains 120 characters. This proposed method, we developed a system which can help future researchers to collect and process handwritten data so they can directly use it in their Handwritten Recognition based researchers and also in many applications.

In our built system, we used some popular handy tools like python [21], OpenCV [22], PIL [23], PyQT [24] which will boost other researchers to modify the process as they want. Our proposed method form is able to collect at least 120 characters within a single form. As well as this form has scanned and pre-process all the 120 characters in different forms like jpeg, grayscale, invert and CSV for multipurpose uses. An example form is shown in Figure 2.

Figure 2: Data collection Form

1.      FORM SCANNING

At the very beginning, we create the form generally which is very tricky. We need to create all cell very carefully with equal size, which will later help to separate the characters automatically.  Under or above every box there have to have some label which will help people to write the character in the specific boxes. If there are lots of characters than selecting the most frequently used character should be placed on the first page of the form. Casting the character under the box found to be more efficient from our own experiences. It is also mandatory to print the form with a good quality printer or photocopy machine. Otherwise, the printed label will be murky and volunteers will fill the form wrongly. Figure 3 showing 2 examples of labeling.

1.                                     (b)

Figure 3: Form with (a) labeled under the box, (b) labeled above the box.

2.      FORM SCANNING

Next step is scanning those forms where the user can use any kinds of applications which will be efficient for them. Usually, any kind of windows scan application takes at least 1 minute to scan 1 form. Also, most of the windows scan application does not support user to save lots of images at the same time. For the sake of, we use “Simple Scan” application on “Ubuntu” with300dpi. Which takes 7-15 second to scan a single page and also have some options for saving and cropping the images to reduce the extra black part of the scanner. Figure 4 showing some example of the scanned form.

Figure 4: Some scanned form on “Simple Scan”

3.      SKEW CORRECTION AND CROP

During scanning the form, it is not possible to scan all the paper in the same position. To overcome this issue, we create a big black boundary which will help to find the biggest contour by canny edge detection [25] to correct the skew of the paper and crop all the images in the same shape, size, and angle which is described in algorithm 1. Figure 5 is showing an example.

Algorithm 1: Find the Biggest Contour and Crop

1. For all images in the scanned directory do:
2.     Convert the image to grayscale
3.     Detect Canny edge
4.     Find all the contour
5.     For all contour do:
6.          Search the biggest contour area
7.     End for
8.     Detect the biggest contour in the image
9.     Crop the image on detected contour area.
10.     Save the file in a new directory
11. End for

1. Scanned form                       (b) Cropped Form

Figure 5: Crop an Image Form

4.      SEPARATE ALL THE CHARACTERS

After detection and crop, all image will be the same size and same angle. First, we will crop the images as row-wise. There will be 10 rows of the cropped image file and this 10-cropped file will be stored in 10 separate folders for the later process, this is explaining in algorithm 2.  Figure 6 showing an example.

Algorithm 2: Row-wise cropping

1. For all images in Skew corrected cropped Images directory:
2.         Start = 0
3.         End = Start + Row height
4.         RowFolder = 0
5.         For 10 epochs do:
6.                 Crop Image in position (start to end)
7.                 Start = End position + distance between 2 rows
8.                 End = Start + Row height
9.                 Save image to RowFolder
10.                 RowFolder = RowFolder + 1
11.         End for
12. End for

Figure 6: Example of row-wise cropping

Each row contains 12 characters. After saving all the row-wise images then we need to perform column wise cropping. This action will separate all the characters individually and store all the images in 120 corresponding folders. This procedure is describing in algorithm 3. Figure 7 showing some example.

Algorithm 3: Separating each character

1. For all folder in row wise cropped directory:
2.         ColumnFolder = 0
3.         For 10 epochs do:
4.          F = 0
5.               For all image in Folder F:
6.                     Start = 0
7.                End = Start + column width
8.                     F = F + 1
9.                     For 12 epochs:
10.                         Crop Image in position (Start to End)
11.                         Start = End
12.                         End = Start + column width
13.                         Save image to ColumnFolder
14.                         ColumnFolder = ColumnFolder + 1
15.                     End for
16.               End for
17.         End for
18. End for

Figure 7: Example of separated Characters

5.      IMAGE SMOOTHING AND NOISE REMOVAL

The threshold applies fixed-level thresholding to a single-channel array. The function is typically used to get a bi-level (binary) image out of a grayscale image or for removing noise, that is filtering out pixels with too small or too large values.

Also, the special values OTSU [26] combined with one of the above values. In these cases, the function determines the optimal threshold value using the Otsu algorithm and uses it instead of the specified thresh. Then used Gaussian blur to smooth the image. Figure 8 showing some example.

Figure 8: Some example of smoothing images.

6.      REMOVING UNNECESSARY INFORMATION

Removing unnecessary information is provide a good result on machine learning algorithms, so we remove all extra bit of information by removing all extra white pixel and this method showing in algorithm 4. Figure 9 showing some example.

Algorithm 4: Removing extra background

1. For all character images in the directory do:
2.     Find where the black pixels are
3.     Create a rectangle around those points
4.     Add 1 pixel to each side of the rectangle
5.     Create a cropped region of the image
6.     Save new image
7. End for

Figure 9: Example of unnecessary information images

7.      INVERT THE IMAGES

After that step, processed images will also have the inverted into the background as black and characters in white. To invert the image, we have to invert all the pixel value of the images. Which will make the lighter part to dark and darker part to light. Some invert images example showed in Figure 10.

Figure 10: Example of inverted image

8.      GRAPHICAL USER INTERFACE (GUI) FOR PROCESS

All the processing step can be done atomically with designed GUI. GUI is shown in Figure 11.

Figure 11: GUI for an automatic process.

3. EKUSH DATASET

Ekush the largest dataset of handwritten Bangla characters for research on handwritten Bangla character recognition. In recent years Machine learning and deep learning application-based researchers have achieved interest and the most significant one is handwritten recognition. Because it has the tremendous application such as Bangla OCR. Also, the Bangla alphabet is the fifth most popular used writing method in the world. For that reason, we are introducing a multipurpose comprehensive dataset for Bangla Handwritten Characters. The proposed dataset contains Bangla modifiers, vowels, consonants, compound letters and numerical digits that consists of 367,018 isolated handwritten characters written by 3086 unique writers which were collected within Bangladesh. This dataset can be used for other problems i.e.: gender, age, district base handwritten related research, because the samples were collected include verity of the district, age group and the equal number of male and female. It is designed to build a recognition method for handwritten Bangla characters. This dataset is freely available for any kind of academic research work.

1.      CONSTRUCTING EKUSH

Ekush dataset of Bangla handwritten characters were collected from 3086 peoples where 50% male and 50% female. Each individual developed 120 characters including 50 Bangla basic characters, 50 frequent Bangla compound characters, 10 modifiers, 10 digits. But after processing the dataset we found more than 55 type compound characters because of Bangla compound characters are confusing and some of are similar. So, when the writer wrote they made mistake to understand which character actually exist in the form. Then we manually checked and found 2 compound character most of the people were mistake to understand the proper character. So, after that, we picked those 2 characters and added in our dataset and finally got 52 types of compound character. These writers were selected from various age, gender, and educational background groups. Figure 12 showing a bar chart of data samples from different ages base on gender.

The collected dataset is preprocessed in three different formats:

1. Foreground and background are inverted so that images have a black background with the letter drawn in white.

2. Foreground and background filled with white so that images have a white background with the letter drawn in black

3. Removed noise and smoothing attempted using the Trash holding and Gaussian blur filter. The proceed dataset is further filtered and after necessary smoothing different format are created including CSV.

The Ekush dataset will available on the various format, depending on the user desired applications as well as will be available without extra information on character images, and with padding added to preserve aspect ratio and also in CSV format. Table 2 showing the details of Ekush dataset base on gender.

Table 2: Number of Character in Ekush by gender

Gender	Modifiers	Basic Characters	Compound Characters	Numeral	Total	Total In Ekush
Female	15580	78615	76912	15622	186729	367018
Male	15087	76209	73928	15065	180289

Figure 12: A bar chart of data samples from different ages base on gender

2.      VISUAL REPRESENTATION OF THE EKUSH DATASET

The class breakdown of Ekush dataset shows in Figure 13, where see that Figure 13 (a) (b) showing Bangla modifiers and digits class as well as seeing that all class are almost equally distributed. The number of the Ekush dataset Modifier and Digits has 30769 and 30687 respectively. From Figure 13 (c) (d) most of the classes are equally distributed, but there are few classes exist where a number of images of that characters are not averagely equal to compare the other classes. While writing the compound characters people made mistake to write the proper character. Then we relabeled some compound character also deleted other as well as collected some incorrect compound characters and added them to the dataset in two new class which we are seeing at the last to Figure 13 (d) Compound character bar chart.

(a) Bangla Modifier (b) Bangla Digit

(c) Bangla Basic Characters

(d) Bangla Compound Characters

Figure 13: Visual Represent of Ekush Dataset

3.      DATA LABELING

The extracted image is saved as the JPEG format and each image has a unique ID or name that represent writer gender, hometown, age, education level, and serial number. In order to storing the writer information which helps to identify that person using an ID. So that this dataset not only uses for handwritten recognition but also help to predict a person gender, age and his or her location as well as it can help investigators focusing more on a certain category of suspects and forensic purposes. This id or name fixed according to the following criteria, Its first one digit indicates writer gender. If it is 0 which means writer was male and 1 means writers were female. After that have writer home district names First 3 or 4 letters, the next one represents age and then their education or occupation level (0 means primary level, 1 high school level, 2 means college level, 3 means university and 4 means other occupation) and the last one is the serial number. And that information are separated by an underscore ( _ ). Here an example.

0_Dha_20_3_00052

Here the first digit one so it was written by a male writer and he is from Dhaka district the next one is 20 which means his age 20 and he is a university student and the last one is the serial number of male data.

4.      POSSIBLE USES OF EKUSH DATASET

This dataset can be used for handwritten character recognition, Bangla OCR, Machine learning and Deep learning base research fields. The prediction of age, gender, the location from handwriting is a very interesting research field. These information’s can be used even for forensic purposes, where it can help investigators focusing more on a certain category of suspects.

4. EKUSHNET

EkushNet is the first research which can recognize Bangla handwritten basic characters, digits, modifiers, and compound characters. Handwritten recognition is one of the most interesting issues in the present time due to its variant applications and helps to make the old form and information digitization and reliable.  In spite of, there is no single model which can classify all types of Bangla characters. One of the most common reasons conducting with handwritten scripts is big challenge because every person has a unique style to write and also has different shape and size. Therefore, EkushNet has proposed a model which help to recognize Bangla handwritten 50 basic characters, 10 digits, 10 modifiers, and 52 mostly used compound characters. The proposed model train and validate with Ekush [27] dataset and cross-validated with CMATERdb dataset. The proposed method is shown satisfactory recognition accuracy 97.73% for Ekush dataset, and 95.01% cross-validation accuracy on CMATERdb dataset, which is so far, the best accuracy for Bangla character recognition.

1.      EKUSHNET ARCHITECT

Proposed EkushNet used a multilayer CNN for classifying Bangla Handwritten Characters. This model used convolution, Max pooling layer, fully connected dense layer and some regularization method like batch normalization [28] and dropout [29].

Layer 1 and 2 are a convolutional layer with a filter size of 32 and kernel size of 5, these two layers also use ReLU (2) activation with the same padding. The output of these layers later connected with max pooling layer 3 followed by 25% dropouts layer 4.

ReLU(X) = MAX(0, X)

(2)

The output of layer 4 than goes into two separate layers- layer 5 and layer 11. Layer 5 is a convolution with 64 filters followed by a batch normalization layer 7. Layer 8 is also a convolution with 64 filters with a kernel size of 5 with a batch normalization layer-8, max pooling layer-9 and 20% dropout layer – 10.

Layer 11, a convolutional layer which takes input from layer 4. This layer has 64 filters with 3×3 kernel size. Layer 12 has the same configuration of layer 11, connected with a max pooling layer 13 with 25% dropouts – layer 14.

At layer 15 both layer 10 and 14 are added together and pass into a new convolutional layer with 64 filters and 3 x 3 kernel size which is connected to a max pooling layer 16 with 25% dropout layer 18.

After all of these 18 operations, the output is flatten into an array and pass through a fully connected dense layer 20 with 2048 hidden units and regularized with 25% dropout. The output of the layer 21 connected with a fully connected dense layer 22 with 122 nodes with SoftMax (3) activation which is also the output layer for the model. Figure 14 showing the proposed EkushNet architect. Table 3 showing the perimeter for the model.

σzj=ⅇzj∑k=1Kⅇzk  for j=1, …k

(3)

Figure 14. Architecture of EkushNet

Table 3. EkushNet model architect summary.

Layer No (type)	Output Shape	Parameter	Connected to	Layer No (type)	Output Shape	Parameter	Connected to
(Input Layer)	28, 28, 1	0	–	13 (MaxPooling2D)	7, 7, 64	0	12
1 (Conv2D)	28, 28, 32	832	Input Layer	10 (Dropout)	7, 7, 64	0	9
2 (Conv2D)	28, 28, 32	25632	1	14 (Dropout)	7, 7, 64	0	13
3 (MaxPooling2D)	14, 14, 32	0	2	15 (Add)	7, 7, 64	0	10, 14
4 (Dropout)	14, 14, 32	0	3	16 (Comv2d)	7, 7, 64	36928	15
5 (Conv2D)	14, 14, 64	51264	4	17 (MaxPooling2D)	3, 3, 64	0	16
6 (Batch Normalization)	14, 14, 64	256	5	18 (Dropout)	3, 3, 64	0	17
7 (Conv2D)	14, 14, 64	36928	6	19 (Flatten)	576	0	18
11 (Conv2D)	14, 14, 64	51264	4	20 (Dense)	2048	1181696	19
8 (Conv2D)	14, 14, 64	256	7	21 (Dropout)	2048	0	20
12 (Conv2D)	14, 14, 64	36928	11	22 (Dense)	122	249978	21
9 (MaxPooling2D)	7, 7, 64	0	8

Total parameters: 1,671,962

Trainable parameters: 1,671,706

Non-trainable parameters: 256

2.      OPTIMIZER AND LEARNING RATE

Optimization algorithms help CNN algorithms to minimize the error. Proposed Ekush net used Adam optimizer [30]. Adam optimization algorithm that can be used to update network weights iteratively in training data. Adam is an update of extension to stochastic gradient descent algorithm. For its better performance, it is widely used in computer vision researches. Proposed EkushNet used Adam optimizer (4) with a learning rate of 0.001.

θt+1=θt-ηv̂t1+εm̂t

(4)

To calculate the error for optimizing algorithm we used categorical cross entropy (5) function. Recent research shows that cross entropy performs better than other function like classification error and mean squared error etc[31].

Li=-∑jt i,   j logpi,   j  (5)

Learning rate is one of the most important hyper-parameters to tune for training convolutional neural networks. If the learning rate is low the classification is more accurate but optimizer will take more time to reach the global optima by reducing the loss. And if the learning rate is high the accuracy may not converge also some time may diverge. So, choosing the best learning rate is more difficult. To overcome this challenge, we use automatic learning rate reduction method [32]. For faster computation, we set a higher learning rate of 0.001 which is atomically reduced by monitoring the validation accuracy.

3.      DATA AUGMENTATION

Deep learning technique performs better if it finds more data. For this reason, data augmentation helps to produce more data artificially. For handwriting characters, recognition data augmentation helps more because a single person can write a character in a different variation. For data augmentation, the images are shifted randomly 10% in height or width or both, also 10% rotation and 10% zoom the images.

4.      TRAINING THE MODEL

The EkushNet model was trained on Ekush dataset with a batch size of 86. After 50 epochs the model got good accuracy. The automatic learning rate reduction formula helps the optimizer to converge faster by reducing the learning rate. End of the training the learning rate reduced by 0.001 to 1.5×10^-5.

5.  IMPLEMENTATION REQUIREMENTS

After the proper analysis of all necessary statistical or theoretical concepts and methods, a list of requirements has been generated that must be required for such a work of Bangla Handwritten Classification. The probable necessary things are:

Hardware/Software Requirements:

• Operating System (Windows/ Linux/ Mac)
• Hard Disk (Minimum 10 GB)
• RAM (Minimum 4 GB)
• GPU (CUDA Supported)
• Web Browser (preferably Google Chrome)

Developing Tools

• Python 3.6 +
• Jupyter Notebook
• Keras
• Tensorflow
• Open CV
• Pyqt
• Sklearn

 

CHAPTER 4

RESULTS AND DISCUSSION

EkushNet was trained and validated on Ekush dataset and cross-validated using CMATERdb datasets and give a promising result on a train set, test set, and validation set.

1.  TRAIN, TEST AND VALIDATION SPLIT

To measuring the model performance, train test and validation split were created. The training set is used to train the model with the known output. Validation set used to check model performance during training time and help the model to tune the hyper-parameters. And test data used to check the final model performance after training.

For training and validation purpose we used the Ekush dataset. The Ekush dataset has 367,018 characters images. 55,052 characters 15% of total used in validation and 311,966 characters 85% used to train the model.  For making the training data equal from all types of characters, we take 85% of modifiers, basic characters, compound characters, and digits separately than concatenated them and make the training set and add other 15% on the validation set.

For testing the model CMATERdb was used which is completely comes from different distribution. CMATERdb has 15,000 basic characters and digits but no compound characters. All 15,000 images were used to measure model performances.

2. MODEL PERFORMANCE

After 50 epochs proposed model got 96.90% accuracy on the training set and 97.73% accuracy on a validation set of Ekush datasets. After training the model it was tested on CMATERdb dataset and got 95.01 % accuracy.

Analyzing the result and confusion Metrix, found that model performs very well to classify handwritten characters. Many Bangla characters are so similar and sometimes it is harder for a human to recognize the character. In those case, the model did a pretty good job. Figure 15 showing the training and validation loss and accuracy.

Figure 15. (a) Training and validation loss. (b) Training and validation accuracy

3. RESULT COMPRESSION

There is no previous work which classifies all types of Bangla characters (basic Characters, numerical characters, compound characters). Table 4 showing some comparison between some previous work on different Bangla characters.

Table 4. Compression between some previous work.

Work	Accuracy	Layer No (type)	Accuracy
Recognition of Handwritten Bangla Characters Using Gabor Filter and Artificial Neural Network [33]	79.4%	HMM-Based Online Handwritten Bangla Character Recognition using Dirichlet Distributions [34]	91.85%
Recognition of Bangla handwritten basic characters and digits using convex hull-based feature set [35]	76.86%	Bangla Hand-Written Character Recognition Using Support Vector Machine [36]	93.43%
Handwritten Bangla Character Recognition Using Neural Network [37]	84.00%	Bengali handwritten character recognition using Modified syntactic method [38]	95.00%
Bangla Handwritten Character Recognition using Convolutional Neural Network [2]	85.36%	EkushNet (Proposed)	97.73% (Ekush) 95.01% (CMATERdb)

 

4. ERROR ANALYSIS

Exploring the error from test set and cross-validation set we considered that model has successfully classified 53,803 characters out of 55,052. On Figure 16 (a) top 6 error from test set and Figure 16 (b) top 6 error from cross-validation are set shown. From this top 12 images, it is clear that the model performs well and top error is caused from incorrectly labeled datasets

(a)         (b)

Figure 16: Error for (a) Validation set (b) Test set

CHAPTER 5

CONCLUSION AND FUTURE SCOPE

1.  SUMMARY OF THE BODY

This research introduces a universal way of processing any languages handwritten character database. That database is possible to develop handwritten character recognition systems for any language. Applying these propose method, researchers will be benefited in different research fields and also help work on automatic recognition of certain characteristics of the writer such as age, gender, location etc. Any purposes of research, the forensic database is the important material so that the processing of data plays a vital role. Besides, providing a model for recognizing Bangla handwritten character as well as a dataset Ekush. The best recognition rate found by the system was 97.73% for Ekush dataset, and 95.01% cross-validation accuracy on CMATERdb dataset.

2. CONCLUSION

This research introduced lead to formulating a diverse repository containing primarily for the Computer Vision and NLP research and called the dataset Ekush. Preliminary experiments using a state-of-the-art recognizer demonstrate the challenge of recognition. This leaves a big room for improvement and stimulates the community to seek new recognition methods. This represents a new and modern performance benchmark for the current generation of classification and learning systems.

To consider the model we understand that Convolutional Neural Network can achieve better performance to classify and recognize Bangla handwritten characters, digits and compound characters. The CNN works very well for handwritten character recognition for its distinguished features. Experiments on a large dataset Ekush which was helped to build the robustness of this method for Bangla handwritten character recognition.

3.  RECOMMENDATIONS

As high configuration computer, GPU then the accuracy would have been better as this model gave the result. Apart from, can produce a better output of this research work by using an efficient dataset.

4.  IMPLICATION FOR FURTHER STUDY

In the future, we will extend our dataset including all kinds of compound characters. it is possible to gain more accuracy by providing a better collection of the dataset. Also, we will make a website where the user can download the form and upload the scan copy which will automatically process those data and added to the dataset after verifying. That website also gives the user to search and download character data by age, gender and districts.

Reference

1. Alom, Md. Zahangir & Sidike, Paheding & M. Taha, Tarek & Asari, Vijayan. (2017). Handwritten Bangla Digit Recognition Using Deep Learning.
2. Md. M. Rahman, M. A. H. Akhand, S. Islam, P. C. Shill, “Bangla Handwritten Character Recognition using Convolutional Neural Network,” I. J. Image, Graphics and Signal Processing, vol. 8, pp. 42-49, July 15, 2015.
3. Saikat Roy, et al. “Handwritten Isolated Bangla Compound Character Recognition: a new benchmark using a novel deep learning approach”. Pattern Recognition Letters, Elsevier, Vol. 90, Pages 15-21, 2017.
4. E. Kavallieratou, N. Liolios, E. Koutsogeorgos, N. Fakotakis, G. Kokkinakis, “The GRUHD Database of Greek Unconstrained Handwriting”,   ICDAR, 2001, pp. 561-565.
5. F. Yin, Q.-F. Wang, X.-Y. Zhang, and C.-L. Liu. ICDAR 2013 Chinese Handwriting Recognition Competition. 2013 12th International Conference on Document Analysis and Recognition (ICDAR), pages 1464–1470, 2013
6. B. Zhu, X.-D. Zhou, C.-L. Liu, and M. Nakagawa. A robust model for on-line handwritten Japanese text recognition. IJDAR, 13(2):121–131, Jan. 2010.
7. B. B. Chaudhuri, U. Pal, “A complete printed Bangla OCR system,” Pattern Recognition, vol. 31, pp. 531–549, 1998.
8. U. Pal, “On the development of an optical character recognition (OCR) system for printed Bangla script,” Ph.D. Thesis, 1997.
9. U. Pal, B. B. Chaudhuri, “Automatic recognition of unconstrained offline Bangla hand-written numerals,” T. Tan, Y. Shi, W. Gao (Eds.), Advances in Multimodal Interfaces, Lecture Notes in Computer Science, vol. 1948, Springer, Berlin, pp. 371–378, 2000.
10.  K. Roy, S. Vajda, U. Pal, B.B. Chaudhuri, “A system towards Indian postal automation,” Proceedings of the Ninth International Workshop on Frontiers in Handwritten Recognition (IWFHR-9), pp. 580–585, October 2004.
11.  U. Pal, A. Belad, Ch. Choisy, “Touching numeral segmentation using water reservoir concept,” Pattern Recognition Lett. vol. 24, pp. 261-272, 2003.
12.  Halima Begum et al, Recognition of Handwritten Bangla Characters using Gabor Filter and Artificial Neural Network, International Journal of Computer Technology & Applications, Vol 8(5),618-621, ISSN:2229-6093.
13.  Nibaran Das, Sandip Pramanik, “Recognition of Handwritten Bangla Basic Character and Digit Using Convex Hall Basic Feature”. 2009 International Conference on Artificial Intelligence and Pattern Recognition (AIPR-09)
14.  N. Das 1, B. Das, R. Sarkar, S. Basu, M. Kundu, M. Nasipuri, “Handwritten Bangla Basic and Compound character recognition using MLP and SVM classifier,” Journal of Computing, vol. 2, Feb. 2010.
15.  Mithun Biswas, Rafiqul Islam, Gautam Kumar Shom, Md. Shopon, Nabeel Mohammed, Sifat Momen, and Anowarul Abedin. Banglalekha-isolated: A multipurpose comprehensive dataset of handwritten bangla isolated characters. Data in Brief, 12:103 – 107, 2017.
16.  R. Sarkar, N. Das, S. Basu, M. Kundu, M. Nasipuri, and D. K. Basu, “Cmaterdb1: a database of unconstrained handwritten Bangla and Bangla–English mixed script document image,” International Journal on Document Analysis and Recognition (IJDAR), vol. 15, no. 1, pp. 71–83, 2012.
17.  Ujjwal Bhattacharya and B. B. Chaudhuri. Handwritten numeral databases of indian scripts and multistage recognition of mixed numerals. IEEE Trans. Pattern Anal. Mach. Intell., 31(3):444–457, March 2009.
18.  Fukushima, Kunihiko. Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biological Cybernetics, 36(4):193–202, 1980.
19.  LeCun, Yann, Bottou, Leon, Bengio, Yoshua, and Haffner, ´Patrick. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998a.
20.  Ciresan, D. and Meier, U. Multi-column deep neural networks for offline handwritten Chinese character classification. In International Joint Conference on Neural Networks (IJCNN), pp. 1–6, July 2015.
21.  G. van Rossum, Python tutorial, Technical Report CS-R9526, Centrum voor Wiskunde en Informatica (CWI), Amsterdam, May 1995. (Python Software Foundation, https://www.python.org/).
22.  Pulli, K., Baksheev, A., Kornyakov, K. and Eruhimov, V. (2012). Real-time computer vision with OpenCV. Commun. ACM, 55(6), p.61.
23. Pillow (PIL Fork), http://pillow.readthedocs.io/en/3.1.x/reference, last access:19 June 2018
24.  Summerfield, Mark. Rapid GUI Programming with Python and Qt: The Definitive Guide to PyQt Programming. Upper Saddle River, NJ: Prentice-Hall, 2008.
25.  Canny, J., A Computational Approach To Edge Detection, IEEE Trans. Pattern Analysis and Machine Intelligence, 8(6):679–698, 1986
26.  N. Otsu, “A Threshold Selection Method from Gray-Level Histograms,” in IEEE Transactions on Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, Jan. 1979. doi: 10.1109/TSMC.1979.4310076.
27.  Ekush: A multipurpose and multitype comprehensive database for Online Off-line Bangla Handwritten Characters, Website: https://github.com/shahariarrabby/Ekush. Last access: 20 Jun. 18
28.  Sergey Ioffe, Christian Szegedy, “Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift”, arXiv:1502.03167 [cs.LG]
29.  Srivastava, Nitish & Hinton, Geoffrey & Krizhevsky, Alex & Sutskever, Ilya & Salakhutdinov, Ruslan. (2014). Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Journal of Machine Learning Research.15.1929-1958.
30.  Kingma, Diederik P. and Ba, Jimmy. Adam: A Method for Stochastic Optimization. arXiv:1412.6980 [cs.LG], December 2014.
31.  Katarzyna Janocha and Wojciech Marian Czarnecki. On loss functions for deep neural networks in classification. Arxiv, abs/1702.05659, 2017
32.  Schaul, Tom, Zhang, Sixin, and LeCun, Yann. No more pesky learning rates. arXiv preprint arXiv:1206.1106,2012.
33.  B. B. Chaudhuri, U. Pal, “A complete printed Bangla OCR system,” Pattern Recognition, vol. 31, pp. 531–549, 1998.
34.  Chandan Biswas et al, HMM-Based Online Handwritten Bangla Character Recognition using Dirichlet Distributions, 978-0-7695-4774-9/12 $26.00 © 2012 IEEE DOI 10.1109/ICFHR.2012.213
35.  U. Pal, “On the development of an optical character recognition (OCR) system for printed Bangla script,” Ph.D. Thesis, 1997.
36.  Riasat Azim et al, Bangla Hand-Written Character Recognition Using Support Vector Machine, International Journal of Engineering Works vol. 3, Issue 6, PP. 36-46, June 2016.
37.  Md. Alamgir Badsha et al, Handwritten Bangla Character Recognition Using Neural Network, International Journal of Advanced Research in Computer Science and Software Engineering 2 (11), November- 2012, pp. 307-312.
38.  Mohammad Badiul Islam et al, Bengali handwritten character recognition using modified syntactic method, NCCPB-2005 © Independent University, Bangladesh