Comparative Analysis and Testing of Deep Learning Neural Network Models for Road Sign Recognition

Introduction. The main problem of neural network models in the form of embedded systems is the limited computing resources, which does not allow obtaining the output result in the shortest possible time. Minimizing the time spent on model training also remains a priority. Therefore, when comparing the accuracy of output data and the learning speed of neural network models, the main attention should be paid to different image sizes. The goal of the work is to identify a model that gives the most accurate results after training, has high performance and allows you to produce the most extensive output data with minimal input. The novelty of the study lies in the fact that it was carried out using a comparative analysis of five neural network models on one framework with input data sizes of 32 × 32,48 × 48 and 64 × 64, the combination of which has not been studied.

Description of research objects. Five of the most common models were selected as objects of comparative analysis: DenseNet, GoogLeNet, LeNet, MobileNet and ResNet50. The study uses the available TSRD dataset, which includes 6164 images of road signs containing 58 categories. The images were combined into two sub-databases, one of which is training, and the second is test. There are 4170 image files in the training database, and 1994 in the testing database. Each image contains an annotation with information about the four coordinates of the sign and its category. Their size varies from 26 × 28 to 491 × 402 pixels. Tensor-Flow framework was also used to run all neural network models. The advantage of the models is the accuracy of the output data with a relatively small number of inputs.

Conclusion. According to the results of the study, the following leaders in output accuracy can be identified: ResNet50 and MobileNet. In terms of learning speed, MobileNet takes first place with an average of 169 seconds on the GPU, ResNet50 takes second place with a difference of 465 seconds in terms of learning speed on GPU. MobileNet data accuracy showed high results of 94.5% on CPU and 94.5% on GPU. In terms of accuracy and training time, the LeNet and GoogleNet models showed similar performance and accuracy. The DenseNet model showed the most negative results in its accuracy and performance. Changing the size of the input data had little effect on the output accuracy and scalability of the training environment. Due to the fact that the data sizes may not affect the accuracy of the result, when using the TSRD data set, it is proposed to use 32 × 32 dimensions as input data sizes that allow you to find road signs. This solution is more economical and does not affect the accuracy of the output data. The experiments conducted during the study showed that the accuracy of ResNet50 is similar to the accuracy of MobilNet. This proves that for the definition of road signs, deeper models with a longer learning time are not in every case preferable to small ones.