Sign Language Alphabets Classification by Convolutional Neural Networks

Abstract

Sign language, which is not a universal language, contains differences in terms of language and communities. Recognizing and translating the symbols of sign languages is a very important method in the communication of citizens with disabilities. Sign language numerals consist of numbers from 0 to 9, while alphabetic letters cover the alphabet ranging from the letter A to the letter Z. An artificial intelligence supported automatic classification system has been developed by using a 24-letter data set in the meaning of Sign language (SL) from the symbols representing the specified alphabetic signs. Recently, Convolutional Neural Network (CNN) based models have performed quite well in computer vision problems. Detailed feature maps were created using the unique automatic distinctive feature extraction structure of CNN methods. In this context, different numbers of hidden layers are defined in the proposed model to capture detail features. The results were evaluated in terms of F1 score, accuracy, recall and precision performance metrics obtained by Adam and Adamax optimization methods with a new CNN model consisting of a total of 15 layers. The proposed CNN model provided 0.99 performance metrics in terms of accuracy, precision, recall and F1 score with Adam and Adamax optimization method. It has been observed that training and test performance measures are close to each other and give satisfactory results in terms of performance metrics. With these results, more effective results can be obtained with CNN models, Capsule Networks structures that keep the spatial relations of the features in further studies.