Extração de características em reconhecimento de parâmetros fonológicos da Língua Brasileira de Sinais utilizando sensores RGB-D

Abstract

The feature extraction in Sign Language Recognition (SLR) is a challenging problem in Computer Vision. In this work, a methodology for feature extraction in Brazilian Sign Language (BSL, or Libras in Portuguese) that addresses some of these challenges is proposed. In this methodology the phonological structure of the language, relying on RGB-D sensor for obtaining intensity, position and depth data is explored. From the RGB-D images we obtain seven vision-based features. Each feature is related to one, two or three structural elements in BSL. This relation between extracted features and structural elements based on shape, movement and position of the hands is investigated. A Support Vector Machines (SVM) is employed to classify elements based on these features. Finally, distances between classified and desired elements are calculated. From these distances, the signs classification is performed. The experiments show that the attributes of these elements can be successfully recognized in terms of the features obtained from the RGB-D images, with accuracy results individually above 80% on average. It can be concluded that the proposed feature extraction methodology and the decomposition of the signs into their phonological structure is a promising method to help expert systems designed for SLR.