Descrição semântica de objetos em imagens baseada na Teoria dos Protótipos

Abstract

This research aims to build a model for semantic description of objects based on features detected in images. We introduce a novel semantic description approach inspired on the Prototype Theory foundations. Inspired by the human approach used for representing categories, we propose a novel Computational Prototype Model (CPM) that encodes and stores the central semantic meaning of the object’s category: the semantic prototype. Our CPM model is used to represent and construct the semantic prototypes of object categories using Convolutional Neural Networks (CNN). The proposed Prototype-based Description Model uses the CPM model to describe an object highlighting its most distinctive features within the category. Our Global Semantic Descriptor (GSDP) builds discriminative, low-dimensional and semantically interpretable signatures that encode the semantic information of the objects using the constructed semantic prototypes. Our semantic descriptor use the proposed Prototypical Similarity Layer (PS-Layer) to retrieves the category prototype using the principle of categorization based on prototypes. In our experiments, using publicly available datasets, we show that: i) the proposed CPM model adequately simulates the internal semantic structure of the categories; ii) the proposed semantic distance metric can be understood as the object typicality score within a category; iii) our semantic classification method based on prototypes can improve the performance and interpretation of CNN classification models; iv) our semantic descriptor encoding ignificantly outperforms others state-of-the-art image global encoding in clustering and classification tasks.