Fashion retrieval in a semantic space: Balancing identity and fashionability

Abstract

Online social networks, such as Facebook and Instagram, are becoming major sources of clothing inspiration. The problem, in this case, is that a substantial time is generally spent searching for specific looks. In this thesis we tackle the problem of searching of looks by using a content-based retrieval approach - given a query image, we find images with similar meanings in a large database of images posted in online social networks. First, we approximate the meaning of a look, through the pieces of clothes that composes it, using a CNN for representation learning and classification. Then, we apply a ranking function in order to sort the images, considering their relevance to the query. Besides, in order to improve the results of the search, according to the user's needs, we produce a new ranking function, considering the balancing of two non-compromise key aspects in fashion retrieval, i.e. visual identity and fashionability. In this balanced search, the user is able to prioritize the similarity of candidate images or their popularity in terms of fashion. Our results show the improvement of the state-of-the-art in fashion retrieval and also show it is possible to build the balanced rank with a little loss in NDCG. The results also show the impact of culture and lifestyle in different countries, making it necessary that the rank is composed with posts related to the same location of user's.