Sketch-finder: uma abordagem efetiva e eficiente para recuperação de imagens com base em rascunho para grandes bases de imagens

Abstract

The cheapening of storage devices for large data volumes has contributed to the emergence of large datasets. However, to have stored information is not enough and to make it really useful. It is important to access needed data quickly and accurately. On this scenario, within the set of multimedia information, the amount of visual information has also been growing and it lacks appropriate and efficient recovery methods. While it is possible to index and retrieve images with traditional methods used for text, based on keywords or tags, the visual media can be best recovered from visual information, since it is the image nature. Thus, this dissertation proposes a new method for sketchbased image retrieval, once that the sketch can be quickly and easily drawn by the user. Among various image retrieval approaches, the use of sketches lets one express a precise visual query with simple and widespread means. The challenge consists on representing the image dataset features on a structure that allows one to efficiently and effectively retrieve images on a scalable system. We put forward a sketch-based image retrieval solution where both sketches and selected contours extracted from the images are represented and compared on the wavelet domain. The relevant information regarding to query sketches and image content has thus, a compact representation that can be readily employed by an efficient index for retrieval by similarity. The use of compressed information is similar to traditional lossy image compression methods and it brings as advantage a small size for the dataset index enabling the indexing of big data. Consequently a smaller and robust index provided by compression makes the answer of the queries faster. To improve the effectiveness of the method, this work also proposes a comparison of the most relevant image contours provided by the query performed in the compressed-domain. This comparison verifies the spatial consistency among the image contours and the sketch. The dataset indexing uses inverted lists either for the compressed information either for the image contours. The use of inverted lists improves even more the efficiency of the proposed approach. Furthermore, with this solution, it is possible to adjust the index size based on the compression rate, in a similar way it is used on traditional lossy image compression reducing quality to gain space. This adjustment affects the index size and reflects on the balance between effectiveness and efficiency that can be easily modified in order to adapt to available resources. A comparative evaluation with a traditional method on the Paris dataset and a subset with 535 thousand samples issued from ImageNet dataset shows that our solution overcame effectiveness of traditional methods while being more than one order of magnitude faster. The approach proposed in this dissertation is also compared to other retrieval methods that use bag of visual features on the Flickr15K dataset. Although these methods have different query objectives and techniques, this comparison places our approach among them. Finally, we put forward a practical mobile application for sketch-based image retrieval for Andoid platform. The application uses the proposed approach of this dissertation and presents an easy and intuitive interface to create a sketch and visualize the results.