Aprendizado ativo para ordenação de resultados

Abstract

Ranking is an essential feature of many applications: from web search to product recommendation systems and online advertising, results have to be ordered based on their estimated relevance with respect to a query or based on a user profile or personal preferences. Ranking is especially important in Web Information Retrieval where a simple query can return hundreds of thousands or millions of documents, but the user will probably only scan the first few results. Learning to Rank (L2R) algorithms use Machine Learning (ML) techniques to provide better rankings than those obtained by classic Information Retrieval models, such as BM25. L2R algorithms need a labeled training set to generate a ranking model that can be later used to rank new query results. These training sets are very costly and laborious to produce, requiring human annotators to assess the relevance or order of the documents in relation to a query. Active learning algorithms are able to reduce the labeling effort by selectively sampling an unlabeled set and choosing data instances that maximize a learning function\\\'s effectiveness. In this work we propose a novel associative rule-based active learning method that can be used to actively select document instances from an unlabeled set without the need of having an initial seed training set. Used alone, this method provides very small and yet effective training sets. Although effective, this method is not easily extended to select more instances if necessary. Thus, another contribution of this work is a round-based second stage selection method can be used to select more instances for labeling as needed. This second stage method initially uses the small sets selected by the first method to train a committee of distinct L2R algorithms. It then progresses iteratively in rounds selecting, for each query, those documents that the committee members most disagree about in ranking. Together, these complementary methods are very powerful and flexible making the resulting combined method applicable to many real-world scenarios. We test our methods with various benchmarking datasets and compare them to several baselines to show that they achieve very competitive results using only a small portion of the original training sets. For instance, in half of the datasets tested, while selecting less than 5% of the original training sets, the proposed combined method achieves results that are better than state-of-the-art L2R algorithms trained on the complete training sets and that also surpass a strong active learning for L2R baseline method.