Aprendizado transdutivo baseado em teoria dainformação e teoria do aprendizado estatístico

Abstract

The machine learning problem is most frequently proposed and solved under the inductive inference paradigm, based on classical inductive principles such as the Empirical Risk Minimization (ERM) and Structural Risk Minimization (SRM). These principles are based on the attempt to create a learning rule given some finite training data. They rely on two assumptions: first, that there are enough observations in the training data and that these data are drawn independently and identically distributed (i.i.d.) with respect to the original data distribution of the problem, and second, that one actually needs to determine a general rule able to estimate points anywhere in the space of the problem. Transductive learning challenges these assumptions, breaking the concept of inducing a general rule and using it to deduce unknown values of a functional dependence. In a transductive setting, one estimates particular values of a functional dependence directly, without necessarily having to estimate the general function itself and, implicitly, all of its possible evaluations in a continuum of points. Furthermore, transductive learninggenerally takes advantage of the information available at the points of interest to enhance its estimate of the problems underlying probability distributions and improve its accuracy. Following Occams Razor principle, transductive learning not only avoids solving a harder problem when there may not be enough information to do so, but it also inherently uses all information available a priori, including that given by the points ofinterest. In this work, we propose a new approach to transductive learning based on a variation of statistical learning theory, where risk functionals are directly estimated in a semi-supervised transductive framework. From that, a new parameter-free method, the essentially-transductive entropic classifier (ETEC), is proposed using concepts from information theory, where the learning problem is solved as a constrained optimizationproblem. In selected contexts identifiable using proposed information indicators, we show that our method outperforms other popular inductive and transductive methods for binary classification problems, such as support vector machines (SVMs) and transductive support vector machines (TSVMs).