Semi-supervised feature selection

Abstract

As data acquisition has become relatively easy and inexpensive, datasets are becoming extremely large, both in relation to the numberof variables, and on the number of instances. However, the same isnot true for labeled instances . Usually, the cost to obtain theselabels is very high, and for this reason, unlabeled data represent themajority of instances, especially when compared with the amountof labeled data. Using such data requires special care, since severalproblems arise with the dimensionality increase and the lack of labels.Reducing the size of the data is thus a primordial need. In the midstof its outstanding features, usually we found irrelevant and redundantvariables, which can and should be eliminated. In attempt to identifythese variables, to despise the unlabeled data, implementing onlysupervised strategies, is a loss of any structural information thatcan be useful. Likewise, ignoring the labeled data by implementingonly unsupervised methods is also a loss of information. In thiscontext, the application of a semi-supervised approach is very suitable,where one can try to take advantage of the best benefits that eachtype of data has to offer. We are working on the problem of semi-supervised feature selection by two different approaches, but it mayeventually complement each other later. The problem can be addressedin the context of feature clustering, grouping similar variables anddiscarding the irrelevant ones. On the other hand, we address theproblem through a multi-objective approach, since we have argumentsthat clearly establish its multi-objective nature. In the first approach, a similarity measure capable to take into account both the labeled and unlabeled data, based on mutual information, is developed as well, a criterion based on this measure for clustering and discarding variables. Also the principle of homogeneity between labels and data clusters is exploited and two semi-supervised feature selection methods are developed. Finally a mutual information estimator for a mixed set of discrete and continuous variables is developed as a secondary contribution. In the multi-objective approach, the proposal is try to solve both the problem of feature selection and function approximation, at the same time. The proposed method includes considering different weight vector norms for each layer of a Multi Layer Perceptron (MLP) neural networks, the independent training of each layer and the definition of objective functions, that are able to eliminate irrelevant features.