Métodos de otimização hiperparamétrica: um estudo comparativo utilizando árvores de decisão e florestas aleatórias na classificação binária

Abstract

Machine Learning has become a fundamental tool for some areas of knowledge. Much of its strength lies in the existing of dierents levels of complexity for a model and existence of adjustments for its training algorithm. These settings allow the achievement of models with lower generatization error. Hyperparameter is a type of variable that controls these functions and needs to be set even before the training procedure is carried out. Therefore the estimation of their optimum values is crutial to obtain a ne model. This work aims to compare the performance of the following hyperparametric optimization methods: Grid Search, Random Search and Bayesian optimization (using Gaussian Process). These three techniques are applied on tuning of hyperparameters from two types of learing models: Decision Tree and Random Florest. For such comparisons, hyperparaments related to tree depth control and decorrelation level between predictors of a random forest were chosen. These two learning algorithms are applied on binary classication problems, using different datasets.The results obtained show that for a same number of model trainings, the Bayesian optimization technique provides better results comparing to the other methods.