Métodos de Krylov aplicados na análise de regressão linear de Big Data

Abstract

The increasing use of massive and complex databases, known as Big Data, renders the optimization of traditional data analysis methods necessary to enable their application to this type of information. Even statistical analysis methods considered simple, such as linear regression, are inefficient when applied in their traditional form to Big Data due to their high computational cost. Consequently, they require adaptations. This paper addresses the application of the Krylov Methods, a class of algorithms, to Big Data to efficiently estimate parameters for linear regression. These methods return an approximation of the solution to the least squares problem at each iteration, being computationally more economical to obtain a satisfactory estimate of the solution compared to other traditional methods, such as the QR decomposition method applied to the least squares problem. Two Krylov methods are presented and studied in the text: the Generalized Minimum Residuals (GMRES) and LSMR, with a strong focus on the latter.

Finally, to evaluate the performance of LSMR, several simulation studies are presented in databases of different dimensionalities, along with applications in real datasets, all considered Big Data. Performance was measured by comparing metrics resulting from the LSMR studies, such as the execution time of the algorithm, with those resulting from the direct method of QR decomposition applied to the least squares problem. In addition, the same approach was employed to correlate LSMR with two other Krylov methods, LSQR and the Conjugate Gradient method. Overall, LSMR performed better regarding execution time, providing estimates of solutions similar to or even better than the other methods evaluated.