Classificação e modelagem de tópicos para documentos de licitação via NLP e deep learning

Abstract

Open government data (OGD) encompasses data on government actions, expenditures, and investments, made accessible and transparent to the public. In Brazil, the Access to Information Law (Law No. 12,527 of 2011) guarantees citizens the right to access information from the three branches of government, Federal District, states and municipalities. OGD promotes transparency and public participation, and is essential for a variety of applications in both the public and private sectors. In particular, public procurement processes – which involve a wide range of documents such as minutes, notices, and errata – are processes that are susceptible to fraud and irregularities, since they directly involve the use of public resources. In response to these challenges, recent research has been developing data-driven applications that strengthen the intelligence of regulatory agencies and facilitate the monitoring of public expenditures by citizens, promoting security in the bidding process. This paper presents a case study of Minas Gerais, focused on applications of Natural Language Processing (NLP) and Deep Learning (DL) to automate processes and detect latent patterns in municipal bidding documents. Municipal bidding processes bring additional challenges, since the lack of standardization among the transparency portals of the municipalities generates a diversity of formats and ways of making the documents available. Our work begins with the construction of LiPSET, a dataset with 9,761 documents from 18 municipalities in Minas Gerais, of which 6,337 were labeled in 4 meta-classes and 13 types, developed with the support of experts. The characterization of LiPSET allowed us to observe the main challenges of the municipal public bidding domain, such as class imbalance and the lack of pattern in the distribution of documents by municipality. We also present LiBERT-SE, a BERT model adapted and trained specifically for public bidding documents, with the potential to serve as a basis for several applications. Two practical applications were developed: the first is an automatic classification of documents that uses heuristic methods and LSTM models, evaluating preprocessing and textual representation techniques to optimize performance; in this application, the proposed methods achieved good results, with values for F1-Macro and F1-Weighted higher than 96% in the best configuration for classification by document type. For the classification of meta-classes using the heuristic method, the result was 91% of F1-Macro. The second application, latent topic modeling in bidding documents, evaluates the potential of LiBERT-SE to identify patterns using BERTopic, a topic modeling methodology based on clustering techniques and contextualized sentence embeddings; in this context, LiBERT-SE consistently outperformed the baselines with statistical significance in the metrics of topic coherence and diversity. In general, the qualitative evaluation identified general bidding themes in the topics generated by LiBERT-SE; however, noises was observed in the words describing the topics. In summary, the presented applications demonstrated practical impacts for document classification, while topic modeling still requires refinement to enhance its application in public interest queries. The LiPSET dataset and the LiBERT-SE model are publicly available and can be used as a basis for future research aimed at detecting patterns in public bidding documents.