Virtualização de grandes bases de dados irregularmente distribuídas e replicadas

Abstract

Large volumes of data are generated every day by experiments, simulations and all sorts of applications. It is common to observe situations where portions of data are irregularly replicated and distributed in different data sources. The independent generation of correlated data and lack of collaboration on sharing these data result in an irregularly replicated and distributed data set. It would be desirable to be able to handle these several pieces of irregular data (replicated or not) as a unique large dataset. This is called data virtualization and is the focus of this work. On this dissertation, we explore a system which is capable of dealing with irregularly replicated data and is able to create a virtual view of the union of the individual irregular portions of data hosted by each data source. We present a geometric model to represent data intervals. The model allows for virtualization of an irregularly replicated and distributed data set. The work also presents a meta-data indexing mechanism to allow the system to process ranged queries submitted to the data set available through the data virtualization system. Two query fragment scheduling algorithms are proposed, based on the greedy andsimulated annealing approaches. These algorithms are responsible for the selection of which server will be in charge of serving each data queryfragment. The algorithms try to minimize the queries' response time and to balance the load between the servers, taking into account their differentservice capacities and the workload to which each server is submitted to at any given time. The performances of the algorithms are compared based on simulation results and the parameter values used were taken from the workload characterization of a real data-oriented application (the Virtual Microscope).