Suporte a reconfiguração dinâmica em aplicações de processamento distribuído de fluxos de dados

Abstract

The scientific and technological advances observed in last decades have as a result the generation of increasing data collections. In some scenarios, these collections are represented as streams, which can be defined as sequences of data in time ordering. Examples include network traffic statistics, sensor measurements used for environments monitoring, data collected by satellites, and in accelerated growth, data available on online social networks and services on the Web. Once data analysis is a fundamental step to understanding and solving various problems, appropriate and efficient tools for processing such collections are necessary. In this context, the use of distributed systems has become popular, providing a robust and efficient solution to build stream processing applications. These applications often run for long time in shared or dynamic environments, being subject to a series of changes that may occur over time, such as faults, increase/decrease of computational resources and variation in workload. The dynamic nature of these applications requires the existence of mechanisms that allow adaptations at runtime, in order to accommodate the changes in the environment. This work proposes a mechanism to support dynamic reconfiguration for distributed data stream processing applications. The mechanism is implemented as a functionality of Watershed, which is a distributed processing system for applications built in the filter-stream programming model. The results of the experimental evaluation show that the mechanism efficiently and consistently supports the proposed reconfiguration operations.