Please use this identifier to cite or link to this item:
http://hdl.handle.net/1843/68565| Type: | Artigo de Periódico |
| Title: | DOD-ETL: distributed on-demand ETL for near real-time business intelligence |
| Authors: | Gustavo V. Machado Ítalo Cunha Adriano Cesar Machado Pereira Leonardo B. Oliveira |
| Abstract: | The competitive dynamics of the globalized market demand information on the internal and external reality of corporations. Information is a precious asset and is responsible for establishing key advantages to enable companies to maintain their leadership. However, reliable, rich information is no longer the only goal. The time frame to extract information from data determines its usefulness. This work proposes DOD-ETL, a tool that addresses, in an innovative manner, the main bottleneck in Business Intelligence solutions, the Extract Transform Load process (ETL), providing it in near real-time. DOD-ETL achieves this by combining an on-demand data stream pipeline with a distributed, parallel and technology-independent architecture with in-memory caching and efficient data partitioning. We compared DOD-ETL with other Stream Processing frameworks used to perform near real-time ETL and found DOD-ETL executes workloads up to 10 times faster. We have deployed it in a large steelworks as a replacement for its previous ETL solution, enabling near real-time reports previously unavailable. |
| Subject: | Ciência da Computação Big data Business intelligence |
| language: | eng |
| metadata.dc.publisher.country: | Brasil |
| Publisher: | Universidade Federal de Minas Gerais |
| Publisher Initials: | UFMG |
| metadata.dc.publisher.department: | ICEX - INSTITUTO DE CIÊNCIAS EXATAS ICX - DEPARTAMENTO DE CIÊNCIA DA COMPUTAÇÃO |
| Rights: | Acesso Aberto |
| metadata.dc.identifier.doi: | https://doi.org/10.1186/s13174-019-0121-z |
| URI: | http://hdl.handle.net/1843/68565 |
| Issue Date: | 20-Nov-2019 |
| metadata.dc.url.externa: | https://jisajournal.springeropen.com/articles/10.1186/s13174-019-0121-z |
| metadata.dc.relation.ispartof: | Journal of Internet Services and Applications |
| Appears in Collections: | Artigo de Periódico |
Files in This Item:
| File | Description | Size | Format | |
|---|---|---|---|---|
| DOD-ETL_ distributed on-demand ETL for near real-time business intelligence.pdf | 743.24 kB | Adobe PDF | View/Open |
Items in DSpace are protected by copyright, with all rights reserved, unless otherwise indicated.