Automatic WLAN control using reinforcement learning for improved quality of experience
| dc.creator | Henrique Duarte Moura | |
| dc.date.accessioned | 2020-05-26T21:41:49Z | |
| dc.date.accessioned | 2025-09-09T00:22:33Z | |
| dc.date.available | 2020-05-26T21:41:49Z | |
| dc.date.issued | 2019-12-20 | |
| dc.description.sponsorship | CNPq - Conselho Nacional de Desenvolvimento Científico e Tecnológico | |
| dc.description.sponsorship | CAPES - Coordenação de Aperfeiçoamento de Pessoal de Nível Superior | |
| dc.identifier.uri | https://hdl.handle.net/1843/33548 | |
| dc.language | eng | |
| dc.publisher | Universidade Federal de Minas Gerais | |
| dc.rights | Acesso Aberto | |
| dc.rights.uri | http://creativecommons.org/licenses/by-nc-nd/3.0/pt/ | |
| dc.subject | Computação - Teses | |
| dc.subject | Aprendizado de máquina. | |
| dc.subject | Sistema de comunicação sem fio. | |
| dc.subject | Redes de computador | |
| dc.subject.other | Computer Science | |
| dc.subject.other | Computer networks | |
| dc.subject.other | Machine Learning | |
| dc.subject.other | Wireless Network | |
| dc.title | Automatic WLAN control using reinforcement learning for improved quality of experience | |
| dc.title.alternative | Controle automático de WLAN usando aprendizado por reforço para melhorar a qualidade da experiência | |
| dc.type | Tese de doutorado | |
| local.contributor.advisor-co1 | Marcos Augusto Menezes Vieira | |
| local.contributor.advisor1 | Daniel Fernandes Macedo | |
| local.contributor.advisor1Lattes | http://lattes.cnpq.br/8758395845049687 | |
| local.contributor.referee1 | Adriano Alonso Veloso | |
| local.contributor.referee1 | Steven Latré | |
| local.contributor.referee1 | Alex Borges Vieira | |
| local.contributor.referee1 | José Marcos Silva Nogueira | |
| local.creator.Lattes | http://lattes.cnpq.br/5445352795551453 | |
| local.description.resumo | Real-time performance demands, such as augmented reality, and high quality video streaming brought new challenges to network control, as users expect high levels of quality to be upheld. Improving the user satisfaction and minimizing customer turnover while still maintaining their competitive advantage can be a significant challenge for service providers. Quality of Experience (QoE) is hard to estimate because it is perceived subjectively by users, as a result of the user’s internal state (e.g., predispositions, expectations, needs, motivation, mood), the designed system characteristics (e.g., complexity, purpose, usability, functionality, relevance), and the context within which the service is experienced (e.g., organizational/social settings, meaningfulness of the activity, voluntariness of use). In this thesis, we focus on Wireless LANs (WLANs). WLAN has become commonplace in office, and campus sites, and are the most common method of Internet access for home uses. WLAN are subject to an already crowded radio spectrum, and the communication deteriorates due to the dynamics of the transmission medium, fading multipath channels, interferences, and misconfiguration. An automatic approach to network control should be developed, because: (i) WLAN users are non-technical; (ii) access providers do not perform direct or online control of the devices in the residences, for legal reasons or because of technical difficulties due to the large number of users; (iii) the increasing demand for more performance by the users; and (iv) the ISP’s need to maintain its competitiveness. To achieve that, a machine should learn from the execution of a particular task to maintain a specific performance metric, and based on past experience, improve the performance while executing a task. Reinforcement learning (RL) guides the learning of an agent using a reward-aware approach. It is very useful in networking, because RL algorithms perform decision making under unknown or very-hard-to-model network conditions. Then, learning algorithms can vary network parameters to meet some performance criteria, such as QoE, providing a more user-centric approach to network services. At the same, the Software Defined Networking (SDN) paradigm can be used to further improve the management of computer networks. SDN advocates the separation of the control and data planes, and the use of a (logically) centralized control. Those characteristics allow the network administrators to have a fuller view of the system, and thus perform global optimizations on the network. To achieve auto-configuration and auto-optimization capabilities with minimal or no human management, two prerequisites must be fulfilled: a control loop that gathers information from the environment and acts on the devices, and some QoE metric representing the user perceived satisfaction. In this thesis, we use Multi-Armed Bandit (MAB), Q-Learning (QL), and Deep QL as control loops, varying from a simple to a complex model. We built two prototypes, one for web browsing and one for video traffic.To evaluate the web browsing application, we use a QoE metric from the literature, composed by three regressors (QoE predictors), one for each of three classes of web sites. We proposed a classification algorithm using semi-supervised learning, which correlates the example of the site classes provided by the authors of the predictor. Next, we developed a QoE predictor as well as a Deep QL control loop for video traffic. results show that the control loops improve QoE. In the web browsing scenario, our best result improved the QoE by 167%, if compared to the baselines, and improved the page load times by up to 6.6 times. In the video streaming evaluation the control loop improves the QoE by 91% compared to the baselines. | |
| local.identifier.orcid | https://orcid.org/0000-0002-6464-0694 | |
| local.publisher.country | Brasil | |
| local.publisher.department | ICX - DEPARTAMENTO DE CIÊNCIA DA COMPUTAÇÃO | |
| local.publisher.initials | UFMG | |
| local.publisher.program | Programa de Pós-Graduação em Ciência da Computação |