Aprendizado por reforço para ajuste dinâmico de dificuldade em jogos de luta

Abstract

Dynamic Difficulty Adjustment (DDA) is a technique that automatically adjusts game parameters, such as items, maps, or opponent behavior, to provide players with challenging and engaging experiences. The goal is to maintain balance and ensure an optimal level of entertainment. This work proposes a reinforcement learning agent for a fighting game to create an opponent whose skill level matches the player's relative ability. To achieve this, two distinct parameters are controlled: Balance and Competitiveness. Balance refers to the difficulty perceived by the player throughout the interaction with the system. Competitiveness is related to the outcome of the task. To address both metrics, a reward function was designed to guide the agent toward having a skill level comparable to the player, thus keeping the match balanced. Additionally, a penalty applied during training was introduced to limit the agent's win rate to the desired level, resulting in an opponent that is neither too weak nor too strong. Furthermore, methods to create robust behavior capable of generalizing against unseen players during training were investigated. Regularization techniques were explored to improve the agent's performance and adaptability. Results indicate that regularization has positive effects on generalization; however, it is insufficient to produce behavior capable of balancing against a diverse set of players. Consequently, a training scheme using self-play was proposed to further enhance the method's generalization capacity without requiring human data or pre-programmed behaviors.