Modelo computacional para controle do movimento de dedos de próteses robóticas baseado em aprendizado por reforço

Abstract

Robotic prostheses currently demonstrate an elevated level of functionality, similar to physiological limbs. However, they still lack control systems that are adapted to user needs. The use of surface electromyography (sEMG) has been demonstrated to enhance user integration with the device, with application in controllers that translate the signal into movements desired by the user. Despite their high level of accuracy, these controllers are not adapted to activities of daily living (ADLs), as the device's movement is standardized and does not provide a smooth transition. To enhance this system, researchers are exploring the potential of translating the signal into joint angles through regression models. However, variations in numerous parameters from user to user necessitate careful consideration when implementing these models. Aiming at the development of myoelectric controllers adapted to the environment and the user for the execution of ADLs, this doctoral thesis presented an evaluation of the use of reinforcement learning (RL) techniques to create a new myoelectric controller that combines a myoelectric classifier and a trajectory controller for the execution of physiological trajectories by the device. The pilot test of the trajectory controller was initiated using an artificial neural network (ANN) that had been trained using RL with the Deep Deterministic Policy Gradient (DDPG) algorithm. In this investigation, the controller was employed to flex a finger with physiological dimensions, executing two predetermined trajectories, sinusoidal and linear, controlling the torque of each joint. An evaluation of the controller's response revealed the presence of diverse strategies employed to fulfill the expected trajectories. The mean error observed was 3.98 mm ± 2.90 mm relative to the target sinusoidal trajectory and 3.22 mm ± 1.42 mm for the target linear trajectory. In addition, it was observed that the controller presented different responses when different proportional inputs for trajectory selection and models with different parameters were used. This indicates the controller's adaptability. An evaluation of the application of the training technique to a real prosthesis was conducted, resulting in the development of a myoelectric controller for the HANDi Hand prosthesis, which was modelled in a finite element analysis (FEA) program. The motion classifier, employed as a trajectory selector, was trained using pattern recognition techniques, with a convolutional ANN based on the Inception architecture. The trajectory controller was designed to execute trajectories with the prosthesis for force grasping, tripod pinch, and two-finger pinch movements. The RL Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm and a network comprising convolutional LSTM cells were utilized in this context. The sEMG signal classifier demonstrated an accuracy of over 96% with data from an amputee volunteer following calibration. The elevated computational cost of the FEA model employed in the training environment was impractical for extended training over thousands of episodes, suggesting that the effective utilization of the proposed methodology necessitates the investigation of simplified models of underactuated actuators. However, the controller with the highest training reward, it was observed that different strategies were employed to execute the movements, both with the training inputs and with the data from amputees.