Análise multimodal da técnica de strumming no violão

Abstract

Audio, video, and motion capture technologies have developed significantly over the past few decades, bringing new methods and perspectives to performance studies. Equipment increasingly compact, portable, and with an excellent cost-benefit ratio contribute to this scenario, boosting the possibilities of researching different techniques used in musical practice. This work performs a multimodal analysis of the strumming guitar technique using multichannel audio, high frame video (replaced in one of the studies carried out with Qualisys optical motion capture system), and a 6 DOF inertial sensor (IMU) to understand the complex gestures of the right hand used in the rhythmic expression. Using data extracted from the execution of popular rhythms - the most common use of this technique - we seek to identify and compare the strategies of different musicians, thus contributing to a better understanding of the strumming. In the first chapter, we present the concepts and tools used in this research. We begin with the strumming technique and its characterization according to psychoacoustic achievements and the generated sound result, followed by a brief review on multimodality and its use in performance studies. Then, we describe the equipment used to extract information (GuiaRT, MetaMotionR, Qualisys, and GoPro), with the approximate procedure for integration acceleration curves and a comparative study with data from MetaMotionR and Qualisys. Next, we discuss the principal component analysis method (PCA), ending with the strategies for the joint data evaluation. Chapter 2 describes the experiments dedicated to two selected rhythms carried out at different times in the research. Initially, we compare performances of Rhythm 1, a simple and regular one, in two situations: without IMU data fusion performed by three musicians and with data fusion performed by one musician. Then, we analyze the performances of the more complex Rhythm 2, bearing arpeggio chords and separation of bass/treble planes, with the sensor in the mode data fusion, comparing three performances by the same musician. We also present a prospective analysis mixing the results of all recordings. The selected tools provided significant and complementary contributions to the analysis of the strumming technique, both in the characterization of different aspects of performances and in the differentiation between musicians. In future work, we plan to increase the number of musicians and the variety of rhythms and deepen the study of correlations between data delivered by the different modalities.