Multi-modal coupling in musical performance

Abstract

The manipulation of acoustic parameters in musical performances is a strategy widely used by musicians to demonstrate their expressive intentions. In the case of instrumental ensembles, the coordination of these manipulations between performers is crucial for the realization of performance. During a group performance interpreters make use of visual and acoustic information transmitted continuously by other interpreters in order to improve their synchronization. We propose to investigate the interpersonal synchronization/coordination in small musical groups and the role of gestural/visual communication among the members of the ensemble. In pursuit of this goal, we adopted two complementary approaches, the analysis of the body movements of the musicians and the analysis of acoustic parameters extracted from the audio signal. One could argue that, in the case of ensemble performances, visual information have a crucial role in the coordination of musical events. However, players are also able to follow other musicians solely by listening to acoustic signals. This performance condition has become trivial as a consequence of the growth in music industry, where the demand for perfection and the need to lower the costs led to an increase in the number of recordings made in studio. Moreover, different types of body movements not directly linked to the production of sound can be constantly observed in any instrumental performance, even in solo performances. In fact, as pointed out by Gabrielsson (2003, pag. 249), the movement performed by a musician, in addition to communicate relevant information to the coordination with others, may also assume many different roles, such as to communicate expressive intentions, to provide information about the artists personality or simply entertain the audience. Over the past years, great efforts have been devoted to the study of these movements and, although there is no consensus on its origins or roles, its existence is undeniable (M. M. Wanderley & Depalle, 2001). This thesis is inspired by findings of previous works (P. Keller, Knoblich, & Repp, 2007; P. Keller & Appel, 2010; M. A. Loureiro, Mota, Campolina, C., & Laboissière, 2012), which argue that musicians playing in ensemble try to adapt their actions to the actions of others, and that the quality of this adjustment could be affected by their movements. In the work we try to review different aspects that integrates the construction of ensemble performance and map their relevance for the adjustment between musicians.