Article by Pierre Saint-Germier (CNRS, STMS – IRCAM), Clément Canonne (CNRS, STMS – IRCAM) and Marco Fiorini (STMS – IRCAM, Sorbonne Université, CNRS) has been accepted for the AIMC2024 Conference (The International Conference on AI and Musical Creativity, 9-11 September 2024, Oxford, UK)
Abstract: Research in Embodied Music Cognition suggests that unless artificial musical agents are embodied in a significant sense, their behavior in collective performance will face significant limitations. As an alternative to the robotic approach, we investigate the hypothesis that Machine Learning may help simulate embodiment in algorithmic musical agents if (i) the data generated by embodied processes bear the mark of that embodiment, (ii) the generative models constructed by machine learning from those data capture relevant aspects of embodiment, such that (iii) the behavior of the agent exploiting such a model inherits some benefits of embodiment for collective music-making. Taking the machine-learning based human-machine co-improvisation software Somax2 as a case study, we provide a theoretical and empirical assessment of this threefold hypothesis. First, we identify by conceptual analysis an aspect of embodiment, capturing the body as a generative constraint on musical performance (as opposed to a multimodal resource for coordination) and propose a method for manipulating experimentally this dimension of embodiment on MIDI representations of musical performances. Drawing on a corpus of MIDI recordings of piano improvisations, we report the result of three experiments showing that (1) our manipulations of embodiment on a MIDI track are understood as manipulations of embodiment by third-party listeners, that our manipulations, when applied to the corpus given to Somax2, make a difference (2) to the experience of human musicians improvising with Somax2 and (3) to the appreciation of the resulting human-machine duets by third-party listeners. Taking into account the various sizes and directions of these effects, as well as the limitations of our experimental setups, we conclude that our data are overall consistent with the hypothesis that Machine Learning helps simulate at least some benefits of embodiment for co-improvisation
