Nicolas Brochec, Tsubasa Tanaka. Toward Real-Time Recognition of Instrumental Playing Techniques
for Mixed Music: A Preliminary Analysis. International Computer Music Conference (ICMC 2023),
Oct 2023, Shenzhen, China. ffhal-04263718f
Abstract: In contemporary mixed music, real-time digital sound processing is often applied to live instrumental performances. However, switching between sound effects often relies on manual computer operation or the performer’s foot pedal, burdening the operator. This research aims to develop a system that automatically classifies instrument playing techniques and switches sound effects according to the result of the classification, reducing the burden on the operator and expanding creative possibilities in contemporary mixed music. To realize such a system, the classification accuracy of existing research is not sufficient. In this study, we focused on the flute and tested various input data formats to improve the accuracy of the classification. The results show that using a number of frames of around 15 with the Log-Mel-Spectrogram (LMS) data format improves accuracy. Furthermore, we have measured the computational times of some classification algorithms to assure that the system can actually be used in real time. We found that Multi-Layer Perceptron (MLP) with LMS data format was the best choice among them because it had high accuracy and was fast enough, while other algorithms had concerns about computational speed.
