GEKIPE - GEste KInect and PErcussion

A film by Aurélien Poidevin about the research project led by Philippe Spiesser, 2016.

"Geste Kinect et Percussion" is a research-creation project that aims to explore the control of virtual instruments based on the analysis of gestures specific to percussionists.

Several Kinect cameras (usually used for games consoles) are used not only to recognise gestures but also to qualify them in a very detailed way (speed, fore-gestures, etc.). These gestures are then matched with Modalys software, developed at Ircam. This allows virtual percussion instruments to be fully modelled and played by sending commands to strike, rub or even blow them. The size, material and shape of the instruments can also be changed continuously.

In particular, the project is studying the relationship between the real physical force brought into play by the percussionist and the quality of the virtual sound in musical relation to this gesture, with the idea of the transparency of the technology serving original musical expressiveness.

In a second phase of the research, the gesture capture system was developed by adding a new sensor to the existing kinect camera: the accelerometer sensors. The addition of generative video made it possible to link images and sound works, creating new artistic and scenographic worlds. This 'hyper' synchronisation between instrumental gesture, sound and image allows the performer to control everything in real time.

This multimedia tool, combining sound and video, is intended to give rise over time to the creation of a new repertoire of works bringing together composers, performers and visual designers, a truly multidisciplinary platform offering the potential for collaboration between different artistic disciplines.