In collaboration with Herui Chen and Peter Zhang.
An alternative MIDI instrument built to promote expressive live performance, powered through a smartphone.
In our task to repurpose an older smartphone, we decided to infuse our project with our interest in music and create an instrument that takes advantage of a modern phone’s built-in sensors and computing power. Our effort resulted in X.E.K.I., a MIDI instrument that offers three parameters of control, enabling the user to have expressive and tactile feedback while using a digital music controller.
Wanting to maintain a high degree of physical motion in the playing of this instrument, the first aspect that was settled was a moving handle at the bottom, making for a motion similar to that of trombone players. The other aspects that came soon after was that of piano-inspired keyboard interface and a violin-like shoulder mount that enabled our desired use of the phone’s accelerometer as a control. It was important for us that each control was assigned to an audio parameter that felt fitting for the motion that was required.
Hardware
After considering various approaches for producing sound using a smartphone, establishing MIDI communication between a Google Pixel 3 and an Arduino microcontroller was the chosen setup. A microcontroller is necessary to process digital inputs which the push-buttons (keyboard) and the rotary encoder (moving handle) require. As this is its only use in our setup, any microcontroller that has at least 15 digital input pins can work, provided that there is a MIDI library available or developed.
Software
The Processing for Android library proved to be the programmable software that we ran on the Pixel 3. Initially playing with the idea of producing sound using the Processing built in Sound library, it quickly became evident that such an approach would limit the audio range of the instrument significantly. We opted to use a fully equipped music app instead, having Processing run in parallel to extract phone sensor data and pass them as MIDI messages to the app. Used along the digital inputs of Arduino, the music application received MIDI data from two separate programs.
The 3D-printed handle for the slider. Two pushbuttons are for sound switching, used to toggle between different audio waveforms in this iteration. A rotary encoder is placed on top to calculate the sliding position thanks to a gear track mechanism.
