Air Guitar via Electromyography

Introduction and Background Info

As the field of music technology progresses, modern advancements in Human Interface devices are increasingly being applied to control and composition of electronic music. The goal of this project is to utilize electromyographical (EMG) sensors and inertial measurement units (IMU) in order to develop a functional MIDI “Air Guitar”. To achieve this goal, two main devices will be used. The first is called the Myo Armband, which is a device containing 8 EMG electrodes packaged into a band worn on the forearm. The Myo also includes a microcontroller and battery, as well as an IMU to determine arm orientation. The Myo’s firmware recognizes five predefined gestures: wave out, wave in, fist, fingers spread, and rest. The Myo incudes a software API written in C++ for third party development. While the Myo provides a wealth of Human Interface options for one hand, in order to mimic the function of a normal guitar the project must include some form of sensing for the other hand. As such, the Invensense MPU-6050 IMU was selected to be worn on the other hand. This IMU provides 3 axes of accelerometer data, and 3 axes of gyroscope data. The IC is commonly used in smartphones and tablets, and its small size and low power consumption make it ideal for wearable applications. Completion of the Air guitar project required some mechanical design of a “glove” which houses the MPU-6050 IMU, and supporting electronics required to provide output to the computer. Following this, a modular approach to software development was taken. The first module is the IMU code, written in C++ for Arduino. The second module is the C++ code to output relevant Myo IMU and gesture data so it can be converted to MIDI. The third and final module is the max patch which receives data from both previous modules, and creates MIDI output based on the data.  

Strum Sensor

Since the left hand “chord” gestures and fretboard position are determined by the Myo Armband, the strumming motion of the right hand is detected by the MPU-6050 connected to an Arduino Nano microcontroller. To securely mount these board to the right hand, a bracket was 3D printed and attached to the back of the hand via a Velcro strap. The image below shows the finished bracket with the Arduino and MPU-6050.

Figure 2: Strum Sensor The MPU-6050 is rigidly attached to the end of the bracket using mounting holes in the PCB. This method of mounting ensures that the IMU cannot move with respect to the user’s hand during use, which is important since any motion would produce erroneous acceleration measurements. The IMU is also mounted as far from the wrist as possible, since most strumming motion occurs at the end of the hand. The Arduino is clipped into the bracket, and is easily removable for upgrades or for easier programming. The mini-USB port fits in a cutout in the bracket, which holds the connector securely during operation. A wireless version of the strum sensor was designed, using the ESP8266 IC to achieve TCP/IP communication. Unfortunately, the added mass of this IC and the batteries to power it would make the sensor cumbersome to wear.