Computer technology can now translate your air guitar moves into music, with no strings attached.

by Elizabeth Quinn

Photo courtesy of Teemu Mäki-Patola Become an instant rock star: A web cam recognizes the orange gloves and software tracks your hands as you rock out.

Forget learning to play the guitar: now you can become a rock star by donning a pair of orange gloves and playing the air guitar. If you haven't been taken seriously in the past as you blast your favorite rock song and impersonate the band's lead guitarist, you may well be now when your antics actually generate music. Finnish researchers are now making it possible for aspiring performers to play an invisible guitar and have their intentions translated into music, thanks to technology involving a webcam and several pieces of software.

The virtual air guitar originated from an EU-funded research project called Algorithms for the Modeling of Acoustic Interactions (ALMA) started in 2002. The project focused on the future of musical instruments and several universities were developing physical sound models: mathematical models of real-world objects that produce sound. The virtual air guitar was developed from the model of a single vibrating guitar string by researchers Aki Kanerva, Juha Laitinen and Teemu Maki-Patola.

"We developed several prototypes of user interfaces for these sound models, the first of which was a very rough and simple air guitar. My main responsibility was the musical intelligence: the module that interprets playing gestures in a musical context, to ensure that the sound you hear is cool guitar music," says Kanerva.

The virtual air guitar is made up of several pieces of software but the first step is an input system to read the movements of the air guitarist. The key components are the orange gardening gloves worn by the performer and a webcam connected to gesture recognition software. The researchers chose the colour orange since it's a colour not usually found in people's clothes and would make the person's hands a unique colour that the software could track as they play the air guitar.

Photo courtesy of Teemu Mäki-Patola This screenshot taken from the webcam software shows the guides and the guitar centreline that it constructs.

For the performer to play a song that has no wrong notes, the gestures of the unskilled guitarist have to be intelligently mapped to guitar chords and playing techniques. The guitarist can press a pedal to choose between chord and solo mode: chord mode will map his movements to one of four chords and solo mode gives him access to a pentatonic minor scale, the most common rock and blues solo scale.

The mapping is performed by the musical intelligence module, a piece of software that determines many parameters like the position on the string where a note is plucked or how sharply a note is plucked. Some of the more important factors are the guitar playing style, if the performer is plucking, fretting or bending the virtual strings, and the musical style, for example hard rock or folk. Since playing a note or chord involves so many different parameters, the set of rules used by the software to convert gestures into parameters is very complex. "Imagine that you are holding a guitar, then swing your right hand over the strings as if you were strumming to make a note. The distance between your hands controls the pitch. You can make slides by moving your left hand along the neck, and vibrato by waving your left hand," says Kanerva. "No two playing sessions are the same, and we've noticed that individuals come to have very distinct-sounding styles."

The sound of the virtual air guitar is produced based on sound models of each of the six strings on a guitar, calibrated to replicate the sound of a Fender Stratocaster guitar. Starting with the sound of a plucked string without any vibration, different filters are applied to the sound wave based on the parameters determined by the musical intelligence module. The sound is then passed through a physical model of a tube amplifier, the amplifier of choice for most guitarists because of its pleasing harmonic distortions, and then through the model of a guitar speaker cabinet. The final added effects are equalization, reverb and delay… and then the air guitarist hears his masterpiece.

Kanerva says that the idea behind the virtual air guitar is not to play actual songs, but to give an explosive, visual show and actually control the sound as well. At present, it can be experienced in an exhibit at the Heureka Science Centre in Finland, but soon it could be integrated into your living room. Kanerva has recently founded the Virtual Air Guitar Company and has begun developing a console game based on the original idea. He predicts its release for Christmas 2007, when it will also be available in the United States. "Our vision is that the virtual air guitar will become known as the new karaoke. Combine this with future instruments from us and singing, and you could start a virtual band with your friends, heading for instant stardom."

For more info:

Virtual Air Guitar
http://airguitar.tml.hut.fi/index.html

Professor Tapio Takala
http://www.cs.hut.fi/~tta/