The Future of Video Game Input: Muscle Sensors

A muscle computer interface allows interaction with a computer without touching an keyboard, mouse or other input device. In tests, a "gesture recognizer" is learning strumming and fretting so a person can play air guitar.

(Image credit: Saponas et al. / University of Washington, Microsoft, University of Toronto)

Motion control and multi-touch have become common in devices ranging from Nintendo's Wii to Apple's iPhone. But a muscle-sensing system could someday allow gamers to play air "Guitar Hero" without a controller, or help harried parents with full hands open car doors.

Electromyography (EMG) sensors can decode muscle signals from the skin's surface as a person performs certain gestures. Researchers attached such sensors to their forearms, and built a gesture recognition library by monitoring muscle signals related to each gesture. The project emerged as a collaborative effort between Microsoft, University of Washington in Seattle, and the University of Toronto in Canada.

Latest Videos From

Researchers have similarly used the muscle sensing system to add user interface features to the Microsoft Surface, a multi-touch tabletop device. Pressure-sensitive painting allowed Surface users to "draw" on the tabletop and apply more color saturation by pressing down harder. Users could even choose to "pick up" virtual photos using a pinching and lifting motion, and then could "throw" the pictures back on the table.

Jeremy has written for publications such as Popular Science, Scientific American Mind and Reader's Digest Asia. He obtained his masters degree in science journalism from New York University, and completed his undergraduate education in the history and sociology of science at the University of Pennsylvania.