The User Interface Software and Technology (UIST) Conference introduced a bunch of amazing new work that’s coming out of research right now demonstrating new ways of using sensors to command different types of interactions. One in particular was interesting: the muscle sensing system. Ph.D Candidate at the University of Washington, Scott Saponas, in collaboration with Microsoft Research and the University of Toronto showcased how an array of muscle sensors on the forearm can map gestures like pressing your thumb and index finger together as an input to do things like change the track in a playlist, open the car trunk and even rock out to Guitar Hero. Really cool stuff. Check out the video to see for yourself!
Paper Abstract
Previous work has demonstrated the viability of applying offline analysis to interpret forearm electromyography (EMG) and classify finger gestures on a physical surface. We extend those results to bring us closer to using musclecomputer interfaces for always-available input in real-world applications. We leverage existing taxonomies of natural human grips to develop a gesture set covering interaction in free space even when hands are busy with other objects. We present a system that classifies these gestures in real-time and we introduce a bi-manual paradigm that enables use in interactive systems. We report experimental results demonstrating four-finger classification accuracies averaging 79% for pinching, 85% while holding a travel mug, and 88% when carrying a weighted bag. We further show generalizability across different arm postures and explore the tradeoffs of providing real-time visual feedback.
via Procrastineering