Tech Sandbox: The Playground of Neural Engineering

If smiling for a video game could be, by itself, enough to make some people happy, then "Unicorb" — a video game controlled by a player's facial muscles — should be regarded as something akin to a cyber fountain of joy. Why? Because the object of the video game is to smile repeatedly in order to keep a unicorn in flight as it jumps through rainbows in a video. Players' facial expressions can be detected and translated into Unicorb controls because, while playing, they must wear Thalia — a wearable interactive system that captures telltale signals from facial expressions.

"Thalia is the muse of joy," said James Wu, a co-creator of Thalia and Unicorb and a graduate student in bioengineering  at the University of Washington. "Many studies have shown that just the physical act of smiling improves your overall mood and mental health."

Thalia and Unicorb were created by Wu, along with three other University of Washington students: Karl Marrett, a graduate student who has a research background in neurology, Tyler Maxfield, a bioengineering undergraduate and Tiffany Youngquist, a bioengineering graduate student. The students created Thalia and Unicorb for a 2014 competition called Tech Sandbox, which was held at the end of a course, also called "Tech Sandbox," offered at the University of Washington in 2014.

Latest Videos From

Watch full video here:

The purpose of Tech Sandbox was explained by Lise Johnson — the instructor of Tech Sandbox and the education manager of the Center for Sensorimotor Neural Engineering (CSNE), an Engineering Research Center funded by the National Science Foundation and based at the University of Washington: "Students are supposed to create projects that demonstrate the core principles of neural engineering, and they should also be things that can work as educational demos. We want them to be fun and interesting and maybe marketable, commercializable."

Also, the Tech Sandbox competition provides a mechanism for encouraging students and faculty to experiment with the CSNE's sophisticated equipment.

What's in a smile?

Initially, the creators of Thalia and Unicorb wanted to incorporate into their game an electroencephalogram (EEG), which measures signaling from the brain's occipital lobe, where visual information is processed. But because electrical signaling from muscles is more robust, the team instead ultimately incorporated into their game an electromyogram (EMG), which measures the electrical activity of resting and contracting muscles.

Although EMGs have previously been used extensively to study facial muscles, the use of EMGs in a video game is novel.

During a game, Thalia's EMG electrodes must be positioned on specific muscles located around each player's mouth, eyes, and other facial areas that move in characteristic ways when people express happiness or surprise. The electrodes capture the muscle player's signaling, which is then amplified and interpreted to control the height and speed of Unicorb's flying unicorn.

"The control signals are designed so you have to use your smiling muscles and surprise muscles," said Youngquist. "The idea is that through facial feedback, we can kind of enforce a state of happiness or positivity in the user."

The creators of Thalia and Unicorb hope that further development of Unicorb may eventually lead to its use as a depression therapy, particularly for young children.

An experiment in progress

The Tech Sandbox competition was first piloted in 2013 as an extracurricular activity. Then, in 2014, it was for the first time offered as part of the Tech Sandbox course. Groups of undergraduate and graduate students enroll in Tech Sandbox as teams and, at the end of the quarter, compete in the annual Tech Sandbox competition.

Although no prerequisites are required to enroll in the Tech Sandbox course, it has, so far, mainly attracted bioengineering students, along with a few students from other disciplines.

"It's kind of an experiment in progress," explained Johnson. "There are no lectures and there is no curriculum. One of the good things about offering it as a course is that there's some sort of incentive for people to do a good job."

Tech Sandbox Benefits

During the 2014 Tech Sandbox course, Johnson requested from each team progress reports so that she could advise them against bad ideas and keep them on track for finishing their devices.

"One of the greatest things you can learn when trying to build a technology in only 10 weeks is what ideas will work and what will not," said Marrett, who aimed to further his understanding of neural engineering through Tech Sandbox. 

"The winning project from last year (2013) has been used in tons and tons of educational and outreach demonstrations," said Johnson, referring to WrestleBrainia 3000 — a game that uses EMG signals from the arms of two human competitors to control arm-wrestling robots. "Everyone had a good time doing it, so we thought that it would be a good thing to continue."

Students in the 2014 Tech Sandbox course benefitted from guidance from Johnson, and advice on their project designs from their teaching assistant Dev Sarma, one of the creators of WrestleBrainia. Also, Tech Sandbox has provided opportunities for undergrads to receive mentoring from teammates who are graduate students; these graduate students, in turn, gain mentoring experience.

"This competition has allowed me to work with graduate students who are quite knowledgeable in their fields," said Maxfield. "It's been quite beneficial to have the group atmosphere and be able to come up with ideas and hash them out logistically."

In 2014, five teams enrolled in Tech Sandbox to win tablets and a chance to demonstrate their understanding of neural engineering. The winning project of the year was vHAB, a virtual reality game that helps stroke patients regain motor control of their hands. Responsive to even the smallest motions made by players, vHAb can be used to help tracks patients' progress and provide them with motivating feedback throughout recovery periods that may be long and excruciating.

The Future of Tech Sandbox

CSNE is currently expanding the unique Tech Sandbox model to partner institutions — MIT, Morehouse College and San Diego State University — to evaluate its potential scalability, particularly for institutions that may be constrained by limited resources or few grad students. Such efforts to improve the adaptability of Tech Sandbox would likely help encourage students from traditionally underrepresented groups to participate in science and engineering.

Keith Roper, an NSF program director who oversees CSNE, said, "The awareness drawn by the Tech Sandbox to cognitive, psychological, and wellness aspects of neural engineering is an outgrowth of interdisciplinary activity at the center, which engages medical and health professionals with engineers to use technology to both advance science and benefit society in important ways."

Editor's Note: The researchers depicted in Behind the Scenes articles have been supported by the National Science Foundation, the federal agency charged with funding basic research and education across all fields of science and engineering. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author and do not necessarily reflect the views of the National Science Foundation. See the Behind the Scenes Archive.