Dual-Eye Augmented Reality Goggles Recognize Faces, Gestures

View full size image

We’ve long been excited about Google’s upcoming Project Glass wearable computer. However, at Mobile World Congress we used an AR headset and operating system that provide an experience which makes Google’s device look like a toy. The product of an experiment by the R&D department at Brilliantservice, a Japan-based app development house, the Viking operating system takes a different approach to wearable computing, relying on gesture controls and projecting a 720p image into both eyes at once.

We had a chance to don a pair of AR goggles, which Brilliantservice threw together from off-the-shelf parts just for testing purposes, and use the Viking OS to draw a painting, open apps and even match names with people’s faces just by looking at them. Written in Objective C, Viking relies on gesture controls as its primary form of input. As we stuck our hand out in front of our face, the cameras on top of the nose bridge recorded our movements and we were able to see a graphical avatar of our hand projected on the heads-up display.

The headset was connected to a laptop, which provided its power and processing capabilities. For the benefit of people visiting the booth and our video viewing audience, the laptop screen mirrored what we were seeing in the goggles. However, in real life, the user would simply wear the goggles, which would have their own power source, and look through them.  

Even though we had to put the headset on over our prescription glasses, we were able to make out all the graphics in the operating system clearly as they appeared on the headset’s lenses. We had to make sure we kept our gaze focused on the lenses, because the images were very translucent. However, the good news is that the graphics didn’t overwhelm our field of vision. It would be very easy for us to wear these all day and walk around in them.

We then pushed the same finger forward to move press a virtual brush down on paper and dragged it around by simply moving our hand with our index finger extended. To close the app, we simply held all five fingers out until a graphical exit arrow appeared on screen, at which point we moved our hand down to exit the app and return to the home screen.

Since this the software is in a very early state, there was very little content in the OS. There were two shortcuts, one for the drawing app and another for a phone dialer, a graphic of a map and a clock with a couple of non-working settings buttons. To scroll up and down the desktop, we had to raise and lower our head; the device does not support horizontal scrolling.

Viking OS also lets you know whether your hand is too close to your face or too far way from it. If your arm isn’t extended enough, a shadow will appear below your hand avatar on screen. If it is extended too far, the on-screen hand will turn red.

The headset was able to perform some very basic facial recognition. When we looked at Johannes Lundberg, the senior engineer responsible for the OS, a box appeared around his head which correctly showed his name. However, when we looked at another booth-goer who was not listed in the system, the software incorrectly identified him as someone else, rather than simply reporting that he was not in the database.

Lund told us he combined Vuzix STAR 1200XL glasses ($4,999), a generic RGB camera and a PMD CamBoard nano depth camera to make the demo headset we used. However, Brilliantservice has no plans to market and sell its own AR goggles. Instead, the company is looking to partner with a hardware developer who can work with the company to design and manufacture a much more attractive headset that uses the Viking OS. To get a better sense of what he envisions, Lundberg showed us a non-working headset that looked much more like a set of sunglasses than goggles.

While the Viking OS itself still has a long way to go, we’re impressed with its immersive, gesture-controlled UI. Putting screens over both eyes is key to providing this experience and we hope more headset developers will follow suit.

Lundberg told us that he is the only person working on the project and only started on it last summer so the amount he’s accomplished in such a short time is truly impressive. He said that, when he finds a partner, he’d like the final device to include voice recognition and a bluetooth style controller for situations where users are in tight spaces and can’t stick their arms out.

We look forward to seeing how the Viking OS and the hardware based on it develops over time.

This story was provided by LAPTOP, a sister site to LiveScience.