'Smart Camera' to Boost Robotic Visual Intelligence

Adam Hadhazy

Date: 18 March 2010 Time: 09:50 AM ET

Weeks after his pet tarantula shot hairs at his face, the patient visited the hospital, by which time his eye was red and inflamed.
CREDIT: The Lancet.

DARPA, the United States military's research and development arm, has announced a new project to develop a "smart camera" that would help robots better understand the world around them.

Called Mind's Eye, the program does not yet have funding or active experiments. But to get the ball rolling, DARPA – which stands for Defense Advanced Research Projects Agency – will host an industry day on April 20 in Washington, D.C. to foster discussion amongst companies interested in the smart camera project.

According to the DARPA announcement, the "Mind’s Eye program seeks to develop in machines a capability that currently exists only in animals: visual intelligence."

Essentially, the goal will be to imbue machine's visual processing systems with a broader understanding of the motives and reasons behind why recognized objects might be moving in such-and-such a way.

Currently, state-of-the-art machine vision research has enabled robots to identify a wide range of objects – the "nouns" in a scene, according to DARPA.

But machines still do a poor job of perceiving the "verbs" in their visual field, or the interplay between these objects, and then forming an overall narrative of the unfolding action.

Humans do this with ease: By looking at a scene, we can mentally form abstract hunches about what the purposes are behind witnessed activity. In other words, instead of just mindlessly watching puppets move randomly in time and space, we can deduce the "strings."

The better to see you with

One of the first uses of a smart camera, according to DARPA's vision, could be beefed-up surveillance systems on small, unmanned aerial vehicles (UAVs) similar to the drones that fly above Iraq and Afghanistan.

Instead of just bringing pictures and video back for human intelligence experts to then visually piece together, a smart camera-equipped drone could infer that an insurgent is planting a roadside bomb.

Similar visual intelligence would also be considered for unmanned ground vehicles (UGVs), according to DARPA.

Nowadays, UGVs defuse bombs and perform other hazardous duties, sometimes in hard-to-reach places. They have also been developed for use as autonomous war machines.