Military to Adopt NFL's Instant Replay Technology

Charles Q. Choi

Date: 01 June 2010 Time: 09:24 AM ET

Alabama's Mark Ingram (22) runs on a 14-yard touchdown as Auburn's Josh Bynes (17) and Sen Derrick Marks (94) defend during their NCAA college football game at Bryant-Denny Stadium in Tuscaloosa, Ala., Saturday, Nov. 29, 2008. Alabama, which is just one win away from a likely BCS birth, won 36-0. AP Photo/Dave Martin
CREDIT:

The same video technology the NFL uses for instant replay during football games could soon help monitor battlefields in Afghanistan.

In the past few years, the amount of intelligence and surveillance video coming in from robots and other sources has increased sharply, overwhelming analysts who simply can't keep up.

For instance, U.S. Air Force drones collected roughly 1,800 hours of video a month in Iraq and Afghanistan in 2009, nearly three times as much video than in 2007, noted Howard Lance, chairman, president and CEO of Melbourne-Fla.-based Harris Corporation, which provides the NFL and Major League Baseball instant-replay technology.

This is only expected to grow as the number of robots increases on the battlefield, as do their capabilities — for example, the General Atomics MQ-9 Reaper Unmanned Aerial Vehicle (UAV) can record in 10 directions simultaneously.

Now Harris is helping the Pentagon with this information overload by helping devise a customized video analysis system that might cut the time needed to analyze trillions of bytes of video from weeks to minutes. After all, U.S. broadcasters handle 70,000 hours daily of video, Lance noted.

"This system began getting deployed at a number of locations last year," said John Delay, director of strategy for Harris's government solutions business unit. "It can work with data relayed from sensors, from a UAV platform, or even from a guy with a cell phone."

Video tagging

The key behind this system, dubbed the Full-Motion Video Asset Management Engine, or FAME, is the use of digital "metadata" tags appended to each frame of video that encode details such as time, date and location in space. Knowing when and where each picture was taken on the battlefield is fundamental to depicting a scene on, say, a Google Earth map, much as knowing where each camera is in a football stadium is key during instant replay to finding the right angle to observe a play.

"As an analogy, you want to not only find a book in a library using the index card system, but you want to be able to find a word on page 36 in chapter 12, and tagging information properly can help you find things instantly to review and analyze it," Delay said.

Additional metadata about the event can then be layered on, such as who is shown and what is happening, as well as links to other documents, satellite photographs, cell phone call recordings, map databases or other files.

"If properly tagged, the speed at which you can garner intelligence is mind-boggling," Delay said. "If you have to watch 1,800 hours of content a day, that's a problem. You want to know which content is relevant to what you're looking for in real-time to speed up the pace of analysis."

A key type of metadata could derive from soldiers narrating what they see using chat programs.

"It'd be like announcing a football game, except here you could also have the system listen in and extract words of interest — say, 'IED' (improvised explosive device) or 'combatant,' which helps with search, retrieval and analysis of data," Delay explained. "You can also imagine drawing a circle around a mosque and saying, 'No, that's not the building we're trying to bomb' for more sophisticated situational awareness."

Smart search

Digital thesauruses aid in data searches, to help make sure that a query for a white truck can also turn up pictures tagged as white lorries. "You can also turn off metadata categories if you're not interested in chat data at the moment, for instance, reducing confusion," Delay said.

One can then view data in ways as rich as depicted with football games on TV, which not only show what is happening from multiple angles, but the identity of teams, the current score, the line of the field where a play started, where the ball needs to go for first down, which quarter and down it is, time remaining, how many yards there are to go, as well as pop-up windows and scrolling data giving details on players and scores from others games and audio commentary detailing plays.

"In the next five to seven years, we think this will become a full-blown enterprise across the defense community," Delay said. "The same architecture invented here could also find use in other fields, such as the medical or public safety communities."