All the robotic drones and battlefield sensors in the world can't win wars if the data deluge leads to information overload for military analysts' brains. That's why the U.S. military has announced it will spend $250 million each year on harnessing the power of "Big Data."
That means creating computer systems that combine "computer speed, computer precision and human agility" to collect and make sense of military intelligence at speeds 100 times faster than today — a power that could also enable military analysts to speedily dig up information from texts in any language. The effort could even lead to truly autonomous war robots capable of making their own decisions on tomorrow's battlefields.
"These visions are not empty fantasy … they are being made possible through big bets the Department is placing on Big Data," said Zachary Lemnios, U.S. assistant secretary of defense for research and engineering. He introduced the effort as part of the Obama administration's new Big Data initiative announced today (March 29).
The Department of Defense also announced plans for a series of related open prize competitions over the next several months — a tactic that has worked well for the Defense Advanced Research Projects Agency (DARPA) and the U.S. Air Force Research Laboratory. About $60 million of the available $250 million will go toward new research projects.
DARPA plans to invest an additional $25 million each year in the Big Data effort. It wants to develop software tools to analyze huge volumes of data as messy as text documents or online messages.
The Big Data challenge can be compared to trying to find an object floating in the Atlantic Ocean's nearly 100 billion, billion gallons of water (roughly 350 million cubic kilometers), said Kaigham Gabriel, acting director for DARPA.
"If each gallon of water represented a byte or character, the Atlantic Ocean would be able to store, just barely, all the data generated by the world in 2010," Gabriel said. "Looking for a specific message or page in a document would be the equivalent of searching the Atlantic Ocean for a single 55-gallon drum barrel."