Computer IDs Culprits with Tattoo Recognition

A new program, still in development, identifies people in photos by their tattoos, scars and other skin markings. The program works even if the person's face isn't clear in the photo. CREDIT: Anthony Ricci | Shutterstock.com View full size image

It's a plot point in countless detective shows and movies. An investigator finds fingerprints at a crime scene, scans the prints into a computer, and automatically finds matches in a huge FBI database. Now, researchers are working on technology making it just as simple for law enforcement to scan in and find not only matches for suspects' photos, but also their body ink.

The face might be the obvious place to start for Facebook, homeland security and other groups interested in automatically identifying people in photos. Indeed, face recognition is one of the biggest areas of research in identification and security. Adding in tattoos and other marks, however, gives law enforcement an edge in using evidence where the suspect's face isn't clear.

"Let's talk about standard police-type action," said Terrance Boult, a computer science professor at the University of Colorado at Colorado Springs and a co-founder of a security startup, Securics Inc. In many police investigations, officers have to contend with grainy, low-quality photos that a bystander might have taken on his phone, or that a store camera captured, he said. "Those photos are often so bad that face recognition wouldn't come even close" to finding a match in a photo database, such as the FBI's, he said. 

To help with these difficult matches, Boult and his colleagues wrote a computer program that examines the tattoos, scars, moles or other skin markings in a new photo, then finds likely matches in a photo database. The program is able to find similar tattoos that are not exactly the same, but which might help identify gang members who get coordinating ink. And it is able to make matches based on eyewitness descriptions that a cop might type into the program. [Digital Interface Tattoo Melds Skin and Circuitry]

Boult gave an example of the kind of description his program would be able to understand: "Well, I saw this guy and he has this skull tattoo on his neck on the left side and a flower tattoo on the right side."

"The idea of detecting markings on the skin and using them as a way to recognize people has emerged as an interesting new research topic in recent years," Kevin Bowyer, a computer scientist at the University of Notre Dame, told InnovationNewsDaily in an email. Bowyer reviewed research papers for an electrical engineers' conference where Boult will present his work later this month and was not involved with Boult's team. "This paper describes work on something of an exotic topic, and introduces improvements that are meant to move past proof-of-concept toward more practical tools," Bowyer said.

A computer program that learns

Boult's team isn't the first to develop automatic recognition of tattoos, scars and marks. Instead, the Colorado researchers built on previous work, making a system that can handle photos taken "in the wild," as Boult calls it — that is, photos snapped by chance, by friends or a security camera. Such photos may not be centered, cropped and evenly lit, like the photos used by previous researchers to test their programs. They better reflect the imperfect evidence investigators may gather about a crime, Boult said. [10 Technologies Poised to Transform our World]

To build in that flexibility, Boult and his colleagues wrote an artificially intelligent program that learns from example. Then they gathered random photos from the Internet to teach their program to find and match tattoos by appearance. "They were real examples from the wild," said Walter Scheirer, one of Boult's colleagues at the University of Colorado and at Securics.

The so-called machine-learning algorithm also allowed the computer program to learn how humans would describe a tattoo using words. The researchers had volunteers choose descriptors for photos of people with tattoos and marks on their skin and then gave those labeled photos to the algorithm as examples. 

"We're trying to deal with witness descriptions because we get those all the time," Boult said. "You want to be able to say, 'Scar, left cheek' and find something."

Not yet ready for the FBI

Boult, Scheirer and their colleague Brian Heflin have a demo version of their program ready to go. "You could use it now," Scheirer said. 

However, the program still needs several more features before it would be used by law enforcement, the researchers say. The team is working on getting the program to recognize more than 100 tattoo descriptors (individual descriptors include "skull," "flower," "flame" and "koi fish"). In a recent paper, they said their program recognizes 15 such words.

They're also trying to ensure the program is able to handle the enormous databases that federal agencies, such as the FBI and the Department of Defense, maintain, in part because their funding comes from a U.S. Army grant for small businesses to perform early-stage research.

"We have some objectives. There's some places we want to make sure it can be used," Scheirer said. "So we want to be sure we can scale up to handle all that data."

Scheirer, Boult and Heflin will present their work Sept. 25 at a conference in Washington, D.C., hosted by the Institute of Electrical and Electronics Engineers. 

This story is the first in a series about exotic biometrics — weird ways that researchers are working on identifying people by their biological features. Face, iris and fingerprint matching are the biggest stars in biometrics right now, but it's worth looking at more unusual ideas, too, Kevin Bowyer, a University of Notre Dame computer scientist, told InnovationNewsDaily. Bowyer served as a reviewer for a biometrics conference coming up Sept. 24. He helped choose some of the research we'll examine in this series, which will not feature his own work.

"It is important to keep track of the new/unusual/not-yet-much-studied things, because this is where the next big things come from," Bowyer said. "Of course, most exotic things never become big.  But history says that some will." 

His own area of expertise, iris scanning, was considered exotic 20 years ago, he added.

This story was provided by InnovationNewsDaily, a sister site to LiveScience. You can follow InnovationNewsDaily staff writer Francie Diep on Twitter @franciediep. Follow InnovationNewsDaily on Twitter @News_Innovation, or on Facebook.