Why Human-Neanderthal Sex Is Tricky to Prove

A female mantled howler monkey in Tabasco, Mexico. CREDIT: Milagros González View full size image

A bundle of recent genetic studies have suggested modern humans had sex with Neanderthals thousands of years ago when the two populations roamed the planet alongside each other. However, the bones left behind by the two species don't bear any obvious traces of interbreeding, and a new study of monkeys in Mexico shows why we shouldn't expect them to.

Researchers examined blood samples, hair samples and measurements collected from mantled howler monkeys and black howler monkeys that were live-captured and released in Mexico and Guatemala between 1998 and 2008. The two monkey species splintered off from a common ancestor about 3 million years ago; today they live in mostly separate habitats, except for a "hybrid zone" in the state of Tabasco in southeastern Mexico, where they coexist and interbreed.

Through an analysis of genetic markers, from both mitochondrial DNA (the DNA in the cells' energy-making structures that gets passed down by mothers) and nuclear DNA, the researchers identified 128 hybrid individuals that were likely the product of several generations of interbreeding. Even so, these hybrids shared most of their genome with either one of the two species and were physically indistinguishable from the pure individuals of that species, the team found.

"The implications of these results are that physical features are not always reliable for identifying individuals of hybrid ancestry," Liliana Cortés-Ortiz, an evolutionary biologist and primatologist at the University of Michigan, said in a statement. "Therefore, it is possible that hybridization has been underestimated in the human fossil record."

The work on howler monkeys was part of the doctoral dissertation of Mary Kelaita, now a postdoctoral fellow at the University of Texas at San Antonio. Kelaita added that the study "suggests that the lack of strong evidence for hybridization in the fossil record does not negate the role it could have played in shaping early human lineage diversity."

When scientists finally finished sequencing the Neanderthal genome in 2010, they revealed that between 1 percent and 4 percent of some modern humans' DNA came from the stocky hominids. This suggested humans had sex with Neanderthals, picking up some genes, and possibly even an immunity boost, from Neanderthals before the population disappeared about 30,000 years ago. But not all scientists are convinced the genetic evidence alone proves ancient interbreeding and a study last year found that even if humans and Neanderthals did have sex, those encounters would have rarely produced offspring.

The scientists of the new study say more work is needed to learn about interbreeding and the factors governing the expression of physical characteristics in hybrid individuals.

The research was detailed online Friday (Dec. 7) in the American Journal of Physical Anthropology.

Follow LiveScience on Twitter @livescience. We're also on Facebook & Google+.