Gene that differs between humans and Neanderthals could shed light on the species' disappearance, mouse study suggests

A protein that helps synthesize DNA is different in modern humans than it is in Neanderthals and Denisovans — our closest extinct relatives — and new experiments in mice genetically modified to express the modern human version hint that this may have made us behave differently.

That discovery, in turn, could shed light on why Neanderthals and Denisovans vanished, researchers propose in a new study.

But the significance of the findings for humans is still unclear.

"It's too early to translate these findings directly to humans, as the neural circuits of mice are vastly different," study lead author, Xiangchun Ju, a postdoctoral researcher at the Okinawa Institute of Science and Technology in Japan, said in a statement. However, this work hints that the variant seen in modern humans "might have given us some evolutionary advantage in particular tasks relative to ancestral humans," such as competing for scarce resources.

Key protein

Previous research found that modern humans diverged from their closest evolutionary relatives, Neanderthals and Denisovans, about 600,000 years ago. It's not clear why modern humans survived while our closest relatives died off.

To search for potential genetic clues to solve this mystery, the researchers analyzed the enzyme ADSL (adenylosuccinate lyase). This protein helps synthesize purine, one of the fundamental building blocks of DNA and other vital molecules.

Related: A braided stream, not a family tree: How new evidence upends our understanding of how humans evolved

"There are a small number of enzymes that were affected by evolutionary changes in the ancestors of modern humans. ADSL is one of them," study co-author Svante Pääbo, Nobel laureate, leader of the human evolutionary genomics unit at the Okinawa Institute of Science and Technology in Japan, and director of the Max Planck Institute for Evolutionary Anthropology in Leipzig, Germany, said in a statement.

ADSL is made up of a chain of 484 amino acids. The version of this enzyme found in virtually all modern humans differs from that seen in both Neanderthals and Denisovans by just one amino acid — the 429th amino acid in ADSL is valine in modern humans but alanine in our extinct relatives.

The scientists noted the ADSL mutation is seen in modern humans and not our closest extinct relatives, and so likely appeared after we separated from the lineage that led to Neanderthals and Denisovans. This led the researchers to investigate the possible behavioral effects of this mutation.

Previous research on lab-grown cells found that the ADSL variant seen in modern humans resulted in a more unstable version of the enzyme that broke down more quickly compared to the one in Neanderthals and Denisovans.

Behavior changes

The new study, published Aug. 4 in the journal PNAS, similarly found that, in mice, the modern variant leads to higher levels of the chemicals that ADSL normally acts on to synthesize purine in several organs, especially the brain. This finding supported the idea that the modern human version of ADSL is less active than the variant seen in Neanderthals and Denisovans.

In experiments where mice learned they could get a drink of water following specific lights or sounds, female mice genetically modified to possess a version of ADSL similar to the kind seen in modern humans were better at getting water than their littermates without this variant were. This might suggest the human-like variant made female mice better at learning to connect the dots between the water and the lights or sounds, or more motivated to seek out the water in some way.

The changes in behavior and ADSL levels seen in female mice with the modern-human variant of the enzyme was not seen in male mice. "It's unclear why only female mice seemed to gain a competitive advantage," study co-author Izumi Fukunaga, a researcher at the Okinawa Institute of Science and Technology, said in a statement. "Behavior is complex."

Statistical tests analyzing Neanderthal; Denisovan; and modern African, European and East Asian DNA found that mutations in the ADSL gene appeared in modern human genomes at higher rates than random variations over time would suggest, making it likely that these mutations provided some evolutionary advantage.

Perhaps running counter to the new findings, prior work found that genetic disorders leading to ADSL deficiency in modern humans can lead to intellectual disability, speech and language impairment, and other problems. This suggests that during evolution, modern humans had to balance the potential benefits of reducing ADSL activity with the problems that could occur from ADSL deficiency, study co-author Shin-Yu Lee, also of the Okinawa Institute of Science and Technology, said in a statement.

Implications unclear

Not everyone thinks the study has direct implications for why modern humans thrived or for why Neanderthals or Denisovans disappeared.

These results in mice "don't say too much about human evolution at this stage," Mark Collard, a paleoanthropologist at Simon Fraser University in Burnaby, British Columbia who did not take part in this research, told Live Science.

However, the strategy of using mice to study the behavioral effects of genetic differences between modern humans and our closest extinct relatives "seems very promising as a way of investigating the evolution of our brain and behavior," Collard said. "I expect we'll see a cascade of studies like this one in the next few years."

Future research can investigate the specific mechanisms by which changes in ADSL activity influence behavior. Scientists can also explore how changes in ADSL activity are associated with other behaviors and how multiple genetic changes might work in concert, the study authors wrote.