It's quick and easy to access Live Science Plus, simply enter your email below. We'll send you a confirmation and sign you up for our daily newsletter, keeping you up to date with the latest science news.
Artificial intelligence (AI) can now transform photos of people into short, highly realistic animations, much like the moving pictures in the newspapers and posters of Harry Potter's magical world.
In these AI-animated clips, faces that were once frozen in time blink, turn their heads and even smile, their movements wavering between astonishingly lifelike and deeply unsettling (and yes, downright creepy).
Genealogy website MyHeritage introduced the animation engine on Feb. 25. Developed by technology company D-ID and known as Deep Nostalgia, it enables users to animate photos via the MyHeritage website, representatives said in a blog post. D-ID designed custom algorithms that recreate the naturalistic movement of human faces digitally, applying those subtle movements to photographs and modifying facial expressions that move as human faces normally do, according to the D-ID website.
When AIs create original video footage, often referred to as "deepfake," they do so using a method called generative adversarial networks, or GANs. This technique pits two AIs against each other; one produces content and the other evaluates how well that content emulates the real thing. Over time, the algorithms drive each other to become better, until the original AI content is very hard to identify as fake.
Such videos can be used in troubling ways: from showing political figures making fake speeches, to adding the faces of famous actresses to the bodies of actresses in pornographic movies, Vice reported in 2018.
However, video clips generated by Deep Nostalgia are just a few seconds long, and the training footage for the AI did not include speech, so as to prevent the creation of deepfakes, according to MyHeritage.
Sign up for the Live Science daily newsletter now
Get the world’s most fascinating discoveries delivered straight to your inbox.
Programmers trained Deep Nostalgia's GAN with sets of "blueprint videos," each representing different combinations of movements for eyes, mouths, eyebrows, cheeks and heads; the AI then learned how these could be applied to photos of different people to achieve an illusion of realistic motion. It assigns different suites of facial gestures to different photos, depending on their subjects' postures and orientations, according to MyHeritage.
Results in the animations can vary, depending on the quality of the original image and how the person in the photo posed. The illusion tends to be most effective when the subject is facing the camera, and the end result can be less convincing when the algorithm has to create digital information to represent something that was missing in the original picture, "such as teeth or ears," MyHeritage representatives said.
Deep Nostalgia may also struggle to realistically incorporate accessories such as hats or glasses, which can obscure parts of the face and head. In those cases, "sometimes the simulated movement works well — and other times it does not," according to the site.
While MyHeritage encouraged users to test Deep Nostalgia with family photos, Twitter users shared examples of some famous faces from the past, such as the poet Emily Dickinson; chemist and X-ray crystallographer Rosalind Franklin; and abolitionist Frederick Douglass.
Frederick Douglass, the mighty abolitionist, was the single most photographed person in the United States during the nineteenth century. Here's how he might've looked in motion. Brace yourself and press play. pic.twitter.com/HOxDK7jGyhFebruary 28, 2021
Mindy Weisberger is a science journalist and author of "Rise of the Zombie Bugs: The Surprising Science of Parasitic Mind-Control" (Hopkins Press). She formerly edited for Scholastic and was a channel editor and senior writer for Live Science. She has reported on general science, covering climate change, paleontology, biology and space. Mindy studied film at Columbia University; prior to LS, she produced, wrote and directed media for the American Museum of Natural History in NYC. Her videos about dinosaurs, astrophysics, biodiversity and evolution appear in museums and science centers worldwide, earning awards such as the CINE Golden Eagle and the Communicator Award of Excellence. Her writing has also appeared in Scientific American, The Washington Post, How It Works Magazine and CNN.
