Inside Movie Animation: Simulating 128 Billion Elements

Simulations based on physics helps make this Disney character incredibly rat-like. CREDIT: Disney/Pixar

Ever wonder how animated films such as The Incredibles get hair, clothing, water, plants, and other details to look so realistic? Or how, like the lion in The Chronicles of Narnia, animated characters are worked into live-action films? If not, the animators would be pleased, since they don't want special effects to distract from the story. Behind the scenes, though, is a sophisticated combination of artistry, computation, and physics.

Traditionally, animation was hand drawn by artists who needed"some of the same magical eye that the Renaissance painters had, to give the impression that it's realistically illuminated," says Paul Debevec, a computer graphics researcher at the University of Southern California. Over the past decade or so, the hand-painted animation has faded as physically-based simulations have increasingly been used to achieve more realistic lighting and motion. Despite this movement toward reality in animated films, the physics of the real world remains a slave to expediency and art: Simplifications and shortcuts make the simulations faster and cheaper, and what the director wants trumps physical accuracy.

In one dramatic scene in the movie 300, which came out early in 2007, several ships collide violently -- their hulls splinter, masts break, sails tear, and the ships sink. Stephan Trojansky, who worked on 300 as visual effects supervisor for the German-based company ScanlineVFX, said just creating the ocean in that scene involved simulating 128 billion elements. “We probably created the highest fluid simulation detail ever used in visual effects,” he said.

"For the fracturing and splintering of the ships," he added, "we developed splintering technology. Wood doesn't break like a stone tower. It bends. To get realistic behavior, you have to take into account how the ship is nailed together. The physics involved is mainly equations that define where the material will break."

Animations of both fluids and solids—and of facial expressions and clothing, among other things—use various computational methods and a host of equations. But there is a tradeoff in the push for more realistic animations – moving closer to reality requires more and more computer power, and becomes increasingly expensive. There are three commonly used methods of computer animation -- break the object being simulated into discrete elements, use sample points from the object, or create fixed cells in space.

Mark Sagar, of WETA Digital, a visual effects company in Wellington, New Zealand, specializes in simulating faces. One technique is motion capture, in which markers are placed on an actor's face, their positions are noted for different expressions, and the positions are then mapped onto an animated character. "For King Kong we mapped the actor's expressions onto a gorilla," said Sagar.

Simulating the face involves interpreting movement in terms of muscle, Sagar said. "We approximate the detailed mechanical properties of live tissue and its layers and layers. You have motion data and start working out what the driving forces are.” Modeling realistic stretching of the skin requires a lot of finite elements—each a small patch of tissue,” he said. "You compute and solve for forces at each point and then sum until you get a balanced equation. It's not sophisticated from an engineering standpoint but produces high-quality results."

Realistic motion is often too complicated for animators to do by hand, said Michael Kass, a researcher at Pixar Animation Studios. "The results can be awful and very expensive." In the original 1995 Toy Story, he said, "if you see a wrinkle in clothing, it's because an animator decided to put in a wrinkle at that point in time. After that we [at Pixar] decided to do a short film to try out a physically based clothing simulation."

The movement of clothing is computed as a solution to partial differential equations, he said. "You start with individual threads. What are their basic properties? Then you consider the bulk properties when [they're] woven. The main physical effects are stretching, shearing, and bending. To a certain degree, you can take real cloth and get actual measurements."

While animating clothing still presents problems, he said, “it's now part of a standard bag of tricks. Our simulations have become accurate enough that we can design garments with commercially available pattern-making software and then have them move largely as a tailor would expect in our virtual simulations."

Animating hair "is in many ways easier than clothing because it's like individual threads,” Kass said. “The difference is that clothing doesn't move like clothing unless the threads interact. In a real head of hair, the threads do interact, but you can get convincing motion without taking that into account."

Illumination is another area in which physics plays a key role in animation. For a long time, says Cornell University's Steve Marschner, "rendering skin was hard. It would look waxy or too smooth." The fix, he says, was to take into account that skin is translucent, which he and colleagues "figured out from looking at a different problem—rendering marble."

As with simulations of fluids, cloth, rigid bodies, and so on, incorporating translucency to model skin involves old physics. "In some cases we have to create the models from the ground up. But sometimes we find somebody in another branch of physics who has solved a similar problem and we can leverage what they've done." For skin translucency, "we were able to adapt a solution from medical physics, from a calculation of radiation distributions inside the skin that was used for laser therapy in skin diseases."

"One of the coolest things you see in a movie is when there is some sort of otherworldly beast or digital character that is sitting in the scene, roaming around, and it looks like it was really there," says Debevec. "The only way you can do that is by understanding the physics of light transport, respecting how light works in the real world, and then using computers to try to make up the difference from what was really shot."

For example, he says, in Narnia "they filmed a lot of it with the children dressed up in their knight costumes and left an empty space for the lion." Then, to get the digital lion just right, "Rhythm and Hues Studios used radiometrically calibrated cameras to measure the color and intensity of illumination from every direction in the scene." The measurements, he adds, "are fed into algorithms that were originally developed in the physics community and have been adapted by the computer graphics community as a realistic way to simulate the way light bounces around in the scene.”

Similar methods are used for creating digital doubles—virtual stunt characters that fill in for live actors. For that, Debevec said, "film studios sometimes bring actors here to our institute, where we've built devices to measure how a person or object, or whatever you stick in [the device], reflects light coming from every possible direction.” The resulting data set, he says, can be used to simulate a virtual version of the person. "There are about 40 shots of a digital Alfred Molina playing Dr. Otto Octavius in Spider-Man 2. It looks like him, but it's an animated character. The reflection from the skin looks realistic, with its texture, translucency, and shine, since it's all based on measurements of the real actor."

"We rarely simulate more than two indirect bounces of illumination, whereas in reality light just keeps bouncing around," Debevec continued. "With no bounces, things look way too spartan and the shadows are too sharp. One bounce fills in perhaps three-quarters of the missing light, and with two bounces you're usually past 95%. That's good enough." Another shortcut, he adds, is to focus just on the light rays that will end up at the eye. "We try to figure out the cheats you can make that give you images that look right."

"There is a long tradition of cheating as much as possible," said Marschner, "because setting up an exact simulation is either not possible or too expensive." “We use physics to get realism,” Trojansky said. "But I am a physics cheater. I use it as a base, but I am interested in the visual effect."

• Next Step: Wild and Disruptive Technologies
• Video: Hologram Fashion Show
• Planet Earth 2007: Top 10 Science Revelations

Editor's Note: This is an edited version of an article that appeared in the November 2007 issue of Physics Today magazine. It is republished here with permission.