Spider-Man 3 - Technical Notes

Millions of people all over the world have thrilled to the excitement of Spider-Man swinging between the iconic skyscrapers of New York City. Director Sam Raimi insisted on honoring a remarkable legacy that includes an internationally beloved property and Sony Pictures Imageworks’ Academy Award® for Best Achievement in Visual Effects, even as he envisioned jaw-dropping new adventures for Spider-Man™ 3.

Raimi charged Visual Effects Supervisor Scott Stokdyk and his team at Sony Pictures Imageworks with making logically impossible new characters, spectacular actions and vengeful villains completely believable and alive on thousands of theater screens. What became the biggest opening movie of all time features about 70 minutes of effects shots, the spectacular work of between 200 and 250 people and more than 900 effects shots.

Imageworks started with the actors’ individual performances, which gave the artists an invaluable foundation for their work even as appearances were transformed to match Raimi’s vision. “Sam really loves getting the real actor on film and that’s part of my visual effects philosophy too, to try to get as much on film as you can,” Stokdyk says.

Tobey Maguire is back as Spider-Man, leaping from vertiginous heights with deceptive ease, alongside such richly emotive humans as Kirsten Dunst as Mary Jane Watson, Bryce Dallas Howard as Gwen Stacy, James Cromwell as Detective Stacy, Rosemary Harris as Aunt May Parker, and J.K. Simmons as Daily Bugle editor J. Jonah Jameson.

Making a surprise appearance is the new Goblin, this time the alter ego of Harry Osborn (James Franco), soaring through the sky and wreaking havoc with an arsenal of deadly inventions. Two remarkable villains are completely new to the cinematic experience, however.

One is Sandman, a destructive and transmogrifying creature formed entirely from individual grains of sand, who in human form is Flint Marko, played by Thomas Haden Church. The other is Venom, an extraterrestrial symbiote which sometimes appears as an insidious goo but which can also take over the body and mind of humans like Eddie Brock, played by Topher Grace.

Raimi, Stokdyk and the Imageworks team have set the bar even higher for themselves and the audience with this third film. “This is the first time you’ll see Peter Parker fighting using his superhero powers but not in his Spider-Man suit,” says Stokdyk, who has spent a total of seven years working on all three films. “In fact, there are a few spectacular fights where the adversaries’ faces can be seen throughout the action. That was an incredible challenge, since it required an exceptional blending of CG and live footage in the same shot and meant we had to work even more closely with the stunt people to help them match our previsualization using motion-controlled wire work. But it also meant we could show more of the actors’ performances, capturing more of the reality.”

Wholly believable digital doubles were achieved using multiple approaches to keeping faces visible to the audience. Some were completely computer generated, a capability Imageworks perfected on Superman Returns to the point where they could be used interchangeably with the filmed actor. This preserved as much of the actor as possible using a system based on Paul Debevec’s LightStage technology, which creates a 3D representation of the individual’s facial geometry and skin textures. Others entirely retained the actor’s face filmed against a blue screen.

Visual effects plate supervisor Sheena Duggal supervised the shooting of the face-replacement elements. “Sheena worked with Spencer Cook and Sam Raimi to determine the eyeline for the CG characters at any moment in the shot, and translated that to marks for the actors to follow during the motion control shoot,” Stokdyk says.

The mix of skillful face replacement with CG is even more technically demanding than the previous films, but worth the effort. Says Stokdyk, “One of our team, John Schmidt, set up a way to extract moves from Tobey or a stunt person. We’d shoot Tobey sitting in a chair acting while his body wasn’t moving to the motion and it actually gave us surprisingly good results.”

Another huge undertaking for Imageworks’ Spider-Man™ 3 team was creating Sandman and Venom, characters that previously could only exist in the imagination. Stokdyk says, “Sandman alone took us almost two years and required computer programmers, animators, visual effects designers and other artists here at Imageworks. He had a whole team devoted just to him, as did Venom, which took nearly as long to create.”

Stokdyk describes the process behind Sandman’s creation. “We knew from the start that sand was a big effects animation challenge tied in with character animation, and it’s the combination of those two things that makes it a level of difficulty exponentially harder,” he says. Stokdyk did extensive sand reference shoots with Raimi and director of photography Bill Pope, capturing every kind of sand motion the script specified, plus every type of motion they could anticipate needing. “Sand has some unique qualities in that it behaves sometimes like a solid and sometimes like a liquid. When you’re doing CG water, you can approximate the volume with different cheats and techniques. But Sandman was a key challenge in terms of how to control particle systems, and being able to get collision detection of those particles, to work into the emotional and character performance. Millions of grains of sand were individually manipulated to convey Thomas Haden Church’s acting.”

To tackle those gritty issues, Stokdyk says, “We brought in a really good team, led by Doug Bloom, Jonathan Cohen and Chris Allen, who took everything they knew about particles and flow and started writing a course set of software. Someone calculated it worked out to ten man-years of software that they’ve written over the last two years. The thing Sam actually wanted was for our sand to be really controllable, but not magical. We have our sand flow in a very physically realistic fashion, like when it’s flowing up to form Sandman’s arm as it’s articulating, but then we had to be able to transition to individual particles falling with gravity.”

“The second part was combining the effects animation of sand flying around with character-driven animation,” says Stokdyk. “What makes that goal difficult is that it takes the normal flow of work – doing animation, to doing effects, to lighting and compositing it – and jumbles it around, because you have to drive a performance, then look at what that does to the effects, and then go back and change the character, in a way we haven’t really had since Hollow Man. It’s the layers that are tied to each other that make it so much more complex.”

Animation director Spencer Cook agrees, saying, “With Sandman, we’re going to a lot of effort to include little bits or chunks of sand falling off of him as he’s developing, and we’re putting that into the character animation as a guide to help the effects animators. That kind of interface is a new challenge, incorporating the effects animation into the character animation, because the effects are such an integral part of who this character is.”

Another vital element for the animators was Cook’s collaboration with actor Thomas Haden Church. Cook says, “I had conversations with Thomas, and we videotaped him doing certain actions to get a really good sense of how he moves and how he’d do certain things. We knew from the start that this character was not a bad guy. There’s a line in the script where Sandman says, ‘I’m not a bad man, I’ve just had bad luck,’ and every step of the way, we tried to impart that sadness to him, that the accident that turns him into the Sandman is a tragedy for him.” Stokdyk says, “We always tried to pay respect to Thomas’ character and his body language. There are many shots where we lined up our cameras and CG body and matched what he did on set.”

As its name implies, the many forms of Venom from its crawling gooey state to the humanoid form share one thing: they are all driven by the creature’s evil nature. Conveying that malevolence with a CG character was a creative highlight for the Imageworks team. “When we first approached the symbiote alien, we had a lot of great input from different departments [on the film] and one person on [Costume Designer] James Acheson’s team came up with this great claw-like design that kind of sparked everybody’s imagination. From the start, the Imageworks team started doing all these animation tests of just dripping goo. It finally coalesced when there was an animation of goo crawling on Spidey’s arm based on a really beautiful, intricate artwork by one of our concept artists, EJ Krisor. It was menacing and predatory, not passive, and we all knew right away that it captured the essence of Venom.”

Cook says of Venom’s goo form, “It really is a malevolent kind of creature, and its goal is to find a host, so we tried to find a simple way to convey that with details like tentacles and fingers reaching. What we tried to impart in the animation is the urgency that it has to find and completely cover and control its host.” For inspiration, Cook says, “We’d look at footage of predators to make Venom more vicious, like documentaries of lions attacking, for example. We’d try to incorporate that into the animation so that there are elements of it that ring true.”

When the aggressive symbiote alien finds a host body, it taps into that person’s darkest qualities, even Spider-Man’s. Says Stokdyk, “It actually makes Spidey a little more blood-thirsty, more of a predator. It feeds on the revenge that he has in his heart.” That change affected the character animation in subtle ways. Cook says, “Black-suited Spidey is much more aggressive, more reckless and more brutal. He can react faster so he’ll do things in sort of a rougher way, whereas classic Spidey is more fluid and graceful.”

Nothing in nature behaves like Venom, so video reference was out of the question. Inspired by the concept art, Imageworks’ Ryan Laney created a special rigging system based on tubes formed by a constantly changing curve. Animators could specify each curve’s velocity, length and thickness, which they then placed on a 3D model in their thousands. Membranes, textures and lighting completed the creation.

All of these characters are part of the film’s climactic battle scene, a spectacle that pushed the Imageworks team and of all their artistic skills to new heights. Stokdyk describes it as “unbelievably complex. It had multiple characters being animated, which is exciting, but again we incorporated more real pieces to make something that you can’t get live and you can’t get CG. We combined those two worlds to get something that’s better than each one of them separately.” Cook got to evoke his favorite monsters in creating the giant Sandman who looms over New York in the film’s final battle. “The giant creature at the end was a lot of fun, because I’m a big Ray Harryhausen fan, and I love the old Jack Kirby comics with the Marvel monsters from Astounding Stories and all that. It’s fun to come up with ways for this giant to move, and make him look like he’s heavy, stomping around and breaking stuff.”

Realistic environments created for Spider-Man™ 3 also were taken to a new level, according to Stokdyk. He says, “In Spider-Man™ 1, we had to learn how to make buildings, and in 2 we learned how to make them better. What we did in this case was something I’ve wanted to do since the first film, which was actually recreate an actual location in New York that people are familiar with.” Production designer Neil Spisak chose a real New York plaza for the film, and Stokdyk’s team gathered all the survey data they needed to recreate it in CG. “We were able to draw from our other building experience to make the best possible replication of this real place that we could,” he says.

Another key locale is a dirty, narrow alley where Peter Parker is being chased at high speed by Harry. In shots nearly indistinguishable from reality, Stokdyk’s team used a mix of reference footage shot in Los Angeles with created CG buildings, focusing on hyper-realistic details such as fire escapes, broken windows, trash cans and pallets on the ground. For some of the film’s most popular scenes, Stokdyk and crew would suspend a cable camera rig known as a Spydercam between two buildings and let it swing, providing great reference footage for Spidey’s swooping glides.

Harry swoops in for this battle on his Sky Stick, a rocket-fueled personal glider that he also uses as a martial arts-style weapon. Franco proved to be very adept at wire work, which helped preserve his performance on film against the blue screen. “There were times where we thought we would use a stunt person, because it was a wide shot, for example, and the character would be fairly small in frame; but the body language and the motion were so much better with James Franco, we ended up using him, even in those wide shots. He was fantastic at it,” Stokdyk says.

Audiences probably won’t even realize it, but another remarkable achievement can be seen for the first time in that same alley sequence: for the first time ever,  a digital double for Peter Parker – no costume or mask – is  performing Spider-Man aerial acrobatics.

Even so, Stokdyk says the biggest challenge in Spider-Man™ 3 has been that “the scope of this movie required that visual effects-wise, we were always ready to turn on a dime and respond to any changes. It was a more complicated movie on many levels than the last two, because there are more characters and more plotlines. Sam did a lot of work making all the pieces come together, and almost immediately we had to respond with the execution of what it took to make those pieces. We had to have an amazing team ready to respond and always on their toes.”