Visual effects played an even more integral part of
Spider-Man 2 than in the original
Spider-Man, comprising over 40 minutes of the completed movie. In addition to taking more screen-time, the effects were designed to bring the Doc Ock character to life, and to remove more restrictions on shot composition and content.
As Visual Effects Supervisor for Sony Pictures Imageworks, my job was to make sure that all of the over 800 visual effects shots and the accompanying R&D were done at the highest quality and on time. I supervised a team of four CG supervisors and over 200 artists at Imageworks. I also gave feedback to the 387 shared composites and wire removals that were contracted to these effects houses: Zoic, Radium, Pacific Title, Pixel Magic, Barbed Wire, Entity, Title House, Ring of Fire, and Cantina.
CG CHARACTERS
Doc Ock proved to be the most difficult and interesting CG character we created because of his animated tentacles, multi-layered costume, and exposed face. In some shots he wore sunglasses. In others, he did not wear sunglasses but the rest of his face was always visible. Therefore, we needed to create accurate expressions and realistic human skin lighting.
|
Secrets of Spider-Man 2: The Nominees Tell All
|
For Doc Ock's CG skin, rather than using popular shading techniques and hand-painted texture maps, Imageworks created a production-ready implementation of the Light Stage system developed by Paul Debevec. With the Light Stage, the reflectance field of a human face is captured photographically. The resulting images can be used to create an exact replica of a subject's face from various viewpoints and in different lighting conditions. To create Doc Ock's face, Alfred Molina was seated in a chair and surrounded with four film cameras running at 60 frames per second. Above his head was an armature that rotated around the chair. On the armature, strobe lights fired at 60 frames per second. We obtained 480 images from each camera of his head in one position, but with unique lighting conditions in each frame. We also had to create a CG unmasked version of
Spider-Man that revealed Tobey Maguire's face, using the same techniques.
After the Light Stage images were acquired, first the light and camera-dependent information (i.e., specular) was separated from the diffuse component of the imagery. Then the images were accessed via a complex lookup function that took into account the additive effects of both CG lights and environmental lighting. On set, HDR fisheye photos were taken for every lighting setup to produce the 360-degree environment map used by lighting artists. Once a CG character's head was surrounded with those global environments, a software algorithm could calculate which of the 480 images from each camera to blend together to create the character's face. On top of this, the lighter could place traditional CG lights to brighten areas of our CG actor. Additional steps in the lighting pipeline were set up to add back in the specular function and account for lighting changes when the face geometry deformed.
We wanted our artists to spend time lighting the characters as if they would in the real world, rather than changing shader parameters to balance skin attributes. This technique took the guesswork out of the initial creation of the CG faces, compared to the technique of painting textures and approximating skin light response (i.e., subsurface scattering, etc.). It is also important to note that this system acquires an effective BRDF (bi-directional reflectance distribution function) at each point on the face, rather than just sampling generic skin and applying that shading model to the rest of the face.
While the core techniques were inspired by Debevec's work (and we used the Institute of Creative Technology's equipment for the acquisition), the real challenge was to make the whole system production-worthy. This skin-lighting technique had to fit in with the hair, cloth, and tentacle lighting pipeline. Also, in the cases where the data acquired from the Light Stage was corrupt, the team came up with an innovative "virtual Light Stage" technique to recreate needed imagery. This involved recreating the Light Stage strobe lighting rig in the computer and using this rig as a multiplier for single-frame paint fixes.
To create photorealistic movement for the actors' faces, Imageworks developed a system based on motion capture. Both Alfred Molina and Tobey Mcguire were scanned and their digital information recorded at the House of Moves using Vicon motion capture cameras. Each actor wore 150 1.5 mm retro-reflective markers in his face. They pronounced selected phonemes, acted out ranges of expressions, and did over 80 muscle actions. Once the captured data was stabilized, Imageworks used it to create the relationships of individual muscles to each other and to the typography of the face through the application of proprietary scripts and tools written for Alias' Maya. The muscles, in turn, deformed the skin. This was a flexible system that let us alter expressions and timing, but we could still use that actor's sampled performance.
To create the digital tentacles, Imageworks started with cyberscans of the Edge FX puppets taken at Gentle Giant studios. We then modeled the appendages by stringing individual vertebrae on cables and adding various claws and devices at the end of these cables. The tentacles were modeled using subdivision surfaces and setup to render, using both image-based rendering and traditional CG light placement.
For Doc Ock's costume, a technique called "object cloth" was used, which simplifies the methods used to create digital costumes. It is different from panel cloth previously used at Imageworks, since a cyberscan model can be used as a cloth rest position. Alias developed new cloth software in Maya for Imageworks during
Spider-Man 2 to implement "object cloth." The simplification enabled the computationally intense task of simulating all the layers of Doc Ock's clothes simultaneously, even when the villain was on top of the fast-moving train. The layers of clothing could not be simulated separately because they interacted with each other.
VIRTUAL ENVIRONMENTS
The goal for this film was to make the virtual buildings more detailed so that the characters could be much closer to the buildings and interact with them. We had expanded CG environment requirements, including: distant cityscapes, high-speed travel through 30 city blocks, close views of streets and their accompanying elements, and situations where the camera was within a foot of a CG building (where individual bricks were modeled). Unique issues were addressed for both daytime and nighttime environment lighting.
The biggest difference between the buildings constructed for the original
Spider-Man and those for
Spider-Man 2 was in the level of geometric detail.
Spider-Man 2 had scenes where the actors had to interact with and damage buildings, which required more geometry and precluded us from using many of the textural tricks used on the first
Spider-Man. Ultimately, the new
Spider-Man 2 buildings looked so much better than the previous buildings that, while the original
Spider-Man buildings were reused, they usually appeared in the background.
Imageworks modelers built and rebuilt 30 new, highly detailed buildings that the team placed in layouts for particular shots, arranging the buildings in various ways to create close-up urban settings with as many as 150 buildings. For shots where the camera pointed toward the street, the team created a full 3D city block with streets, intersections, bicycle racks, newspaper stands, and other props. Motion-captured 3D people hail cabs, look up, and point at
Spider-Man and Doc Ock on the side of a building.
For room interiors, fish-eye photos taken of various office and apartment interiors were taken apart and reassembled in 3D shader space and rendered on a 2D plane. The photos were bracketed so that we would have plenty of dynamic- range material to use for different lighting scenarios. Shader improvements were made so that these pseudo 3D interiors could be lit in a convincing way to make the light appear as if it came from outside the building. Additional work was also done on window treatments and newly added blinds and curtains. The interior improvements were especially apparent in nighttime shots.
Lighting techniques also were updated - ambient occlusion became a standard part of the pipeline, and multi-bounce ray-traced reflections allowed for more indirect sunlight bounce. Corner offices were treated as light transmitting areas, even to the extreme of having window blinds casting shadows on adjacent exteriors. An innovative "vector lighting" technique was introduced, which allowed us to add hundreds of lights to nighttime scenes and to also get fast lighting feedback on daytime lighting. The idea behind "vector lighting" is to cache all the geometry information in 2D that is usually done in a renderer, and then to perform all the shader calculations in a compositing system. For environments in
Spider-Man 2, where there were massive amounts of geometry, this technique took the burden off the renderer and made the lighting time for designated "vector lighting" shots geometry-independent.
OTHER VFX WORK
New to our pipeline was the notion of rendering our CG elements into a log10 image with the same exposure-response curve as that of the Kodak 5279 film stock that the film was shot on. All lighting calculations were made in floating point linear space, then "exposed" into a log10 range using an empirically derived falloff function. In our composites, both our CG elements and the film plates were typically treated exactly the same - brought into linear floating point, then output to the film recorder back as log10.
In many fight sequences, we developed a CG destruction pipeline and used it to shatter building facades, dent metal on the train, shatter glass windows, and destroy wooden train tracks. This pipeline was set up to work on 2D plate elements or 3D CG elements. Software was developed to shatter solid volumes in a natural, cellular fashion. A ray-tracing function was adapted to spread the shattering effect out from the impact point.
To create the energy ball that Doc Ock manipulated in the movie we used Renderman's line primitive to be able to create thousands of fur-like surface events that mimic what is seen with spectral photography of the sun's surface. One of the most complex parts of this setup was creating tools for the character animators to choreograph the action when the tentacles affected the motion of the prominences and vice versa.
CG water was created to add a dynamic storm-like situation in the final segment of the movie. Because the energy ball had to affect the water flow in nonlinear directions, a custom version of Arete water software was written for
Spider-Man 2 that was able to accept vector flow fields as input. These maps were created in Houdini, and accounted for complex interactions between deforming geometry and the water surface.
For the burning building sequence, Tobey was shot on a set containing a safe amount of pyro, and then additional plates with an unsafe amount of flames were split-composited together with the Tobey plate. CG Embers were added for effect & composition.
