'The Avengers': How ILM assembled the visual effects
Any kid can tell you that one of the great pleasures of any toy is unpacking that sealed box for the first time and putting all the parts and stickers together.
Visual effects house Industrial Light & Magic has given EW’s CapeTown a video that might give fans of The Avengers that same feeling.
The collection of behind-the-scenes clips reveals everything from the simulated Mark Ruffalo they created to morph into The Hulk, to the fact that most of New York in the movie was a digitally painted backdrop.
So watch the video, then allow ILM visual effects supervisor Jeff White to provide a deeper explanation of what it shows:
And for other behind the scenes videos like this one, check out ILM’s YouTube page.
0:00 – It begins with what looks like a stop-motion puppet show, but this was just a gag to kick off the reel – not something they used as a starting point for a sequence in the movie. “It’s just a funny bookend on the whole thing,” says White. The video was created for various presentations, including awards submissions and for demonstration at SIGGRAPH.
“It’s something we originally created to show people the behind-the-scenes stuff,” White says. “It’s helpful for people to see more detail as far as how the effects are put together. And we try to go a little beyond just the layers building up and really fly through it, or zoom in to show the specific details.”
0:16 – “The way we tried to solve the problem of creating a digital green person was to start by making as close a replica to Mark Ruffalo as we could and solve all the difficult problems there,” says White. “Then we were able to re-sculpt that into the format of Hulk.”
You can see the digital model of Ruffalo in the upper left corner. And later in the video, he springs into action.
Next Page: The world’s deadliest tackling dummy …
0:30 – Not all The Hulk’s activities were digitally rendered. Here we see an element from the sequence where the monster smashes through the S.H.I.E.L.D. helicarrier in out-of-control pursuit of Black Widow, and a giant green shape actually crashes through a practical set.
“That was built by the special effects supervisor Dan Sudick,” says White. “When [Hulk] is chasing somebody, he’s not going to worry about smashing through a wall to get to them. Dan built this steel armature on a track that he could pull at running speed all through the set. What it gave us was a lot of fantastic destruction — real glass and real steel beam blasting apart. That makes our job of integrating a digital character much easier.”
0:56 – After seeing a kind of rhythmic snap-together of the helicarrier model, we crash through the windows to see a clip from the sequence where a hovering jet is machine-gunning the green monster.
“We do a zoom in to a super high-res, 8K render – a still of the Hulk that allows you to really get in there and see all the detail in the eye and the pores and the skin, and the little microhairs on the nose – all this incredible detail that doesn’t necessarily get appreciated onscreen during an action sequence,” says White. “But if it wasn’t there, you could pick it out right away.”
Next Page: New York City wasn’t built in a day …
1:03 – “When you’re doing something like the Hulk, people have a pretty good idea that’s CG,” says White. “But our goal with New York City was to really make the effects unnoticeable.”
So this part of the video may surprise people: We see Iron Man flying through Manhattan, but what many might assume to be just a digital character layered over a background plate of the city turns out to be an entirely animated sequence, right down to the traffic on the street below.
“For iron man flying around the city, we wanted him down around the valley of buildings and in New York you can’t take a helicopter that low. You can’t get to 500 feet above the tops of the buildings. That makes it pretty restrictive to get the plates.”
When in doubt, just build the city yourself.
Next Page: Strange rain falling on the city …
1:30 — What is this chubby rain falling onto the buildings around Stark Tower?
White explains that these orbs are a way to demonstrate how they created their digital New York “playground.”
“The approach was to build everything off of real photography. We actually did the largest photography shoot we’d ever done on a city at the start of the project — four photographers working for about six weeks.” The crew captured 275,000 images in the midtown area during that time.
“The rig we used is very much like the Google street view camera, where you get that full 360 degree sphere, except ours are super high-resolution. All the spheres dropping is just an illustration of all the different photography elements we used for textures on the buildings.”
Next Page: Peeping in through skyscraper windows …
1:53 — The massive photography project gave ILM “a pretty good base to start with as far as stone and concrete and those flat surfaces,” White says. But it’s not quite enough.
“The challenge is with a moving camera you have to put in all the visual cues that are lost. That means every window has to be replaced with a CG window, that would give us the reflections of the environment, the reflections of the sun, and it would add a random window blind at a random height. And the room interiors also had to be visually created.”
They did that by plugging in images of their own workspace. “There are 20 offices at ILM for all of New York. If you looked around and zoomed right in through the windows, you could see right into our offices,” White says.
“Plus everything on the ground that needs to parallax – or change perspective – like cars, street lamps, and trees, has to be painted out of the photography and put back in as digital models,” he adds. “We built about 200 New York City set dressing props, like hot dog stands and mailboxes. Pretty much everything we could find in the original photography.”
2:10 — Here’s the digital Ruffalo in action.
“This was the time he was deciding to turn into Hulk instead of it happening to him. It was conceived on set as a very fast, two second – Boom! And he punches the [leviathan] in the face,” White says. “But once we got back and started animating that, it just didn’t work. It was too fast. Couldn’t understand the action of what was happening. So we ended up having to replace him digitally for that entire shot.”
Next Page: Six heroes, one major fight sequence …
3:04 – “We called this the tie-in shot,” White says. “It was supposed to be the first time in the film where Joss wanted to see them all working together. It was probably the first shot we started on and the last one we finished because it took so much coordination and planning.”
It begins with Scarlett Johansson as Black Widow, riding on the back of an actor in a black and white motion-capture suit, which becomes a sky-bike-riding Chitauri warrior. From there, we follow Iron Man as he flies by firing his repulsor beams, on his way to help Captain America in the middle of a melee down on the street.
“We shot a plate of Captain America fighting aliens, but it slowed the camera down so much that we needed to replace that part with a digital Captain America,” White says.
So the only flesh-and-blood characters to appear are Johansson as Black Widow, Jeremy Renner atop a building, sniping aliens with his bow and arrow, and Thor – riding one of the leviathan with Hulk.
Next Page: “Puny god …”
4:01 — What!? Tom Hiddleston didn’t do his own stunt as Loki in this sequence? Well now I don’t know what to believe …