The Decision to Go With Completely CGI Characters: “We saw it as an opportunity because in Pirates 1, the characters had to be seen in the picture costumes and then transitioned into graphics and back, so the most sensible working methodology was to photograph the characters in their picture costumes and then in post we matched their performances as closely as we could. But it was a very manual process and there’s a lot of interpretation, especially with some of the large coats. Where is that limb exactly and how is that character posed? There’s some ambiguity there. We had to just make it up and do the best we could with it. I thought that worked out generally pretty well. But going into 2, Hal [Hickle] and I sort of discussed, ‘Well, since the characters are always computer graphics all the way through, what can we do to help make that process of recovering their performance better?’ We thought, ‘We can put some kind of tracking marks on them that will make it a little less ambiguous, exactly where the bones are, so that we can get good, faithful representations of their on-set performance.’
The thought was, ‘Well, is there something we can do that can get us close to motion capture quality but without all the stuff that comes along with motion capture?’ If you’ve seen what a motion capture rig looks like, with a big array of 16 infrared cameras that really can’t be occluded and there are a whole bunch of lighting constraints because you don’t want… Cameras are looking for the infrared markers and you don’t want to have so much infrared already in your scene that it confuses the cameras. You can’t obscure the view of the cameras. There’s a whole bunch of equipment that you have to take along with you if you’re gonna do this. So it looked like it’s not really practical to do motion capture because it’s just such a large footprint on set. You can’t really take it out on location. It was clear early on that whatever we did had to be robust enough that we could take it out into Exuma and shoot with it standing in two feet of water and in pouring rain, and all of these sort of remote places.”
We got together with software R and D department to talk about what can we do to make this process easier. Ideally, we want to be able to get what amounts to motion capture data but we can’t afford the footprint of motion capture and all of the constraints that come along with it. So the R and D guys scratched their heads for a little while and they came back with a proposal for what we’re calling IMO Cap. IMO Cap is a lightweight, low footprint, on-set motion capture technique that involves kind of a motion capture-like suit, and then two relatively small witness cameras that can just be on a still camera tripod. That’s really about all that happens on set. Then there’s a clever piece of software that our guys wrote that is smart enough to know about range of motion and where a limb was on a previous frame, so kind of knowing what plausible position would be on the next frame. It’s robust in that it tolerates some occlusion in the cameras and some other errors, and the process still gives us good, reasonable results. That was probably the big technical innovation that was on this picture was this whole on-set motion capture technique.”
Being Presented with Next-to-Impossible Tasks: Has Verbinski presented him with ideas for visual effects that were almost impossible to do? “Oh yeah, routinely,” said Knoll. “Davy was the probably the biggest challenge in 2. To do a completely believable character and he’s got a major role. You see him close up under the most difficult lighting conditions possible.
One of the things that we had really working for us, a big advantage we had on Pirates 1 was that the skeletal characters, by their nature, they’re only seen in this very stylized lighting environment. They’re only seen in the moonlight when its conditions are sort of favorable to making the characters look good. Now we knew going in to Pirates 2 that Davy was going to be seen at night, he was going to be seen indoors. He was going to be seen outdoors in daylight and pouring rain. He had to work under every possible lighting condition. He had to do it in close up. That was a bit of a scary prospect because on the last picture we did have the characters were always in monster lighting. It would have been hard to make them look good in broad daylight. That was one of the constraints in Pirates 2. The artists, I think, did a fantastic job in putting in loads and loads of detail and making him look great.”
Page 3: Studying Sea Creatures and the Work of Digital Animators
