The ménage à trois between K (Ryan Gosling), his companion hologram Joi (Ana de Armas) and Mariette (Mackenzie Davis) in Denis Villeneuve’s Blade Runner 2049 is a stunningly realised sequence. It involves the real human Mariette interacting with K, while also ‘merging’ at times with Joi.
The sequence involved significant visual effects planning and execution, undertaken by overall visual effects supervisor John Nelson and Double Negative, with Paul Lambert as Dneg’s VFX supervisor. In the first piece of my coverage of Blade Runner 2049, John Nelson breaks down how they made these amazing shots possible (Nelson recently spoke at length about the visual effects of Blade Runner 2049 at the VIEW Conference).
Settling on a hologram look
John Nelson: We looked at Joi early on in design. We didn’t really like the way most real holograms looked. We didn’t want it to look too much like Tinkerbell and made up of light. It was a conscious decision in the movie to try to have the film really feel analogue and not digital. For the most part, we tried to make things that felt like they could be photographed. As opposed to being made up on a computer screen, things could be/would be photographed.
With Joi, we tested a bunch of looks. What we finally came up with was, if you look at a glass or you look at a bottle, you see the front-side surface of the glass or the bottle, then you see the back surface of the bottle. What we called that was the back-face. So what we did is we came up with a way of mapping the back-face as if it were clothing projected from the back, on her. Then we would see it only from the front. It’d be like we see the reverse of it.
It wouldn’t be like the front just projected on the back-face. It would be the back-face reversed. Transparent through the front-face. What that gave us was a really good feeling of volume for her. She has volume in her. We add it through her face for a while, but it seemed to be getting in the way of the performance. So it’s mostly in her body. It’s subtle but it’s very nice because what that gave us was, we got all the realism of photographing the real Ana, the real Joi. But then we could make her feel volumetric by using the back-face and using this. You see it when she comes on. She’s still on the ceiling but you see it when she turns into someone else and twirls and after she goes on the emanator. You see it up on the roof in the rain.
Prep’ing and shooting
I shot a bunch of tests early on in pre-pro. The original storyboards had them dancing when this was going on. I said, ‘Well, how am I going to put this together?’ What I’ve done in the past, you know, in other effects work that I’ve done is that I was able to generate a count and have everybody sort of sync into that count. What we did with Mariette is real. She’s a replica but she’s real whereas Joi is virtual. What we did is, for that, we would shoot Mariette and Ryan. Then Denis would pick a take that he liked. Then I would spend five minutes with the script supervisor. We would go through and we would actually time, on a stopwatch, where all the major actions were done.
Then I would line up, with an iPad, the first image with DOP Roger Deakins’ camera. Roger’s camera, of course, is not a motion controlled camera. He’s just very good at repeating and being close. So we would line up with this iPad so Ana could see, ‘Okay, let’s get you lined up as best you can.’ Then we’d pull away the iPad because we didn’t want her to look at it. We wanted her to look at Ryan. We would give her directions, ‘Okay. Get ready now. You’re lifting your hand. Now you’re touching his face. Now you begin to walk around him.’ That got us close. Not exactly because humans are humans and actors are not repeatable robots. They all have their own way of putting their performance on. That got us close.
We also had, while we shot it, we had witness cameras, GoPros in every corner of the room because it was a very small room. Then we also had at least one C-300 off to the right angle from Roger’s camera. Those witness cameras gave us a view of both women in three dimensions from all different angles. We used those to basically roto our three dimensional scans of the women to be in exactly the right place. Then, when the women didn’t line up, we would project the textures onto one of the digital version and then move it over. Until it lined up. In some instances, when that was too much of a shift, we would go to a full-on CG version. We tried to use 2D because the 2D version looks more real because it is.
Putting it together
What we found, when Joe Walker, the editor did the first rough cut, we found these moments of magic where the eyes line up for a moment. Immediately I said, ‘Well, whatever we do, we can’t lose all this good stuff, right? We only want to make it better.’ What I came up with is that in lots of the shots, we can start out of sync but I want as many shots as possible to end in sync, right? You get these moments and what I’m really proud of is you get these moments where you get both performances of the women. You get Joi’s performance, you get Mariette’s performance saying a line. Then you get Joi’s performance saying a line. Then they merge. Then what I would call the third woman throws a look at Ryan. Or when he sticks his hand around her waist and pulls her towards him, you see, you know, Mariette look at him. Then you see Joi look at him. Then the third woman looks at him.
I think there are about three to four shots in a sequence where it’s both actresses and the third woman. It’s really very interesting because each shot has the story arc that goes through it. It’s cool because they don’t ever completely line up. But they line up for moments and it’s really magical when their eyes line up. At least from my point of view. I’m really proud of it because it doesn’t really look like a big CG scene. It just looks like this emotional crazy scene, which is good. Which is what it should look like. When they kiss, it’s okay for them to be out because the motions are really heavy but even when they kiss, you see Joi’s eyes are one place and Mariette’s eyes are in another place but they sort of – it’s like a Picasso painting where the eyes merge. It’s some really nifty stuff going on in there because she’s electrical. She begins to move into Mariette, you see Mariette’s hair sort of move over like static electricity.
You see the eyes of one in the eyes of another. We love the Sheba thing with the hands behind. It’s like he’s being caressed by four hands at the same time, behind his head. We would use these things where we would map our reality onto CG and then look for how to make the interpenetration work without looking too weird. That scene took a lot of work. A lot of Saturdays where we would just spend the whole Saturday working on every shot in that scene. I remember flying up to Vancouver and spending a solid two days with Double Negative and Paul Lambert, the visual effects supervisor, in the screening room in Vancouver. Just saying, ‘Okay. This shot. How does this shot begin? Now when it goes to here, where should it be here? This is how it ends.’
Check back soon for more Blade Runner 2049 coverage on vfxblog, and at VFX Voice.
vfxblog 