When Alex Garland’s Ex Machina won the Academy Award for Best Visual Effects in 2016, it was perhaps considered a major surprise in the VFX community. But the win also highlighted just how crucial the visual effects were in bringing the story of the A.I. robot Ava, played by Alicia Vikander, to life.
Now Garland and Ex Machina’s Oscar-winning visual effects supervisor Andrew Whitehurst from Double Negative have re-teamed on Annihilation, about a group of scientists who investigate a mysterious quarantined zone called the Shimmer. While there was a larger visual effects effort required for this new film – which involved the Shimmer itself, an array of mutated creatures and an alien humanoid being – the VFX again remain highly integrated into the storytelling and crucial in realizing Garland’s vision.
vfxblog caught up with Whitehurst for an in-depth discussion about making the film and the visual effects (realized by Double Negative, Milk VFX, Nvisible and Union VFX), including shooting in real locations in the UK, designing with Mandelbulb 3D fractals, filming with practical effects and stand-ins for much of the creature work, and delivering a beautifully choreographed ‘dance’ for Annihilation’s stunning conclusion.
***Warning: this interview contains major plot spoilers***
vfxblog: Let’s talk about the Shimmer itself – how was its kind of petrol slick look designed?
Andrew Whitehurst: The way that was described in the screenplay was like a heat haze, but with prismatic effects, and distorting the background but having a surface quality so that there was a tangible moment when someone passed through it. We look at so much reference. We looked at a lot of paintings, Turner for example. With the way that Turner painted skies, we thought maybe there was something in that because there’s so much strong colour in Turner’s skies. We thought, ‘Okay, maybe we could grab that.’
And we looked at natural phenomena, we looked at things like nacreous clouds and prismatic optical effects you’ll get on contrails of aeroplanes if they’re made of the right kind of water vapour and the sunlight hits them in the right way. We thought, ‘Well maybe that’s something we can use.’
None of those things felt ultimately like they tied into the rest of our film. So what we’d been doing simultaneously was the design of the most alien manifestation of the alien that you see in the chamber at the end, which is based on a Mandelbulb 3D fractal shape. We got really into the shapes that you can get from mucking about with Mandelbulb. I guess the shapes we liked are almost like a Gaudi cathedral. The geometry is procedural, but it has this organic quality, but also this architectural element to it. I think Gaudi is probably the best analogy I can think of.
Because we’d been working on this other thing in parallel, we thought well, what we should try and do is try and get Mandelbulbs in the film as much as we can, they could be our design motif for the alien presence throughout the film. One of the places we could look at doing that would be the Shimmer, or specifically, the exterior views of the Shimmer. So what we did was we unwrapped a Mandelbulb, made it into a wall, and then had multiple different Mandelbulbs running at different speeds, so you always have this churning shape that has some structure in it, which would ebb and flow continually.
That gave us the physical form of it, and then in terms of the look of it was playing with how much we wanted to refract the background, and how much do we want to make it feel like a petrol slick, how much do we want to make it feel like those paint jobs you get on sports cars that give you rainbow colours at glancing angles. We spent the most time balancing those elements to get the finished result.
vfxblog: Was the original Mandelbulb geometry done in Houdini?
Whitehurst: It was done in Houdini. So when I started on the film, I was using Houdini on my laptop, and I put together a Mandelbulb in that, and we started playing with it in the production office. And so, Alex would tell me, you know, ‘It’s moving a bit fast, can we slow it down?” So we’d tweak things. I could show him quite quickly with that, without having to run it through a render farm at DNEG.
Also, the art department wanted Mandelbulb shapes, especially for the chamber at the end of the film. What we did was to sit with the art department and come up with a Mandelbulb shape that they liked in Houdini, and then produce high-res geo from that, which we exported out. They’d then have that cleaned up to a shape that could be milled on a CNC machine, and from that sculpt moulds were made, and then multiple fibreglass sections were produced from the mould formers that were then built to make the set. So the set itself actually is a physical Mandelbulb.
Art department would take other Mandelbulb shapes that we’d made so many of the the spores and lichen that you see dotted around the landscape throughout the film also have Mandelbulb shapes in them.
vfxblog: The Shimmer is like a wall around the area but while they are inside you also see so many instances of it. What kind of compositing challenge was that to have the Shimmer always present?
Whitehurst: We knew that the design problem that we had was that the Shimmer is just described as the Shimmer in the screenplay, but you actually need the Shimmer to perform two narrative functions. You need it to do shock and awe out front when they first see it and when they’re walking towards it, like, ‘What the hell is this thing?’ But once you’re inside it, you can’t have that throughout the entire movie because it’s going to completely lose its impact. It’s very distracting, so we needed to think of something else to do inside of the Shimmer that we could dial up and down to taste depending on how present we wanted it to feel in any given scene. In some scenes you just want to be focusing on just the performances, so you want to dial the shimmer right back, and in others you want to remind the audience about the strangeness of the landscape, so there you’d make its presence felt more.
We actually came up with a few approaches. On one of the camera prep days, we all brought in things like prisms and crystal balls, and interesting pieces of glass from home. We got Panavision to give us every weird, old lens that they could lend us for a day, and Andy Lowe the gaffer got some amazing old theatrical lights that were designed to add lighting effects for theatre. We spent a day firing weird lights through the pieces of glass, shooting footage with our old lenses. We got these amazing flares and aberrations and strange shapes, and we shot a lot of that. We later went through that footage and picked the more interesting elements, and we used those to add optical strangeness to the shots, particularly when we see things like the sun coming through trees.
In areas of mist we would add subtle rainbow shapes into the background, and whenever you had water, like puddles or the scene where they go down the river, we would add rainbow effects to break that up the specular highlights. It’s always there, but it’s hopefully not drawing too much attention to itself, so it’s something that we hope subliminally sinks into the viewer as they watch the movie.
vfxblog: There are so many exterior scenes in the film, where was it actually filmed?
Whitehurst: It was a ten week shoot; five weeks out on the road, and five weeks on various stages at Pinewood. It’s all shot in the UK, but we shot in many different locations. The fishing hut is in Windsor Great Park. The lighthouse and the breach at the end is Holkham in Norfolk. The base is actually a former US Air Force base, Upper Heyford, that was one of their cold war bases where they stored nuclear bombers. That was helpful, because all of the fixtures and fittings are actually American there, but otherwise it was the art department who did an absolutely amazing job of turning rural England into the bayou.
Mark Digby, the production designer, and Michelle Day, the set decorator, did such a great job designing the fishing hut, for example. They visited Florida and Louisiana, and took tonnes of reference photos. It was one of their reference photos where they found this fishing hut that was half submerged, and said, ‘This is amazing,’ and everybody agreed, ‘Yes, it is amazing.’ So the fishing hut was redesigned to be this half submerged structure.
That was built in Windsor Great Park. Art department not only built the hut, they also fabricated fibreglass tree trunks for Cypress trees, which we would then top up in VFX to add more trunk and the canopy. We also changed the more distant tree lines so that it felt more Florida and less Oxfordshire, and added digital foliage here and there as well.
vfxblog: That’s where we see the alligator. What’s great about those shots is that it’s almost filmed just as if it might be there for real. How did you achieve those shots?
Whitehurst: It’s interesting because the design process for each of the creatures was different. For the bear, we led the design and then worked with special effects. The alligator was designed by Tristan Versluis, from the creature SFX team, and then built as a full size four and a half metre gator. We scanned that, and that was the basis of our digital build. What the practical build also meant was that on set we had a full size creature that was great lighting reference, and it also gave the cast something to engage with.
The other thing that it allowed was Haley Williams’ special effects crew to also get casts of the gator sculpt and build bucks from them. So for the shot where the gator slides into the water, Hayley’s team built an articulated buck that was weighted, and had a cable attached to the front of it. The SFX teams sank metal plates with a pulley on in the lake so that when they towed the line, the gator buck slid into the water and submerged in a very naturalistic way.
All of the ripples that you see running away from the gator that then go and break against the bank and disturb the foliage there, all of that’s real. Think of the nightmare that trying to do all of that interaction in CG would be: you’d have to end up replace every bit of foliage that’s in contact with the water. It would also have been hard for the DP, Rob Hardy, to properly frame up the shots because there wouldn’t be anything in frame if we hadn’t had the buck.
It’s the same story when the gator leaps out of the water – Hayley’s team built a modified air ram that could be fired under water, pushing another gator buck into the air and then slap itself down onto the bank. Again, in the finished shots, it’s entirely digital, and we also had to replace grass in the foreground because it starts walking up through the grass and pushing it aside, but a lot of the splashing that you see at the beginning is practical, augmented with CG water.
I’m always very keen, wherever it’s possible, to have something practical in camera, particularly where we can get interaction, whether it’s the lighting interaction, or whether it’s actually physically interacting with the environment. I just think it gives us such a huge leg-up when we start doing CG work. Even though it’s a pain to paint it out, I think the advantages that you get from that physical thing physically being there really outweigh the effort of having to do the paint-work to remove it.
So, for the gator, we had as much practically there as we could. The bits where we couldn’t have anything practical was the actual section where it’s advancing on the cast and they’re shooting at it, because the creature’s too big really to have somebody in a suit. With that we had to get the cast to react to nothing, and say, ‘Okay, eye lines are about here.’ And then we assembled the sequence in the edit, and worked out what worked better and what didn’t.
vfxblog: When they’re interacting with it once it’s been killed, is that more of a practical gator or is that still CG?
Whitehurst: It’s a practical gator that they’re interacting with. Now, the whole story of the film, in terms of actually working on it, was that every design decision that we made had to be deliberate, but because we wanted a sense of strangeness ramping up, and until you’ve actually got the entire movie on a timeline in the edit and you can watch it, you’re not fully aware of where some scenes bump as being too weird, or not weird enough, and you need to make adjustments. The gator was originally a pure albino gator. That was what Tristan built, because that was the brief, and that’s what our original CG version was. Then when we started working, and we looked at how the rest of the film was going, we realised that the gator needed to look sicker. So quite a distance into post production we started adding more sickness to the gator; it’s got some areas of pigmented skin, and other areas where it’s albino, like vitiligo.
We added lesions and some other illnesses to it, around the eyes we changed the skin so it was cracked and bleeding, and other things like that. So even in the moments toward the end of the scene where the cast are physically interacting with the practical build that Tristan’s team did, we still ended up having to augment that with all of this extra skin damage and the rest of it, because that’s what the scene actually needed once we put it together and had it located in the cut.
It was still practical, particularly the parts where Gina’s character is holding the jaws open. That was a large amount of practical gator there. The inside of the mouth is all CG because we ended up having to add more rows of teeth.
vfxblog: Let’s jump to the bear, firstly where it attacks Cass outside the camp?
Whitehurst: In terms of preparation, we got some really beautiful storyboards from Martin Asbury which gave us a starting point. Then when we were physically shooting it, it was down to Jo McLaren, the stunt supe and her team, working with Tuva Novotny, who plays Cass Sheppard. Tuva did as much as she could being yanked forward on a cable. That’s how the moment where the bear reaches down and bites her and chucks her to one side was done; that’s Tuva being hauled around on a cable. Then the shot you see a couple of moments later through the night vision goggles is a stunt performer being pulled on a cable across the grass that we then added a CG bear onto later.
We shot some other shots for the sequence and we actually started animating some of them that made the sequence a bit bigger where you glimpsed more of the bear and that kind of thing, but actually in the edit, everybody felt, ‘We should just make less of it here. We should let this be a shock, and then it’s gone.’ Then obviously we’ve got the second major bear scene later, and we thought, ‘Well that’s where we should reveal it for what it is.’ The decision was made to limit the first appearance of it, to a literally just blink and you’ll miss it moment.
vfxblog: That second appearance, I could barely watch because it’s so tense. Tell me how that was actually filmed and pulled off.
Whitehurst: DNEG designed the bear, and we did an initial digital sculpt of that, which we passed to Tristan, and Tristan and his team did a full sized head and neck practical build. That was silicone and painted and absolutely beautiful. From the same mould we also built styrofoam versions of the bear with some foam padding to suggest the scale of the back that a performer could wear and move around the set. It gave us a sense of the scale and the size of it, so that when we shot the scenes, we, again, helped Rob Hardy and the actors to have something physically there that was the right kind of size.
We painted it dark grey so that we weren’t getting weird spill, but it was blocking lights where it should do, because the whole scene is lit with these very source-y work lights that it’s permanently walking in front of, so if we knew that we wanted something practically there because it’s blocking the light out of the rest of the room and trying to create that in post would be practically impossible on our budget. Also when it’s getting closer to the characters, you’re getting a sense of the occluding shadows on them as well. We don’t want to be doing that kind of stuff in post, because to my mind it never really works.
So yes, we had a physical form in a buck costume for when it’s walking around, and then for the moments where it’s nuzzling against the characters, that’s when Tristan’s puppeteered head was used. So that’s how the majority of the scene was shot was with those bucks, and then we would shoot clean plates, and use them to then paint out the bucks, and then we’d put the CG bear in. At the sequence finale, where it gets shot, again that started with casts from Tristan’s bear head build. Hayley’s team built three heads that were painted and filled with gore and guts and Lord knows what, and then blew them up with squib dets, because that’s what Hayley likes to do.
We filmed these elements from the angle of the final shot, so a lot of the bits that you see flying about in the final comp are practical from that buck getting destroyed. So again, wherever we could use something real, we did. In some of the shots it made the animation harder because we were tied into a physical timing that, maybe if we’d been doing everything with no in camera reference, we might not have chosen.
There was a fair bit of to and fro between the edit and animation. We would try things out in blocking the animation, and I’d go down to the edit, and say something like, ‘You know we’re trying some ideas out here and in this shot it would be great if we could get another 36 frames on it, because we would really like to just have this head turn take a beat longer.’ And they’d adjust things, and send a new cut back, often with our playblast animation blockouts cut in. And the reverse was true too, with Barney Pilling, the editor, and Alex making adjustments to the cut for pacing and so asking us to adjust our animation to fit with that. It was a back and forth for three months until we felt that we’d locked down the sequence from a performance perspective.
One of the other benefits of having the physical performer on set, was that Barney could cut the sequence before we had any animation at all, and get a sense that, ‘Yeah, this is going to work. We’ve got all the beats pretty much. Okay, let’s do a first pass of animation and see where we’re at.’ That’s what we did, and then we just iterated on that. as things settled down froma n animation perspective, we started to being able to swap out the playblasts of the animated bear with fully rendered versions.
And then obviously we did huge amounts of comp work to clean up the backgrounds and then put the bear in wrestling the actors or the foreground elements, putting the flares back and all the weird aberrations from the lenses that we used back in, and all of that kind of stuff, just to seat it in the plate. So the practical onset suit did create a huge amount of work, but I still think it was worth shooting with it there because of what it gave us.
vfxblog: If we jump to the end of the film, we talked about the Mandelbulb creation, but what about the actual humanoid being that we see?
Whitehurst: The idea was to have it as a dance almost, so the humanoid creature was attempting to mimic or understand what Natalie Portman as Lena is doing, and so Bobbi Jene Smith, who’s a dance choreographer, she’s very well known for turning everyday human activities into performance. She worked with two dancers – Sonoya Mizuno, who played Kyoko in Ex Machina, and is also one of the students at the beginning of Annihilation.
Sonoya was the humanoid, and Kristen McGarrity, doubled for Lena. Those two worked with Bobbi, and over a couple of weeks they choreographed this scene, and they just did it in the production office. So they’re in the central area of the production office, with masking tape on the floor, taped out to form a circle that was the diameter of the lighthouse set build.
Then, I think on the last afternoon of the first week, they said, ‘Okay, we’ve got something to show”. Alex and Rob, the producers, and I, were all invited, and we sat down outside the tape ring. So you’ve got two dancers wearing tracksuits acting this scene out, and it was amazing. Right there, with no costumes, no nothing, in an office at Pinewood. Just the physicality of it, and the beauty of the movement, and the timing of it was mesmerising.
I looked at it and you thought, ‘Well you know, we’re really going to have to go some to screw this up now.’ I mean it’s amazing what we’ve already got. Two dancers, not even in their costumes, and it’s already amazing. So the whole design of that sequence boiled down to Bobbi’s choreography, and then Rob and Alex figuring out how to shoot it.
For the design of the humanoid – Jonathan Opgenhaffen did most of the concept work on that – and we looked at a lot of references how different cultures, particularly very old cultures, had represented the human form. We looked at a lot of cycladic sculpture and took reference from that.
We actually went through very few iterations at the concept art stage. We quickly got to a stage where we said, ‘Well, this works, we should build it.’ We knew that Sonoya was going to be the humanoid on set, so we scanned her and we built the humanoid model to match her proportions, because we were going to try and do what we did with Ex Machina, where we body tracked the performance and that’s what we would see, so there wouldn’t be an extra animation step.
So on set Sonoya was wearing a black all-in-one skin-tight costume with tracking markers on it, and that’s how she performed the whole scene up to the moment where she’s on fire. From that moment on, Sammy Sheldon Differ, the costume designer, had another suit made that had panels of LED lights that could be trigger individually. It was actually interesting, because of the way they were wired together, it actually inhibited Sonoya’s movements, making them not so fluid, which was good, because from the performance standpoint, this character is now on fire, and it felt like the Humanoid was really struggling to move, and it was true, Sonoya was struggling to move, because she was trussed up with all of these LED lighting cables.
But again, we had the LED lighting, which put the interactive light onto the set, so when we add the CG fire onto the CG humanoid, we’re not having to also completely relight the entire environment as well. Also the LED light panels add really amazing flares that we kept and then added the same style of flares digitally, where we were adding CG flame. It had this very otherworldly quality. It was very beautiful.
Then the whole section where the humanoid crawls down, back into the chamber, that was thought up on the afternoon we shot it, and it was one of the few times we shot overtime on the movie. Originally, as soon as the door closed on the lighthouse and Lena was outside, you never saw the humanoid again. At least that’s how it was in the original screenplay.
But the thought was, ‘Actually no, we should make the Humanoid’s demise into a moment.’ And so it was just designed, shot, performed, all on the fly, and it’s one of my favourite moments in the movie now. The moment where she crawls up on to that central dais in the chamber, and the whole chamber is ablaze, and she leans back in this moment of submission, and the frame whites out, that’s absolutely one of my favourite shot in the movie, and it was just made up on the spot.
Nothing was practically set on fire, so all of the flames you see are CG. So in those last shots we were pretty much replacing the entire background with CG because we were relighting it so much due to the amount of flame.
vfxblog: Before she’s on fire when she’s just interacting with Natalie Portman’s character, what were some of the challenges of rendering and integrating a character that looks like that with the sort of rainbow colours, and the sort of Shimmer type feel?
Whitehurst: It was very tough because of the way that the grade was pushed. When it was shot it had a very evening, tungsten feel to it, and in the DI it was pushed to being something much more neutral and white. The effect of that amount of colour correction was that when we were adding a lot of these rainbow colours, we were losing them, they were effectively getting graded out.
So we ended up working with Asa Shoul, the colorist. He gave us DPXs of a, ‘Here’s my pass of what I want this to look like,’ and then I just went through in NUKE and made some grade nodes to duplicate that as best I could in Nuke and sent the shots back to him and asked, ‘Is this close enough as a base, that you can then work with?’ We’d tweak things, and then eventually the decision is made: yes, that is okay.
Then that meant that we could maintain all of those extreme chroma values when we comp the Humanoid back on top. The iridescence pass was rendered as a separate layer so that we could have full control of that in the comps. We had a lead metal base look, and then this extra layer for the iridescence that we could dial up as needed, and Alex’s taste was definitely towards more psychedelic at that point, so we really turned that up to 11. But because we had pre-graded the plates, it meant that all of those colours were preserved, and then did survive into what you see on screen.
vfxblog: At some point I think I saw on Twitter that you said this was one of the hardest projects you’ve worked on – what were the biggest challenges?
Whitehurst: There was just such a huge amount of design work. Because it’s an escalating tale of oddness and strangeness, when you see things bump in the edit, you have to change them so that the whole thing flows. That meant the whole design process wasn’t always designing upfront and executing on that, we were actually re-designing a lot of things quite late in the day.
An example of that is on the beach when Lena walks to the lighthouse and it’s covered in crystal trees. That wasn’t the original plan. Someone in the art department had found these really beautiful photographs taken underwater of divers entering the water, and the thought was well on the beach what we do is effectively invert them so we would have all these humanoid shapes emerging out of the beach with columns of blobby sand surrounding them.
Art department built bucks of these blobby columns for lighting interaction on the beach. That’s what we shot with, and we built and look-developed these models in CG, and we started to put them in the shots. We were sitting in the editing suite and we realised that they didn’t feel right. When we saw them in the context of the rest of the movie we understood that they didn’t feel like they were connected to anything else in the film. So, what were we going to do?
The scene prior to that beach scene, Lena’s walking through a forest, and we Lidar’d that forest, in case we needed to track it. So we’ve got these Lidar models of these trees, and obviously Lidar doesn’t cope very well with volumetric type materials like hair or leaves. So the Lidar models were beautiful trunks and branches, and then there’s a weird blobby mess where all the leaves should be. But actually they had a very pleasing, sculptural quality to them, and we thought, ‘Well maybe there’s something here we can use.’
So we cleaned up the leaf mess so that they were renderable, but they still had the same blobby shapes of the Lidar artefacts. Then did the look-dev again to make them crystalline, and then painted out the bucks that we put on set, and put in the trees, and that’s what you end up seeing. Then after we’ve done that, we thought, ‘Well, it looks a bit sparse.’ So then we designed and added those shards blasting out through the sand around the trees, with the thought that the trees start out of these shards and then they grow into the trees.
All of that was really quite late into post, but it felt right. It was the right thing to do for the film. There was a lot of that – it just meant a huge amount of work for everybody. But you had to remember that we didn’t actually have tonnes of money, so we couldn’t just continue to chuck money at the problems. We had to be as smart as we can. That’s the story production-wise, and post production-wise of the movie. We all went on a massive journey pretty deep into the Shimmer ourselves to make this movie about going into the Shimmer.
vfxblog 