Some of the most intensive work for visual effects artists on any project can be when they are asked to design characters, environments and even shots – often from scratch. That creative process is a mix of concept design, devising a shooting methodology and technical R&D. All of those things were required on director James Bobin’s Alice Through the Looking Glass, when he called on Sony Pictures Imageworks to help realize the Oceans of Time sequence.
When the film was released in May, vfxblog spoke to Imageworks visual effects supervisor Jay Redd, who worked alongside senior visual effects supervisor Ken Ralston. Back then, no imagery from the Oceans of Time sequence was available, which made telling the story of how the shots were created difficult to tell. However, Imageworks has now delivered a great video breakdown (see below) which prompted us to now post our interview with Redd. There are also images from and links to a paper (available for free download) presented by the Looking Glass Imageworks team at DigiPro 2016 which dives into the use of Houdini and other tools to complete the shots.
Hope you enjoy revisiting this incredible work from the film.
Redd: Oceans of time came from just a really abstract idea in the script. In the first act of the script, time wasn’t even in the script. Time was an idea, but time wasn’t a person, and there wasn’t an environment to go into. And that’s actually James’ addition, the idea of making time a person. So as the concept of time travel was going to be in this film somehow we all got together and said okay what can we do that’s new? We all know about all these time travel tropes, which is, you know, there’s time-lapse and you disappear, or you hyper drive and you get somewhere else, or there’s a wormhole that opens up and you travel down that at lightning speed, or there’s some kind of magic that happens. Or, you know, there’s always different ideas. We wanna do something different, and we thought everybody says that, let’s do something new. What hasn’t been done before?
vfxblog: So how did you decide where to take it?
Redd: In the script there was this line that says, and I’m paraphrasing here, that said: Alice us in the Chronosphere and travels in the Oceans of Time. Okay well that’s kind of cool, that’s a little poetic, right? It’s a nice idea. The Oceans of Time, well what if we actually just took that and said what does an ocean of time look like? James gave us some things to go on, and one of the things I like about working with James is he says, here’s some ideas. I’m not gonna tell you exactly what it is, I don’t even know if I have it in my head, but he’ll give a few pointers and say run with it. So he gave it to Ken Ralston and I and we had a lot of kind of fun and freedom with our team to go explore some ideas on this.
His one main thing was that he wanted to have the Oceans of Time full of memories or moments in time. Now those could be, you know, that’s wide open. How do we visualize that? Does that mean they’re little, they’re still frames? Are they moving, are they 3D animation, are they what? How do we create all of these moments? The key for him is he wanted audience members to remember something that may have happened in the first film, or maybe something that happened just moments before in the current movie that we’re watching, right? How is that gonna look? Moments in time, or memories, things that Alice has experienced or things other characters have experienced. So immediately in the background we’ve got our team basically digitizing hundreds of thousands of pictures, clips, little sequences from the film that we, the first Alice, from the current one, we’re working with editorial, gathering every image we possibly can that relates to the movie. Because we know we’re gonna have to populate some world, some environment with all of these hundreds, if not thousands, really, of memories.
vfxblog: Can you talk about that gathering process?
Redd: We needed everything to be organic. That was one of the big important things of this film, is it shouldn’t feel like a high-tech experience. It shouldn’t feel like The Matrix, it shouldn’t feel machined, right? It needs to feel organic. The whole world. And this is what Tim Burton did in the first movie with Ken and those guys is they made the whole movie feel organic and alive. You know, there’s not really a lot of technology if you think about it in Wonderland. They don’t, you just don’t see much, especially in the first movie. Everything kind of stops at, kind of stops at this kind of a Victorian end really. And so in this film we pushed a little further with Time’s Castle of course, with all the clockworks that are there, and all the complexity. Steam power, that was where we drew the line. And then add a nice dose of magic and Wonderlandian inexplicable physics and you’ve got something.
So we started looking at just masses of images. Maybe they lived in clouds, maybe they lived in the air, maybe they wanted something that felt like, that it was huge. You know, the vastness of your head. All the memories, all the experiences you’ve ever gone through. How do you travel through that without it looking like A like a science film, you know, or a microscope that’s you know like a Siggraph tech demo, or how do you make it look like something cool?
We actually started going towards the organic qualities of oceans. We really started looking at that, thinking well there’s a nautical theme in the film. Alice knows how to captain a ship as you know you see in the first part of the movie. So the Chronosphere started taking on some design elements from a ship and maybe levers and whatnot, rudders and stuff, right? So it’s like well why don’t we go down that path a little bit and see what we can do? So we started doing just some simulations of waves with our effects team. The trick was how do we art direct these things? Always needed to control nature, right? We would need to have a wave crest right at this particular time. And how do we see the image in that wave?
Well we worked with a small team, and just really our previous group, and put together some very rough ideas. And just kinda started projecting some of these images onto the waves themselves just to get a visual. How does this feel? Is it alive and interesting? Well yes it was, immediately, but we wanted to take it another step further which was how do we get the images inside the wave? We don’t want it to feel like it’s a drive-in movie theater or feel like it’s just a cheap projection gag. Everything needed to feel volumetric and massive and dimensional, which is why when you see it in 3D you can actually see into the waves, and every moment and memory is dimensionalized so it actually feels like a full-on volumetric kind of hologram feel. It’s beautiful in 3D, it’s something else, almost a different film. That’s why I plugged that in the beginning, because it feels so different.
vfxblog: What then were some of the technical solutions you had to come up with for Oceans of Time?
Redd: We had to develop some tools for our animators to help navigate this world, and to again to have artistic control over timing, over scale, over movement, over speed. And that was a big joint production between our effects group and our animation group and our animation tools department over at Imageworks, putting something together that allowed our animators to work quickly, allowing us to put stuff in front of James quickly, and give art direction for all of us.
So the further we took this, this Oceans of Time really became a character in a way. And there’s this really tight relationship with speed and scale of things. Like you move too fast, everything goes too small. If you move too slowly it’s not exciting. Right? So every shot kind of had to have a lot of care taken to it. Like, how fast do you move the Chronosphere through this space? How big is the Chronosphere against a big memory? Sometimes you’ll see a two hundred foot wave, a three hundred foot wave, sometimes the waves will be forty feet, it all depends on what the storytelling needs are.
And so questions started coming up the more you get into this design, like well how does she actually get into one world and from one world? Well we discovered that by accident. She kind of gets bumped into one of these memories, into the water thinking oh she’s gonna drown, but actually what that does is it pushes her into that moment, that moment she’s able to go back in time to that period in time.
And so that kinda got us out of this oh good we’re not gonna do the tunnel thing. She actually enters the ocean of time where all these moments happened simultaneously or together, you know, at least in the vastness of it. And getting out of these moments is the Chronosphere allows you to do that. It scales up and down, that was kind of a big challenge to figure out how to make that look cool and not kill her. That was a difficult one because it’s kind of made of blades and rings, how we get her in and out of that thing. So there’s all these things you look at, and it’s kind of thrown away in the film but it was actually a lot of work to design this thing, and figure out how to open the gates, open the ring, and have her control it.
vfxblog: Anything viewers should take particular note of when they’re watching this sequence?
Redd: It may not have been apparent, but these are kind of the subtle visual storytelling things we like to do, is that the first time she flies into the ocean, she enters the Oceans of Time as the castle kind of liquefies and melts away in front of her, is that you enter the Oceans of Time and it’s this massive space, but it’s relatively calm. Alright? And in one of our first tests we thought well there’s a sky above, what does the sky mean? What if she goes into the sky? That doesn’t go anywhere. Well let’s put an ocean, let’s flip that ocean and put it on top as well so we’re now actually completely surrounded. And so there’s no way out except through a memory, not gonna really escape, right? And that kind of suddenly made the world feel like it’s a totally different thing, and it’s not really a tunnel. We do have a forward and backward. But in time travel in this world there’s no future because the future is happening all the time. You can only go back to the past. So we never had to figure out what was in the future, everything had already happened. So, but our vision of the future was just kind of a white horizon, it was glowing off in the distance because it’s always being made.
As the film goes forward and time breaks down, literally, time himself and the grand clock starts to decay because Alice is disintegrating time, basically, the ocean becomes more and more rough. So every time she visits the ocean of time, or uses it to travel from one place to another, it becomes more rough. There’s more lightning, there’s more wind and spray, and white water. So by the end of the film when they’re being literally chased by rust to time’s castle, the thing is just complete chaos. And the waves are actually now spilling on themselves and forming funnels and all these things, and that’s what kind of forces them to get through back to the time’s castle.
vfxblog: Just more on the tech side, what R&D was required in terms of tools used?
Redd: It was a big mixture between Houdini simulations and being able to adapt some of that geometry back into the Maya world and allow our animators, by using deformers and other sets of tools to give them control over a bunch of crest waves how to make things bigger or smaller. So it’s more than rudimentary too. It was enough to be able to scale things up and down and have our animators have the animation sensibility, because, and it’s no slight to our effects people at all but we wanted to work in an animation capacity because we’re also animating the Chronosphere. And to give those things weight, and to be able to control the banking and all these things, you know it’s a very tight marriage of all of these elements. So yes, it’s a cross-development of Houdini, Maya, and a bunch of proprietary tools in between.
We worked in Arnold to be able to render all this stuff because it’s all refractive. There’s also a ton of 2D preparation to get all these memories working, you know to process this stuff and dimentionalize things. So there’s a lot of optical flow work done with the little sequences, because the actual, the images themselves actually affect the waves that they’re inside of. So when you see a Jabberwocky wing come across inside the wave that actually affects the spray of water on the wave. So there’s a lot of complex stuff that’s happening in each one of these shots. So it all started in 2D preparation of getting these plates ready and these shots ready, all the way back in, you know it was a chicken and egg scenario too because some of these memories had to be in the oceans of time, but we hadn’t even made the shots yet, right? So we couldn’t even finish some of the shots until we finished a big chunk of the movie that came before it, if that makes sense. Because as she goes back in time she’s visiting moments that just happened, you know, three minutes ago, five minutes ago. So we had, we were kinda literally chasing ourselves in this little time loop for quite some time.
vfxblog: When you’re filming the plates with Mia Wasikowska, was she able to visualize anything while those sequences were being filmed?
Redd: Yeah she was. Now we had done some previs on this (Halon also carried out postvis for the shots). We didn’t go in totally blind. We went in a bit, with one eye closed. We did a bunch of previs initially knowing, you know, knowing what the rough design of the Chronosphere was. We knew we had to cover it with effects and things later, so the basis of it is a six axis rig with some basic controls for her to be able to react to, and then we on set, we would have our previs with us and we would show it to her and we’d kinda walk through what was going on, but we would set targets up around on the blue screen in that three sixty stage where we were, or the you know two eighty stage where we were, and then we’d set different targets with numbers. And so we could call out while we were filming. You know, she’s gotta react and we could time lights with Stuart Dryburgh, our DP, and James could yell out to her. Because sometimes the sound didn’t matter so much because we had wind going and fans going and it’s like crazy set with all this interactive light and whatnot.
And we could say okay look at number one, which is the Jabberwocky, and now down below, he’s diving, now three, go to number four, okay look behind you. You know, and we would do that sequence, but from a wide angle, a close-up, behind, in front of, right? So we’d get this series of things so that we could also have, because we knew that when we get into editorial, into the post-vis process, we would discover some more exciting things. Because when you start intercutting with full CG shots, some wide angles and whatnot, and you wanna cut into, you know she’s just come from a three sixty, we wanna go wide now and watch her do that, and then cut back into a POV or shoulder, you know there’s all this variety.
So we went in with a lot of coverage on that, and the six axis rig was puppeteered. And so it had, it was great, this little waldo system it was a couple of joysticks, so we could give instant feedback, James could direct it or I could or Ken, whoever was doing it at the time, to make that work against blue with the wind. And of course she had a cable on for safety because I put her up about, I don’t know, about six feet up in the air. Six, seven feet up in the air to get enough coverage, because we wanted our cranes to be able to dive and go around, you know, up and down and do some nice booms and whatnot.
So the goal was though is to leave the shoot with a ton of material, which we ended up having a ton of material. So that’s kind of how that went. We had rough boards, we had rough previs, and then magical things happened on stage which you wanna just explore. And so we were, we had a lot of freedom and I guess we’re kinda wild on some of that stuff. And then you know there were a couple of pieces that we ended up getting in our additional photography a year later, just a few insert shots that we wanted. That’s kind of how that process went. We didn’t use motion control or anything, it was all kind of free-form which kinda made it I think more exciting in a way.
vfxblog: I feel like the Chronosphere itself might have been a slight animator’s dream, but also a very tricky one in that there were lots of things to move about and intricacies and also keeping that old school look like a watch, you know, steam powered or whatever it was. Can you tell me a bit about the challenges of animating the actual thing?
Redd: Yeah that’s a good point that you bring up. So when we saw some of the initial designs, it went through many different designs. There were the Chronosphere kinda started feeling like a bubble, there was almost like some stained glass looks, there were more things like cages that would unfurl and keep her safe. The only thing we wanted to say is that there’s almost like a different kind of weather inside the Chronosphere.
We wanted to stay away from the Time Machine movie. I don’t know how old that is, about fifteen, eighteen year old one.
vfxblog: The film by Simon Wells?
Redd: Simon Wells, right. In that movie he’s sitting in that thing and it turns into kind of a bubble while all this time stuff’s going around. We actually didn’t wanna do that, and ended up ironically having some reflection of that. But really the influence was an Astrolabe, which actually is a kind of an astronomical tool for aligning and watching eclipses for planets, the moon and whatnot. So that was a little bit more of the influence, was kind of space and time. But the challenge was it’s gotta scale up and down rapidly. She’s gotta hold it in her pocket and be able to hide it, and then be able to kinda throw it on the ground and have it unfurl, which she discovers accidentally of course. But so it’s got a little bit of its own magic intelligence. Once she steps out of it it shrinks down.
For the animators working with animation supervisor Troy Saliba, Troy and I have done a few movies together so we have a little bit of a shorthand. Troy’s really great at keeping things weighted and with mass. It was a tough thing, we did many, many tests of making this thing feel heavy and clunky, but also how can it float and hover. So we just kept thinking about metal work and Slinkies. When you crumple up a Slinky and it kind of holds a shape but it might retain its own tension, those are things that influenced us, like a spring action unfurling, or the kind of twisting that’s there. And those rings actually became a bit of a character as well. They had to be sped up, they had to slow down, they had to be removed on some shots because they’d just get into the way of face, so we took a lot of creative liberty with where they were in between the audience and her.
It was a tricky little object, but it was also really cool because again it was this kind of steampunky brass and ironwork thing that felt like it maybe a thousand pounds but could somehow fly. And then all the effects works on top of it, again we wanted electrical kind of Tesla coil looks and arcs. There’s a little bit of magic dust in there, but it’s got its own kind of plasma vibrations and stuff there.
Imageworks animation supervisor Troy Saliba will be speaking and delivering a workshop next week at the VIEW Conference in Turin, Italy.
vfxblog 