In July 2006, I got the chance to sit down with Jonathan and Stu, two of the three founders of
The Orphanage, the San Francisco visual effects firm (and software development shop) that is just now branching out into computer-generated animation. (The third founder is Scott Stewart, who works in LA.)
A very abridged version of the conversation ran in The Hollywood Reporter's
`Future of Entertainment' special issue this month, but I wanted to post the full transcript here.
What did we talk about? The Orphanage's plans to compete with Pixar...the permeation of visual effects shots into every kind of movie...YouTube and TV commercials...Video remixing and audience participation...whether videogames are the future of cinema...HD DVD and Blu-ray...whether model-making will survive...and pre-viz.
Here it is... Would love to hear your comments.
@ @ @ @ @ @ @ @ @ @ @ @ @ @ @ @ July 10, 2006
Scott Kirsner: Is there a convergence happening between the worlds of live action visual effects and computer-generated animation? When The Orphanage starts making CG animated movies, it makes me wonder whether it’s a similar pipeline, and a similar skill set – that we might not see as much white space between the two worlds in the future.
Stu Maschwitz: Sony’s doing that right now. They’re very much about moving people from their visual effects pipeline into their animation pipeline. They’re completely intertwined.
We definitely want synergies between the two, but we don’t see them as the same animal.
They have different needs. In some ways, we’ve been trying to make our visual effects gigs train us for the CG animation thing. What it means for someone in visual effects to do lighting is to just match some live action lighting, for a character that’s in a live action scene. But getting to do a job like “Sin City,” it’s the opposite. We’re putting an already-photographed character in a CG scene. So lighting means really fleshing out the cinematography of an entire environment. And that’s something you have to do at Pixar, but not if you’re doing a “Jurassic Park” sequel.
“Sin City” was the first show where we had a layout department. And that’s a department that we’ve maintained ever since. That’s an experience that merges this technical knowledge of cameras with the aesthetic concern of making a shot that could be in a movie. And that’s a department that a visual effects company suddenly finds themselves needing, with these “Sin City,” “Sky Captain”-like projects. But Pixar has needed that from Day One.
There is a little bit of a colliding thing happening.
Jonathan Rothbart: Doing character animation in a visual effects film is not so distant from something you’d do in an animated feature.
SM: And effects animation – a lot of the stuff that we do with fluid simulations, and particles, and smoke and fire – all the hard things in CG [visual effects] are just as hard in animation.
JR: That’s an interesting thing that Stu mentions. With all these CG features coming out, there’s different levels of quality that you see. Our goal is to go after the Pixar-level quality.
You go watch “Cars,” and when they’re slaloming through the prairie, the dust they’re kicking up is pretty near photo-real, but more importantly, it’s realistic looking, but they’ve been able to art direct it to their needs. And that’s exactly what you have in visual effects -- you want an explosion, but a particular kind of explosion. You want it to look real, but the director has to be able to art direct reality.
SM: The misconception may be that in an animated feature, because it’s not necessarily photo-real, all that detail doesn’t need to go into it. But it does. It’s just stylized.
SK: So what differentiates live action from CG, if you look out a few years? Is it just that one has a more fantastic aesthetic, and the other is slightly more grounded in looking out the window and seeing, say, a palm tree.
SM: There’s this phenomenon called the uncanny valley. As you make a CG character look more and more realistic, it gets more and more appealing, until it goes into this dip, where it falls just short of being realistic. It gets really creepy. I think we are a company that wants to straddle the uncanny valley. We want to be completely realistic with our CG visual effects for feature films, and we want to completely author a very stylized world for our CG animation. Genndy [Tartakovsky’s] work is extremely stylized, and part of what makes it so cinematic is that it has such a unique signature style to it. There’s no place in Genndy’s style for creepy CG, semi-photo-real humans. So I don’t think anyone would ever confuse a Genndy movie with a live action movie that was just more fanciful. I don’t think we’re going in that direction, of making a movie where we’d be confusing the borderline between live action and animation.
SK: In terms of your core, live action effects business, what has changed between 1999 and 2006, aside from more powerful processors and Moore’s Law? What’s different about the kinds of things you’re asked to do?
JR: Well, they’re always harder, and bigger. That’s the funny thing about our business – the machines always get better, the tools get better, and the things you did last year are pretty easy to do this year. However, this year, that’s not what somebody wants. They want something much larger that’s going to push the envelope.
SM: I think that’s Ed Catmull’s Law. [Catmull is a founder of Pixar.] As Moore’s Law increases, you just make up for it by trying to do harder and harder things. He is one of the ones you hear quoted, saying that render times have remained a constant.
JR: Over the years, I’ve never felt we had a glut of render power.
SM: You’re always just riding up against that threshold, of just barely being able to see your shot overnight.
JR: We push the envelope to do more with the tools we have. Before, there were things you wanted to do, and you just couldn’t.
SK: It feels like there always is that edge that people teeter on, in terms of wanting to do a shot that people haven’t seen before, and not wanting it to be too conspicuous, where people say, that character, or that explosion, is a visual effect.
SM: We call that work. (Laughs.) We call that showing up to work in the morning. There’s an interesting tendency, this odd thing that CG seems to have initiated in film, which is a real willingness of people to over-extend their capabilities. With the first computer-generated effects in movies, we were just so excited to see them that it didn’t matter that they weren’t that good. “Jurassic Park” was an exception – it’s still great by today’s standards. But after that, the floodgates opened, and it was almost like the audience didn’t mind looking at bad CG, because it was new, even if it wasn’t photo-real. The audiences are over that, but somewhere, the word got out to directors that anything’s possible. Our industry is a little bit a victim of its own success. You work the miracle on one show, and everyone expects the miracle on the next.
JR: The films that we think of in the past as the major visual effects films, that really pushed our industry forward, have just a miniscule amount of visual effects work by today’s standards.
SM: Jurassic Park had 35 CG dinosaur shots in it.
JR: “Superman Returns,” which I just finished, had 1500.
SM; “Superman” probably had 35 dinosaur shots in it that were just cut.
JR: (Laughs) Yeah, that sequence went away.
SK: It seems that there’s a lot of tendency now that if something doesn’t work out on set, or doesn’t look the way the director envisioned, suddenly that becomes a visual effects shot.
SM: (Incredulously) No….
JR: Yeah, absolutely.
SM: No one would go into the production of any movie today without some idea about incorporating visual effects. It’s officially ubiquitous, at any budget level.
JR: We did our own small movie -- a romantic comedy. We were excited for it to have no visual effects in it. It’s called “Griffin and Phoenix,” starring Amanda Peet and Dermot Mulroney. Sure enough, we had to do some cameraman removals, a little window help —
SM: -- Making the lights on the Christmas tree twinkle a little more. Why not? You’re doing a digital color correction, and your files are already there.
SK: Lately, it seems to have become the Liquid Paper of the movie industry.
SM: There’s definitely that. But it gets back to that thing of art directing reality. It used to be that if you wanted a particular type of sunset, you had to wait. Now, you don’t. Now, you can retroactively apply whatever sunset you’d like.
JR: We worked on “50 First Dates” – not what you’d consider a visual effects film. But we did some green screen car comps, because it was much cheaper to shoot [on a green screen stage] than shooting on a bridge. There were these golf ball shots, where Adam Sandler is hitting a golf ball, and all these kids are hitting it on the green. Instead of going out, and trying to get the golf balls to land exactly where they want, just do it as a CG shot, and save yourself some money, and direct it exactly the way you want.
SM: In “X3,” major sequences take place in San Francisco, and not a single thing was ever shot here, except for a couple of plates. There’s definitely a certain type of visual effects that’s making it more and more OK to shoot your whole movie in Vancouver and call it New York.
JR: “Superman” was shot in Sydney, which stands in for New York City. There are hundreds of visual effects that pass by, and you don’t even realize they happen. Those are “helper” visual effects, where it’s easier and more cost-effective to do it later on, and the director has the opportunity to art direct it like he wants. It has helped the filmmakers manage their films’ look better.
SK: I’m fascinated by the new competition in the industry. You started The Orphanage in the era when competition was still pretty scarce. It used to be that if you had an effects movie, you could only go to Industrial Light & Magic, or Digital Domain, or Rhythm & Hues. My sense now is that it’s gotten more competitive, because you’ve got five kids in a basement who just graduated from University of Illinois, and they’re willing to underbid like crazy to get work.
JR: That’s definitely reality, and then you have the Canadian companies, where the Canadian government gives tax subsidies for having your shots go up there. It’s an extremely competitive field. You search to find the sweet spot where you can position yourself, so that you’re reducing the field for your particular kind of work. Our goal is always to try and reach up to the ILM’s and the Sony’s and the DD’s and the Rhythm + Hues’ of the world, because those guys are on a certain playing field.
The difference for them is that in the old days, they used to take on an entire show, and do every shot, from the most difficult work down to a simple paint-roto shot. Now, they do the most difficult work, and the other stuff goes elsewhere. But the fact is that there are only a handful of shops that can get a certain kind of shot done —
SM: -- In a certain quantity. We’ve always tuned our size so that we can take on a sizable chunk of a movie, and yet not be at an uncomfortably large size, if the work ebbs or flows. We’ve very specifically set out to differentiate ourselves by quality, and to be able to drink from that firehose when it does come. It’s in no way easy, but we did go out and actively try to do it.
JR: We also like to have a diverse set of incomes, as far as doing commercial work, visual effects, and production. We do feature visual effects, and now animation. We’re trying to diversify our portfolio, so we can move the pieces around, and never feel any valleys during the course of the year.
SK: On the commercial front, looking at some of the stuff that you’ve done, it seems that maybe ad agencies are looking to visual effects to dial up the “wow” factor for a commercial. In the old days, you didn’t see visual effects in commercials, except for the Super Bowl. Is that a function of TiVo, where commercials now have to be bigger and faster and more visually impressive to get people to pay attention to them?
SM: I definitely think there’s that. You’re hoping to get that commercial where people will want to point it out on YouTube to their friends, as opposed to trying to fast-forward through it. But the other thing that has happened is kind of similar to what has happened to feature visual effects, which is that agencies know now that they can get a pretty impressive commercial for a fraction of what it used to cost. When I first started, it was not unusual to work on a commercial that had a million-dollar budget, and now that’s unusual. And the degree of complexity has just skyrocketed. It’s another funky struggle. There’s this high echelon of really amazing commercials out there, where people are sparing no expense and trying to wow everyone, and then there’s this playing field of commercials that have challenging visual effects in them, but people have started to take it for granted that they can get something like that for a fraction of what it used to cost.
SK: People passing around commercials via e-mail, or sharing them on YouTube, makes me wonder whether we’re getting into a world where commercials really have to live or die based on whether it’s an entertaining minute of video.
SM: We love that.
SK: At some point, why even buy the commercial time?
SM: I just directed a Navy SEALs commercial where seeding it to the Internet, and having it be something that people wanted to download, was an active part of the conversation. Yes, we’ll buy airtime, but we expect people to share it with each other.
JR: I have TiVo, and I certainly stop on the commercials I find entertaining, and race through the ones I don’t. I find that well-done, entertaining, story-type commercials are just as entertaining as the show they’re in.
SK: It does seem like maybe we’re seeing some evolution, where there are fewer commercials today where you’ve got the housewife standing in front of her washing machine, talking about Tide.
SM: That’s only a good thing. I was watching the World Cup, and it was almost like the level of attention in the room increased during the commercials, where you have the little Italian kid running around in a field, pretending to play with all the great players. You love watching those spots, and you look forward to seeing them.
We’re just starting to see the beginnings of the shake-up. Still, on the client side, the big, corporate clients live in a world of PowerPoint: “I would like to communicate the following message to you about Advil.”
JR: They want a big logo in the middle of the screen.
SM: Bless them, ad agencies are trying to warm them up to the idea that you need to zig when you think you need to zag.
SK: The other idea that’s interesting is when advertisers let their customers submit their own video, or direct their own commercials. Or when Richard Linklater invited people to edit their own version of the trailer for “A Scanner Darkly.” Do you feel like that’s a legitimate new direction, where you have consumers engaging more with the content?
SM: One of the things that we do at The Orphanage is that we package up some of our technologies and we sell them through a sister company called Red Giant Software. That’s a community of people who are out there with their FinalCut Pro systems, who are very in touch with one another. A product lives or dies based on its word-of-mouth in that world. And I became really aware of how, if I go out and just snoop around in that world, and observe what’s going on, and then every once in a while pipe up and try to answer a question or solve a problem, I’m viewed as this outsider looking in.
So what I did is I started a blog, and became a more active part of those things. Suddenly, there was this friendly thing – a warm embrace of what we had to offer, because we were perceived as giving back to the community. I think people are slowly learning that. Microsoft lets their employees blog.
JR: Certainly, blogs and podcasts are opening that up. You’re finding the people out there who are capable of doing it. “Four Eyed Monsters” is fantastic. They’re so great.
SK: I haven’t seen that one.
SM: It’s this couple who made a film that played at Slamdance, and they’ve been making video podcasts about the trials and tribulations of making and then releasing their film.
JR: It’s very engrossing.
SM: More people will see this podcast than will ever see their film. But they’ve worked out a system where you can go to their Web site, and add your name to a list, and when it gets to a critical mass, they will put a screening in your area. They can go to the theater, and say, “We have this list of this many people who say they will show up, and as we’ve seen from these other cities, 60 percent of them actually do show up. Can we have a screening?” It’s kind of a no-brainer for the theater to say yes. They’re side-stepping distribution, by just distributing it themselves in this very grass-roots way. But they’re doing it by feeding free entertainment to the world on their iPods, which is super cool.
JR: It happens to be incredibly entertaining, in four-minute spurts, which is why it works.
SM: It’s like open source software. If you give it away, somehow, you’ll get it back.
SK: The Hollywood creative community seems to have a hard time with that. With open source, the community is helping to build the product. When you float that idea by someone in the movie industry, they’re not so keen. Could a director imagine a scenario where all the raw footage of a movie is available, and someone else would be able to make their own cut? Maybe their 90-minute cut would be better than the director’s 120-minute cut. That idea of letting other people contribute is scary.
JR: Yeah, you have a level of ownership to what you’re doing. It’s a scary proposition to release that out for other people to take hold of.
SM: Some people want that level of participation, and a lot of people don’t. Asking someone to come see a movie, and pay $10 for the privilege, is like, “OK, dude, I’ve got this great story I’ve got to tell you. First, give me ten bucks.” Then I tell you the story, and I get to the end, and ask you, “How do you want it to end?” “Why are you wasting my time? You don’t even know how the story ends.” You want someone to grab you by the lapels and tell you a story.
SK: But a different scenario is that you’ve already paid the ten bucks to see “Superman Returns.” Maybe after the theatrical run is over, it’s just open season on the movie, where anyone can do their own cut. I might do a cut that’s just twenty minutes of my favorite visual effects shots.
SM: But I don’t think that’s what’s happening with the “Snakes on a Plane” phenomenon. It’s different than people wanting to re-edit their own version of the movies. People aren’t all closet filmmakers, but everyone has an opinion. What happened with “Snakes on a Plane” is that it was a temporary title, and they were going to rename the movie. But somehow it got out that the working title was “Snakes on a Plane,” and that’s funny. Then, someone makes a little movie poster with Sam Jackson’s face, and suddenly you have a hundred people saying, “I can’t believe you would call it something else.” Then you had Sam Jackson saying that.
SK: I don’t want to focus too much on remix culture, but I do think that you’ve seen some interesting examples, like when someone did their own edit of “The Phantom Menace” and took out Jar-Jar Binks. There are some interesting, playful things that people do with movies that could give them life after the theatrical release, in many different versions. What if you could get someone to buy two different DVDs from the studio?
SM: I think you’re onto something there, but I don’t think it’s going to be limited to re-editing. If you amalgamate all those phenomena, like “Snakes on a Plane” and General Motors letting you edit your own version of their pick-up truck commercial –
SK: -- And all the anti-global warming people started to make all these mocking commercials.
SM: But if they hadn’t, it wouldn’t have gotten any attention. So for a while, they made the active decision not to pull those things down, because they were getting a ton of traffic.
You’re right to call it the remix culture. But with “Snakes on a Plane,” or cutting Jar-Jar Binks out, someone is making a point. It’s not going to be a version of “Superman” where someone is going to come back and say, “I think this is a better version.” Someone is going to find some way to edit “Superman” so it has a gay pride message – which maybe wouldn’t be that difficult. They’re going to find some way to make a point with it, and that is where Hollywood and ad agencies and content owners are going to have to decide if they want to try to stop people from making a point using their tidbits.
JR: My guess is, knowing the Hollywood community, that their first choice is going to be to not let someone use their material to make their own points.
SM: But if you clamp it down – if GM had clamped down on the environmentalists – you instantly turn them into martyrs.
SK: We’re in a world now where the machinery of moviemaking is increasingly accessible, and you can make movies for under $1000. I wonder if you might see more interesting creative ferment in that world. On the flip side of that, you have the studios saying, let’s make fewer movies, and spend more on each one.
SM: The whole formative impetus for The Orphanage was that accessibility – the collision of high-quality digital cameras that you could buy at The Good Guys, plus off-the-shelf hardware and software that let you do ILM-quality visual effects.
JR: Or doing editing on my Mac on the airplane flying to New York.
SK: Given all that cheap technology, and the inventiveness and off-the-cuff-ness you get in that world, it sometimes seems that the established movie industry is less and less able to do those kinds of clever, fresh things.
JR: The nice thing about the Web now, and things like YouTube, is you can actually get access to [all kinds of small videos and movies].
SM: I’ve been thinking a lot about those iFilm and Atom Film sites. iFilm is the one that has tried to swing into the YouTube mentality, but they still put a lot of obfuscation between you and the content, whereas YouTube and Google don’t at all.
The only thing people seem to be willing to pay for is stuff that has pre-sell. Like, I missed that episode of “Lost,” so I’ll download it from iTunes. Or, I remember vividly all those great episodes of “Knight Rider,” so I’ll download one for $1.99 to watch on the plane.
For my part – and I’m fully willing to be proven wrong – I feel like it’s going to exhaust itself a little bit. I think there will always be some little clip of something that’s so funny you want to show it to somebody. No one’s ever going to want to pay for that. I wouldn’t send you [the popular YouTube video] “Evolution of Dance” if it cost you 25 cents to watch it. I don’t think that’s going to change. But there is a movie that’s been giving itself away for free in ten-minute segments, and if you want to pay, you can get ahead in the story.
I’m very excited for the distribution barrier to break down.
There are some filmmakers out there who are doing short science fiction things — there was a “Star Wars” fan film where people all over the world, who’d never met each other, were contributing visual effects work to it. You can go on the Web site and download the box art, which you can print out yourself. You can burn a DVD. George Lucas lets people make those movies, as long as they don’t make a profit, so they have to give all this stuff away. They got a lot of attention, because they blogged the whole thing. They posted the footage they shot. “Look, we got 30 Storm Troopers to show up, because we put a thing on the Internet saying, if you’re a Storm Trooper, show up.” People followed it along, and seeing the final product was like the end of a long process of seeing how the thing was made.
Peter Jackson picked up on that same thing with “King Kong.”
SK: Do you see other changes happening on the Internet, given the increasing prevalence of video and film content on the Net, and more people having portable devices that can show video?
JR: The cool thing about the iPod is that it’s a really cool way for people who travel, or are busy at work, to have a chance to watch stuff. But to be honest, I still like my big widescreen TV at home.
SM: There’s really stupid stuff happening now, where you can spend $20 to download an encrypted, low-resolution version of a movie that’s available on DVD for $21, or $16.
I love my video iPod, and I didn’t think that I would. I’m a complete crack addict with it.
SK: What sort of stuff are you watching?
Stu: I’m listening to a lot of podcasts where people are hypothesizing about the exact thing we’re talking about now. To me, that’s why I’m willing to predict that it will kind of blow over. Remember back in the mid-1990s, with virtual reality? Everyone knew what it was supposed to be. We had seen “Lawnmower Man,” and we’d seen pictures of people with goggles, and we knew that there were high-powered computers somewhere that would kind of make it possible. But everyone was like, “It’s all about virtual reality. We just have to wait until the computers get there, and it will be awesome.” Disney said, “We’ve got the money. Let’s do it.” They built this Aladdin magic carpet ride. They got a supercomputer, and you’d put goggles on, and sit on a Persian rug, and hold the edges, and if you pulled this way or that way, you’d fly. They did it. They proved that it was do-able. And then what happened was, it didn’t happen, and we kind of forgot about it.
Then, completely quietly, without anyone even realizing it, [the videogame] Doom came out. It was virtual reality, without the goggles. You realized that we got distracted by the goggles. If you put some headphones on, and turn off the lights in the room, and play Doom, it’s way more virtual reality than what [Disney’s] supercomputer was doing.
I think that’s what going to happen here is, we’re all going to be talking about how to monetize Google Video, and then the thing that is actually the thing is just walking right past us, completely unnoticed.
SK: While we’re talking about videogames, a few years ago, you’d talk to people in the movie industry about interactivity. They’d tell you that people don’t want to pick an ending for a movie, or tell you what’s going to happen in the next reel. And yet, while they were saying that, the videogame industry was growing like crazy, and it has kind of turned into interactive movies – these almost cinematically-real environments, where you’re controlling your path through a story. A game like Grand Theft Auto is better than CG-animated movies were fifteen years ago.
SM: I’m hooked on Grand Theft Auto. But I hate actually playing the game. I just like going around in the environment, and seeing what kind of sh** you can cause. I really enjoy that. But the actual game -- I can’t be bothered. What I really hate is the part where I go into a room, and it fades out, and I know I’m going to be in for watching fifteen minutes of bad mo-cap animation, and no lip sync.
Games make more money than movies, but that doesn’t mean that they are movies, but better.
JR: There’s a reason games have clung to two or three different media. You’ve got your sports games, your car games, your kill-everyone games.
SM: With certain kinds of entertainment, like immersive worlds, games will let people do that better than movies ever could. “Star Trek” has had years to create this very pervasive world, and a whole community people who gravitate towards it in this Renaissance Fair sort of way.
Movies used to be your only recourse, if you were into swords and sorcery. You just had to wait for someone to make the next epic fantasy movie. But I wouldn’t confuse that with the general [idea] of movies being subjugated by videogames in general.
JR: There’s definitely a different level of appeal.
SK: But if you want to talk about young men, who are so attractive to studios and advertisers…
JR: There’s definitely an age group. Things like [the videogame] World of Warcraft are time-sucks. They’re immersive, and it takes ten hours to go do stuff. You need lots of time.
SK: I wasn’t suggesting that games will replace movies, but they are that sort of more interactive form of entertainment. And there still is a separation of the studio making the movie, and someone else making the game based on the movie. And they’ll cross their fingers and hope that it works out. Then, on occasion, you have a game company licensing the rights to make a movie based on something like World of Warcraft, and hope that it works out.
JR: More often than not, the films that license the rights to the game companies, those are usually not the games that people fall in love with. The only time I’ve seen it be extremely successful are the James Bond games, where they take the character and gave him missions, and they did a really good job with that. You don’t often see people loving the Superman game or the Spider-Man game. They’re trying to force you into that movie, and I don’t think people enjoy the experience as much.
I think the thing that people love about World of Warcraft is that they create these characters that are 100 percent their own. They own them. They decide what they’re good at, what they’re not good at, what they can do, what they can’t.
SK: Do you guys have a vision of at some point building a games business here? The LucasArts people have been saying for years that there are some synergies in sharing assets between visual effects and game development.
JR: For us as a company, it’s about stages. We find games interesting. A lot of us like games, and understand that there’s money to be made in games. It’s out there, but we’re just trying to handle the things we’ve got in our vision right now.
SM: If we were to do it, we’d approach it the way we’re approaching animation, which is with a huge amount of respect and deference to the people who actually know what they’re doing in that world. I believe that there are good games, and bad games. The good games are good because of good creative leadership. A really good friend of mine was working for a company called Double Fine. They had this really cool, unique game called Psychonauts. I don’t know if it was hugely successful financially, but it was really well-reviewed. It was on lots of ten-best lists for the year. I think we have too much respect for it to cavalierly say, “We’ve got computers, so we should be doing that, too.”
JR: You need to understand that good work doesn’t come easy. It takes certain skill sets, and you can’t just assume that because you’re good in one industry, you can roll into the next one, just because you have the assets you can share from visual effects.
SK: But you do hear that “convergence” word a lot, with regard to visual effects and gaming…
SM: Except, what are you making? How you make it, absolutely there’s convergence. But game design is a whole thing. It comes down to gameplay. You can have the best graphics in the world, but gameplay is the game world’s version of story. When we talk about Orphanage Animation Studios, we won’t shut up about how story is the most important thing. Everyone is sick of hearing it. And Pixar also won’t shut up about it, and they’re living proof that it’s absolutely true.
JR: Rocket Science [Games] is a perfect example. They did this game called Obsidian. They were this gaming company in the Bay Area, and now most of them are in the visual effects industry. It was this company that had a ton of cash, and they made this game Obsidian, which was just beautiful. They made an incredible-looking game, with beautiful artwork. But the game sucked, and nobody wanted to play it.
Without the gameplay, you’re lost.
SM: I would also point out something else about games. Something really interesting is happening right now. There’s a console war between Sony and Microsoft, and Nintendo is going to win it, with less technology and more fun, more silly games – games that are much less like movies, and more like games. You watch. [Nintendo’s] Wii is going to blow up, and the PlayStation 3 is going to fall on its face. It’s in trouble.
SK: While we’re talking about conflicts in the marketplace, what’s your read on the high-definition format wars? You talk to the HD zealots, who tell you that absolutely everything is going to be in HD.
SM: It will, but you won’t be getting it on anything round or shiny. You’ll be getting it through a dedicated or not-so-dedicated box in your living room that’s hooked up to very fast pipes. I directed a commercial for Toshiba’s HD DVD player, so I should be telling you that HD DVD is going to win, but…
JR: I have had the feeling that Blu-ray missed its opportunity for sure, and the PlayStation 3 is really in trouble, given that they’re trying to charge something like $600 for the console, which nobody is going to pay for, and Blu-ray as a disc is not really ready for prime-time. I’m sure you’ve heard the comparisons between Betamax and VHS. [HD DVD] was first-to-market…
SM: I have the HD DVD player, because I directed that spot, and it doesn’t seem to work.
I’m a DVD collector. That’s part of my job. I love the commentary tracks, and the supplemental materials. I watch the commentary on the worst movie ever, in hopes of gleaning some cool piece of information about it. But I get all my DVDs through Netflix. If I like a movie enough, I’ll buy it, but inevitably, I wind up buying movies more because I need them for reference for some visual effect we’re trying to do.
Netflix is awesome. I think nothing of returning a movie one day, and maybe getting it back two days later. Right now, the turnaround is that I return a movie on Day One, and get another on Day Three. I think that, a year from now, I could download an HD movie in that period of time, to my TiVo or whatever it is. Netflix is even rolling out a beta of a dedicated box that you would have hooked up to your home theater set-up. They were very smart to not call themselves DVDs-By-Mail. They’re the first ones to tell you that they knew they weren’t always going to be reliant on the postal service.
JR: When it comes to those DVD collections, moving into HD there’s the frustration of buying all new ones, like when I went from Laserdisc to DVD.
SM: That’s the thing, rather than buy new DVDs, I would so much rather Netflix them, and I’d prefer to digitally Netflix them, rather than Netflix them [by mail.]
With videogames, I get them through GameFly, which is like Netflix. If you hold onto a game long enough, and feel like you should buy this one, you just don’t return it, and they charge you a used price for it. Sometime in the next week or so, they send you the box it came in, so you have that. It’s kind of effortless.
SK: Do you think that there’s still going to be a place, in five or ten years, for model-making and optical effects, stuff that’s hand-built or hand-painted?
JR: I hope so.
SM: Miniatures, for sure, for the same reason that, in 2050, there will still be a 9-millimeter pistol: because it can kill someone. You see the sci-fi movies where everyone’s shooting lasers, and you go, a gun would still work, and it’d be cheaper than a laser gun.
JR: Some things are still cheaper. It’s really difficult and expensive to make digital explosions, and explode a car, and have it go into a million pieces and flip over, and have the tire fly off. It’s better done with models, and it looks better. I don’t think that some of the [digital] work we do is at the level where it replaces the model. More often than not, the effects I see that are some of the best ones out there have models incorporated in them.
SM: People herald “King Kong,” for example, as this victory of CG, because the title character is so expressive. But all of the jungle environments that they’re running around in are really elaborate miniatures.
JR: Or “Lord of the Rings.”
SM: We’re seeing a really great emergence of the proper understanding of how miniatures and CG relate to one another.
There’s no longer a differentiation between what’s a matte painting, and what’s a miniature. You start with some photography of the live action, and some of the miniatures, and some painted pixels – which may be a photo-collage from some aerial photo shoot you went on. What’s a matte painting anymore? We don’t even really use that term anymore. We call it a digital environment.
JR: Given the age we’re in with visual effects, where everything has to be so photo-real and undetectable, it’s almost necessary to start from a photograph as it is. All of our matte artists, the first thing they do when they have a shot is they start running around with a camera for the first two days, taking pictures to support the shot they’re doing.
SM: We get a reminder sometimes that we can kind of outsmart ourselves. With “Sin City,” we had a bunch of splattering blood we were trying to do. After two weeks of fluid dynamics development of splattering blood, the guy who was putting the final shot together and the CG supervisor said, “We just need some reference for what this should look like.” They went out in the parking lot with a bowl of milk, and started splashing it. The milk was white enough that they could get a good key off of it, so the compositor put that in the shot, just to see how it might play. He put it in dailies, and we all said, “That’s the best fluid simulation we’ve ever seen.” That afternoon, we were out in the parking lot with a bowl full of eggnog – this was around Christmas -- and a hockey puck. Throwing a hockey puck into a bowl of eggnog became how we splattered the Yellow Bastard’s blood all over the barn at the end of “Sin City.”
It was so much faster and so much easier. It was real-time. It was in the shot that day.
When we worked at ILM, Jonathan needed a waterfall element for a shot in “Star Wars,” and everyone wanted to do it some complicated CG way. I had just gotten my digital video camera. It was a beautiful day, and we took our lunch hour to go down to the Golden Gate Bridge and shoot the bay, and comp it in. We got yelled at.
JR: But nobody bothered to argue about the shot anymore. Realism works.
SM: Dirty tricks work.
JR: The thing I talk to artists about here is, that’s a part of filmmaking. They do it on the set everyday. Tricks with lighting, tricks with sound. Why should visual effects be any different?
SM: For “Sin City,” we had the world’s hokiest miniature stage set up downstairs. The room was blacked out, we had a couple lights, and a table covered with baking soda. If you spritz it with water, baking soda gets this crusty shell on top. They had little action figures, and they’d walk them through it – and it looked like perfect little footprints in snow. Then they’d take digital photos of them, and map them onto CG surfaces. That’s how all the snowscapes in “Sin City” were done.
SK: Can we talk about pre-viz. Do you guys do pre-viz work?
JR: Pre-viz has become its own industry now. We don’t typically do it here, and it’s not from a lack of interest. I did pre-viz at ILM. I basically started building that department there. Two of our junior supervisors here were part of the pre-viz group for George, up at the ranch. But now you’ve got these companies that are devoted to going on set, and going on location. They’ve got a pipeline, and a system built up for it, and they have a reputation for doing it, and a cost structure that works. It’s hard to fold a visual effects company into that design.
ILM had a much larger [pre-viz group], and I think they shrunk it down.
SM: What’s interesting to me about pre-viz, is how I think of it as a director working on commercials. I always do pre-viz working on my commercials, and I can do it myself, with whatever rudimentary knowledge of computer graphics I have. I can do my own animatic that’s pretty cool looking, and I think that should only get easier. There’s a storyboarding program out there now, and if you buy it, it comes with what looks like a PlayStation controller. They realized that that’s something you’re used to doing. I can play Grand Theft Auto, and walk my guy over to the car, and he’ll open the door, and get in, and suddenly I’m driving the car. Any three-year-old can do that, and yet to animate that in a pre-viz, you’d wind up with this horrible animation, and it would take days.
SK: Do you see more pre-viz happening for more movies? It still seems like it’s stuck in this place, where it’s just for the big effects sequences of the big-budget movies.
JB. Absolutely. Certainly, on “Superman,” I saw every bit of pre-viz on that film, and it was not just the effects sequences. They didn’t pre-viz every scene, but they did a lot of talking scenes, and simpler scenes that didn’t have blown-out action and visual effects. They were working out a lot of blocking.
SM: There have been cases of people going in to try to get someone to give them their first directing gig, and they’ve gone in with storyboards for the whole movie. You could show it to the executives, and it takes a big part of their risk aversion away.
I’ve heard of genre filmmakers, making low-budget horror movies, who are using this FrameForge [3D] Studio to storyboard every scene. They’re using it as part of trying to get funding. FrameForge has a PlayStation controller with it.
SK: Is that what you use?
SM: I use SketchUp. But if I could do my pre-viz with Grand Theft Auto, I would do it. My favorite game on the PlayStation is Driver. It’s all based on “Bullitt.” You’re driving around San Francisco in this big American muscle car, and after you have your chase scene, you can go into film director mode, and you can direct the chase scene – move cameras around, and edit it. To me, that’s my idea of a good time.
It wouldn’t be crazy to think that someone could go into a meeting at a studio, and show them that: this would be the chase scene at the end of my movie. And I’ve pre-vized it in PlayStation with a bag of Doritos in my lap.
There are all these Half-Life 2 mods, where people take the Half-Life 2 game engine and tune it to do these crazy, weird things. Every once in a while, I pester the guys here who are really into games, and ask them, “When am I going to be able to pre-viz my movie using Half-Life?” Half-Life is [in high-definition], the simulations are great, and the explosions look great. If I could just spend an afternoon painting texture maps for my characters, so they looked how I wanted them to look, and take a few pictures from the location scout I just went on, and map them onto that environment, and spend the rest of the day crashing cars and doing whatever else the scene demands, I could imagine putting that on a DVD, and going into a room and saying, “Here’s the first fifteen minutes of the feature film I want to do,” and having a bunch of people vividly gripped in entertainment.
JR: The truth is, it can be an incredibly entertaining tool. The Spider-Man pre-viz we saw was a board-o-matic -- it wasn’t even 3-D. When we saw that, nobody wanted to move. It was the first chase sequence in Spider-Man. It was purely 2-D, but the story was coming through completely. It was amazing.
Getting a movie made, or trying to get your idea across to the studio – pre-viz is a perfect tool. I don’t see any reason why it wouldn’t grow.
SK: You’ve talked about doing features, like “Griffin and Phoenix.” Is that a future direction, developing more of your own feature projects?
JR: It was the initial idea of the company.
SM: It’s not a future direction. It’s just that we finally had some success last year.
JR: It’s a hard direction. When we started the company, we were just kind of mimicking George and his Lucasfilm idea, which is feature production, visual effects, and technology as three legs to the company.
SM: And your visual effects division cuts its teeth and stays current by working for whoever can pay, but they’re available to you as your own personal backlot for moviemaking.
JR: And it develops the technology that you go and sell later.
The production side has been the most difficult to expand on. It’s something we’re interested in, and are trying to grow, but it’s a tough business.
SK: If you think about what’s not possible today – or barriers you find yourself running into – what are some of the things you think will be easier in five or so years?
JR: Well, you constantly find ways to make this stuff easier and faster, but then we constantly come up with new ways to make it more difficult. It’s an endless chase. You want that, because nobody really wants to be doing the same thing they were doing three years ago, and the audiences aren’t as interested in seeing it. You’re always trying to find the newest edge for what you can do.
What we aspire to be doing is what Weta Digital and ILM and Sony are doing with digital characters and digital humans, which is really pushing the envelope: King Kong and Gollum, and Sony’s digital Superman. The digital human is probably the furthest away, because we all know it so well, and it’s so hard to reproduce. There are definitely scenes where I know a character is digital because I saw the dailies. But there are shots that go by where I don’t think most people know it’s a digital character.
SM: We don’t sit around and think about what we would like to have, because it’s such an all-consuming task to think about how to get done the work you have in front of you, given the tools you have now.
Whenever I think about a tool that would make my life easier, I always think about the limitations that it would impose on me, and how I wouldn’t be satisfied. We’re always going to be the Formula One team, where you wouldn’t think of racing the car without the guy who built the car waiting in the pit to fix the thing that broke. You have to be at that bleeding edge to stay competitive.
I can’t help but feel like there’s going to be a time where a certain type of visual effect will be somewhat akin to that Grand Theft Auto experience – when I drive the car towards the thing, the car crashes, and it looks more or less like a car crash. There could well come a time when that would be good enough for a car crash that maybe isn’t in the foreground of a shot. There might be a visual effects toolkit where it’s more pre-fab.
SK: That’s an interesting idea, that if you’re trying to make a sub-$1 million movie, and you have a car that needs to do a Thelma-and-Louise thing over a cliff.
SM: If you can set up a play in Madden football, then you probably know as much as you need about the PlayStation controller to set up that visual effects shot.
JR: This is why the tools always need to move forward. You’ve got a car going over a cliff, and then the director says, “I wanted the wheels to wobble.” And you say, “Ooh, that machine doesn’t do that.”
SM: It’s almost like the system is designed to prevent anything from ever making our lives easier. As soon as you get competent enough with fluid dynamics to do a raging torrent of water going down a street in New York, then someone is going to say, “Sweet. Now I need a tidal wave to knock over an ocean liner.”
JR: Directors never want to do the same thing that somebody else has done, so they’re always going to push you to do something different.
SM: There are a couple specific things I think you’re going to start to see. You’re going to see in-camera motion capture characters. So you’ll be able to put an actor in a scene, and the motion-track targets won’t be visible to the camera, but they will be visible to the sensing device. You’ll be able to put a digital prosthetic on them, or replace them with a CG character, and yet you never had to shoot a clean plate, because the person was really there.
You’ll see some pretty amazing technologies, in terms of when a computer is able to track in real-time an entire scene – every pixel of the motion, so that you’re building a computer version of it – which will let you age a character digitally over time. You’ll start to see a lot of directors using high-definition monitors, where there’s a rudimentary version of the visual effect already happening on it. So when you pan the camera up from the girl to King Kong, you’re framing King Kong into your shot, not a bunch of plusses on a green wall.
What has been a motto for us is keeping visual effects out of post-production. They’re thought of this thing that happens later. They really need to be a big part of planning and a big part of production.
Historically, a director would need to be an expert in visual effects if they wanted to push visual effects forward. So you’d have the Zemeckises and the Spielbergs who’d get inside it and understand it, and they’d know what they were doing when they pointed the camera at some empty space, and they could infuse their creative energy into it while imagining what was going on.
Going forward, [directors will be able to] produce good results if [they’re] not a visual effects expert, using some of these tools that let people intuitively use the tools of movie-making to design visual effects.
JR: I’ve seen directors who understand visual effects and those who don’t. It’s difficult for them to visualize something being there that isn’t, or a green-screen set.
SM: For me, when I get on a set with a crew that’s used to working in a particular way, I’d often myself saying, “Guys, give me some room – there’s going to be this thing in the frame.” Now, when I direct a commercial, I have this stand where my laptop goes, and it’s always going, looping some version of the pre-viz. Or I’ll feed it to the playback operator, so that he can be cutting in or overlaying shots. I’m trying to find these ways to communicate to the crew or the actors.
SK: Do you think there’s going to be a continuous line, from the pre-viz stuff to the final effects shots? Today, it seems that there’s this gulf between the cheap-and-dirty pre-viz, and you start from scratch with your final effects shots.
SM: Eventually. We’ve been battling this since ILM days. People would say that an animatic for a space battle looked pretty good. But if we didn’t start from scratch, we’d be putting our train on a certain set of tracks.
JR: And you’re in a constant state of Band-Aids. Your hands are tied in some ways. Currently, the way pre-viz is designed is to get it done quickly, and show a lot of versions. You don’t want too much detail, because it can be distracting from what you’re trying to get across. If you just try to push that into the visual effects shot, it is already built for something different. So you’re finding yourself constantly trying to retool what’s already there. Plus, there’s inherent laziness in pre-viz. It’s not a bad thing. It’s just the nature of the game.
SM: No matter how responsible you’re trying to be, you’ve got the director standing over your shoulder saying, “If only that was a little bit smaller.” And so you scale it down, but you’ve created something that breaks its prospects for directly feeding it into the pipeline. The visual effects artist has to be the responsible adult, and say, “If you want it smaller, it should be a little farther away, and a little higher, and a little further to the left.”
SK: It sounds like you’re talking about the difference between really adhering to the building code when you’re building something to last, versus building a set for a movie.
JR: It is very much like that. You can break out the staple gun when you’re doing pre-viz.
SK: It seems like a relatively new trend, with big-budget summer movies, that a studio will chop it up and farm the visual effects out to three or four different places.
JR: That is the norm. That is the standard. It’s relatively new. Even a place like ILM, which could command an entire film, or Sony, which is now the biggest facility out there – they can’t dictate that that happens. The standard now is that the studio signs a VFX supervisor to a production – not from a facility – to run their show. Even ILM, if their effects supervisor is going to be the effects supervisor for the show, they are actually loaned out to the production, so they can supervise the other facilities as well. It’s just the economics of visual effects, and I don’t see it going any other way.
SK: Is that necessarily good or bad?
JR: Films are inconsistent, as far as the effects. You’re trying to find what the different facilities do best, and you don’t always get that. Some shots look rough, and some look spectacular. That’s one of the bigger drawbacks. Plus, there’s a strain on the process of production.
But [the studio] can bid a bunch of shops against each other. Everyone’s price point is different, and you know what you can get from each facility.
SM: It’s horses for courses. You know that you can only afford the big guys for this hero sequence, and you may have a bunch of wire-removals that you can give to somebody else.
SK: It’s interesting that you guys are in the renovated building here in the Presidio, and ILM is over in the new building.
JR: (Laughs) We love our renovated building.
