EddieOffermann dot com

I just read that Ubisoft bought Hybride Technologies - a large video game company buying a medium-size visual effects house.  The article had a lot of talk about the future of video games and movies and seemed to miss the point when it quoted recent reviews of "Beowulf" that said it was like a "long video game scene".

The point was that wasn't a positive statement...

But it got me thinking: there's been so much talk about how video games will become as good as movies - and it's a pet peeve of mine that writers can be so ignorant.

What??? You say?

Sure.  Video games will eventually look as good as what you see when you go to see Hellboy this weekend.

But when that happens, movies won't look like that any more.

Now eventually, and we're not talking in the next year or two, we're talking a decade down the road perhaps, video games will start to look a lot like what you see when you walk outside (except of course that there will be zombies or terminators in the video game and may or may not be killer cyborgs and diseased undead prowling your real-life neighborhood).  By then, feature films may well be predominantly 3d - but your video games probably won't be.  Another five or ten years and then maybe the playing field will be levelled.  Except that the movie theaters will have vibrating chairs, a better sound system than you're likely to have at home, misters and blowers and scent generators, actuators to lift and tilt the chairs.  Oh - and even better 3d and higher res displays than consumer televisions can deliver (what, you thought HD was the best it got?  Please.  Not even today.  But don't count on a 4k home television standard any time soon.)

Oh, and their screens will always be bigger than yours.

The constant talk about video games 'replacing' movies on some level just sounds like people talking 100 years ago about radio replacing books.  The internet hasn't even replaced books - though it has put printed encyclopedias out of business, but then how many people ever kept a consistently updated set of encyclopedias in their homes?  If you ever even owned a set, chances are you owned one that still talked about the USSR and the apartheid government of South Africa.  If the internet replaced *that* bookshelf dominating monstrosity, I'd say that's not even surprising.  Public libraries had been replacing private encyclopedias for decades already.

Ubisoft's purchase of Hybride is motivated by the same thing that's making companies like Digital Domain say they want to get into video game production: fear. The fear that they won't be prepared if things go that way.  The fear they're going to be missing out on something.  They see something like the release of the new GTA and it looks like video games are the road to making billions.  It's just not true, any more than a small multimedia house in New Orleans should look to Wall•E as the future of their company.  Ubisoft doesn't need Hybride to understand how to make better CG. And the guys at Hybride probably can't help much with getting their ideas into the game engine: because that's an entirely different discipline.

My point is, video games will change.  But I don't think they'll "converge" with feature films any more than television "converged" with feature films.  Television is still a poor substitute for going to the movies and as much as we make the decision to sometimes catch something when it comes out on television/pay-per-view/dvd/bluray/what-have-you, as a society the rise of television has done nothing to diminish movie attendance.  Now we just expect more entertainment, and predominantly entertainment that we can enjoy just by sitting on our ass with a bucket of popcorn in our lap or a plate of nachos on the coffee table and no video game controller anywhere in site.  A theater full of people aren't going to be strumming their Guitar Hero guitars or thrashing around with their Wii controllers, and next year's Batman installment isn't going to require you to learn to drive the batmobile.

Unless of course you want to.

I was talking to a friend today about what I'm doing with regards to managing data through an animation pipeline using XML. The more I work with it and the farther I get into the project, the more flexible and powerful the whole thing seems. Of course the goal to doing the implementation in Python is that virtually every software package in the vfx industry is python-friendly - so once the core routines are written, everything from Nuke and pyShake (the python plugin for Shake - if you haven't seen it yet, check it out here) to Maya, Houdini and RealFlow will be able to make use of them. I think most places are doing that these days, with a few nods to TCL/tk here and there - but broadly supported scripting languages are King and open description formats like XML are Queen.

My friend marveled at how nice it would be if one day, a couple years from now, everything was able to talk that smoothly: that a character animated in Maya could be pulled into Houdini, for instance, as something other than an OBJ sequence or a separately rigged character that you had to tediously (or with a lot of specific coding) link to exported channel data.

I wonder if that interoperability thing will ever extend beyond each individual studio's implementation. Everybody has a way of getting software to talk amongst themselves, some solutions being more elegant than others, but when you invest in creating something as elaborate as this it becomes your own proprietary tool. If you develop a tool that an animator can take an animated character with a complex rig on it, arbitrarily select additional elements that were never *really* meant to be animated and animate them anyway, and the modeling team can modify the model and issue a new version of it - and the animation gets seamlessly transferred over to the new model, even able to be read into RealFlow, substituting a different set of low poly independent objects that are driven by the data in that XML file: you don't put that pipeline tool on the internet for everyone to download for free.

That tool becomes your secret weapon. As a studio with an investment in a powerful and unique proprietary tool, even charging for it may not mean as much to you as the edge you gain during the heat of production.

Being XML based and implemented in Python does put my current project a wee bit closer to being an open standard, though. Even Shake will take Python scripts now - and they're really powerful in it and getting more so as development continues. The readability thing for XML is a gigantic plus, and the way it represents data is great. I can build a module that will write out the translation of a locator in both world and local space, as a baked set (every frame has a value) and as a set of keyframes (values only for those frames where the value was explicitly set by the artist), as well as screenspace UV values - so the same XML file could reconstruct a scene for a lighter to light and render from or another animator to tweak the animation curves, or for RealFlow to drive low-poly proxy objects with to disturb a drifting mist, or for a compositor in Toxic to link an effect to. And it's all one XML file - not a half dozen formats (often multiple versions of each) and a hundred-unit sequence of geometry exports.

I am officially in crunch...

It's mini-crunch, though, because we're just working to get things together for the trailer. Most of the staff isn't actually working overtime, but there's a few of us... the dynamics/fluid guys, an animator or two, the cg supe and me, piecing together the necessary bits to get the shots ready for the trailer.

I wish I could be a little more clear on what exactly it is I'm doing... When the trailer comes out in a couple weeks I can at least indicate then what segment of the film I'm addressing - I'd rather leave that part as a surprise for when the trailer hits (not to mention that talking about things like that would get me in hot water).

In a sufficiently vague sort of way, I can say that there's a segment I'm working on right now that's around 76 shots long, each ranging from 50-300 frames, that has a common element that's travelling between them. Our tracking team has been tracking that element, and its progress through each individual scene, for each of those 76 shots (and at least that many more as well that I'm not dealing with yet).

The particular task I've been working on is basically a continuity script. It's a MEL script that gathers all of that data: cameras, animated and matchmoved elements, and stitches them together end to end in a master shot. This way, the element that's moving through the scene will maintain an accurate location relative to the CG environment.

To coincide with what's happening in each shot, it's necessary on that big grandaddy master shot to make adjustments: turn something here, nudge something there... and that affects the path of that common element throughout all of the individual shots.

Do you see where I'm going with this?

Basically, if we imagine that I'm talking about a man driving a car across the salt flats in Utah (not at all what's happening in this film), there are features on the horizon that we want to be consistently and realistically moving in relation to the view of the car. If the driver swerves to miss a rock, we want to change the path he's taking across the salt flats (even though in the original plates, that driver would have been sitting in a stationary vehicle in front of a bluescreen, with some hydraulics underneath making the car tilt and sway to add realism). Our view of the car from the camera should stay the same... Car and camera must maintain their relationship... but the position, speed, orientation, etc of the car and camera combined will need to change. If there's a guy on one of those sail-carts out there, we need to make sure his path is realistic and that he's in the right place in the background of each shot...

Remember that the car, the driver, and the camera are often the only real things here... the sail-cart, other things on the horizon, in fact the entire salt flats... all are cg or augmented with cg.

So I've been developing tools for funneling data from all of these shots... everything from camera data to object matchmoves, world-space offsets for each shot, and animation data... stringing it together into a seamless master scene, manipulating the paths and locations of elements in that master scene and transferring that data back into the individual shots. If we need our hypothetical salt-flats driver to swerve this way and that across several separate shots, I'm making sure that it doesn't just "kinda look right" but that we can get that data integrated into the object and camera motion of several live camera plates.

If somebody didn't ask me if I could do it, I'd probably have never thought it could even be done.

If I'm kinda scarce around here in blogland, it's because I'm catching a much-needed nap in between long shifts.

After a fairly exhausting week, tonight was my night to really kick off a weekend... and what better way to kick off a really world-class weekend than to go see an old friend perform with a band in Hollywood?

Went to Relax Bar out into the sorta sleepy grungy part of the Boulevard of Broken Dreams (better address and directions at the link above)... Weird little venue, that. Meredith and I both were having college flashbacks of places we used to go hear bands back in the day. Difference for her was that here she was performing in one.

That Meredith would be Meredith Meyer, who some old Orlando friends of mine may know. It's not every day you find someone you've known in a different place and time that's found a working niche for themselves elsewhere - but there it is. It was exciting to me to get to hear her tonight, and doubly good to see and hear just how beautiful she sounds live. Am I gushing? *shrug* Maybe. But it's deserved.

I'd love to go on more about the evening: it was excellent. Found another couple bands that I really enjoy hearing live. The whole night was good, but a couple of those bands were ones I'd go hear again and again - and in a city packed to the rim with excellent bands, that's saying something.

As I'd mentioned before, this blog is going to get more technical and I'll be spawning off a personal blog shortly. For now, bear with me non-technical readers: I'm about to talk geek for a bit. When I was teaching at FullSail, a lot of my "curriculum development" time, as well as a substantial portion of my free time, was spent doing R&D on advanced topics - things that are done at a handful of visual effects studios around the world, or even things that nobody was doing yet except in university settings.

Things like relighting using what's called a bent normals pass (a special render pass that contains information about the direction a point on the face of an object is pointing) - a process that enables you to gain a fair bit of control over how the rendered object appears to be lit, long after the frames have left the render farm. Other experimental tools like an algorithmic forest and crowd generator, or today's topic: retexturing in the comp, instead of at rendertime.

There are a couple of tools for doing this already: basically, you render out a diffuse color pass, full intensity colors, no shading, but instead of finished shaders, the elements are textured with these especially bright and colorful red/green gradients. The color of each pixel corresponds to its relative location on the UV map, so even after the texture is wrapped around an object and that object viewed from an arbitrary camera position, each pixel still holds information about its original position in UV texture space.

Now, most of the tools I've seen for this sort of thing will apply one texture to one UV pass. Certainly an exciting demonstration, but hardly practical: a typical scene will often have dozens of texture maps at the very least, frequently hundreds, even thousands. Naturally, each of these may be rendered as a separate UV pass, and each separate element brought into the comp: but that can make for some pretty unwieldy scenes and a headache to maintain all those passes in the pipeline. Re-render an animation, and the number of potential UV-passes can quickly make each iteration into a big undertaking.

Of course, if you're dealing with 8-bits per channel UV-coordinate textures, like a typical JPEG, then your possible texture size to project onto that is fairly small. 256x256 to be exact, and for practical reasons, maybe only about half of that. Hardly anyone even uses textures that small in games these days, let alone feature films. If you crank those UV-coordinate textures up to 16-bpc, now you wind up with some pretty impressive texture space: each of the red and green channels can now represent up to 65,536 coordinates: the equivalent of 16 4096-pixel-wide textures. Even if we want to be similarly conservative like in our above estimate, you can see where I'm going with this...

So my tool, which I call "OpaqueWhite ReMap Plus" uses nice, big UV coordinate textures, all generated by your rendering application in 16 or 32 bits per channel, rendered as described above as an unshaded UV-coordinate pass and is designed to accept up to 16 separate textures at a time, anywhere from 256x256 to 8192x8192. I provide a collection of high-res, high-bit-depth UV-coordinate textures for users to set up their render with, and a tool that is able to interpret the data from that render pass and automatically re-plot the texture in the render-specific UV space.

There's a lot more stuff going up on the site shortly. It's ALMOST THERE, I swear. To those that might be interested, you can learn a little more about and (soon) download my Shake Tools here.