EddieOffermann dot com

I've been doing a bit of coding specifically to make it easy for Big Blue Ceiling to be deployed into a facility that already has an established file structure.

By itself, this is a natural approach - and not that complicated. Because everyone with *any* sort of organization will have arranged their projects by job and by shots on that job, even though the exact heirarchy may be different it's easy to predict where the *gaps* in that heirarchy may exist and allow those to be configurable by the end user.

For instance, Company A, using predominantly linux machines may have a path to their publishing directory like this:

/FACILITY/cg/PROJECT/SUBPROJECT/shots/aa010/pub/

Company B may use a path on their Macs more like this:

/Volumes/Share/projects/PROJECT_SUBPROJECT/shot/aa010/publish/

It's not exactly brain surgery to identify that in the first case, the meaningful root directory is "/FACILITY/cg/" and in the second it's "/Volumes/Share/projects/" - then there's identifying the delimiter between "show" and "subshow" or "campaign" and "spot" or whatever your nomenclature is for project & subproject info. Maybe they're grouped as folder/subfolder, maybe they're companion folders.  Maybe there's no concept of "subshow" at the facility and we can treat both the delimiter and the subshow as empty.

But this got me to thinking: what I'm doing is creating a visual effects pipeline that is essentially filestructure agnostic.

A large part of a formalized visual effects pipeline is putting the filesystem behind an abstraction layer from the user.  Camera data moves along from artist to artist with each artist knowing nothing more than what shot they're working on. Someone will have specified, early on, what assets will be needed in the shot: ideally, when the shot comes to the animator, they pop open Maya and click a button that makes it populate their scene with the camera, models, set geo, rigged characters, etc.

They don't need (nor do they really want) to know that the model is called "candyBarDancer_v09.mb" and it's stored in "/Volumes/Share/projects/CANDY_DANCE/shot/cd090/publish/rig/".  They don't need to know it, they don't want to know it, and I believe they shouldn't even be *aware* of it because that information is nothing but a distraction from what they're trying to do.

When they're done, they can save a copy of their scene wherever they like for their own edification.  But the data needs to travel to the next artist - and they shouldn't have to tell the next artist where they saved it: again, they shouldn't even *know* this information because the data that gets written out isn't directly usable by them anyway. It's the formal vfx pipeline toolset that manages the handoff and makes sense of the data. Not the artist.

When the pipeline is properly structured, it's perfectly reasonable for multiple artists to work on the same shot at the same time: all the while getting regular updates of animation being done by other artists, receiving model changes relevant to their scene, etc. - all without having to be conscious of where this data is coming from and without having to do anything more than click an "update scene" button.

But here's the big deal!

By building the pipeline software itself around a filesystem agnostic abstraction layer, it allows us to link up established remote facilities. Company A in Los Angeles and Company B in England could be working on the same shots on the same shows, can still actively collaborating using something like Big Blue Ceiling coordinating between their facilities. The fact that their filesystems may have different naming schemes and different heirarchies, even calling the job and shots by different names, isn't going to affect their ability to actively communicate with one another.

Think about it: you have a small studio in Santa Monica and you're working on a medium budget feature with only a handful of effects shots. One of the sequences, however, needs a large set extension and a matte-painted cityscape in the background. You task your team to get to work on the foreground elements of the set extension and contact a small studio in Austin, passing along the requirements for the elements and having them install the toolset for using the shared effects pipeline. They can easily fit it into their filesystem and in no time, they're up and running.

As your artists work on the foreground elements, they register with the system to receive notifications as elements are updated. They click a shelf button to update their scene and as they build the foreground, they see the background elements and matte painting slowly taking shape: right there in their scene.

There's no messy FTP'ing of files back and forth, and no complicated backend server synchronization to set up. They don't even use the same file heirarchy: but that doesn't matter because structure and naming conventions are handled by the pipeline software, not the artists, so there's no confusion.

This is the promise of truly good pipeline design.

And Big Blue Ceiling is going to make that a reality.

As long as we're here, let's take a look at a commercial that I look on fondly, because a lot of what's described above I already wrote once when at Asylum and was used to create this spot.

I've been working on the spec for Big Blue Ceiling and thought I'd share at least an introduction to what the project will deliver. I've changed jobs recently (still in the vfx software/pipeline/td world) and that, among other things, has interrupted the project a bit but I'm ramping back up on it now.

Below are some thoughts on what a pipeline is and how I envision Big Blue Ceiling.  There is so much more to say, but much of it will have to remain unsaid until the project is closer to launch. I believe it will change the world and transform how and where we do business.

What is a visual effects pipeline?

A vfx pipeline is a collection of tools, processes and standards that permit and ensure the following:

-    Efficiency of communication of assets

o      Models

o      Animation

o      Camera data

o      Textures

o      Lighting information

o      Simulation data

o      and anything else necessary to assemble a given scene

-    Standardization

o      Filenames

o      Storage hierarchy

o      Version management

o      Tools

-    Show, Sequence and Shot Management

o      Structure

o      Dependencies

o      Assignment and Evaluation

A formal visual effects pipeline in a traditional effects house serves to automate the transfer of data by abstracting it. Separating camera characteristics, lens and movement data, from its strict representation inside the software package. Treating models, rigs and animation and separate elements that can be updated individually: for instance, animation may begin before a character design is finalized and the models updated on the fly, per shot or across the entire show without losing the animation data or requiring a complicated manual process to transfer that animation onto the updated model or rig.

On the compositing side, it may involve providing an additional set of facility-wide tools that standardize certain effects or provide a particular look.  Tools for grading and color management may be developed for the facility or the particular show and distributed between the artists to ensure that different compositors are able to deliver visually similar shots with minimal headaches.

For the entire facility, a formalized pipeline will ensure that assets and media are located with predictable names and stored in standard locations, along with maintaining project history and asset version information. If the last animation, the last comp, or last week's version of a model are preferable, it's easy to return to that version - often this can be made to happen at the level of producers without having to send the shots back to an artist to make the necessary adjustments.

What then is a cloud-based visual effects pipeline?

With the emergence of cloud computing, many common applications are moving onto a new platform: instead of being contained by a conventional desktop operating system or local server infrastructure, they are being moved onto platform-agnostic, internet-based hosted application clusters. Popular examples that many users are familiar with are such services as Google Docs, the office suite provided online as part of Google’s service offerings, or Photoshop.com, Adobe Corporation’s online photo editing and gallery hosting service. Medical information, insurance software, procurement & fulfillment systems, and tons and tons of software development have all left localized corporate infrastructures and moved onto the Cloud.

For visual effects and other CG projects such as animated features or game development, the process is a little more complicated. It’s still impractical to move software like Maya, 3D Studio Max or Houdini off of workstations and onto a web-based platform. Pipeline management has been complicated by the sizes of files that are often involved, but more recent developments in existing internet infrastructure have brought fast broadband into the range of slower local area networks, opening up the option of intelligent management and delivery of data by software such as Big Blue Ceiling. While many facilities are hesitant even to share projects with other facilities because of the combination of file transfer times and manpower overhead involved in delivering hard drives or preparing data before and after manual transfers over FTP. A Software-as-a-service or cloud-based system such as Big Blue Ceiling handles the typical database and project management aspects of a traditional in-house pipeline, as well as managing data for effortless transfer between facilities or between facility and remote artist.

What is a “Software as a Service” solution?

Software as a service (SaaS, typically pronounced 'sass') is a model of software deployment whereby a provider licenses an application to customers for use as a service on demand. SaaS software vendors may host the application on their own web servers or upload the application to the consumer device, disabling it after use or after the on-demand contract expires. The on-demand function may be handled internally to share licenses within a firm or by a third-party application service provider (ASP) sharing licenses between firms.

What is Big Blue Ceiling?

Big Blue Ceiling is a special subcategory of SaaS solutions referred to as SaSS (Software as a Secure Service). Software as a secure service (SaSS) is a derivative of software as a service. SaSS denotes a class of software as a service which emphasises security, not only in the link to and from the service, and the storage of any content by the software providing the service, but also in the security of the user in terms of the ability to make consistent backups and restores of any data stored in the service, in a non-proprietary format. In other words, security in transmission, storage and control over the user's own data.

In the context of a Cloud-Based Visual Effects Pipeline Software-as-a-Secure-Service, Big Blue Ceiling is a method of providing the sort of sophisticated visual effects pipeline normally restricted to larger, established visual effects houses, not only enabling it to be used by smaller studios but at the same time decentralizing it in such a way that it is as efficient for a group of broadly spread artists working in different locations as it is for a tightly integrated team of artists in a centralized studio.

Why would I be interested in a cloud-based, software-as-a-service solution for my visual effects pipeline?

The traditional rationale for outsourcing of any IT system involves applying economies of scale to the operation of applications, such that a service provider can offer better, cheaper, more reliable applications than companies can themselves. Several important changes to the way people work have made the rapid acceptance of cloud-based solutions possible and these changes are even more notable when we consider the visual effects community.

I.               High performance computers are widespread: Most cg & visual effects artists not only have a home computer but have one capable of performing functions far in excess of the basic needs of checking email and consuming online media.

II.             Processing power is a commodity: In the vanishing past, innovations in hardware were considered strategic advantages. From optical printers in the analog days to the first digital to film transfer processes of the TRON era, for many years hardware was king. More recently, proprietary applications and software tools were viewed as strategic. Today, people know it’s the business processes and the data itself: customer records, artist techniques, pricing information, and good effects design. Computing and application licenses are cost centers, and as such, they’re suitable for cost reduction and outsourcing.

III.           “Insourcing” pipeline systems requires expensive overhead including salaries, health care, hardware, software and OS management, liability and physical building space: not to mention unproductively reinventing the wheel!

IV.          Applications have tended to standardize: with a few notable exceptions, most people spend most of their time using standardized applications. A handful of standard 3d software packages dominate the market and they’re capable of exchanging data in a small set of standardized exchange formats. This means that a comprehensive solution for managing digital assets will work for a wide range of projects and facilities. Purpose-built, facility-specific pipelines should be regarded as dinosaurs.

V.            Web systems are incredibly reliable: Despite sporadic outages and slow-downs in the past, most people today are more than willing to use the Internet as a critical component of their business. As of early 2010, the visual effects community is behind many other information-related industries in moving out of centralized work clusters and into the cloud, largely because there has been no sound approach to managing the complex assets and often considerable data transfers involved.

VI.          Security is sufficiently well trusted and transparent: Secure communication no longer requires a complicated VPN setup or, worse, leased lines. Confidential client data can be handled even remotely with little risk.

VII.        Bandwidth of wide-area networks has grown drastically following Moore's Law (more than 100% increase each 24 months) and is reaching the bandwidth of slow local networks. Added to network quality of service improvement this has driven people and companies to trustfully access remote locations and applications with low latencies and acceptable speeds. Additional layers of asset data management by Big Blue Ceiling additionally boosts apparent speed of access while boosting quality of the artist experience.

VIII.      Cloud-based “Software as a Service” solutions have the effect of democratizing software, allowing small and medium businesses to have access to functionality formerly the domain of large enterprises. Big Blue Ceiling provides pipeline services at a level that small and even midsize companies could never afford to develop on their own, with a continually evolving set of tools and features unmatched even by in-house proprietary systems.

Who can benefit from Big Blue Ceiling?

Simply put, nearly everyone!

An entirely centralized studio can use Big Blue Ceiling and continue working as a centralized facility, leaving ongoing pipeline development to the Big Blue development team, regularly rolling out new features, support for additional packages and powerful artist tools, while letting the studio focus on creativity and project execution. And that centralized studio can rest easy knowing that if they need to expand, open up other locations, or cooperate on projects with artists and other facilities around the world, a set of tools are already in place for them to make sharing data completely effortless!

Smaller, newer facilities, perhaps formed just to accomplish a single project such as an animated short or independent animated feature can benefit enormously from a convenient slip-on pipeline like Big Blue Ceiling. It’s low entry cost provides world-class effects facility capabilities at a budget that nearly any production can absorb. Your cost savings just in artist hours transferring files and maintaining versioning, will easily exceed the entry cost of the service for small projects.

Loose collectives of artists will benefit from the Big Blue toolset as well, whether they’re seasoned industry professionals working purely for the love of the art or students collaborating on a project, access to the class of data management tools provided by Big Blue Ceiling should easily catapult any group’s efficiency.

Mid-size and larger facilities will find the toolset handy not only for their in-house work but for the ability to quickly and easily add artists in remote locations, or to easily permit secure access to their assets and production database by producers and vfx supervisors who may be offsite.

Additionally, tools that monitor times to complete tasks are able to chart them against production cost estimates, easily indicating unexpected burdensome cost centers, suggesting areas that estimates might be adjusted or manpower requirements reconsidered to adjust schedules. These are benefits that very, very few pipelines can provide, even in large established facilities.

It's been a while since I updated here, but I thought I'd throw out a tidbit from something I've been working on.

Part of any vfx pipeline project involves making your different software pieces communicate with each other. Part of that, of course, is being able to write common file formats. Sometimes that's for sharing generic data, such as through XML, or basic channel information that can be written to a tab or space-delimited text file sometimes referred to as a chan file.

Other times, though, you'll want the packages to communicate more intimately, and Maya provides a method to closely integrate different software packages with it as long as the other packages have the ability to do port-level communication. If you haven't written IP protocol software before, this can be a little daunting, so I'm going to share a couple short code tidbits that'll make talking to Maya a little easier. I'm assuming some basic MEL experience, and some background in Python wouldn't hurt either, though you should be able to follow along with any amount of coding background. I'll be skipping the tedious details though, so if you need a programming refresher that's beyond the scope of this post.

First, you'll want to pick the port that Maya is going to listen on. For my example, I'll use 7777 - basically because it's not used by another widely used protocol, and because it'll be easy to identify in the example code.

In Maya, either in the script editor or as part of another MEL script, you'll want to execute:

commandPort -n ":7777";

Now, this will open a port on 7777 which I should give you a little warning about: ANYTHING that is sent to that port will be executed by Maya, as long as it is running, and until that port is closed. This includes quitting maya, saving or deleting your scene, or even executing system commands. So you may want to have the data coming across that port filtered by a MEL procedure first, or passed to a command. In that case, you'll want something like this:

commandPort -n ":7777" -pre listenOnPort;

In the former case, anything you send to port 7777 will be executed by Maya. In the latter case, you'll want a procedure like:

global proc listenOnPort(string $argv)
     {
     string $args[];
     int $argnum=`tokenize $argv $args`;
     // Code to process $args should follow here...
     };

Now of course, you need some way to communicate with Maya. This is where it's actually a little tricky - since you'll need a socket client to issue commands. I'm providing only the most basic example: this one is asynchronous and one-way only. There is no way for this routine to receive any feedback from Maya.

#!/usr/bin/python2.5
import socket
import sys
HOST = '127.0.0.1'
PORT = 7777
CMD=sys.argv[1]
try:
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
except socket.error, msg:
    sys.stderr.write("[ERROR] %s\n" % msg[1])
    sys.exit(1)
try:
    sock.connect((HOST, PORT))
except socket.error, msg:
    sys.stderr.write("[ERROR] %s\n" % msg[1])
    sys.exit(2)
sock.send("%s\n" % (CMD))
sys.exit(0)

By calling:

talktomaya.py "select -all"

You will be able to send the indicated command to Maya to be executed, or in the case of the -pre example, pass the command to the procedure for parsing.

More sophisticated implementations may require sending to machines other than localhost (127.0.0.1) as in the above example, and that's left as an exercise to the reader. It's a simple matter to send commands to a range of machines across a collection of IP addresses. Examples of this might include having individual users register their interest in a particular piece of data - and when that data is released, those users are notified automatically that the data is available, say an updated model or a revised animation, and giving them the ability to then update their scene with the new data. It might also be useful to centrally request an update from all users who are currently working on assets required for a particular shot - and obviously many other options are available when you can communicate directly with the maya instance of each user in a facility.

The tricky bit is that I hadn't seen a clear example of a script to send instructions to Maya - and there it is, in its most basic form.

In order to really make the best use of this article, you may need to visit my actual blog if you're reading this on a mirror such as my livejournal, xanga pages or my LinkedIn, or via an RSS feed, since it contains embedded quicktime movies and such that don't always play nice across those mirrors.

You might also want to pick up a set of glasses like I'm linking over on the right, since you can't make much sense of anaglyph images if you don't have anaglyph glasses! (If you've swiped polarized glasses from a movie theater, those won't work with this.) If you already have a pair of them stashed away in a drawer somewhere, you can play along already! The cheap "throwaway" cardboard ones that sometimes come with comics or advertising materials will work, but the lens quality of real framed sunglass-style glasses will improve the experience.

Note that this article is an attempt to make a pretty technical subject as accessible as possible, so as much as possible I'll be keeping it straightforward and just noting a few things that seem important. There's much more technical information to be written about at another time, but after talking to a friend that's dealing with the same things himself on another project, and realizing that each studio will run into the same issues again and again until somebody writes this stuff down, maybe it was time to get a little of it onto the internet. Since I know the people that find this article are going to range from complete novices to experienced vfx industry professionals, the tone of the article may swing a bit. Rest assured, if it momentarily becomes too mundane or too technical, it'll roll back towards the middle in another paragraph or two!

To make something clear up front, when I refer to stereo films/cg/animation/movies/etc, I'm referring to stereography - the process of creating a 3d image in the mind of the viewer by the presentation of two or more 2d images. I tend to refer to projects as "stereo" projects vs. "3d" projects since I work in visual effects where we think of "2d" as a department and 3d as a variety of cg software package: texture and digital matte painters, roto and paint artists, compositors are 2d artists, and even on a stereo film, the "2d" department does just as much work as they would on a traditional "flat" film.

After working on an in-development feature a few months back, as well as the Hannah Montana 3D concert feature, and the upcoming "My Bloody Valentine: 3D" flick, one thing was really painfully clear: almost nothing about 3d presentation has really been entered into the record. What's been learned by the relative handful of people working on stereo movies has been kept to themselves. Each new team to confront the integration of live action with stereo cg has to invent the whole process from scratch all over again. It's like the beginning of cinema, which makes it exciting and frustrating at the same time.

If you're in the industry, the time to come to terms with stereo production is now: in five years time, we shouldn't expect to see a lot of flat films coming out of the major studios.

So let's jump in and look at a clip now and then talk about what it shows.

Consistent convergence and interocular distance between elements

There are two things that we discuss when talking about stereo cameras: interocular distance and convergence. These are what makes a stereo camera setup different from a traditional camera. All the same old things are there: film speed (though generally this is a CCD or CMOS sensitivity level, not an actual film speed), aperture, shutter angle (or something like it), and field of view. But interocular distance and convergence are new concepts to most people, even though 3d movies have been around for over 80 years.

The video above shows a rotating checkerboard with two geometric objects on it. The interocular distance on the shot is about human-eye normal (that is, the distance between the two "cameras" used in the CG scene is about the natural distance for human eyes). I should also note that this distance is constant in this shot. Some camera rigs, such as the Pace rig that's been quite popular in stereo film production recently, have the ability to vary the interocular distance while the shot is in process (usually in conjunction with a focus pull). When I first heard about this being touted as a promotional point, I was terrified: often, we consider ourselves lucky in visual effects if we were able to get accurate lens info, and now this? Turns out, it's not as hard to deal with as you might think - even if you don't have metadata from the shoot that pairs up interocular distance with each frame. I'll deal with the specifics of that in another post.

The convergence in the above shot is on the cone. Convergence is the distance in front of the eye-pair that each camera's view crosses the other. If you hold your finger about a foot in front of you and look at it, your eyes are converging on your finger. If you then shift your focus to an object beyond your finger, you will notice the image of your finger splitting into two images: this is because your eyes are no longer converging at your finger but rather at another object in the distance.

This is an important thing to note, here, that when composing a scene for stereo presentation, you should always try to set your convergence on the element in frame that you most want to draw the viewer's attention to. Failure to do this will increase complaints from your audience about headaches and eyestrain as they (usually) unconsciously try to compensate! You can feel and see the effect of this by playing the above video and trying to stare at the foreground edge of the checkerboard. The good news is that if this doesn't happen in camera, it's easy to make it happen in post. The dual plates that represent the two eyes of the camera can be panned horizontally until the desired convergence element merges. If the same element exists in both plates at the same position when overlayed, that's where the convergence is! Again, this is something I should cover in more detail in another article - but this info will be sufficient for many.

In the above shot, all of the elements were rendered with the same interocular distance and convergence distance. They appear solid, the geometric objects appear to rest on the checkerboard, and all is right with the world.

Let's see what happens when we try to fool Mother Nature:

Inconsistent convergence and interocular distance between elements

This is the same render of the checkerboard, but in the render of the elements that are sitting on it (which still exist in the same location as before) the camera pair converges just a little bit in front of where it did before. The effect of this is to make them appear farther away from the viewer. It's a subtle effect: our brain still wants to perceive the elements (which obscure the table and thus *must* be in front of it) as being closer to us, but it's fighting with itself: there's a sense that the elements are somewhat indented into the table. The cone is also harder to focus on because it's no longer the point of convergence.

More to come in a later entry...

As I'd promised earlier, I'll be doing a quick run-through of how matrix multiplication can be used to rotate, scale and translate coordinates through 3d space. First, though, I'm going to write a little bit about a tool I've been working on and just launched. Sorry: the actual code for it is proprietary, so I can't share it, but a brief discussion of the tool and what it needed to be able to do will help illustrate why matrix transformation is so important, even when you're not creating a 3d package from scratch or developing a game engine.

I needed a way to quickly create a usable "right eye" camera to correspond to a "left eye" camera. I needed to be able to specify interocular distance (the distance between the left and right eye), as well as the convergence distance (where the eyes were focusing). Our eyes naturally "converge" on objects that are in front of us - this is part of how we judge relative depth as this convergence affects the apparent relationship if the dual images our eyes perceive of *other* objects that are not at the point of convergence. But that's a subject for another entry.

Anyway, the tool needed to be able to take a tracked "left eye" camera and apply that tracking data to generate the "right eye" camera using convergence and interocular information that was either derived from a track, noted on set, or dynamically assigned in Maya. I wanted a right eye camera that could be created instantly and that would automatically follow any changes I made to the left eye camera. If I smoothed that camera's path, I needed the right eye to follow suit, if I performed some dramatic transformation to that camera's path, perhaps extending its animation or re-animating an object that was part of the track while conforming the camera to that object's new path: I needed the right eye camera to keep up. If a cg element was being added, moving toward the camera, I needed to be able to selectively lock the convergence to "look at" that object's depth without diverting the cameras to actually look directly at it - similar to changing the focus depth.

It's all about control and instantaneous response, with a camera whose path is known with some degree of certainty and a second camera that will follow that one in a way that will produce realistic cg and will enable artists to quickly and accurately duplicate the second camera position when the original interocular distance and convergence information is not known.

Matrix multiplication, as a mathematical operation, isn't complicated - it's simple multiplication and addition, just repeated a number of times to generate the required result matrix. Since there are already more-than-ample tutorials online about how to actually *perform* matrix math (if you were cursed to do it by hand or develop procedures to support it in programming environments that don't already have matrix math support) I won't be covering that in detail. If my simple explanation doesn't quite do it for you, put "matrix multiplication" into Google if you're stumped.

I'm also going to avoid covering application in a specific language. Most recently, I was doing this in MEL for a project I'll discuss in a moment, but there are Python libraries for doing matrix math, TCL/tk support for it, PHP and Perl support - you'll rarely find yourself with no built-in or easily-added matrix support, though you may want to write (as I did) a number of routines to make it a little more accessible.

To the left, you can see the standard format of a translation matrix. This matrix (when multiplied by a coordinate matrix as represented on the right-hand side of the equation) will translate that coordinate into a new space, offsetting it by (tx, ty, and tz). In the typical way of matrix multiplication, the element in each row of the coordinate matrix [x y z 1] is multiplied by a column of the translation matrix, as shown here:

The rows of this matrix are then added together (in typical matrix multiplication fashion) to give the resulting (x',y',z') location.

But there's more to matrix transformation than simple translation. If you wanted to find the new (x', y', z') for a translation like this one, simple addition would be enough. But manipulating points in 3d space is rarely that simple!

Fortunately, it's just as easy to scale a point using matrix math! Always away from the origin - we'll talk about how to scale a point away from somewhere other than the origin shortly. All matrix operations work with a similar setup, so you'll get used to seeing a similar notation here.

This matrix is multiplied in the same fashion as the one we see above, rows against columns, with row sums producing (x',y', z') for the new location.

I'll admit, though. I don't use matrix math for scaling things very often. You know what I do use it for, though?

Rotating a point in space with respect to the origin! Isn't that exciting? I LOVE rotating things! Well, ok, so it's not that big of a deal - but when we start combining some of these things, it can turn a complicated object tree in your scene into a relatively simple expression.

I have to warn you, though, rotation is a bitch. Ever notice how your favorite 3d software has this whole "rotation order" thing? That's because if you rotate something 10 degrees in X, then 15 degrees in Y, it's not the same as rotating it in Y first and then in X. There's also a different matrix for each axis of rotation, so let's take a look. Same matrix math process as translation and scaling, but from this one we get rotation around the Z axis.

And now here:

That one rotates in X! And lastly, as you'd expect, there's a matrix for rotating around the Y axis:

Now, combining these matrix transformations together is as simple as multiplying the matrices with each other. Now, you build the series of multiplications in order from right to left, but they're carried out from left to right. For instance, for "zxy" rotation order, you would create an expression similar to the following: (I'll use MEL for this example: $xRot, $yRot, and $zRot are each 4x4 matrices that already contain transformation data for x, y and z rotation):

matrix $r[4][4] = $yRot * $xRot * $zRot;

Provided proper matrix variables are supplied, MEL supports basic matrix operations with some limitations that you'll find as you stretch your legs.

These can be strung together into much longer expressions generating much more complex matrices. To rotate an object about a point other than the origin, for instance, subtract the values of that coordinate from the coordinates of the object (transform it in {-cx, -cy, -cz} where {cx,cy,cz} was the center to rotate it around), perform the rotation, the transform it back {+cx,+cy,+cz}.

It may take some time learning to visualize and plan out a complex matrix transform - but what makes it powerful is the ability to combine any number of transformations into a single operation. Some basic trigonometry and a well-applied matrix transform can accomplish all kinds of things!

Matrix math is a bitch. And it's nigh impossible to find any good resources online. Every "transform matrix tutorial" or "matrix multiplication lesson" online explains in detail how to multiply matrices together.

That hasn't been my problem for about 25 years now. I get it. This is how you multiply a matrix with a vector. This is how you multiply two matrices together. Neither of those things actually *accomplishes* anything if you don't know what the data in those matrices represents or what adding, subtracting or multiplying them would do.

My problems always center around something more like this: I have a point at <10,20,30> with orientation <15,30,45>. Where is the point if I rotate it 15 degrees in y then translate it 100 units in the X axis?

What's screwed up is that I've so frequently solved this by doing trig that I know off the top of my head that pi/180 is estimated at 57.29577951. Some people pride themselves on the number of digits they know pi to: I find the radian conversion number to be far more handy. But for reference, pi to me is 3.14159265. Note the 8 significant digits behind the decimal number: can you tell what decimal significance I grew up programming in?

The thing is, I'm working on a number of tools for managing stereo cameras in Maya. I need to quickly determine from the transforms of one camera (derived from a track of live footage) the location and orientation of a second camera if we know (or can estimate or otherwise determine) the interocular distance (the distance between the eyes) and the convergence (how far in front of these cameras their line of sight intersects). Determining the angle of the second camera relative to the first is a straightforward bit of trig: think Pythagoras and arctangents and it'll come to you, or go find a good trig reference online - this part's easily found in a million places. Diagram that out for yourself, making an equilateral triangle with cameras at each base corner with convergence being the length of a line bisecting the triangle into two right triangles. If it's not immediately clear, work on it a bit: it might wake up parts of your brain that haven't seen the light of day since highschool.

The tricky bit is figuring out where a right-eye camera would be if it was offset along a line that isn't quite perpendicular to the left-eye camera.

I'm very nearly there - based on a given camera, I can find a point offset along that camera's X axis and apply a convergence factor to the resulting camera location by canting it back in along the Y. But rotating a known transform matrix by an additional value in the y axis *before* doing that x translation... still fussing with that bit. I can make that happen less elegantly - by actually transforming nulls through space in Maya - but I'm searching for the magical transformation matrix that'll make it happen in a simple series of math statements.

It's late. It'll probably be plenty clear to me in the morning.

UPDATE: Got it! Sleep did wonders - my tired brain just didn't quite get the matrix arrangement right before.  I'll write an entry this weekend covering concatenated matrix transforms.  Damn handy.  Damn powerful.

Installing pyShake on the laptop today.

It's every bit as painful as it was when I installed it on my old desktop machine.  Maybe a liiiiiittle bit better since at there's a configure for it now, but installing prebuilt binaries really spoils you.

If you've never installed a traditional unix/linux/macosx application,  you're in for a treat. It's one thing if it's your own application, as well, since you'll know what's going on: but when it's someone else's...

Well, you never *really* know what any application installer is doing.  Data gets socked away hither and thither, but mostly you don't know about it.

Doing a configure/make/make install sequence usually displays TONS of data.  There will be a thousand lines of successful completions, another thousand of warnings of varying severity (with usually alarming-sounding messages that may or may not be important - and very frequently aren't), and maybe even a few outright failures that *also* may or may not be important.

A truly great build will usually ease your mind a little.  Several of the warnings you get for things like boost::python will reassure you that the error you're seeing happens to nearly everyone and shouldn't alarm you.

Somehow, that doesn't make me feel entirely at ease. It's like your girlfriend telling you, "It's ok, honey. It could happen to anyone."

Then there are errors like this:
Please ensure that LDFLAGS is configured properly, or specify --with-boost-root

Thanks.  I'll do that.

I just read that Ubisoft bought Hybride Technologies - a large video game company buying a medium-size visual effects house.  The article had a lot of talk about the future of video games and movies and seemed to miss the point when it quoted recent reviews of "Beowulf" that said it was like a "long video game scene".

The point was that wasn't a positive statement...

But it got me thinking: there's been so much talk about how video games will become as good as movies - and it's a pet peeve of mine that writers can be so ignorant.

What??? You say?

Sure.  Video games will eventually look as good as what you see when you go to see Hellboy this weekend.

But when that happens, movies won't look like that any more.

Now eventually, and we're not talking in the next year or two, we're talking a decade down the road perhaps, video games will start to look a lot like what you see when you walk outside (except of course that there will be zombies or terminators in the video game and may or may not be killer cyborgs and diseased undead prowling your real-life neighborhood).  By then, feature films may well be predominantly 3d - but your video games probably won't be.  Another five or ten years and then maybe the playing field will be levelled.  Except that the movie theaters will have vibrating chairs, a better sound system than you're likely to have at home, misters and blowers and scent generators, actuators to lift and tilt the chairs.  Oh - and even better 3d and higher res displays than consumer televisions can deliver (what, you thought HD was the best it got?  Please.  Not even today.  But don't count on a 4k home television standard any time soon.)

Oh, and their screens will always be bigger than yours.

The constant talk about video games 'replacing' movies on some level just sounds like people talking 100 years ago about radio replacing books.  The internet hasn't even replaced books - though it has put printed encyclopedias out of business, but then how many people ever kept a consistently updated set of encyclopedias in their homes?  If you ever even owned a set, chances are you owned one that still talked about the USSR and the apartheid government of South Africa.  If the internet replaced *that* bookshelf dominating monstrosity, I'd say that's not even surprising.  Public libraries had been replacing private encyclopedias for decades already.

Ubisoft's purchase of Hybride is motivated by the same thing that's making companies like Digital Domain say they want to get into video game production: fear. The fear that they won't be prepared if things go that way.  The fear they're going to be missing out on something.  They see something like the release of the new GTA and it looks like video games are the road to making billions.  It's just not true, any more than a small multimedia house in New Orleans should look to Wall•E as the future of their company.  Ubisoft doesn't need Hybride to understand how to make better CG. And the guys at Hybride probably can't help much with getting their ideas into the game engine: because that's an entirely different discipline.

My point is, video games will change.  But I don't think they'll "converge" with feature films any more than television "converged" with feature films.  Television is still a poor substitute for going to the movies and as much as we make the decision to sometimes catch something when it comes out on television/pay-per-view/dvd/bluray/what-have-you, as a society the rise of television has done nothing to diminish movie attendance.  Now we just expect more entertainment, and predominantly entertainment that we can enjoy just by sitting on our ass with a bucket of popcorn in our lap or a plate of nachos on the coffee table and no video game controller anywhere in site.  A theater full of people aren't going to be strumming their Guitar Hero guitars or thrashing around with their Wii controllers, and next year's Batman installment isn't going to require you to learn to drive the batmobile.

Unless of course you want to.

Read this question on cgtalk tonight and thought I'd kick the question and my response out here since I've repeated much this same advice many, many times.

I use Boujou for matchmoving but am tracking 640x480 camera resolution all of the time... I often have shots with handheld very shaky camera and motion blur. Boujou has trouble tracking it. Is it just that there's to much blur/shakiness or would a higher camera resolution help?

and my response:

Well, it looks like there are a number of issues to overcome:

First, the quality of the footage: handheld/shaky/moblurred footage is just awful to work with. If you're just the artist, ie, you get what's handed to you: you work with what you have. There's no telling Mann, Fincher or Stone they have to reshoot because your job is hard. Their job is harder and it's them the client paid for, not you. If it's for your own projects, ie, you're the director or at least the vfx supervisor on a smaller project: you have to take this into your hands (so to speak) and fix it before it gets past the camera! Since you're working with 640x480 footage, I have to assume you can address it at this stage: these are obviously personal projects, not features or broadcast. Fix the source first! Don't shoot/allow to be shot footage that's going to suck to do post on!

Second, the resolution of the footage: Until you've worked in HD or film res, it's hard to appreciate just how much easier it is to work with on a tracking level. A dot that is so small it'll disappear into noise at NTSC is an obvious crosshair at 2k. Smudges on the wall and skin blemishes become usable tracking points. If you have the option to work with higher resolution footage, insist on it!

Third: Lighting. Again, this is only something you can influence if you're shooting your own projects, but remember to light for contrast not final levels. More light means faster shutter speeds means less motion blur. Light for the type of shadows you want, not how dark you want the scene when you're shooting for visual effects. If your key and fills are set properly, you can always make the shot darker when grading. This isn't as much of an issue when shooting film or deep color digital like the Arri Digital cameras since their shadow detail is good enough to boost for tracking detail. But if you're shooting with consumer or prosumer cameras, you've got jack but noise in the shadows and the low light levels will mean excessive blur and bad noise overall.

Fourth: Manual vs Auto. Boujou has a reputation as being the fire-and-forget solution for people that don't understand tracking. The really hard shots are going to need manual solutions. I don't mean hand-keyed (necessarily) but we're talking careful selection of usable tracking detail and improved setting of constraints. Personally, I recommend SynthEyes - unbelievably good price point and an unbeaten feature set. I know there are shots that people say they can click a button and get a solve on in Boujou that they can't get in SynthEyes, but the reverse is true far more often than it's not. When the footage is hard to work with, throw away the automatic solutions. Give them one run at it if you've got the time and processor power to spare, then move to something where you have some control.

Fifth: Tracking techniques. Track the center, not the edges. When doing a supervised track of moblurred features, follow the center of those blurs.

Sixth: Rendering techniques. Moblurred footage is hard to render to match because what you have to match is a particular slice of time. Think of it this way: you're placing tracking points on each frame, generally spaced apart at the rate of 24, 25 or 30 frames per second. Between frame 1 and frame 2, there's a fraction of a second where anything could happen. The camera could jump up a couple centimeters and back down for instance, before hitting frame 2. This will show in the motion blur but not in the keyframes. You can't easily fix for this! (See tip 1). More often, though, the problem is the phase of the render. How to address this will depend on your renderer. You may be able to adjust this in the renderer, you may have to adjust it in the actual animation keyframes depending on what 3d software you're working from. Basically, you need control of the shutter timing and shutter offset: when in that frame it opens and how long it stays open.

For high end vs. low end facilities, #6 can often be the difference between them. For all the skill you can buy from freelancers in tracking, modeling, animation, textures, etc: having the resources to do test renders and tweak that setting (and having someone on staff who can make sure you're addressing this) makes all the difference.

photo credit: Capture Queen ™I have a side project for iPhone that I'm working on. There's a patent filing in process, incorporation paperwork to do, all that good stuff - so I won't be doing a lot of specifics about the project until it's ready to go. It's basically a location-based-services thing but when we've looked over what everyone seems to be doing with LBS and social networking and the like, it all seems to be thinking very inside the box. Just like a few years ago there was a rash of "... on the internet!" patents where things we'd done forever in the wetworld were being done online and called "visionary", now there's a rash of similar "... on a mobile phone!" patents and websites that are no more groundbreaking than when we did them online or on our feet. It's like patenting driving in nails with the side of a hammer. It works, but it's not especially groundbreaking nor is it even ideal.

We really feel that this project is going to move mountains though, and change the way people view the real world around them, not just when they're in front of a computer.

The tendency towards developing for the iPhone is that the iPhone has arguably the best SDK there is: and a substantial and rapidly growing marketshare. Still, though, that limits our deployment: it's the largest selling smartphone-style device, but it doesn't represent the majority of the market. No-one does.

Today I've been evaluating Mojax, a cross-platform mobile development environment. It looks promising. It promises cross-platform compatibility, including access to a number of necessary core services like GPS, and currently runs on all phones supporting J2ME (Java on Mobile), all color Blackberry phones, and shortly Windows Mobile devices from WinMo 2003 onward, any mobile running Brew V2 or later, and Helio devices. Just the J2ME support gets me onto most smartphones - so this effectively would get the product onto every mobile device there is with gps capability.

This excites me because I like the prospect of running the application on every feature-capable mobile phone without having to separately develop for Blackberry, HTC, Samsung, and Nokia. Getting the product out simultaneously for iPhone, Windows Mobile, PalmOS and Blackberry would be fantastic. There are some annoyances with using J2ME midlets, but I'll take the slight user experience tradeoff for being able to deliver an application at all without breaking the bank