Motion Vectors (Everything About Them)

Note: All information on this page is based on the 2012 Extension 1 release.

Motion Vectors are the result of an analysis telling you where every pixel of the current frame is going to or is coming from (in the previous or following frame). There can be 2 types of Motion Vectors: Forward (next frame) and Backward (previous frame). Forward vectors are more frequent, and when no information is provided as to whether it is forward or backward, you can assume it is forward.

The way motion vectors are stored is through the Red and Green channels of an image, and there are essentially two formats: Absolute and Normalized.

Absolute Motion Vectors

Absolute Motion Vectors is the format that all tools that support motion vectors inside of Flame and Smoke are expecting and generating. This format implicitely means that the vectors are stored in a float or half-float image format (Open EXR), and the particularity is that the pixel no movement is set to 0 in the Red and Green channel, with colour information being positve and negative using the whole float colour range, so colour values can get way beyond what your monitor can display. The usage of a histogram is necessary to understand what is going on with these types of files. The main advantage is that they can be of a really high precision and are self contained in terms of vector magnitude.

Normalized Motion Vectors

Normalized Motion Vector need to be converted to the Absolute format to be used with any Flame and Smoke tools supporting Motion Vectors. There are currently two tools applying the conversion: Motion Convert and Motion Blur. In both tools, the conversion requires manual interaction because, in essence, normalized vectors signify that the pixel displacement has been compressed in a limited colour range, so you have to manually tell the conversion mechanism what the scale of the displacement is, with the Maximum Displacement field, which acts basically as a multiplier.

Normalized Motion Vectors can be found in any bit-depth type of file including half or full float. Its particularity is that the pixel no movement is half of the colour range, so for example, in float it will mean 0.5 (for 8-bit it will mean no movement to 128), and the colour range is clipped in between 0 and 1.

Normalized Motion Vectors can also support information in the blue channel, allowing you to set the pixel displacement magnitude automatically. If this is the case, you need to select the Blue Channel Magnitude option instead of using the Maximum Displacement field.

How do I know what Motion Vector format I'm dealing with?

• Look at the file bit depth, if it's either 8,10 or 12-bit, it is Normalized.
• If it's float or half-float (16 or 32-bit), then a colour histogram is your best friend. If the file only contains colour information between 0 and 1 then it's Normalized, otherwise if any negative values are present it's Absolute.

Where do I get Motion Vector information?

• 3D renderers such as Mental Ray are able to generate this render pass allowing you to deal with Motion Blur or Timewarp effects on the beauty pass at the compositing stage. It's best to ask for Absolute Motion Vectors in Open EXR format (the mental ray render pass name is mv2DToxik).
• For any live footage, you will need to create the Motion Vectors using the Motion Analysis node. It's best to render them and use them as clips inside your schematic since they can be processing-intensive.

What can I do with Motion Vectors?

Motion Blur:

The Motion Blur node and module require a Forward Motion Vector input to perform its Motion Blur effect.

3D Blur:

The 3D Blur Batch node can do Motion Blur as well as a Depth of Field type of Blur, and also requires a Forward Motion Vector input.

Timewarp:

The Timewarp node and module can take Forward and Backward Motion Vector input to use the Motion Frame Interpolation mode. If no inputs are provided, it will generate its own based on the Motion Analysis settings found in the Frame Interpolation section. When the timewarp value is below 100% it will use that information to create new frames to allow you to create smooth slow motion timewarps; and if the value is above 100% it will then use the same information for frame interpolation and Motion Blur trails effects.

Pixel Spread:

The Vector Warp mode requires a Forward Motion Vector to allow pixel warping of the input.

What problems can I encounter with Motion Vectors?

• Precision: this is mostly a side effect of Absolute Motion Vectors, but too high precision can, in some cases, create effect artefacts that would show up as either aliasing or even object dislocation. The solution is to reduce the vectors' precision by softening them. The Motion Blur node contains a Vector Softness field to accomplish this, but for usage with any other node you could use a Blur node to do the same.
• Magnitude: depending on the scale of the 3D scene, you can end up with Absolute Motion Vectors that have a really low magnitude, which make them hard to use. Boosting the colour range by remaping the actual colour range of the vectors to a larger colour range footprint using a colour histogram (like the CC module or Batch node) could help in these cases.