One of the hardest problem facing plug-in developers is designing effects that work at the native pixel aspect ratio of any source media fed to the plug-in. FxFactory plug-ins are based on compositions that mix together Core Image units and other Metal-based geometry and shaders, often assuming that all inputs and outputs process pixels with a square aspect ratio.
Why is native aspect ratio a problem in the first place? Host applications usually feed video frames to plug-ins at the size and pixel aspect ratio determined by the timeline or sequence. For example, if your effect is applied inside an anamorphic 1080i sequence in Premiere Pro, the plug-in is asked to process one field at a time. Each field has resolution of 1440x540 pixels. That is obviously quite different from its ultimate display size of 1920x1080. If your composition draws a circle inside that 1440x540 field, by the time the circle is scaled to the final display resolution you end up with a nasty-looking oval:
Each video app behaves differently. Motion and After Effects always field-double their images. The same 1440x540 field is fed as a 1440x1080 image (notice the doubling of its height). Final Cut Pro supports plug-ins only when wrapped inside Motion Templates. It borrows most of its behaviors from Motion, including the habit of field-doubling all frames before processing.
To make effects development as simple as it can be, FxFactory has always provided an easy way out by enabling the following options under the Renderer section:
With both options on, you are asking FxFactory to scale all inputs to the composition to be field-doubled (1.) and transformed to square aspect (2.):
The method described above is widely used by many FxFactory plug-ins because it greatly simplifies the design and development of the composition. Unfortunately the method implies that images are being scaled automatically to have the required pixel aspect ratio, and therefore it is the source of filtering artifacts. Before discussing that problem in depth, let’s list scenarios where it is (and always will be) desirable to continue enabling those options:
When designing high-quality effects, the automatic scaling performed by FxFactory may introduce unwanted filtering artifacts. These are worse with interlaced media than with progressive media, and they are worse in Premiere Pro than other video apps. Final Cut Pro, Motion and After Effects always field-double their media. For example, a plug-in that processes 1440x540 fields at their native resolution will be faster than an identical effect that relies on FxFactory to upscale fiels to 1920x1080 images, and downscale the output of the plug-in back to field-size.
Life doesn’t have to be hard all the time!
If your plug-in does not render any geometry and does not use filters that rely on a geometric distance (such as a blur radius) you can safely disable the Renderer options discussed earlier. Color correction filters are a common example for a category of effects that can trivially support native pixel aspect ratio.
You can test this problem very easily. Take a simple pass-through filter:
Enable the Renderer options discussed earlier:
This instructs FxFactory to scale inputs to the composition from native to square-pixel aspect when rendering a frame. The output of the composition is then scaled from square-pixel to native pixel aspect.
Find a test graphic that is designed to help you spot filtering artifacts, such as this grid:
Apply the passthrough filter to get the following result:
Notice the unwanted softness introduced by the filter because of the automatic scaling, even if the composition is not actually doing anything with its input.
There is no general approach to handling native pixel aspect ratio correctly. The most important step is actually understanding what's going on behind the scenes. Then, depending on the effect you are designing, one of the following techniques may be used:
This technique can be summed up as adding a normalization step to your composition:
Unfortunately, some of the stuff mentioned above can be very tricky to figure out, and it certainly comes in the way of FxFactory's goal of keeping things as simple as they can be. That's why FxFactory includes improved versions of built-in effects (Core Image units) that handle pixel aspect ratio. You can spot these in the Library because they have identical names to the built-in units, except for a superscript that hints at their different nature:
The main difference between our versions and Apple's is that our units include an extra "Pixel Aspect Ratio" input. By virtue of doing that, we have created units that are capable of honoring non-square pixel aspect ratios.
When creating a new filter in FxFactory, notice that the composition you start with looks like this:
Notice the Gaussian Blur unit being used accepts the native pixel aspect ratio as an input. Feed it the Pixel Aspect Ratio value obtained through the FxFactory Effect node.
The FxCore application only uses images with a square pixel aspect, making it hard to predict what may happen when the same composition is fed video frames at their native aspect.
When attempting to design and test a composition that handles native aspect ratio, it is necessary to test the plug-in inside a video application with media of various characteristics to know if it can truly handle non-square pixel aspect.