FaceTracker is a plugin node for Foundry Nuke created for facial tracking without mocap rigs and markers. The tracking information can be later used for retouching, adding scars, relighting, face replacement, aging and de-aging, etc. For best results we recommend using it with FaceBuilder. FaceTracker is similar to GeoTracker but in addition to model's position it also tracks facial gestures. Our tracking algorithms are stable contrary to neural networks solutions, precise, allow full manual control and don't require any kind of on-stage preparation like facial motion capture rigs.
We want to bring latest achievements of the scientific world to the production pipelines, meaning you have the best algorithms and approaches at hand when you use our tracking plugins. Speaking of the speed, we can't guarantee you the realtime tracking, but we are really close to it, so you wouldn't need to wait long.
You don't need to export and import anything from and into Nuke, the whole tracking process is happening inside: you create a FaceTracker node and start tracking using a 2D Viewer, and checking the results in 3D Viewer when needed.
You don't need to meddle with sliders and knobs, you just create 'pins' on the face mesh in the Viewer and drag them to the corresponding position. The face model is adapting while you drag the pins.
While we try to make everything full automatic requiring as little as possible human work hours, we still focus on precise tracking results, so you can control the process and adjust the results and keyframes to a full degree.
All our nodes including FaceTracker are available in Nuke Indie. You need to install the 12.2v4 version of Nuke and 2020.1 version of KeenTools package or newer. There are no functional limitations compared to the standard Nuke.
Once the tracking is done, you can export the animation into a CSV file with ARKit-compatible FACS blendshapes, just like the Epic Games' Live Link Face app provides you, so you can later use it with your own tools or integrate into Unreal Engine-based pipeline. You can also load this file into our FaceBuilder for Blender add-on to transfer animation to a different person for example.
You don't need to manually pin faces anymore! At least, not from the scratch. Our face alignment is not yet always 100% accurate, so it requires some manual adjustments sometimes, but it radically reduces the time of shaping the models. So much, we’ve actually stopped doing this manually in our team while testing and we don’t want to go back!
Usually you can just rely on our own tracking algorithms, but just in case, you can improve tracking quality using tracking data from other trackers. For example, you can import tracks of Nuke's built-in Tracker node and FaceTracker will take them into account with the highest priority.
You can mask out some polygons of the geometry that you want to exclude from tracking. For example some reflective parts of skin or eyeballs. Though usually you don't need to do this, since FaceTracker deals with them pretty well on its own.
Another way to improve tracking (in those rare cases when it actually fails) is excluding occlusions from tracking. To do it you can connect a roto node to the Mask input of FaceTracker and (re-) launch tracking.
Our plugins are written with C++ using low-level Nuke API. So you can be sure you're working with tools with native-like experience.
It's waiting for you here.
Sure! They're on the examples page.
We assume that you're going to use FaceTracker along with FaceBuilder, using FaceBuilder's output as an input model for FaceTracker. But you can easily use FaceTracker without FaceBuilder, check the question about a custom model below to find out how.
Yes, the workaround is described in the Custom Model question below. You don't need a license for FaceBuilder to export a default model.
You can export the default model of FaceBuilder, and then modify the shape keeping vertices count and order. Then you can import it back, and connect to FaceTracker. It uses vertices to understand what is what in the model (e.g. nose, lips, eyes, etc). So, you can manually move vertices if FaceBuilder's result doesn't suit you, remove or add faces, change normals and UVs, wrap a scanned model with our default model, etc.
Not at the moment, but we are considering adding support for custom blendshapes in future.
Not really, but it'd be easier if you can find a neutral expression to start from.
No, the tracking information is saved frame by frame, so you can abort tracking at any moment keeping the tracking information you have already got intact.
Yes, the licenses for FaceBuilder and FaceTracker are independent. You need both to work with both plugins.
You can use Nuke's built-in WriteGeo node with ABC format. After pressing "Execute" button specify the frames you want to export.
Yes! You'd only need to install Nuke 12.2v4 or newer.