Face Landmarks
To make it easier for you to mix-and-match our different examples, this template has been converted to an importable asset in the Asset Library with the name Face Landmarks
.
The Face Landmarks template allows you to track 93 points on the face which we call Face Landmark. The template also provides additional functions to attach objects, check the distance between points to trigger an effect.
Tutorial
Guide
This template comes with several helper scripts to help you to attach your objects to a single point or the center of multiple points. You can also use the distance between the points to create different interactions.
Take a look below for a description of each helper and information about their settings.
Pin To Face Landmarks
This is the main script of the template which allows you to attach your object(s) to a single point or the center of multiple points.
Since Face Landmark uses 2D space to track the points on the face we need to get the depth of each point with the help of the Head Binding component.
This script gets the depth from Head Binding and then uses that to convert each point position from 2D transform into 3D transform. With that being said you need to always have a reference of Camera
as well as Head Binding
on the script.
If you want to learn more about how to convert 2D transform to 3D transform, please take a look at the Screen Transform API page.
To attach your objects to Landmarks, just place any content as a child of the PinToFaceLandmarks
script in the Scene Hierarchy
panel.
The sections below walk through each customizable setting in the PinToFaceLandmarks
script.
Placement Mode
There are two options available for placing the objects on the Landmarks.
Stick In Between Points: This option attaches all the child content of the script to a Face Landmarks position.
Spawn At Each Points: This option distributes the child objects along all the Face Landmarks points.
You can see the difference in a gif below:
Points Mode
Points refers to each placement of the landmarks as you can see in the diagram below:
There are two options available for selecting point(s):
Custom Points: In this option you can specify a specific point(s) for the object to be attached to. Take a look at the Face Landmarks diagram which you can also find in the Asset Browser
panel named Landmarks Points Guide [LOOK_AT_ME]
to find a desired point(s) and then add the number into the Points
array input.
If you put more than one point, it will find the center between points and place the content in the center.
Predefined Points: When predefined points are selected you will get a Defined Points
dropdown. With this drop down menu, you can select one of the options to attach the content to a desired place.
Predefined points is just a short hand for a subset of the points available.
Offset
You can offset all of the child content from the Face Landmark point using the Offset
input.
FaceLandmarksDistanceTrigger
This is one of the helper scripts which allows you to trigger an action based on the distance of two points. You can use the output of the script to move, scale, rotate or change the opacity of an object. Additionally, you can also use it to create custom face events such as eye blink.
Like before, to use the script, we need to specify the Camera
and the Head Binding
so the script can access the landmarks.
The sections below walk through each customizable setting in the FaceLandmarksDistanceTrigger
script.
Getting Distance
The first thing we’ll do is define two sets of landmarks to calculate distance from. Like before, you can use the Points
mode to select between predefined points or you can have custom points.
This option is the same as PinToFaceLandmarks
scripts. To learn more about it please read the guide above in the PinToFaceLandmarks
section.
Min Distance / Max Distance
With the two points specified, this script can provide raw values between two points.
You can easily see the raw distance on each frame with use of the Print Distance
checkbox. By checking the checkbox, you can see the value in the logger panel.
If you don't see your logger. Click on Add New Panel Button on top right corner of the Lens Studio and then select the logger.
As you can see this value can be varied based on the points that you select. To make the values easier to use, we will convert the raw value to a value between 0 to 1 using the Min Distance
and Max Distance
values.
Min distance is the minimum distance between two points that we specify. For example, to find a minimum distance between upper and lower lips is to close your mouth and see the Distance
value in the logger. That value is your Min Distance
value.
The same way works for Max distance as well. For example, to find the maximum distance between upper and lower lips, open your mouth and see the Distance
value in the logger. That value is your Max Distance
value.
This workflow is not only for lips and can be applied to eyelids and more.
Depending on the user using the Lens, these values might be different so don’t choose an extreme value.
Smoothing
By checking the Smoothing
checkbox you can use the slider to make the value more smooth and less bumpy, but it may result in a delay.
Custom Triggers
You can enable the Custom Trigger
option to specify a trigger that will occur when the distance hits the Min Distance
, and when it hits the Max Distance
.
You can use this custom trigger with Behavior script to create events and triggers that did not exist before in the Lens Studio! For example you can get the distance between two eyelids and create a blink trigger event.
To learn more about the blink trigger you can look at the Eye Right or Eye Left example in the template.
You can choose a name for the Min Trigger
to get called when the distance value is equal or less than the Min Distance
number.
The same way also applies to Max Trigger
which you can specify a name to get called when the distance value is equal or greater than the Max Distance
that we explain earlier.
FaceLandmarksDistanceResponse
This script allows you to get data from the FaceLandmarksDistanceTrigger
and convert that value so you can apply the value to Move, Rotate an object, drive blend shapes, or make an object disappear and appear.
you can add more features to this script since this is just a JavaScript file and expand the script based on your needs.
The sections below walk through each customizable setting in the FaceLandmarksDistanceResponse
script.
Distance Script
To make the script work, you need to have a reference to FaceLandmarksDistanceTrigger
so this script can receive the values from that.
Mode
There are three modes available.
Transform Mode:
This mode allows you to move or scale your object based on the values that received from the FaceLandmarksDistanceTrigger
.
Blendshape Mode:
This mode allows you to drive a blendshapes based on the values from the FaceLandmarksDistanceTrigger
.
Opacity Mode:
This mode allows you to make your object appear or disappear based on the values received from the FaceLandmarksDistanceTrigger
.
Previewing Your Lens
You’re now ready to preview your Lens! To preview your Lens in Snapchat, follow the Pairing to Snapchat guide.