Face Mesh
The Face Mesh resource provides a 3D mesh that will mimic the user’s facial expression in real time. It can be used anywhere a mesh may be used. For example, you can use the Head Binding component and the Face Mesh resource to add a 3D face mask on the user. Take a look at the Face Mesh Template for examples on how to use Face Mesh.
Adding a Face Mesh
You can automatically add a Face Mesh to a user's head by clicking + > Face Mesh in the Objects Panel.
Alternatively, you can add the resource alone by pressing + > Face Mesh in the Resources panel.
You can then use this resource in the Mesh field of a Render Mesh Visual to display it in your Lens.
Face Mesh Customization
To change the setting of the Face Mesh, select the Face Mesh resource in the Resources panel, and adjust its settings in the Inspector panel.
You can set which Face in the camera drives the Face Mesh by setting the Face Index field. The first face detected by the camera has a face Index of 0, and the second face has an index of 1, and so forth.
You can also specify which part of the Face Mesh should be rendered by enabling or disabling the Face Geometry, Eye Geometry, Mouth Geometry, Skull Geometry or Ear Geometry checkboxes.
When using the Skull or Ear options, make sure to use the UV1 option for the textures in your material, as the UV0 option is used for the Face Mesh without the whole skull.
If you're using a Graph material with Material Editor, you can use the UV1 node:
You can also add Face Mesh with the Skull option directly by adding Head Mesh:
External Face Mesh
In addition to the built in face mesh, you can also provide your own custom mesh for the users face to map to through External Face Mesh.
Through External Face Mesh, you are able to:
- Transition between the built-in face mesh and the custom face mesh
- Add additional Blend Shapes to modify the shape of the face mesh.
In order for your custom mesh to map correctly to the user’s face, make sure to use the same UV map as the built in Face Mesh. You can download both the UV maps, as well as the 3D model below
To add your own custom mesh, first import your mesh into Lens Studio by dragging your file into the Resources panel. Then, select the Face Mesh resource in the Resources panel, and in the Inspector panel, select the field next to External Face Mesh. In the pop-up, choose your newly imported mesh.
Once imported you will have additional options:
- External Scale: scales the custom mesh
- Expression Multiplier: how much the user’s expression will affect the mesh
- **Mapping UV:**the UV map to use to map the mesh to Face Mesh
Hiding Parts of the Face Mesh
In some cases you may want to hide parts of the Face Mesh. You can do this by using an Opacity Mask on a PBR material and use that material for the Face Mesh, similar to how you would on other meshes.
To create a PBR material, in the Resources panel, press + > PBR.
Then select your Face Mesh object in the Objects panel, and in the Inspector panel, set the material of the Render Mesh Visual, to be the new PBR material.
Then, in the Resources panel, select the PBR material, and set its Blend Mode to Normal so the material can have transparency. Next, enable the Opacity Texture checkbox and a Texture field will be enabled.
You can set this Texture field to your own texture. The texture should correspond to the Face Mesh UV, which you can find in the next section.
For example, if we use this mask:
The final Face Mesh will look like:
Download example Face Mesh Opacity Textures
Download the Face Mesh UV
The Face Mesh UV file provides a reference on how 2D textures are applied to the Face Mesh. These textures can be used to modify how your Face Mesh appears. For example, you can use it to create an opacity mask texture to hide parts of the Face Mesh from appearing in your Lens as in the previous section, or create a normal map texture to add additional details to the Face Mesh. Take a look at the Materials guide for more information on how different textures can be used to create a material in Lens Studio.
Download Face Mesh UV Reference
When the Skull or Ear property is enabled, you will need to use the UV1 set as these properties contains additional polygons.
Download the Face Mesh 3D model
You can also download a 3D model of the Face Mesh to view your content in other 3D software. For example, you can use it as a reference to paint on in software such as Substance Painter. Take a look at the Substance Painter texturing guide for more information on how to create textures to use on a material in Lens Studio.
Download Face Mesh with Skull 3D Model
Download Face Mesh with Skull 3D Model and Ears
At this time it’s not possible to replace Face Mesh object in the Lens Studio with other face mesh 3D models outside of the Lens Studio.