Upper Body Tracking
When developing Lenses for the front camera, developers often need to use 3D body tracking for tasks like neck movements, attachments, and collisions. While Face Mesh, Body Mesh, and full 3D Body Tracking can address these needs, 3D Body Tracking is typically more suited for tracking a person who is fully in frame. To optimize for selfie use cases, a lighter version has been introduced - Upper Body Tracking 3D.
Upper Body Tracking 3D
Upper Body Tracking 3D asset provides a subset of humanoid attachment points, such as the hips, spine bones, neck, head, shoulders, and upper arms. Other than this you may add to scene and use Object Tracking 3D component as usual.
Upper Body Mesh
This feature also introduces a specialized version of Body Mesh called Upper Body Mesh.
When you add the Upper Body Mesh in the Scene Hierarchy panel, several scene objects are automatically added to your scene. These include:
- Head Component
- with a Head Mesh attached to it (which is a Face Mesh with the skull enabled).
- 3D Object Tracking configured with Upper Body Tracking asset
- And Upper Body Mesh attached to it.
This creates a full, seamless-looking mesh of a person’s upper body, ideal for developing selfie (front camera) lenses. It can be used in various cases, such as an occluder for different body accessories or a collider for physics, and it allows for seamless texturing of the face and neck.
Please note that Upper Body Mesh doesn’t support external mesh.
Changing Face / Body index
To change the body index of a combination of the Upper Body Mesh and Head, follow these steps:
- Set the
Face Index
on theHead Mesh
asset. - Set the
Face Index
on theUpper Body
Mesh asset. - Set the
Face Index
on theHead Binding
component.
Texturing Upper Body Mesh and Face Mesh
You can find a reference model and UV map layout here. Use any 2D or 3D editor of your choice to create textures for the model.
Once your textures are ready, follow these steps to apply them to the Upper Body Mesh:
- Import the exported texture files into Lens Studio.
- Create a new material that allows you to adjust the mesh UV index. For example, use a new Uber PBR material and name it Body Material.
- Configure the material by setting the input parameters with the imported textures.
- Select the Upper Body Mesh scene object and assign Body Material to the render mesh visual component attached to it.
To apply textures to the Head Mesh:
- Duplicate Body Material and rename it to Head Material.
- Select Head Material and change texture UV to
Mesh UV 1
for all texture inputs. - Assign Head Material to the Head Mesh.
You need a separate material because We need to use a second UV channel for the Face mesh.
Try combining Upper Body Mesh with skin segmentation map applied to the camera to create tattoo effects.