Cloth Simulation
Overview
This guide covers the main concepts of how cloth simulation works in Lens Studio and Snapchat. This guide is script heavy. However, you can also review the Cloth Simulation Template to see an example of what you can do with Cloth Simulation.
Cloth simulation
Cloth simulation renders and simulates the real-world transformation of clothes in real time. Lens Studio now includes the realistic cloth simulation feature. Cloth simulation allows you to make cloth move with human actions (move, stretch and bend), collision, gravity, air friction, etc. Each single point on cloth may move in different directions due to the combination of different forces.
Getting Started with Cloth Mesh
You can import your own mesh for Cloth Simulation. You’ll need to paint at least one vertex color on your own custom mesh to follow specific objects or set different simulation properties in Lens Studio.
In Blender, click on your mesh and switch to Vertex Paint Mode. Choose colors to paint on the mesh. You can pick different parts of the mesh based on different vertex colors in Lens Studio.
With different vertex colors, you can bind vertices to Pivot Points in Lens Studio. The bind parts won’t be affected by cloth simulation, but are stationary relative to Pivot Points. For example, you can bind the red part of the mesh bandana to the Neck Pivot Point object. Thus, the red part of the bandana remains relatively static to the Neck and the black part will have the cloth simulation effect.
You can see more details on how to import 3D objects into Lens Studio here. We recommend a maximum of 3000 triangles and single-sided mesh with two-sided material for the cloth mesh.
In the case of a tutu we use a simple flat mesh with only 1388 triangles. The result is a relatively good cloth simulation with good performance.
3D Body Tracking and Cloth Mesh
To create a cloth mesh for 3D Body Tracking, there are several FBX 3D models in the Cloth Simulation Template. You can find them in the Resources -> 3D Body Tracking References.
With the bindPoints FBX, you have a reference for the cloth binding points. They are the spheres in the screenshot. Combining bind points and 3D body tracking, you can then bind different cloth vertices to the body tracking points. For example, with the jacket, you can bind the green part to the right arm bind point, the yellow part to the left arm bind point, and the red part to the neck.
With Body_Mesh_T_Pose FBX, you can have a reference for the body mesh occluder. You can use it for cloth occluder.
With the Collider Guides FBX, you use it as the reference for body part colliders in the Lens Studio.
When creating cloth mesh for 3D body tracking, use the FBX files as the reference for cloth transform. In order to have the right binding and occlusion, click here to see how to export and import 3D objects into Lens Studio.
You can also find the 3D Body Tacking Prefab in the Resources ->Prefabs in the Cloth Simulation Template. For details, see the examples in the Cloth Simulation Template.
Getting Started with Cloth Component
Clothes are added to a scene using a Cloth Component added to an object. To create the Cloth component. Go to Objects > + Scene Object. With the Scene Object selected, in the Inspector panel, select Add Component -> Cloth.
Cloth Component UI
- Cloth mesh: attach imported mesh and cloth materials
- Shadow mode: choose shadow mode for cloth mesh
- Update normals: enable/disable normals update
- Debug mode: enable/disable debug draw for cloth mesh
Simulation
- Iteration: means iteration simulation is done per step and increases constraint stiffnesses.
- FrameRate: ensures that simulations are the same speed on different devices.
- Bend mode: Isometric or linear
- Gravity: is the external gravity force vector. Usually, the value is -Y for normal gravity.
- Mass: is mass of each vertex on cloth mesh.
- Stretch Stiffness: controls the hardness of the cloth when stretching it and sets the stiffness of edge distance constraint to preserve length. The smaller the value is, the softer the cloth will be.
- Bend Stiffness: controls the hardness of the cloth when bending it and sets the stiffness of triangle bending constraint to preserve the isometric shape. The smaller the value is, the softer the cloth will be.
- Friction: is air friction, which controls how fast to reduce the motion to zero. Friction is coefficient to damp motion. The smaller the value is, the harder the cloth will calm down.
- Max Acceleration: limits maximum acceleration
- **Enable Colliders : determines whether collision repulsion is enabled with collider models.
- Repulsion Stiffness: means the hardness of repulsion constraint. The smaller the value is, the softer the cloth will be.
- Repulsion Offset: means the offset when points on cloth are too close to colliders. This is an additional offset to avoid close penetration.
- Repulsion Friction: means the collision friction to damp relative motion.
- Add Collider: attach colliders.
Vertex Binding
- Color: choose vertex color for binding
- Color Mask: enable/disable RGBA channels for vertex color
- Follow Object: attach follow object
Physics Collider Component
To make the collision effect with the cloth mesh, use the Physics Collider Component in the Lens Studio.
We have four different shape type:
- Box Collider: adjusts the xyz size.
- Capsule: adjusts the axis, length, and radius.
- Levelset: imports from Resources.
- Sphere: adjusts the radius.
With the Show Collider field enabled, you can also draw the collider in the Scene.
You can add the Physics Collider Component to different tracking, like the head, hand, or even 3D body tracking. Here is an example of 3D body tracking with full body part colliders. See the examples in the Cloth Simulation template.
Add the ClothVisual Component with Script
To set up the ClothVisual Component, create a new Cloth Simulation Script. In the Asset Browser
panel, select + -> Script. With the newly added script selected, go to the Inspector panel to replace the text in your script resource with the following code:
// @input Asset.RenderMesh mesh
// @input Asset.Material material
//Create ClothVisual Component
var clothVisual = script
.getSceneObject()
.createComponent('Component.ClothVisual');
//Simulation Settings
clothVisual.bendMode = ClothVisual.BendMode.Isometric;
clothVisual.mesh = script.mesh;
clothVisual.mainMaterial = script.material;
clothVisual.gravity = new vec3(0, -100, 0);
clothVisual.stretchStiffness = 0.1;
clothVisual.bendStiffness = 0.1;
clothVisual.friction = 0.1;
clothVisual.repulsionEnabled = true;
clothVisual.repulsionOffset = 0.1;
clothVisual.repulsionStiffness = 0.1;
clothVisual.repulsionFriction = 0.1;
clothVisual.iterationsPerStep = 60;
clothVisual.frameRate = 30;
clothVisual.debugModeEnabled = false;
clothVisual.drawCollider = true;
clothVisual.updateNormalsEnabled = true;
clothVisual.mergeCloseVerticesEnabled = true;
//Initialize ClothVisual Component
clothVisual.onInitialized = clothInitCallback;
clothVisual.updatePriority = 0;
function clothInitCallback(clothVisualArg) {
clothVisualArg.resetSimulation();
}
Add a new scene object in the Scene Hierarchy
panel and rename it to “Cloth Simulation” (Scene Hierarchy panel > + > Empty Object
). In the Inspector
Panel, add the Cloth Simulation script to the object as a new Script Component. (Inspector panel > + > Cloth Simulation
). Now you can attach your own mesh, material, and colliders.
Fixed points on cloth mesh
In cloth simulation, you need some points on cloth mesh to be bound to specific objects, like the head or body. Otherwise, with the gravity property, the cloth will only fall down. You can pick points based on the vertex color to transform with binding objects and attach/detach flexibly.
Cloth mesh may be very large and there may be multiple vertices on it. We support a flexible way to get vertex index via vertex color on cloth mesh.
Make fixed points following objects with scripts
Here is the script to pick points based on the vertex colors. Note that we set red vertices to follow the FollowObject as an example:
// @input SceneObject followObject
// different vertex colors
var red = new vec4(1.0, 0.0, 0.0, 1.0);
// different color mask (picking which color channel to compare)
var colorMask = new vec4b(true, true, true, true);
var redIndices = [];
//Create Cloth Visual Component
var clothVisual = script
.getSceneObject()
.createComponent('Component.ClothVisual');
//Initialize ClothVisual Component
clothVisual.onInitialized = clothInitCallback;
clothVisual.updatePriority = 0;
function clothInitCallback(clothVisualArg) {
// binding all red vertices to the Scene object: follow_obj
redIndices = clothVisualArg.getPointIndicesByColor(red, colorMask);
for (var i = 0; i < redIndices.length; i++) {
clothVisual.setVertexBinding(redIndices[i], script.followObject);
}
clothVisualArg.resetSimulation();
}
VertexSimulationSettings
The VertexSimulationSettings provides possibilities to set specified vertices with different simulation settings on mass/stretchStiffness/bendStiffness/friction. Like fixed points on cloth mesh, you can select the vertex color to modify based on the vertex colors available in your mesh. Notice that, for each vertex simulation property, the global weight property is available. Global weight determines how much global settings influence the Cloth Simulation Script. If you set Global Weight to 0.0, you are using the absolute value of the property in the Vertex Controller on this vertex. If you set it to 1.0, you ignore the property value in the Vertex Controller. When you disable the vertex controller, all the vertex properties are 0.0 by default, so the mesh will be simulated by all global simulation settings.
Let’s take Bend Stiffness as an example.
let G = globalSimulationSettings.bendStiffness
let G.VertexWeight = globalSimulationSettings.bendStiffnessVertexWeight
let V[i] = perVertexSimulationSettings[vertexIndex].bendStiffness
let V[i].GlobalWeight= perVertexSimulationSettings[vertexIndex].bendStiffnessGlobalWeight
In the simulation, the final result for Bend stiffness will be bendStiffness[i] = G * V[i].GlobalWeight + V[i] * G.VertexWeight;
Add Vertex Simulation Settings with script
Here is the script to pick points based on the vertex colors. Notice here we customize yellow vertices with Vertex Simulation Settings as an example:
//Create Cloth Visual Component
var clothVisual = script
.getSceneObject()
.createComponent('Component.ClothVisual');
// yellow vertex colors
var yellow = new vec4(1.0, 1.0, 0.0, 1.0);
// different color mask (picking which color channel to compare)
var colorMask = new vec4b(true, true, true, true);
var yellowIndices = [];
//Initialize ClothVisual Component
clothVisual.onInitialized = clothInitCallback;
clothVisual.updatePriority = 0;
function clothInitCallback(clothVisualArg) {
yellowIndices = clothVisualArg.getPointIndicesByColor(yellow, colorMask);
// create a vertex settings set specific simulation settings value on it
var vertexSettings = ClothVisual.createVertexSettings();
vertexSettings.stretchStiffness = 0.75;
vertexSettings.bendStiffness = 0.75;
vertexSettings.stretchStiffnessGlobalWeight = 0.0;
vertexSettings.bendStiffnessGlobalWeight = 0.0;
// set this vertex simulation settings to all the yellow vertices with
for (var i = 0; i < yellowIndices.length; i++) {
clothVisualArg.setVertexSettings(yellowIndices[i], vertexSettings);
}
// we can also use getVertexSettings(vertexIndex) and modify vertex settings.
clothVisualArg.resetSimulation();
}
Next Steps
Now that you’ve learned the basics of Cloth Simulation, try making your own cloth. See different examples in the Cloth Simulation Template.