Gaussian Splatting
Gaussian splatting, introduced in the 2023 paper “3D Gaussian Splatting for Real-Time Radiance Field Rendering”, is a modern approach for modeling and rendering 3D scenes. It stands out from Radiance Field methods like NeRFs for its quick setup (training) and faster rendering times, while maintaining or improving visual quality, and being easier to understand and modify.
The data is a cloud of “Splats”, generated automatically from a set of photos from different angles.
Importing Gaussian Splatting
In the Asset Browser
panel, click on +
, then Import Asset
. Next select, the gaussian splatting .ply
file you want to import.
You can drag a .ply
file directly into the scene hierarchy to import the asset and create a new visual.
You can see the number of splats, and size that the compressed Gaussian Splatting asset file takes up inside the Lens, in the Inspector
panel when your asset is selected.
PlayCanvas provides a powerful 3D Gaussian Splat editor called SuperSplat. Additionaly, they provide a guide to authoring PLY files.
Gaussian Splatting Component
This component allows you to display the Gaussian splatting itself. Add it to any scene object to see your splats in your scene.
This is similar to how you would use the Render Mesh Visual
component to display 3D meshes.
Gaussian Splatting Material
Optionally, you can override the default Gaussian Splatting material with a custom one, to create special effects.
- Create a new material + shader graph and in the
Asset Browser
panel. - In the shader graph, set the
Shader
node toGaussian Splatting
mode. - Use the material in the
Gaussian Splatting
component.
In Gaussian Splatting mode, there are two new inputs that can be overridden. They are applied in the Vertex shader stage.
Splat Center
: change the location of the splatGSplat Point Size
: specify the size of the splat, in pixels.
Examples for custom Gaussian Splatting materials
Colorize
Discard by world position
Point Cloud
Wavy Fruit
GSAF - Gaussian Splatting Animation Frames
GSAF is a feature of Lens studio that allows the user to import a series of .ply files as frames of a single animated GS asset, effectively creating a volumetric video.
Importing into Lens studio (Creating the asset)
-
In the
Asset Browser
panel, click on+
thenGaussian Splatting from .ply files
. -
Select the .ply files to use as frames.
-
There are 4 sliders for compression / quality trade-off, one for each attribute, with a range from 4 to 16 bit. Higher settings for these attributes will yield better visual quality at a cost of the GSAF file size and import time. The default configuration is recomended for most use-cases.
-
The import process should take a few seconds, depending on the .ply files’ size, quantity and the attributes settings.
-
Once importing and compression is done, you can inspect the newly added GSAF asset. It displays the number of splats and size that the compressed Gaussian Splatting asset file takes up inside the Lens. You will notice a new field which will indicate the total number of frames for this GSAF file as well.
Placing the GSAF visual into the scene (Creating the visual)
-
Placing the newly created GSAF is done by using the same Gaussian splatting component for a single ply file - just drag the asset into the scene hierarchy.
-
When a GS visual component has a GSAF asset attached, a few additional fields appear: Active Frame to indicate the current active frame number for this Gaussian splatting animation file as seen on the image below. Auto Play check to automatically load the visual in play mode FPS Animation speed in Frames Per Second