Skip to main content
Version: 5.x

Gaussian Splatting

Gaussian splatting, introduced in the 2023 paper “3D Gaussian Splatting for Real-Time Radiance Field Rendering”, is a modern approach for modeling and rendering 3D scenes. It stands out from Radiance Field methods like NeRFs for its quick setup (training) and faster rendering times, while maintaining or improving visual quality, and being easier to understand and modify.

The data is a cloud of “Splats”, generated automatically from a set of photos from different angles.

Importing Gaussian Splatting

In the Asset Browser panel, click on +, then Import Asset. Next select, the gaussian splatting .ply file you want to import.

You can drag a .ply file directly into the scene hierarchy to import the asset and create a new visual.

You can see the number of splats, and size that the compressed Gaussian Splatting asset file takes up inside the Lens, in the Inspector panel when your asset is selected.

PlayCanvas provides a powerful 3D Gaussian Splat editor called SuperSplat. Additionaly, they provide a guide to authoring PLY files.

Gaussian Splatting Component

This component allows you to display the Gaussian splatting itself. Add it to any scene object to see your splats in your scene.

This is similar to how you would use the Render Mesh Visual component to display 3D meshes.

Gaussian Splatting Material

 

Optionally, you can override the default Gaussian Splatting material with a custom one, to create special effects.

  1. Create a new material + shader graph and in the Asset Browser panel.
  2. In the shader graph, set the Shader node to Gaussian Splatting mode.
  3. Use the material in the Gaussian Splatting component.

In Gaussian Splatting mode, there are two new inputs that can be overridden. They are applied in the Vertex shader stage.

  1. Splat Center: change the location of the splat
  2. GSplat Point Size: specify the size of the splat, in pixels.

Examples for custom Gaussian Splatting materials

Colorize

Discard by world position

Point Cloud

Wavy Fruit

GSAF - Gaussian Splatting Animation Frames

In the next section we will address a feature of Lens studio that will allow the user to import a series of .ply files from a designated folder.

These files will be imported into Lens studio in such a manner that it will compress and provide a single GS file, that can be animated using a simple animation script.

Preparing the .ply folder for import

  1. The .ply frame sequence files will all need to be under the same folder.
  2. This folder MUST only contain ply files in order for Lens studio to initiate the GSAF Asset settings.
  3. PLY files naming convention should be in order sequentially to avoid frame jumps in the GSAF final output. For example: Frame_001.ply, Frame_002.ply, etc.

Importing the GS files into Lens studio

  1. Once the folder with the .ply files are sorted, in the Asset Browser panel, click on + then Import Asset.

  2. Next select the first .ply file within the folder you want to import the .ply frame files.

  3. Once Lens studio recognizes that the .ply is within a folder with only .ply files a new message-box will pop-up. Enable the checkbox that indicates to convert all .ply files from the same folder. (Checkbox should be enabled by default).

  4. Below the checkbox there are 4 attributes for Gaussian Spaltt import and compression capabilities and you can see it will display each of these attributes with a range from 4 to 16 bit. Higher settings for these attributes will yield better visuals of the Gaussian splatt at a cost of the GSAF file size and import time.

You can also drag the .ply file directly into the asset browser to initiate the GSAF process of Lens Studio.

  1. The import process should take a few minutes depending on the .ply files’ size, quantity and the attributes settings. And should be added to the asset browser once the process is complete.

  2. Once importing and compression is done, you can inspect the newly added GSAF, the number of splatts and size that the compressed Gaussian Splatting asset file takes up inside the Lens. You will notice a new field which will indicate the total number of frames for this GSAF file as well:

Placing the GSAF file into the scene

  1. Placing the newly created GSAF is done by using the same Gaussian splatting component for a single ply file.
  2. Once added to the scene, an additional field to indicate the current active frame number for this Gaussian splatting animation file as seen on the image below.

Animating the newly imported GSAF file

Once the GSAF asset is placed, the way the animate it can be achieved using a dedicated script.

In your Asset Browser panel, press the + > JavaScript to add a new Javascript file to your project. Double-click the file to open it, and paste the following code in the Script Editor panel.

Next, select the object with your Gaussian Splat, press Add Component in the Inspector panel, and add your newly added script.

//@input Component.Text textComp
//@input float frameDuration {"widget":"slider", "min": 0.01, "max": 2, "default":0.333}
//@input int maxFrames {"min": -1}
//@input bool pingPong

var gsVisual = script.sceneObject.getComponent(
'Component.GaussianSplattingVisual'
);

function getFrameCount() {
var numFrames = gsVisual.asset.getNumberOfFrames();
if (script.maxFrames > 0) numFrames = Math.min(numFrames, script.maxFrames);
return numFrames;
}

var forward = true;
function nextFrame() {
var numFrames = getFrameCount();
var cur = gsVisual.activeFrame;
var next;
if (forward) {
if (cur < numFrames - 1) {
next = cur + 1;
} else if (script.pingPong) {
next = numFrames - 2;
forward = false;
} else {
next = 0;
}
} else {
if (cur > 0) {
next = cur - 1;
} else {
next = 1;
forward = true;
}
}

gsVisual.activeFrame = next;

next += 1; // UI should display 1-based index not 0-based
//print("frame := " + next);
if (script.textComp) {
var gsfps = Math.round((1.0 / script.frameDuration) * 100) / 100.0;
script.textComp.text =
'Frame #' + next + ' / ' + numFrames + '\n' + gsfps + ' fps';
}
}

var interval = script.frameDuration; // in seconds
var timeNextSwap = getTime() + interval;
script.createEvent('UpdateEvent').bind(function (eventData) {
var delta = getTime() - timeNextSwap;
if (delta >= 0) {
nextFrame();
timeNextSwap = getTime() + Math.max(0, interval - delta);
}
});

Once the script is added to the component, The component will look something, like this:

Take a look at the Scripting section to learn how to add scripts to your project.

Script parameters:

  1. Text Component: Shows the Current frame of the GSAF rendered (For debug purposes)
  2. Frame duration: Set the time in which every GSAF frame will be rendered. Lower duration means the entire GSAF will be rendered faster.
  3. Max Frames: Sets the number of frames to be rendered (Default: -1 which will rendered all of the GSAF’s frames)
Was this page helpful?
Yes
No

AI-Powered Search