Skip to main content
Version: 5.x

World Tracking Planes

World Tracking Planes provide Tracked Planes based on the surfaces your device detects. This allows your experience to match your environment. This example uses the World Tracking Planes API to manage planar scene objects that have collision and occlusion. There are two example experiences: spawning static objects and launching physics objects.

This feature works on Snapchat and Spectacles. However, if you are building for Spectacles, consider the WorldQueryModule which allows you to raycast to the world without waiting for World Mesh, and reduce performance cost.

World Tracking Planes API

The World Tracking Planes API consists of three events:

  • WorldTrackingPlanesAdded: Triggered when plane(s) are newly detected
  • WorldTrackingPlanesRemoved: Triggered when plane(s) are no longer detected
  • WorldTrackingPlanesUpdated: Triggered when currently detected plane(s) are updated

Each event returns an unordered array of Tracked Planes. This array gives us a position, orientation, and size of the plane. Learn more about these events and see sample code in the API documentation.

To use the World Tracking Planes API in our TrackedPlanesManager.js, we must be sure to:

  1. Add the Device Tracking component to the camera and set it to world.
  2. Write deviceTracking.worldOptions.nativePlaneTrackingType = NativePlaneTrackingType.Both at the top of the script.

How This Example Works

The World Tracking Planes API gives us Tracked Planes, but we want collision. So in this example, the TrackedPlanesManager.js updates an array of in-scene plane objects that have collision on them.

The plane object prefab has the PlaneObj.js script on the root, which refers to the plane object's render mesh visual, collider, debug text, and materials to use based on whether the plane's orientation is horizontal or vertical.

TrackedPlanesManager.js orients the PrefabPlaneObj [EDIT ME] root and scales the Unit Box (cm) child to match the correlating Tracked Plane returned by the World Tracking Planes API events. (We don't scale the root because we want the DebugText child to stay the same size.)

The code that scales the Unit Box (cm) scene object works because the mesh is a unit box - 1 cm cubed. If you drop a non-unit mesh into this field, scaling will be off.

Customize the look of your planes by editing and applying changes to the PrefabPlaneObj [EDIT ME] prefab:

  • Change the texture by selecting the current pattern material in the Asset Browser panel->Materials folder and dragging a new texture into the Inspector Panel->Graph Parameters->Pattern field.
  • Create new materials for your planes.

There are several lens-side operations going on to enable the examples below. Because we have box colliders to work with, and the event Tracked Planes have a height of zero, TrackedPlanesManager.js sets a shallow height of .5 units for the instantiated plane objects. Their child Occluder object functions to hide what is behind the plane.

PlaneObj.js also stores a reference to the Tracked Plane that it's based on. TrackedPlanesManager.js uses this to make sure our in-scene plane objects match the constantly changing array of TrackedPlanes returned by the World Tracking Planes events.

The World Tracking Planes API is seeking to create a static scene of the most accurate planes. That means that the plane objects in the scene are always shifting. You cannot rely on a given plane to be ever present; parenting new objects to plane objects is not recommended.

Compatibility

Lens Studio 4.19 and up

PlatformHorizontalVertical
iOS
Android
Spectacles

Limitations

The World Tracking Planes technology performance varies based on the scene layout, lighting conditions, and the player's movement in the scene. Performance notes:

  • Requires stereo cameras on Spectacles (won’t work well if one camera is covered)
  • Roughly 1.5 seconds to initialize
  • 2-3% of distance accuracy
  • 0.7-2.5m range
  • Needs textured surfaces (not white wall)
  • Needs reasonable lighting conditions (won’t work in twilight)

Examples

The example comes with two examples. These examples can be found in the Scene Hierarchy panel under their names. Try enabling the example scene object by pressing the tick-box to the right of their name.

You can push the Lens to Snapchat or Spectacles. Currently, planes appear incorrectly in the Lens Studio Editor Interactive Preview mode of the Preview panel. Stay tuned for this bug fix!

Tap To Spawn Example

This example allows you to spawn any prefab in the world. TapToSpawn.js gives you customization as to how to add some randomization and offset relative to the plane orientation(s) you'd like to spawn.

Set Up Object Spawning

First, create a prefab. Then modify the example to spawn our own prefab in the world. To do that, select one of the Tap To Spawn [EDIT_ME] objects under the Tap To Spawn example.

Then in the Inspector panel, replace the prefab input with the prefab we just created. Adjust the TapToSpawn.js properties to customize how and where your prefab(s) is spawned:

  • Prefabs: Tapping will spawn a random prefab from this list.
  • Surface Type: Spawn on a Horizontal, Vertical, or Both surface(s).
  • Min Offset From Plane: The minimum distance in unit cm from the plane to the spawn position.
  • Max Offset From Plane: The maximum distance in unit cm from the plane to the spawn position.
  • Up Axis: Choose Plane Normal to align your prefab's up axis with the plane's up axis (i.e. the plane normal). Choose Unchanged for your prefab to spawn as it appears in the editor.
  • Fwd Axis: Choose Random to rotate your prefab to a random angle around the up axis. Choose Unchanged for your prefab to spawn as it appears in the editor.
  • Min Scale: The minimum % original scale for the spawn scale.
  • Max Scale: The maximum % original scale for the spawn scale.

To eliminate randomization:

  • Add only one prefab to the array
  • Set the min and max offsets to the same value
  • Set the min and max scales to the same value

Launcher Example

This example shows how we can launch physics objects onto the world tracking planes and see them interact.

Set Up Object Launching

First, create a prefab. Make sure your prefab has a Physics Body component on the same scene object as your Render Mesh Visual component. Add these settings to you Physics Body component:

  • Set the Shape Type to match your Render Mesh Visual.
  • Select Fit Visual or set the collider size manually.
  • Select Dynamic.

Then modify the example to launch our own prefab in the world. To do that, select the Launcher [EDIT_ME] object under the Launcher example. Drag your new prefab into the Launcher.js Prefab field. This script has four modifiable properties:

  • Prefab: the prefab to spawn and launch
  • Launch From Obj: the transform to use for the launch position
  • Timer: the duration of time in seconds between launches
  • Force: the force applied to the prefab on launch. This vec3 is applied in space local to the Launch From Obj.

Previewing Your Lens

You’re now ready to preview your Lens experience.

Was this page helpful?
Yes
No

AI-Powered Search