Device Tracking
World AR technology offers several tracking solutions through the use of a device tracking component placed on a scene object that also has a camera component. The most relevant and modern tracking solution is to use a device tracking component set to World tracking mode. This will enable the usage of additional World AR technologies like instant world hit test, World Mesh, and depth texture.
Device Tracking
The Device Tracking
component allows you to place objects in your Lens Studio experience that will be locked to the world. When run in Snapchat, the content will appear in front of you at the start of the Lens. The Device Tracking
component has three modes, and we suggest you use the World tracking mode:
- Rotation: Users can rotate their phone and have the objects stay locked relative to their orientation. Users will not be able to walk towards, away from or around the content.
- Surface: Users are able to walk towards, away from and around the content, in addition to being able to rotate their phones. When using this mode, the 0,0,0 position of your Scene represents a horizontal surface in front of the device's initial position. When the Surface mode is selected, you have the option to also enable
Use Native AR
. WithUse Native AR
enabled, the device's native AR tracking capabilities will be utilized to enhance tracking. - World: Users are able to walk freely through the scene without focusing on a surface. When using this mode, the 0,0,0 position of your Scene represents your device's initial position. This mode solely relies on the device's native AR tracking capabilities and falls back to a Rotation experience if native AR tracking is not available. Note that while in the World mode, the units in the scene reflect centimeters in the real world.
Adding Device Tracking
To add Device Tracking
, first select the Camera
object in the Objects
panel. Then, in the Inspector
panel click the Add Component
button and select Device Tracking
.
When Device Tracking
is enabled, you'll see a green grid in your Scene
panel indicating that Device Tracking
is turned on. Any object in the scene on the layer of the camera will now be attached to the world.
Choosing Device Tracking Mode
By default, Surface
mode is used in the Device Tracking
component. To change this, in the Inspector
component, press the Tracking Mode
drop down and select an option. The preferred option is to use World tracking mode as it is the modern and offers access to additional World AR technologies like instant world hit test, World Mesh, and depth texture. It is also simpler to use as it uses ARCore/ARKit SLAM tracking capabilities and no additional setup is required.
Using Surface Mode
Surface mode is useful for when you want to allow users to walk up to your content, as well as walk around it. For experiences that encourage the user to more freely walk around (like a portal) or for larger content, consider enabling Use Native AR
which will utilize the device's native tracking capabilities if available.
Surface
mode tracking works best when content is scaled roughly human size or less. For anything bigger, consider enabling Use Native AR
as well which will utilize the device's native tracking capabilities if available. When Surface
mode is enabled, content in your scene is attached to the ground. Surface mode does not understand or detect walls or obstacles (like a couch or coffee table).
Positioning Object in Surface Mode
When you use Surface
tracking mode, Manipulation
can be added to any object allowing the user to move, rotate and scale objects. To add Manipulation
follow the Manipulation guide.
For an example of the Device Tracking
component in-use, please refer to the Static Object Template.
Choosing Tracking Point in Surface Mode
As you move your object around on a surface, the further it gets from the origin and the less stable the tracking can become. Telling the Device Tracking system what the tracking target should be can significantly help this. Each time you start your Lens you can reset the device tracker's target with a very simple script. Create a new script named SetSurfaceTrackingTarget.js and add the text below to its contents.
// -----JS CODE-----
// @input Component.DeviceTracking deviceTrackingComponent
function onSurfaceReset(eventData) {
if (script.deviceTrackingComponent) {
script.deviceTrackingComponent.surfaceTrackingTarget =
script.getSceneObject();
}
}
var worldTrackingResetEvent = script.createEvent('WorldTrackingResetEvent');
worldTrackingResetEvent.bind(onSurfaceReset);
With the object you want to be the tracking target selected, add a new Script
Component and set it to the SetSurfaceTrackingTarget script you just made. Make sure the event dropdown is set to Initialized
. This object will now be the device tracker's target.
Using Rotation Mode
Rotation
mode takes the device's gyroscope rotation and applies it to whatever object it is added to. The primary use case for the Device Tracking
component with Rotation
mode is to have the scene's camera rotation be driven by the device's gyroscope. This will allow you to place content all around the user in 3D space and give the user the ability to freely look around the scene. It is useful for when you want to surround the user with content, but not necessarily allow them to walk towards it.
For an example of Rotation
mode in-use, please refer to the Look Around Template.