Overview

World AR experiences refer to Lenses designed for the outward-facing camera, looking out at the world — in other words, not a selfie experience. Using various AR technologies, user can place virtual objects on real-life surfaces. With that in mind, in order to create believable, truly immersive experiences, it is critical that tracking technology can accurately track objects, people, things, and even reconstruct a space — like a room, street, or building — in 3D and in real time.

Tracking

There are the fundamental tracking capabilities that allow you to track your experience to the real world.

Device Tracking

Using tracking information from the device, you can have your AR experience tracked around you (3DoF), on a surface like a floor, and even movement through spaces (6Dof).

Marker Tracking

You can track your experiences directly on an image. Knowing the image in advance, allows you to integrate your AR experience accurately.

Object Tracking

Using computer vision, you can track in 2D (screen space) various known things: e.g. pets, body, specific parts of the body, and more.

3D Object Tracking

You can track certain things like the body, hand, etc. in 3D space for better fidelity and understanding.

Custom ML Tracking

You can add your own custom trackers using SnapML

World Understanding

In addition to various trackers, you can have understanding of what you're tracking: whether it be through reconstruction, location information, and more. These features often extend and leverages the various trackers.

World Mesh and Depth Textures

World Mesh provides a real-time 3D reconstruction of the real world based on what your device sees. This allows your experience to better match and respond to the environment that it’s in.

Custom Location

You can scan a World Mesh in advance, and track AR experiences uniquely on it.

City Scale AR

You can track AR experiences to specific locations in cities around the world.

Map Component

You can display where the user is on a map to help guide them to an experience.

Spatial Persistence

You can enable Lens users to save data to a physical location in the world.

Landmarkers

You can track experience to famous Landmarks around the world.

Pet

You can track track cats, dogs, or both using Object Tracking 2D.

Additional Techniques

In some cases, you might have some specific requirements like performance, information gathering, or access platform specific capabilities. You can integrate the following capabilities, in addition to the ones listed above.

World Tracking Planes

You can understand horizontal and vertical surfaces without waiting for World Mesh by relying on Tracked Planes.

Instant World Hit Test

You can instantly get information about the world using Depth Texture, and when World Mesh is available, get a more accurate information.