Custom Location Creator Lens

The Custom Location Creator Lens allows you to use your mobile phone to create Custom Locations out of different locations and landmarks around the world.

With World Mesh 2.0, Lens Developers can take advantage of new features such as:

• Higher quality Meshes.
• Location Meshes are now in color.
• The Lens no longer requires a LiDAR powered phone, meaning iPhones 6s and up are supported, along with high end Android devices.

The Custom Location Creator Lens is a part of Lens Cloud, a collection of backend services, built on the same infrastructure that powers Snapchat.

To learn more about Lens Cloud and how to power a new generation of AR experiences that are more dynamic, useful, and interactive than ever before, please visit Lens Cloud Overview

This feature is currently targeted for use with structures up to three meters in height. For best practices, avoid any objects that can move and objects that have a large amount of shiny or reflective surfaces.

Scan this QR code within the Snapchat app to get access to the Custom Location Creator Lens:

Video Walkthrough

To assist first time users with Custom Location Scanning, Snap AR Engineer Eric Nersesian has created a visual walkthrough on how to scan your first Custom Location with the Creator lens.

New Custom Location Guide

You can also favorite this lens in order to make it easy to find again later on. When you are at the location of your Location, open the Lens just scanned and follow the screen instructions.

Start Page

On the start page, you can create a new Location, view existing Locations, or change the scanning settings.

If you already created Custom Locations that you want to use in Lens Studio, then you can choose to View your Uploads to find the Custom Location and its ID to load it up in Lens Studio.

Settings Page

The Settings page allows you to change the settings of the scanning lens. These settings will be saved between lens sessions on your device.

Default settings are recommended for first-time users.

If you need a larger mesh, set Create Mesh Max Size to Max. If you are scanning a large space and still running out of mesh too quickly, then try lowering Create Mesh Resolution to Low. In the opposite situation, if you are scanning a smaller location that needs a more detailed and accurate mesh, then increase Create Mesh Resolution to Max. If you are scanning a space that needs a large localization area (the area that Snapchatters will start your Lens off at), then increase Capture Perspectives Max to Max to gather more perspectives.

Create Mesh

The Lens now has a real time feedback to help you create the best Location Mesh for your Custom Location. The feedback mechanism will monitor the rotation and acceleration of your device to let you know if you are moving too fast, moving too slow, rotating too fast, or looking too far up/down which can disorient the device tracking.

Once you have arrived at a location you wish to scan, make sure you are close to your structure, and able to move around the structure. Hit Done when you finish, or the Lens would automatically jump to the next step once the Storage Limit is reached.

Beware of the amount of storage you’re using as you scan the mesh to make sure you capture the essence of the mesh. Doing a few tries is strongly recommended.

This is no minimum on the size of the mesh you can create. The mesh is usually used to align and place objects in Lens Studio. You can test content placement on the mesh with the Content button which will randomly spawn flowers, and will spawn flowers at your finger touch as well.

The Question button will bring back the informational model if you need to review any directions. The Exit button will bring you back to the beginning if you want to exit this scan before completing. If you are doing a combined scan, which is the default scan, sim to create just the mesh that you need for object placement in Lens Studio and then proceed to the next scanning phase so that you don't waste unnecessary captured perspectives in the create mesh phase.

Create Mesh Prompt	Mesh Selected	Content Selected

Capture Perspectives

After a mesh has been created, the Lens will prompt you to record potential perspectives of your audience. Make sure to stand where you expect your audience to be when using your lens and press Next. Afterwards, press Done or wait for the max storage to be filled. With both your mesh and perspectives finished being scanned, you will be able to preview it before you submit.

A perspective will be captured every time the phone moves more than 10cm. Make sure to wave your phone around in locations where your lens users will be and capture as many views of your Custom Location from each location as possible. You must fill the minimum and are encouraged to get as close to the max as possible.

Perspective view shows where you were standing when you captured that perspective and what direction that perspective is looking at. The perspective visuals will minimize when they are far from you to not clutter the view. The features view will show you where all the computer vision features are positioned on your Custom Location. More visually distinct objects will generate more features. Each perspective is looking at a subset of those features. Use the help button to review the information modal and the exit button to go back to the beginning before finishing the scan.

Perspectives prompt	Perspective Captured	Features Selected

Lens Testing

Click on Accept and you’ll be able to see the mesh you just scanned overlapped on top of the target object. Test the Lens to see how well the mesh fits. If the mesh doesn’t fit quite well, make sure to go through the steps again until best results have been achieved.

You should familiarize yourself with the Best Practices section to see what are some of the best conditions to record Custom Locations.

Afterward you’ll be prompted to add a title to the Location.

By default, test mode is turned off and you can view your mesh and the minimized perspectives. Features testing will show you the Computer Vision (CV) features and minimized perspectives while it does a pulse testing of the tracking. This means if it’s tracking successfully, you will see the features and perspectives and at the bottom, text will tell you that it’s tracking and it will countdown to reset the tracking.

Once the tracking gets reset, based on what you are looking at, either features will be recognized or not. If not, it will tell you to keep moving around to find a perspective that will recognize features. If it is tracking, the visuals will show and then countdown to reset tracking. Use this testing mode, by moving around to all the areas that you want your lens users to be able to view your Custom Location and check that features can be recognized in those areas.

Lens Testing Prompt	Off Selected	Features Selected

Uploading

The next step is to name and upload you mesh to the cloud.

This process might take some time, please be patient and do not exit the Lens while the Custom Location is uploading.

After uploading, you will have a Location ID for you scanned mesh. This ID is used later in Lens Studio to download this mesh. If the upload doesn’t complete in 30 seconds, please press the Retry button. The download is usually under 1-2mbs, but you might have to wait longer in poor data reception areas.

Try to be succinct with your name, and keep it under 20 characters.

Once your ID is shown, either write it down, enter it directly into Lens Studio, or use the View Your Uploads to access your ID again.

Add a Title	Uploading	Landmarker ID

View Existing Locations

You can also go to the start page of this Lens and click on View My Uploads to see the ID of locations you’ve scanned.

The list is in reverse chronological order.

Press View on the Location that you want to view more details on. You will want to be on the location where you made the initial scan and you have access to the same testing modes as in the Lens Testing phase of Create New Location.

For a better user experience, it is advised to return to the location under different lighting, and weather conditions to make sure the Location tracks well under varying conditions.

Custom Location Creator	Uploads	Landmarker Localized

Import an Existing Location

You can upload other Location that have been created by other Lens Creators. To import another Custom Location;

1. Press the View Your Uploads button to view your existing Locations.
2. Press the + button to import a Location.
3. When promted, enter the twelve character Location ID for the Custom Location you want to import.
   1. If that Location is invalid or already in your list, the Lens will tell you and prompt you to enter another Location ID.
4. Enter a name to the imported Location.

After a Location has been imported, you can select View to see the Location mesh.

In order to view the Location mesh on device, you will need to be in the same physical location of the Custom Location and view the Location from the perspectives that it was captured.

Incremental Scanning

Incremental Scanning allows you to capture multiple new scans with different lighting conditions or from new perspectives. The workflow is similar to capturing perspectives when creating a new Custom Location.

If you are not in the same physical location and not tracking the Location mesh, the + button for incremental scanning will be greyed out.

1. Open up an existing Location.
2. Localize and begin tracking the Location by being in the same physical location and perspective as the original scan.
3. Press the + button to add a new incremental scan.
4. Press the Capture button to start scanning the location.

Import Mesh to Lens Studio

After you have created a unique Location ID, you can import you mesh in Lens Studio.

Please make sure you are connected to the internet before importing a Custom Location, or opening a project with a Custom Location imported.

1. Open an empty project in Lens Studio.

   

2. Navigate to the Objects panel and type in Custom Location.

   

3. Input your custom mesh ID.

After you input the ID, you’ll see the location mesh and its tracking asset in Lens Studio.

The scanned mesh would not have wrapped UV information, in order to add materials and textures to this mesh, we recommend you either use the ‘Surface Position’ property in the material editor, or create a custom mesh resembling the same shape, and work with that mesh instead.

Export the Mesh from Lens Studio

The mesh produced via the Custom Location Creator Lens can be exported as an OBJ file from Lens Studio. This is beneficial if you wish to make alterations to the mesh, such as refining it or overlaying it with a more detailed scan from another photogrammetry tool.

To export the mesh from Lens Studio, right click the Location Mesh resource in the Resources panel and select the “Export to OBJ".

Best Practices

When scanning Custom Locations for Location-based AR experience, there are some things to keep in mind in order to provide the best experience for your content.

Scanning Motion

• Make sure that your device is in constant motion.
• Ideally, move your phone in a steady “sweeping figure 8” motion in front of you for a few seconds, while moving around in the space you are scanning. This will significantly improve the quality of the mesh you are getting.
• Avoid remaining static in space and rotating your device around one spot.
• Avoid extreme vertical viewing angles (pointing up and pointing down) and avoid fast camera motions.

Positioning

• The maximum range of the meshing is 5m, and the minimum is 20cm. Try to move within 1-2m of objects you’re scanning, moving closer/further as needed to correct errors.
• Make sure to cover as many viewpoints as you can: this will help fill holes in the mesh.

Lighting Conditions

• Make sure to scan with enough ambient brightness. Avoid scanning at dusk or night-time.

Other Best Practices

• If you are scanning thin objects, set the “Mesh resolution” setting to “Max”.
• Avoid scanning reflective/glass objects: the meshing quality is expected to be low in those cases.
• Observe the resulting mesh as much as possible while you are scanning, and try to correct potential artifacts by capturing as many perspectives as you can, and moving around and closer to the problematic areas.