Touch and Interactions

Introduction

You can add interactivity to the Lenses you create in Lens Studio by handling user touch input events. Your Lens can play a sound, trigger a character animation, and more by listening for events when the user touches the screen of their device.

Touch handling is an essential part of any interactive experience. In this guide we will learn how to handle touch events and drive different effects.

This guide assumes you are familiar with the basics of scripting in Lens Studio. To learn more about scripting, please visit the Scripting Overview.

---

Scene Events

This section describes simple full screen touch events. For more precise touch detection please check out Interaction Component. Lenses can respond to events which trigger when the user touches the screen. For example, your Lens can play a sound any time the user touches the screen.

TouchStartEvent - this event is triggered when the user starts a screen touch.

TouchMoveEvent - this event is triggered when the user moves their finger while holding a screen touch (i.e. while dragging on the screen).

TouchEndEvent - this event is triggered when the user ends a screen touch (i.e. when the user lifts their finger off of the screen).

TapEvent - this event is triggered when the user taps the screen (i.e. when they quickly start and stop a touch).

While starting working with tap and touch move interactions in your lens you may notice that Snapchat ui also reacts to those events. To avoid this you may enable touch blocking.

If your Lens does not use touch blocking, user touches will trigger default actions - such as double-tapping to switch the camera, and long-pressing to trigger Scan

To learn more about events in Lens Studio, please visit the Script Events Guide.

---

Touching the Screen​

The following guide will walk you through setting up a screen-based Touch Started event.

Let’s start with creating a New -> Script in the Resources panel .

Then drag this Script from Resources into the Objects panel, that will automatically create a new Scene Object with this script attached.

Next, double click on the Script in the Inspector panel to open the Script Editor panel.

And finally copy and paste the lines of code into the Script:

script.createEvent('TouchStartEvent').bind(function (eventData) {
  Studio.log('Touch Start');
});

script.createEvent('TouchMoveEvent').bind(function (eventData) {
  Studio.log('Touch Move');
});

script.createEvent('TouchEndEvent').bind(function (eventData) {
  Studio.log('Touch End');
});

script.createEvent('TapEvent').bind(function (eventData) {
  Studio.log('Tap');
});

Now save the script and try pressing and releasing mouse buttons in the Preview panel and see the output in the Logger panel!

Touch Event Arguments

As you may notice, script events mentioned above pass in a parameter called eventData.

This argument allows us to obtain more information about the touches for each specific event type, such as touch position in screen coordinates and touch id.

Let’s see how we can utilize these to drag a 3D object on the screen:

• First add New Sphere in the Objects panel by clicking on the + button
• Replace the code in the Script.js with the code below and save the script

//@input SceneObject sphere
//@input Component.Camera camera
//@input float depth = 40

var transform = script.sphere.getTransform();

script.createEvent('TouchStartEvent').bind(function (eventData) {
  updatePositionFromScreen(eventData.getTouchPosition());
});

script.createEvent('TouchMoveEvent').bind(function (eventData) {
  updatePositionFromScreen(eventData.getTouchPosition());
});

function updatePositionFromScreen(screenPos) {
  transform.setWorldPosition(
    script.camera.screenSpaceToWorldSpace(screenPos, script.depth)
  );
}

• Attach Script.js to the Sphere using Script Component
• Set Sphere and Camera inputs of the script in the Inspector panel

• Lastly, touch and move your mouse in the Preview panel to see this super simple implementation of dragging 3D object on the screen.

Preview the Project

Preview the Lens on your device. You should see a sphere rendered on the screen. When you touch the sphere and move your finder - it follows your touch position.

---

Interaction Component

Lenses can respond to touch events that trigger when the user touches an object. This is useful for cases when the user should interact with a specific object in the scene. For example, when the user taps on a 3D character in your Lens, that character plays an animation. This type of interaction is implemented in the Interactive Tap Template.

Interaction Component is the main building block for creating robust interactive experiences by allowing you to detect touches within the bounds of a specific 2D or 3D object.

Additionally it allows to filter touches by depth, see the section Depth Filtering below.

Adding To Scene

Interaction components can be added to a Scene Object with the next couple steps:

• Select Scene Object in the Objects panel
• Click on Add Component button and select Interaction Component
• Configure component properties.

Set Up

As you may see, the Interaction Component has several properties amongst which there is a Camera and array of Mesh Visuals.

Camera - camera used for detecting interactions. Should be the camera that renders corresponding Mesh Visual objects. Most of the time this can be left empty and the Interaction Component will determine the right Camera automatically.

Min Touch Size - Sets the minimum bounding box size used for detecting touches. Value range is [0,1], relative to screen width.

Depth Filter - when enabled, interaction events will be invoked only on the “closest“ object, where order is defined by distance from camera, camera render order and scene hierarchy.

Mesh Visual array - one or multiple Mesh Visual components to detect interactions with. InteractionComponent will automatically use Mesh Visuals on the same SceneObject, so in that case you can leave these fields empty.

Scripting Interaction Component

Interaction Component allows access to events from script and add callbacks to them. You may check out the list of available event registrations here

Using Interaction Component events is the preferred way of scripting interactions in a scene.

Let’s create a simple script example using Interaction Component events:

• In an empty project create a new Sphere in the Objects panel
• Add an Interaction Component to the Sphere Object
• Create new Script in the Resources panel and add it to the Sphere Scene object via Script Component.
• Finally double click on the created script in the Resources panel to start editing code.

Please note that Interaction Component events have a different type and api than Script Events. Please refer to api page for more details

/** @type {SceneObject} */
var sceneObject = script.getSceneObject();
/** @type {InteractionComponent} */
var interactionComponent = sceneObject.getComponent('InteractionComponent');
interactionComponent.onTouchStart.add(onTouchStart);

/**
 * @param {TouchStartEventArgs} eventArgs
 */

function onTouchStart(eventArgs) {
  print(
    '[Tapped on] ' +
      sceneObject.name +
      ', [Touch position] ' +
      eventArgs.position +
      ', [Touch Index] ' +
      eventArgs.touchId
  );
}

Depth Filtering

At this point we have an example that detects touch on the 3D object. Let’s build upon it and create 2 copies of a Sphere object and assign different materials to it.

You may notice how touches are detected only on a Bounding Box of the object that is closest to the camera if Interaction Component has Depth Filter checkbox enabled.

Touches are detected by mesh bounding box not the mesh itself

Interactions for Screen Transforms

Interaction Component also works with 2D objects that use Screen Transform, for example Screen Image or Text. This can also be used for Screen Transforms with no visual content, like the bounding area of a scrolling list as shown in the UI Scroll View Custom Component available in the Asset Library.

Screen Transform Component provides of useful API functions that allow to script precise interactions with Screen Transform and build various UI elements.

Let’s build an example that will allow us to manipulate screen objects on screen from scratch:

• Add a new Screen Image in the Objects panel, customize image texture to your liking.
• Add an Interaction Component to the Image scene object
• Create a new Script in the Resources panel with the code below and attach it to the same scene object via Script Component

// Requires Screen Transform
// Requires Visual
// @ui {"widget":"group_start", "label":"Drag"}
// @input bool dragX = true {"label": "X"}
/** @type {boolean} */
var dragX = script.dragX;
// @input bool dragY = true {"label": "Y"}
/** @type {boolean} */
var dragY = script.dragY;
// @ui {"widget":"group_end"}
//@input bool scale = true
/** @type {boolean} */
var scale = script.scale;
//@input bool rotate = true
/** @type {boolean} */
var rotate = script.rotate;
/** @type {boolean} */
var isActive = false;
/** @type {{id: number, position: vec2}[]} */
var touches = [];
/** @type {Number} */
var touchCount = 0;
/** @type {vec2} */
var startPos;
/** @type {vec2} */
var startDir = new vec2(0, 0);
/** @type {vec3} */
var startScale = new vec3(1, 1, 1);
/** @type {quat} */
var startRot = new quat(0, 0, 0, 0);
/** @type {vec2} */
var offset = new vec2(0, 0);
/** @type {SceneObject} */
var so;
/** @type {ScreenTransform} */
var st;
/** @type {InteractionComponent} */
var ic;
/**
 * initialize
 */
function onStart() {
  so = script.getSceneObject();
  st = so.getComponent('ScreenTransform');
  ic = so.getComponent('InteractionComponent');
  ic.onTouchStart.add(onTouchStart);
  ic.onTouchMove.add(onTouchMove);
  ic.onTouchEnd.add(onTouchEnd);
}
/**
 * @param {TouchStartEventArgs} eventData
 */
function onTouchStart(eventData) {
  if (touches.length == 0) {
    isActive = true;
  }
  if (isActive && touches.length < 2) {
    var touch = { id: eventData.touchId, position: eventData.position };
    touches.push(touch);
    if (touches.length > touchCount) {
      touchCount = touches.length;
    }
    if (touches.length == 1) {
      if (dragX || dragY) {
        var pos = st.screenPointToParentPoint(touches[0].position);
        if (pos != null) {
          offset = pos.sub(st.anchors.getCenter());
        }
      }
    } else if (touches.length == 2) {
      startDir = touches[0].position.sub(touches[1].position);
      startScale = st.scale;
      startRot = st.rotation;
    }
  }
}
/**
 * @param {TouchMoveEventArgs} eventData
 */
function onTouchMove(eventData) {
  if (isActive) {
    for (var i = touches.length - 1; i >= 0; i--) {
      if (touches[i].id == eventData.touchId) {
        touches[i].position = eventData.position;
        break;
      }
    }
    if (touchCount == 1) {
      if (dragX || dragY) {
        var pos = st.screenPointToParentPoint(touches[0].position);
        if (pos != null) {
          pos = pos.sub(offset);
          pos.x = dragX ? pos.x : startPos.x;
          pos.y = dragY ? pos.y : startPos.y;
          st.anchors.setCenter(pos);
        }
      }
    } else if (touches.length == 2) {
      var dir = touches[0].position.sub(touches[1].position);
      if (rotate) {
        var startAngle = Math.atan(startDir.y / startDir.x);
        var newAngle = Math.atan(dir.y / dir.x);
        var newRotation = startRot.toEulerAngles();
        var angleDiff = startAngle - newAngle;
        if ((dir.x < 0 && startDir.x > 0) || (dir.x > 0 && startDir.x < 0)) {
          angleDiff += Math.PI;
        }
        newRotation.z = newRotation.z + angleDiff;
        st.rotation = quat.fromEulerVec(newRotation);
      }
      if (scale) {
        var magChange = dir.length / startDir.length;
        st.scale = startScale.uniformScale(magChange);
      }
    }
  }
}
/**
 * @param {TouchEndEventArgs} eventData
 */
function onTouchEnd(eventData) {
  if (isActive) {
    if (touchCount == 1) {
      if (dragX || dragY) {
        var pos = st.screenPointToParentPoint(touches[0].position);
        if (pos != null) {
          pos = pos.sub(offset);
          pos.x = dragX ? pos.x : startPos.x;
          pos.y = dragY ? pos.y : startPos.y;
          st.anchors.setCenter(pos);
        }
      }
    }
    for (var i = touches.length - 1; i >= 0; i--) {
      if (touches[i].id == eventData.touchId) {
        touches.splice(i, 1);
        break;
      }
    }
    if (touches.length == 0) {
      isActive = false;
      touchCount = 0;
    }
  }
}
/**
 * clamps center to parent bounds
 * @param {vec2} pos
 * @returns
 */
function getClampedPosition(pos) {
  pos.x = dragX ? pos.x : startPos.x;
  pos.y = dragY ? pos.y : startPos.y;
  if (limitToParent) {
    pos.x = Math.min(Math.max(-1, pos.x), 1);
    pos.y = Math.min(Math.max(-1, pos.y), 1);
  }
  return pos;
}
script.createEvent('OnStartEvent').bind(onStart);

Here is how the attached components should look:

Make several copies of Screen Image scene objects to create a fun sticker lens!

As you may notice script above uses screenPointToParentPoint api function of a Screen Transform to calculate local position as we manipulate objects on the screen. This allows us to develop such useful ui elements as sliders, image carousels etc.

Check out the Manipulate Screen Transform Component from the Asset Library for more advanced version of this example!

---

Touch Blocking

There may be times when you want your Lens to receive all touch events to prevent the user from unintentionally triggering other Snapchat actions while using your Lens. For example, if your Lens has a rapid tapping interaction you may want to override Snapchat's default behavior of switching cameras on double-tap.

In these instances, you can make use of Touch Blocking. With Touch Blocking, you can override some or all of Snapchat's default touch behaviors.

Basic Use

To override the Snapchat app's default touch events, add the following line to a script bound to the "Lens Turned On" event:

global.touchSystem.touchBlocking = true;

With this line included, your Lens will override all of the app's default screen touch events.

The global property global.touchSystem is a TouchDataProvider object used to manage touch blocking exceptions.

Touch Blocking Exceptions

You may not want to override all of the default Snapchat touch events. For example, you may want to override the double-tap event, but still allow the user to swipe to Stories or Chat. In this instance, you can make use of Touch Blocking Exceptions.

Usage

To make an exception for a specific type of touch, add the following code to a script bound to the "Lens Turned On" event:

global.touchSystem.touchBlocking = true;
global.touchSystem.enableTouchBlockingException('TouchTypePan', true);

In the example above, the Lens overrides all of the default touch events, but makes an exception for screen swipes. Enabling this exception allows the user to still swipe into another view of Snapchat (like Stories or Chat) while the Lens is active.

You can replace the string TouchTypePan with any of the Touch Types listed in the section below.

Exception Types

You can enable or disable the following Touch Type Exceptions:

TouchTypeTap: Enabling this exception allows Snapchat to continue handling the tap gesture.

TouchTypeDoubleTap: Enabling this exception allows Snapchat to continue handling the double-tap gesture.

TouchTypeScale: Enabling this exception allows Snapchat to continue handling the two-finger pinch gesture.

TouchTypePan: Enabling this exception allows Snapchat to continue handling the pan gesture.

TouchTypeSwipe: Enabling this exception allows Snapchat to continue handling the swipe gesture.

Related Documentation

Please refer to the guides below for additional information:

• Scripting Overview
• Scripting Events