Skip to main content

World Query Module

Overview​

The World Query Module enables you to place objects on real-world surfaces instantly and accurately. This lightweight solution is specifically designed for Spectacles' wearable platform.

Why Use World Query?​

Traditional surface detection APIs like hitTestWorldMesh or DepthTexture.sampleDepthAtPoint can be computationally expensive for wearable devices. The World Query Module provides:

  • Instant placement: Fast hit testing for real surfaces
  • Lightweight performance: Optimized for wearable devices
  • Surface analysis: Depth and normal sampling at specific locations
  • Smoothing options: Results can be smoothed across multiple hits

How It Works​

The World Query Hit Test performs ray casting against real surfaces by:

  1. Computing a depth map for the current view
  2. Intersecting the ray with depth data to find hit points
  3. Determining surface position and normal vectors
  4. Returning null if the ray falls outside the field of view

Performance Limitation

Due to the low, 5Hz update rate of the underlying depth data, the hit test works only for static or slowly moving objects. Consider use cases accordingly.

Basic Surface Placement​

Setup Requirements​

Before implementing World Query, ensure you have:

  1. Project configured for Spectacles with Interactive Preview enabled
  2. Spectacles Interaction Kit imported and initialized
  3. A target object to place on surfaces

Implementation Steps​

Follow these steps to implement basic surface placement using the WorldQueryHit API:

Step 1: Create a new TypeScript or JavaScript file with the required imports

Step 2: Set up the hit test session with filtering options

Step 3: Handle hit test results and object placement

Step 4: Configure user interaction for spawning objects

Complete Example​

// import required modules
import {
  InteractorTriggerType,
  InteractorInputType,
} from 'SpectaclesInteractionKit.lspkg/Core/Interactor/Interactor';
import { SIK } from 'SpectaclesInteractionKit.lspkg/SIK';

const WorldQueryModule = require('LensStudio:WorldQueryModule');
const EPSILON = 0.01;

@component
export class NewScript extends BaseScriptComponent {
  private primaryInteractor;
  private hitTestSession: HitTestSession;
  private transform: Transform;
  private lastHitResult: any; // Store last hit result for trigger end callback
  @input
  indexToSpawn: number;

  @input
  targetObject: SceneObject;

  @input
  objectsToSpawn: SceneObject[];

  @input
  filterEnabled: boolean;

  onAwake() {
    // create new hit session
    this.hitTestSession = this.createHitTestSession(this.filterEnabled);
    if (!this.sceneObject) {
      print('Please set Target Object input');
      return;
    }
    this.transform = this.targetObject.getTransform();
    // disable target object when surface is not detected
    this.targetObject.enabled = false;
    this.setObjectEnabled(this.indexToSpawn);

    // Set up trigger end callback
    this.setupTriggerEndCallback();

    // create update event
    this.createEvent('UpdateEvent').bind(this.onUpdate.bind(this));
  }

  setupTriggerEndCallback() {
    // Get all interactors and set up trigger end callbacks
    const allInteractors = SIK.InteractionManager.getInteractorsByType(
      InteractorInputType.All
    );

    for (const interactor of allInteractors) {
      interactor.onTriggerEnd.add(() => {
        // Only place object if we have a valid hit result and this is the primary interactor
        if (this.lastHitResult && this.primaryInteractor === interactor) {
          this.placeObject();
        }
      });
    }
  }

  placeObject() {
    if (!this.lastHitResult) return;

    // Copy the plane/axis object
    const parent = this.objectsToSpawn[this.indexToSpawn].getParent();
    const newObject = parent.copyWholeHierarchy(
      this.objectsToSpawn[this.indexToSpawn]
    );
    newObject.setParentPreserveWorldTransform(null);

    // Set position and rotation from last hit
    const hitPosition = this.lastHitResult.position;
    const hitNormal = this.lastHitResult.normal;

    let lookDirection;
    if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
      lookDirection = vec3.forward();
    } else {
      lookDirection = hitNormal.cross(vec3.up());
    }

    const toRotation = quat.lookAt(lookDirection, hitNormal);
    newObject.getTransform().setWorldPosition(hitPosition);
    newObject.getTransform().setWorldRotation(toRotation);
  }

  createHitTestSession(filterEnabled) {
    // create hit test session with options
    const options = HitTestSessionOptions.create();
    options.filter = filterEnabled;

    const session = WorldQueryModule.createHitTestSessionWithOptions(options);
    return session;
  }

  onHitTestResult(results) {
    if (results === null) {
      this.targetObject.enabled = false;
      this.lastHitResult = null;
    } else {
      this.targetObject.enabled = true;
      // Store hit result for potential trigger end callback
      this.lastHitResult = results;

      // get hit information
      const hitPosition = results.position;
      const hitNormal = results.normal;

      //identifying the direction the object should look at based on the normal of the hit location.

      let lookDirection;
      if (1 - Math.abs(hitNormal.normalize().dot(vec3.up())) < EPSILON) {
        lookDirection = vec3.forward();
      } else {
        lookDirection = hitNormal.cross(vec3.up());
      }

      const toRotation = quat.lookAt(lookDirection, hitNormal);
      //set position and rotation
      this.targetObject.getTransform().setWorldPosition(hitPosition);
      this.targetObject.getTransform().setWorldRotation(toRotation);
    }
  }

  onUpdate() {
    this.primaryInteractor =
      SIK.InteractionManager.getTargetingInteractors().shift();
    if (
      this.primaryInteractor &&
      this.primaryInteractor.isActive() &&
      this.primaryInteractor.isTargeting()
    ) {
      const rayStartOffset = new vec3(
        this.primaryInteractor.startPoint.x,
        this.primaryInteractor.startPoint.y,
        this.primaryInteractor.startPoint.z + 30
      );
      const rayStart = rayStartOffset;
      const rayEnd = this.primaryInteractor.endPoint;

      this.hitTestSession.hitTest(
        rayStart,
        rayEnd,
        this.onHitTestResult.bind(this)
      );
    } else {
      this.targetObject.enabled = false;
    }
  }

  setObjectIndex(i) {
    this.indexToSpawn = i;
  }

  setObjectEnabled(i) {
    for (let i = 0; i < this.objectsToSpawn.length; i++)
      this.objectsToSpawn[i].enabled = i == this.indexToSpawn;
  }
}

Integration Steps​

  1. Save the script and add it to your scene
  2. Create a target object to place on surfaces
  3. Set the Target Object input in the script component
  4. Test in preview by moving the mouse and tapping, or send to Connected Spectacles

This example demonstrates the simplest method for spawning objects on surfaces using hand gestures (point to move, pinch to spawn).

Check out the World Query Hit - Spawn On Surface and Surface Detection assets in the Asset Library for ready-to-use implementations, or download them as unpacked packages from the GitHub repository.

Semantic Hit Testing​

Ground Detection​

World Query can identify specific surface types, such as ground surfaces. This enables automatic object placement based on surface classification.

Experimental Feature

You need to enable Experimental APIs in your project settings to use semantic hit testing functionality.

Classification Implementation​

Enable semantic classification to detect ground surfaces:

import {
  InteractorTriggerType,
  InteractorInputType,
} from 'SpectaclesInteractionKit.lspkg/Core/Interactor/Interactor';
import { SIK } from 'SpectaclesInteractionKit.lspkg/SIK';

const WorldQueryModule =
  require('LensStudio:WorldQueryModule') as WorldQueryModule;

@component
export class HitTestClassification extends BaseScriptComponent {
  private hitTestSession: HitTestSession;
  private primaryInteractor;

  onAwake() {
    this.hitTestSession = this.createHitTestSession();

    this.createEvent('UpdateEvent').bind(this.onUpdate);
  }

  createHitTestSession() {
    const options = HitTestSessionOptions.create();
    options.classification = true;

    const session = WorldQueryModule.createHitTestSessionWithOptions(options);
    return session;
  }

  onHitTestResult = (result: WorldQueryHitTestResult) => {
    if (result === null) {
      // Hit test failed
      return;
    }

    const hitPosition = result.position;
    const hitNormal = result.normal;
    const hitClassification = result.classification;

    switch (hitClassification) {
      case SurfaceClassification.Ground:
        print('Hit ground!');
        break;
      case SurfaceClassification.None:
        print('Hit unknown surface!');
        break;
    }
  };

  onUpdate = () => {
    this.primaryInteractor =
      SIK.InteractionManager.getTargetingInteractors().shift();
    if (
      this.primaryInteractor &&
      this.primaryInteractor.isActive() &&
      this.primaryInteractor.isTargeting()
    ) {
      const rayStartOffset = new vec3(
        this.primaryInteractor.startPoint.x,
        this.primaryInteractor.startPoint.y,
        this.primaryInteractor.startPoint.z + 30
      );
      const rayStart = rayStartOffset;
      const rayEnd = this.primaryInteractor.endPoint;

      this.hitTestSession.hitTest(rayStart, rayEnd, this.onHitTestResult);
    }
  };
}

Additional Resources​

API Reference​

Was this page helpful?
Yes
No