Camera Module
Spectacles offers APIs to retrieve the camera frame – what the user is currently seeing – to better understand and build experiences around the user’s real-world environment.
This is an Experimental
API. Please see Experimental APIs for more details.
Privacy Note: Accessing the camera frame in a Lens will disable open internet access for that Lens. The camera frame contains user information information that may not be transmitted outside the Lens. For testing and experimental purposes however, extended permissions are available to access both the camera frame and the open internet at the same time. Note that lenses built this way may not be released publicly. Please see Extended Permissions doc for more information.
Requesting + Receiving Camera Frames
The CameraModule enables developers to request the camera frame. To begin receiving frames, construct a CameraRequest, specifying the exact camera of interest (Spectacles have multiple cameras from which to choose: Left_Color, Right_Color, or SystemDefault_Color) and the desired resolution of the received frames (via imageSmallerDimension, which indicates the size in pixels of the smallest dimension to return).
For example:
let cameraModule = require('LensStudio:CameraModule');
let cameraRequest = CameraModule.createCameraRequest();
cameraRequest.id = CameraModule.CameraId.Left_Color;
let cameraTexture = cameraModule.requestCamera(cameraRequest);
createCameraRequest may not be invoked inside onAwake event.
The requestCamera method takes a CameraRequest and returns a CameraTexture. CameraTexture contains the opaque texture handle that refers to the underlying camera data, and can be used as an input into an MLComponent or uploaded to a remote service via RemoteMediaModule.
To receive a notification each time the frame is updated, use the CameraTextureProvider which can be accessed via cameraTexture.control. The `CameraTextureProvider` has an `onNewFrame` event that can be utilized as shown below:
let onNewFrame = cameraTexture.control.onNewFrame;
let registration = onNewFrame.add((frame) => {
// Process the frame
});
script.onStop.add(() => onNewFrame.remove(registration));
Getting Camera Information
The Camera API also provides information about the cameras used on Spectacles in order to understand how the camera transforms 3D space into 2D space. This information is provided by the DeviceCamera which is accessible via the DeviceInfoSystem.
// Select the camera
let camera = global.deviceInfoSystem.getTrackingCameraForId(
CameraModule.CameraId.Left_Color
);
// Retrieve camera properties
let focalLength = camera.focalLength;
let principalPoint = camera.principalPoint;
let resolution = camera.resolution;
let pose = camera.pose;
Additionally, DeviceCamera includes project and unproject methods that makes it easier to convert 3D points into 2D points and vice versa.