Class CameraKitSession

A CameraKitSession represents a single rendering pipeline connecting an input media source to output <canvas> elements. When a Lens is applied to the session, CameraKit uses the Lens to transform the input media into rendered output.

CameraKitSession is the primary object that applications interact with when integrating the CameraKit SDK.

A CameraKitSession instance is obtained by calling CameraKit.createSession.

Example

const cameraKit = await bootstrapCameraKit(config)
const session = await cameraKit.createSession()

Properties

output playing events metrics keyboard

Methods

applyLens removeLens play pause mute unmute setSource setFPSLimit destroy

Properties

Readonlyoutput

output: { live: HTMLCanvasElement; capture: HTMLCanvasElement }

CameraKitSession renders video output to a <canvas> element. In fact, each session contains two canvas outputs corresponding to the RenderTargets used by Lens creators, when using LensStudio to create a Lens.

The live output renders content suitable for the Lens user (e.g. it may contain additional UI elements applicable only to the person applying the lens). The capture output renders content suitable for sharing with other users (e.g. sent to the other members of a video call, or saved to disk for sharing later).

For many lenses, these outputs are identical – but each lens is free to render differently, based on its own use-case.

playing

playing: { live: boolean; capture: boolean }

Indicates whether or not the session is currently rendering. If false, rendering is stopped. Otherwise the value indicates which output is being rendered.

Readonlyevents

events: TypedEventTarget<CameraKitSessionEvents> = ...

Add event listeners here to handle events which occur during the CameraKitSession.

Note: Applications may want to handle the error event, and check the contained error type -- if the type is LensExecutionError, this means the current lens was unable to render and CameraKit will automatically remove the lens.

Example

cameraKitSession.events.addEventListener('error', ({ detail }) => {
  if (detail.error.name === 'LensExecutionError') {
    console.log(`Lens ${detail.lens.name} encountered an error and was removed. Please pick a different lens.`)
  }
})

Readonlymetrics

metrics: LensPerformanceMetrics

Use this to measure current lens performance.

Readonlykeyboard

keyboard: Keyboard

The Keyboard API enables applications to handle keyboard input requests from lenses. When a lens requests a keyboard, the app displays it in its preferred UI. As users type, the app sends the text back to the lens for display. The lens can also request keyboard dismissal, prompting the app to remove the displayed keyboard.

Methods

applyLens

  • applyLens(lens: Lens, launchData?: LensLaunchData): Promise<boolean>

  • Apply a Lens to this session.

    This method will download (and cache) the Lens executable, and then use that Lens for rendering. If the session is currently playing, this will immediately update the rendered output. Otherwise, the new Lens will be used when session playback in resumed.

    Calling applyLens replaces any prior Lens – only one Lens is allowed at a time (per session).

    NOTE: Errors may occur after the Lens is applied. If the Lens encounters errors while rendering, Camera Kit will automatically remove the Lens from the session and emit a LensExecutionError event. Applications may want to listen for this error and, for example, prevent the Lens from being selected again by the user.

    session.events.addEventListener("error", ({ detail }) => {
      if (detail.error.name === "LensExecutionError") {
        preventFutureLensSelection(detail.lens);
        showMessage("We're sorry, but the Lens you selected encountered an error. Please choose a different Lens.");
      }
    });

    Parameters

    • lens: Lens

      The Lens to apply to this session.

    • OptionallaunchData: LensLaunchData

      This can optionally be provided to pass some initial data to the Lens – only certain Lenses expect launch data.

    Returns Promise<boolean>

    A promise which can have the following results:

    1. Resolved with true: the Lens has been applied.
    2. Resolved with false: the Lens has not been applied, but no error occurred – this can happen if a subsequent call to applyLens interrupted the Lens application.
    3. Rejected: the Lens has not been applied because an error occurred. This can happen if:
    • The Lens ID cannot be found in the LensRepository (use LensRepository to load the Lens before calling this method)
    • Lens content download fails, or the download of any required lens assets fails.
    • An internal failure occurs in the Lens rendering engine when attempting to apply the Lens.

removeLens

  • removeLens(): Promise<boolean>

  • Remove a Lens from this session.

    When a Lens is removed, rendering continues if the session is playing. It will just render the session input directly to the outputs without any image processing.

    Returns Promise<boolean>

    A promise which can have the following results:

    1. Resolved with true: the session's rendered output has no lens applied.
    2. Resolved with false: the current lens has been removed, but a subsequent call to applyLens means that the session's rendered output will still have a (new) lens applied.
    3. Rejected: the lens has failed to be removed. This can happen if an internal failure occurs in the Lens rendering engine when attempting to remove the lens.

play

  • play(target?: RenderTarget): Promise<void>

  • Start/resume session playback – LensCore will begin rendering frames to the output.

    If no source has been set for the session, calling play() will update the playing state, but no actual image processing will occur until setSource() is called.

    Parameters

    Returns Promise<void>

    Promise resolves when playback state has been updated. If no source has been set, this means play will resolve before any frames are processed -- but once a source is set, frames will immediately begin processing.

    Example

    const cameraKitSession = await cameraKit.createSession()
    await cameraKitSession.setSource(mySource)
    await cameraKitSession.play()

    // If you call `play` before `setSource`, the call to `play` will resolve but playback will only begin once a
    // media source has been set.

pause

  • pause(target?: RenderTarget): Promise<void>

  • Pause session playback – LensCore will stop rendering frames to the output.

    Parameters

    • target: RenderTarget = "live"

      Specify the RenderTarget to pause playback. May be either 'live' or 'capture'. Default is 'live'.

    Returns Promise<void>

    Promise resolves when playback has stopped.

mute

  • mute(fade?: boolean): void

  • Mute all sounds (default SDK state is unmuted).

    Parameters

    • fade: boolean = false

      Do we want audio to fade out?

    Returns void

unmute

  • unmute(fade?: boolean): void

  • Unmute all sounds.

    Parameters

    • fade: boolean = false

      Do we want audio to fade in?

    Returns void

setSource

  • setSource(source: CameraKitSource): Promise<CameraKitSource>

  • Set the media source for this session.

    Sessions may only have one source at a time - if setSource is called multiple times, subsequent calls replace the prior source. Setting the source does not trigger rendering (that’s done by session.play()). If the session is already playing, setting the source will immediately begin rendering the new source.

    The CameraKit SDK provides implementations for various common sources, which applications can create using the following functions:

    Important: Once a source has been set for a session, it cannot be set again, even if it has been replaced by another one. You must provide a new instance of CameraKitSource to CameraKitSession.setSource. If you want to reuse the existing source, you can use its CameraKitSource.copy method to create a new instance.

    Parameters

    • source: CameraKitSource

      A CameraKitSource object representing input media (e.g. a webcam stream, video, or some other source of image data), which CameraKit will supply to Lenses in order for them to render effects on top of that source.

    Returns Promise<CameraKitSource>

    Promise is resolved when the source has successfully been set. If the session was already in the playing state, the Promise resolves when the first frame from the new source has been rendered. The resolved value is the CameraKitSource object attached to the session.

  • setSource(
        source: HTMLVideoElement | MediaStream,
        options?: Partial<CameraKitDeviceOptions>,
    ): Promise<CameraKitSource>

  • Set the media source for this session.

    Sessions may only have one source at a time - if setSource is called multiple times, subsequent calls replace the prior source. Setting the source does not trigger rendering (that’s done by session.play()). If the session is already playing, setting the source will immediately begin rendering the new source.

    The CameraKit SDK provides implementations for various common sources, which applications can create using the following functions:

    Important: Once a source has been set for a session, it cannot be set again, even if it has been replaced by another one. You must provide a new instance of CameraKitSource to CameraKitSession.setSource. If you want to reuse the existing source, you can use its CameraKitSource.copy method to create a new instance.

    Parameters

    • source: HTMLVideoElement | MediaStream

      A CameraKitSource object representing input media (e.g. a webcam stream, video, or some other source of image data), which CameraKit will supply to Lenses in order for them to render effects on top of that source.

    • Optionaloptions: Partial<CameraKitDeviceOptions>

    Returns Promise<CameraKitSource>

    Promise is resolved when the source has successfully been set. If the session was already in the playing state, the Promise resolves when the first frame from the new source has been rendered. The resolved value is the CameraKitSource object attached to the session.

setFPSLimit

  • setFPSLimit(fpsLimit: number): Promise<void>

  • Set an FPS limit.

    This may be useful to reduce CPU/GPU resource usage by CameraKit if, for example, the input media source has a low FPS – CameraKit would then not try to render more frequently than the source produces new frames.

    This may also be useful to gracefully degrade performance in situations where lowering FPS is preferable over alternatives.

    Parameters

    • fpsLimit: number

      A maximum FPS, rendering will not exceed this limit

    Returns Promise<void>

    Promise is resolved when the limit is successfully set.

destroy

  • destroy(): Promise<void>

  • Destroy the session.

    The session will become inoperable. Frame processing stops, and any session-scoped graphical resources are freed.

    Returns Promise<void>

Settings

Member Visibility

On This Page

Properties
Methods
MMNEPVFCICPMFPCPTTAAATR