Skip to main content
Version: 5.x

Sentiment Analyzer

Sentiment Analyzer is available in Lens Studio Asset Library. Import the asset to your project, create new Orthographic camera and place the prefab under it.

The Sentiment Analyzer template demonstrates how you can utilize the Emotion and Yes/No classifiers to enrich any vocally driven lenses with Speech Recognition. The template contains several helper scripts that you can use to trigger visual effects without the use of scripting.

To get to know more about Speech Recognition, please visit the Speech Recognition Guide to see more detailed explanations about the concepts and scripting. We recommend learning the Speech Recognition Template first to get a general understanding before diving into this template.

Guide

The template shows how to use voice command detection with Speech Recognition. This template supports two intent models:

  • Emotion Classifier: Anger, Disgust, Fear, Joy, Sadness and Surprise.
  • Yes/No Classifier: Positive Intent and Negative Intent.

Transcription and keyword detection are now available for the English, Spanish and German languages. Voice Command Detection is only available for the English language. Transcription limitations include for example new names for things, slang words or acute accents. Please keep the following in mind when working with Sentiment Analyzer;

  • Do not play sound or speech from the Lens while activating the microphone to capture sound.
  • Try to avoid background noise and far device distance while activating the microphone to capture sound.
  • If the microphone is muted for more than two minutes, the transcription won't continue after unmuting, and you’ll need to reset the Preview panel to enable it.
  • If you were previously logged on to MyLenses and are having trouble seeing the preview in Lens Studio, logout from MyLenses and login to MyLenses again.

Here is how to logout and login to MyLenses.

VoiceML Module

The main asset used for Speech Recognition is VoiceML Module. We can find it in the Asset Browser panel. We attach it to the scripts in each example to configure settings for Speech Recognition.

In the bottom of the Preview panel, click the microphone. Test with your voice to see the blue vertical volume meter in action to ensure you are not muted. Then try speaking anything to see the transcription and scene object react to the voice events.

If the voice event - On Listening Enabled is successfully called. We can see the Listening Icon pop up. Now try to speak to the microphone. The icon will animate when in listening mode. Animation will pause when getting the final transcription results. And turn red when getting an error.

Click on Speech Recognition in the Scene Hierarchy panel. In the Inspector panel, we can then see here we are using SpeechRecognition.js in the Script Component. This is the main script we are going to use for speech recognition. Now let’s go through the details.

Notice here for the first section, we attach the VoiceML Module to the Speech Recognition Script Component for Speech Recognition configuration and voice input in Lens Studio.

Basic Setting for Transcription

Now let’s go through some basic settings for transcription in the next section.

  • Transcription: final transcription.
  • Live Transcription: live and slightly less accurate transcription before we get the final, more accurate transcription.

Try to change the setting to see the difference.

Transcription Text

Here you can set the transcription result to a Screen Text with Transcription Text enabled.

In this example, the Screen Text object is under Orthographic Camera->Speech Recognition Result UI->Transcription Text.

Speech Context

We can also add speech contexts to the transcription and boost some of the words for specific transcription scenarios. Use this when transcribing words which are rarer and aren’t picked up well enough by Snap, the higher the boost value will be, the more likely the word to appear in transcription.

With useSpeechContext setting enabled, we can then attach the Speech Contexts object to it.

In the Scene Hierarchy panel, click on the Transcription Example -> Speech Contexts object. In the Inspector panel, we can see here we have a Speech Context Script Component attached to the object!

Add New Phrase to Speech Context

With Speech Context Script Component, we can add words to the phrases and set a boost value for the phrases. To add a new word, click on the Add Value field and input a new word you want to add.

Notice here the phrases should be made of lowercase a-z letters. The phrases should be within the vocabulary.

Add New Speech Context

Or we can add a new Speech Context Script Component with a different boost value.

The range for boost value is from 1-10, we recommend you’ll start with 5 and adjust if needed (the higher the value is, the more likely the word will appear in transcription)

Use Command

Now goes back to the Speech Recognition Script Component. Here we can see Use Command is enabled for Emotion Classifier and Yes_No Classifier.

  • Emotion Classifier: Anger, Disgust, Fear, Joy, Sadness and Surprise.
  • Yes/No Classifier: Positive Intent and Negative Intent.

Error Code for Command Responses

There are few error codes which NLP models might return:

  • #SNAP_ERROR_INCONCLUSIVE: if two or more keyword categories
  • #SNAP_ERROR_INDECISIVE: if no keyword detected
  • #SNAP_ERROR_NONVERBAL: if we don’t think the audio input was really a human talking
  • #SNAP_ERROR_SILENCE: if too long silence
  • Anything starting with #SNAP_ERROR_: Errors that are not currently defined in this document and should be ignored

The Lens can only detect one voice command from each intent model. However both intent models can return valid results. For example, “Yes, I feel happy” will return Joy and Positive Intent as the results. “I feel happy” will return Joy for Emotion Classifier and #SNAP_ERROR_INDECISIVE for Yes No Classifier.

CommandResultHandler​

With Use Command enabled, we attach the Command Result Handler Object to the CommandHandler field. In the Scene Hierarchy panel, click on the Speech Recognition -> CommandResultHandler object.

Send Behavior Triggers with CommandResultHandler​

​ When each command is detected, we can assign multiple Behavior Scripts. Try to click on the Add Value field to attach new Behavior Scripts.

Here in this example, we are attaching different behavior scripts to trigger different VFX effects. You can find the behavior scripts under Emotion Classifier Behaviors object and Yes No Classifier Behaviors object.

For example, when the anger command is detected, the behavior under Speech Recognition -> CommandResultHandler -> Emotion Classifier Behaviors -> Anger Behavior object will be triggered. The behavior script will call the triggerVFX function in the Camera -> Sentiment VFX Object -> VFXTrigger Script with the anger texture as the parameter. As a result, VFX will emit anger texture as the main texture.

Command Text

Now let’s go back to the Speech Recognition Script. Here you can set the command result to a Screen Text with Command Text enabled.

Voice Events

Let's go back to the Speech Recognition Script and take a look at Edit Behaviors. Notice here if we enable Edit Behaviors, we can then attach Behavior Scripts to different voice events and trigger different visuals. Here we have 6 different voice events.

  • On Listening Enabled: Trigger when the microphone is enabled.
  • On Listening Disabled: Trigger when the microphone is disabled.
  • On Listening Triggered: Trigger when changed back to listening mode.
  • On Error Triggered: Trigger when there is error in the transcription.
  • On Final Transcription Triggered: Trigger when it's a full result, or partial transcription.
  • On Command Detected: Trigger when any command is detected.

In Lens Studio Preview the microphone button is not simulated on the Screen. When we reset the preview, once the Speech Recognition is initialized, On Listening Enabled will be triggered automatically. To learn how to use the microphone button, try to preview the Lens in Snapchat! Please check the details in Previewing Your Lens.

Debug Message for Voice Events

Notice here if we enable the Debug in Voice Event Callbacks. In the Logger, we can then see the Voice Event being printed out when it gets triggered.

Send Behavior Triggers with Voice Events

For each Voice Event Callback, we can assign multiple Behavior Scripts. Try to click on the Add Value field to attach new Behavior Scripts.

Here in this example, we are attaching different behavior scripts to change the visuals for the listening icon and screen texts.

Behavior and Tween

In the example, when the voice command is detected, you might notice here we are also triggering four tween animations:

  1. TWEEN_SETSCREENTRANS_DETECTED - Orthographic Camera -> Voice Enabled UI -> NLP Command Text Object -> First Script Component.
  2. TWEEN_SETCOLOR_DETECTED - Orthographic Camera-> Voice Enabled UI -> NLP Command Text Object -> Second Script Component.
  3. TWEEN_SETOUTLINE_DETECTED - Orthographic Camera-> Speech Recognition Result UI -> NLP Command Text Object -> Third Script Component.
  4. And we also pause the Listening Icon animation.

To learn more about Behavior, check out the Behavior Template and Tween!

Previewing Your Lens

You’re now ready to preview your Lens! To preview your Lens in Snapchat, follow the Pairing to Snapchat guide.

Once the Lens is pushed to Snapchat, you will see a hint: TAP TO TURN ON VOICE CONTROL. Tap the screen to start VoiceML and the OnListeningEnabled will be triggered. Press the button again to stop VoiceML and the OnListeningDisabled will be triggered.

Please refer to the guides below for additional information:

Was this page helpful?
Yes
No

AI-Powered Search