Sentiment Analyzer
Sentiment Analyzer
is available in Lens Studio Asset Library. Import the asset to your project, create new Orthographic camera and place the prefab under it.
The Sentiment Analyzer template demonstrates how you can utilize the Emotion and Yes/No classifiers to enrich any vocally driven lenses with Speech Recognition. The template contains several helper scripts that you can use to trigger visual effects without the use of scripting.
To get to know more about Speech Recognition, please visit the Speech Recognition Guide to see more detailed explanations about the concepts and scripting. We recommend learning the Speech Recognition Template first to get a general understanding before diving into this template.
Guide
The template shows how to use voice command detection with Speech Recognition. This template supports two intent models:
- Emotion Classifier: Anger, Disgust, Fear, Joy, Sadness and Surprise.
- Yes/No Classifier: Positive Intent and Negative Intent.
Transcription and keyword detection are now available for the English, Spanish and German languages. Voice Command Detection is only available for the English language. Transcription limitations include for example new names for things, slang words or acute accents. Please keep the following in mind when working with Sentiment Analyzer;
- Do not play sound or speech from the Lens while activating the microphone to capture sound.
- Try to avoid background noise and far device distance while activating the microphone to capture sound.
- If the microphone is muted for more than two minutes, the transcription won't continue after unmuting, and you’ll need to reset the Preview panel to enable it.
- If you were previously logged on to MyLenses and are having trouble seeing the preview in Lens Studio, logout from MyLenses and login to MyLenses again.
Here is how to logout and login to MyLenses.
VoiceML Module
The main asset used for Speech Recognition is VoiceML Module
. We can find it in the Asset Browser
panel. We attach it to the scripts in each example to configure settings for Speech Recognition.
In the bottom of the Preview panel, click the microphone. Test with your voice to see the blue vertical volume meter in action to ensure you are not muted. Then try speaking anything to see the transcription and scene object react to the voice events.
If the voice event - On Listening Enabled
is successfully called. We can see the Listening Icon pop up. Now try to speak to the microphone. The icon will animate when in listening mode. Animation will pause when getting the final transcription results. And turn red when getting an error.
Click on Speech Recognition in the Scene Hierarchy
panel. In the Inspector panel, we can then see here we are using SpeechRecognition.js
in the Script
Component. This is the main script we are going to use for speech recognition. Now let’s go through the details.
Notice here for the first section, we attach the VoiceML Module
to the Speech Recognition Script Component
for Speech Recognition configuration and voice input in Lens Studio.
Basic Setting for Transcription
Now let’s go through some basic settings for transcription in the next section.
Transcription
: final transcription.Live Transcription
: live and slightly less accurate transcription before we get the final, more accurate transcription.
Try to change the setting to see the difference.
Transcription Text
Here you can set the transcription result to a Screen Text with Transcription Text enabled.
In this example, the Screen Text object is under Orthographic Camera->Speech Recognition Result UI->Transcription Text
.
Speech Context
We can also add speech contexts to the transcription and boost some of the words for specific transcription scenarios. Use this when transcribing words which are rarer and aren’t picked up well enough by Snap, the higher the boost value will be, the more likely the word to appear in transcription.
With useSpeechContext
setting enabled, we can then attach the Speech Contexts object to it.
In the Scene Hierarchy
panel, click on the Transcription Example -> Speech Contexts
object. In the Inspector panel
, we can see here we have a Speech Context Script Component
attached to the object!
Add New Phrase to Speech Context
With Speech Context Script Component
, we can add words to the phrases and set a boost value for the phrases. To add a new word, click on the Add Value
field and input a new word you want to add.
Notice here the phrases should be made of lowercase a-z letters. The phrases should be within the vocabulary.
Add New Speech Context
Or we can add a new Speech Context Script Component
with a different boost value.
The range for boost value is from 1-10
, we recommend you’ll start with 5
and adjust if needed (the higher the value is, the more likely the word will appear in transcription)
Use Command
Now goes back to the Speech Recognition Script Component. Here we can see Use Command is enabled for Emotion Classifier and Yes_No Classifier.
- Emotion Classifier: Anger, Disgust, Fear, Joy, Sadness and Surprise.
- Yes/No Classifier: Positive Intent and Negative Intent.
Error Code for Command Responses
There are few error codes which NLP models might return:
#SNAP_ERROR_INCONCLUSIVE
: if two or more keyword categories#SNAP_ERROR_INDECISIVE
: if no keyword detected#SNAP_ERROR_NONVERBAL
: if we don’t think the audio input was really a human talking#SNAP_ERROR_SILENCE
: if too long silence- Anything starting with
#SNAP_ERROR_
: Errors that are not currently defined in this document and should be ignored
The Lens can only detect one voice command from each intent model. However both intent models can return valid results. For example, “Yes, I feel happy” will return Joy
and Positive Intent
as the results. “I feel happy” will return Joy
for Emotion Classifier and #SNAP_ERROR_INDECISIVE
for Yes No Classifier.
CommandResultHandler
With Use Command
enabled, we attach the Command Result Handler
Object to the CommandHandler
field. In the Scene Hierarchy
panel, click on the Speech Recognition -> CommandResultHandler
object.
Send Behavior Triggers with CommandResultHandler
When each command is detected, we can assign multiple Behavior Scripts. Try to click on the Add Value
field to attach new Behavior Scripts.
Here in this example, we are attaching different behavior scripts to trigger different VFX effects. You can find the behavior scripts under Emotion Classifier Behaviors
object and Yes No Classifier Behaviors
object.
For example, when the anger command is detected, the behavior under Speech Recognition -> CommandResultHandler -> Emotion Classifier Behaviors -> Anger Behavior
object will be triggered. The behavior script will call the triggerVFX
function in the Camera -> Sentiment VFX Object -> VFXTrigger Script
with the anger texture as the parameter. As a result, VFX will emit anger texture as the main texture.
Command Text
Now let’s go back to the Speech Recognition Script
. Here you can set the command result to a Screen Text
with Command Text
enabled.
Voice Events
Let's go back to the Speech Recognition Script and take a look at Edit Behaviors
. Notice here if we enable Edit Behaviors
, we can then attach Behavior Scripts to different voice events and trigger different visuals. Here we have 6 different voice events.
On Listening Enabled
: Trigger when the microphone is enabled.On Listening Disabled
: Trigger when the microphone is disabled.On Listening Triggered
: Trigger when changed back to listening mode.On Error Triggered
: Trigger when there is error in the transcription.On Final Transcription Triggered
: Trigger when it's a full result, or partial transcription.On Command Detected
: Trigger when any command is detected.
In Lens Studio Preview the microphone button is not simulated on the Screen. When we reset the preview, once the Speech Recognition is initialized, On Listening Enabled
will be triggered automatically. To learn how to use the microphone button, try to preview the Lens in Snapchat! Please check the details in Previewing Your Lens.
Debug Message for Voice Events
Notice here if we enable the Debug
in Voice Event Callbacks. In the Logger, we can then see the Voice Event being printed out when it gets triggered.
Send Behavior Triggers with Voice Events
For each Voice Event Callback, we can assign multiple Behavior Scripts. Try to click on the Add Value
field to attach new Behavior Scripts.
Here in this example, we are attaching different behavior scripts to change the visuals for the listening icon and screen texts.
Behavior and Tween
In the example, when the voice command is detected, you might notice here we are also triggering four tween animations:
TWEEN_SETSCREENTRANS_DETECTED
-Orthographic Camera -> Voice Enabled UI -> NLP Command Text Object -> First Script Component
.TWEEN_SETCOLOR_DETECTED
-Orthographic Camera-> Voice Enabled UI -> NLP Command Text Object -> Second Script Component
.TWEEN_SETOUTLINE_DETECTED
-Orthographic Camera-> Speech Recognition Result UI -> NLP Command Text Object -> Third Script Component
.- And we also pause the Listening Icon animation.
To learn more about Behavior, check out the Behavior Template and Tween!
Previewing Your Lens
You’re now ready to preview your Lens! To preview your Lens in Snapchat, follow the Pairing to Snapchat guide.
Once the Lens is pushed to Snapchat, you will see a hint: TAP TO TURN ON VOICE CONTROL
. Tap the screen to start VoiceML and the OnListeningEnabled
will be triggered. Press the button again to stop VoiceML and the OnListeningDisabled
will be triggered.
Related Guides
Please refer to the guides below for additional information: