Skip to main content
Version: 5.x
Supported on
Snapchat
Spectacles
Camera Kit

Audio Analyzer

Audio Analyzer allows you to drive different effects based on the power of the audio signal in certain frequency ranges.

The Audio Analyzer asset can be found in the Lens Studio Asset Library. Import asset and drag included prefab to the Scene Hierarchy panel.

Choosing the Audio Input​

The asset is set up to analyze audio data from the File Audio Track. Select the Audio Analyzer[EDIT ME] scene object in the Scene Hierarchy panel to explore all the options of the Audio Analyzer script in the Inspector panel.

Audio from Microphone​

Set Input Type to Microphone, set Microphone Audio with the Audio From Microphone asset if it’s not set already.

You can add Microphone Audio asset to the project by clicking on the + button in the Asset Browser panel.

Make sure to provide access to your microphone in Lens Studio by clicking on the microphone button at the bottom of the Preview panel.

Then try making some sound to see the reaction of the scene objects.

Audio From File or Sound​

Set Audio Track input with the imported audio file or a Licensed Music from the Asset Library

Enable Play Audio checkbox to create audio component and play specified audio track.

Set a number of loops to play.

Please match sample rate with your audio track sample rate, (44100 for Licensed Music)

Audio From Audio Component​

Another way is to specify Audio Component to get audio data from. In this case audio analyzer will process the audio track of this audio component, stop and start as the audio component starts or stops.

Play or stop this audio component as needed, for example with a Behavior script.

From Script with API​

You can even generate your own audio buffer and set a custom script with certain api as an input.

script.getAudioFrame(Float32Array audioFrame) : vec3 - function that writes audio samples into audioFrame array and returns it shape
script.sampleRate : Number - sample rate of this audio buffer

Set input via api​

You may also choose not to set Input and do it later from script or a Behavior script.

audioAnalyzerScript.setInput(input, sampleRate);

Mel Spectrogram Settings​

Sample Rate: Set number of samples per second (has to match your audio input sample rate).

Frame Size: Length of the window, which is the length of frameSize and then padded with zeros to match fftSize.

Hop Size: Set number of samples between successive fft segments.

Fft Size: Set length of the fft window.

Num Mel: Set number of mel bins.

Min Frequency: Set min frequency.

Max Frequency: Set max frequency.

Driving Effects based on Audio Analysis​

By default the example analyzes audio data and outputs power of signal in all frequency bands in the range from 0 to 1.

The bands (or bins) are defined by splitting the interval from the minimal frequency to the maximum frequency into a certain amount of bins on the mel (logarithmic) scale. This scale is used as it better represents the way the human ear perceives the sound.

Asset contains aAudioAnalyzerHelper script that allows it to perform certain responses to the change of band value.

Audio Analyzer: reference to the audio Analyzer script.

Get Band Value: specifies which value to take.

  • By index: get band value by index.
  • Average: get average value across all bands.
  • All Bands: get all band values. Can be used to set the array property of the Material or VFX asset or passed as parameter to the script component api function.

Smooth Value: allows to blend band value with previous value.

  • None: no blending.
  • Fixed step: each band value fades off by a fixed step per update.
  • Lerp: interpolate between previous and next value.

Response Action: perform certain actions based on the selected band values

Call Api Function: allows to interpolate between start and end values and pass result to the api function.

  • Value Type: set the type of value.
  • Start Value: start value that corresponds to band value = 0.
  • End Value: start value that corresponds to band value = 1.
  • Script: script with api.
  • Function Name: api function name.

Set Material VFX Parameter

VFX or Material: asset to set property of

  • Parameter Name: asset parameter, if you select All Bands - this expected to be array parameter
  • Value Type: set the type of value
  • Start Value: start value that corresponds to band value = 0
  • End Value: start value that corresponds to band value = 1

Example: script on the Beats scene object that sets float array parameter from the band values

Set Transform: Allows to set scene objects transform

  • Scene Object: scene object to set transform
  • Transform: set Position, Rotation or Scale
  • Local: if enabled sets local transform, otherwise - global
  • Mode: allows to specify Start -  End values or Offset from the current transform value

Using Audio Analyzer Api​

Open AudioAnalyzer.js file to see available api functions. You can call them from a different script by referencing Audio Analyzer script component

// @input Component.ScriptComponent audioAnalyzerScript
var bands = script.audioAnalyzerScript.getBands();
var numMel = script.audioAnalyzerScript.getNumMel();

AudioFrameToTexture script provides an example of how to write current audio frame into a texture and use it in the material or VFX editor.

Previewing Your Lens​

You're now ready to preview your Face Lens. To preview your Lens in Snapchat, follow the Pairing to Snapchat guide.

Was this page helpful?
Yes
No