Immersive ML Generation

Immersive ML is similar to ML Face Effects in the sense that both aim to create stylization effects. ML Face Effect does this only for the face region, whereas Immersive ML does this for the entire frame (i.e. including person, objects, pets, and background).

Launching Immersive ML

1. Navigate to the GenAI Suite in the top right corner of Lens Studio.

   

2. In the panel, select Immersive ML.

   

User Interface Overview

• Create new Effect Toolbar: Contains tools for creating and editing Effects:

• Gallery: Displays all Effects you created and their Statuses.

• Search and Filters: Use Search and Filters to find wanted Effect.

Creating an Effect

Text Prompt

• Input your own prompt: Enter the prompt into a prompt field.
• Use the ‘Surprise me’ feature: Tap ‘Surprise me’ to use a random prompt.

Image Prompt

• Input your own image: Upload the image from your computer.

Generation

• Press Generate when you are ready.

• Check upon the Status bar for info.

Immersive ML Preview and Editing

Effect preview

• You can review how the effect is applied to various sample images of models.
• You can adjust the prompt to edit the effect.
• Press Update Effect and wait for the result. The new effect will be created and displayed in the Gallery view.

Delete Effect

• You can also delete the effect.

Create a Model

Import to Project

• Press Create Model Button to start creating a model for your effect.

• The message indicating progress of creation will appear on a progress bar.

• Model is ready message will appear once a model is trained.

Create a Lens

Import to Project

• Press Import to Project button to start creating a Lens with your effect.

Preview the Result

You can see a preview of the Lens your are building in the preview panel.

Once satisfied with the result save your project and push the Lens for testing on Snapchat. Take a look at the Pairing to Snapchat guide to test your Lens on device, and the Publishing guide to learn more about sharing your creation.

Prompt Tips and Hacks

• Immersive ML currently is best used for artistic stylizations, and not realistic transformations.

Words and Phrases

• Context Words and Phrases: It is best to divide long sentence descriptions into context phrases separated by a comma symbol.
  • Works best: cute cartoon anime, kawaii romantic colors, immersive, photorealism
  • Works worse: Create a cute cartoon anime with kawaii romantic colors, combining immersive elements and photorealism

Descriptors

• Style: Describe the style you want to achieve.
  • Example: oil painting portrait, stoic, oil texture, pixel style, game consoles, graphic design, retrofuturistic, vintage style, vector art, detailed simple outlined minimalism, comic art, hand drawn illustration, poster,flat illustration, children’s picture book illustration, 3d papercuts, surrealistic cartoons, inspired by paper cut-outs art and paper sculptures, punk illustration, nightcore, flat outlined illustration, dnd art style, digital illustration, CG, risography effect, charcoal courtroom sketch
• Colors: Play with colors and palettes you want to emphasize.
  • Example: dark sepia color pallet, illustration in bright colors, style in full pastel colors, realistic usage of light and color, pastel color palette theme, nightcore, magical pink, soft colors
• Patterns: Adding pattern descriptors can create a sense of texture and depth, making flat surfaces appear more dynamic and interesting.
  • Example: heavy thick brush strokes, thatch textures, geometric patterns, floral patterns, herringbone patterns, chevron patterns, polka dots, stripes, plaid, houndstooth, animal prints, paisley, abstract patterns, crosshatch, swirls, damask, zigzag, checkerboard, gradient, lattice, argyle, marble, wave patterns, brick patterns, mesh, hexagonal patterns, brocade, tartan, quilted patterns, leaf patterns
• Light and Shades lines: Include descriptors for special lighting and shading details if needed.
  • Example: realistic usage of light, natural light, soft focus, blur, detailed shading, soft flush, flat shading, precise lines, with clear lines, anti-form, flat outlined illustration
• Mood: Mention mood if needed.
  • Example: happy, cheerful, calm, relaxed, mysterious, intriguing, energetic, excited, sad, melancholic, romantic, dreamy, tense, suspenseful, playful, fun, warm, cozy, lonely, isolated, adventurous, bold, reflective, thoughtful, angry, intense, joyful, celebratory, peaceful, harmonious

Advanced Prompts

• Special words.
  • Close-up: Ensures the image is tightly framed around the subject, highlighting specific features and details.
  • High quality: Emphasizes that the generated image is clear, sharp, and visually appealing.
  • Detailed: Stresses the importance of including a lot of small, intricate elements.
  • Flat shading: Instructs the AI to use a shading technique that lacks gradients, giving the image a more stylized and simplistic look.
  • HD: Combines high resolution and extreme detail for a highly realistic and visually impressive output.
  • Maximalistic: Indicates a style that is rich in detail and complexity, often with elaborate and decorative elements.
  • High definition: Ensures the image is created in high resolution, enhancing clarity and detail.
  • Fine luster: Adds a touch of shine or gloss to the image, giving it a polished and refined look.
• You can increase/decrease weight (importance) of certain words/phrases/characters by adding certain symbols to the prompt. The rules are as follows:
  • (word) - increases importance of the word by a multiplier of 1.1
  • [word] - decreases importance of the word by a multiplier of 1.1
  • (word:float_number) - changes importance of the “word” by a multiplier of float_number ([0 .. +Infinity])
  • Examples:
    • (Old) person - Make the person slightly older
    • Bright [sunny] (autumn:2.0) day - Make the day slightly less sunny but much more “autumn-y”
    • A (dark blue:0.5) suit - Make the suit twice as bright as before

More Examples

Below are some more examples of good prompts that are within the 200 character limit:

• 3d cubism illustration Cubist representation with bright colors illustration, vibrant
• low-poly style art low-poly game art polygon mesh jagged blocky wireframe edges
• Flat 2D Art simple flat color 2-dimensional style highly detailed
• Mecha Anime futuristic technologies detailed mechanical designs highly expressive
• comic graphic illustration, art graphic novel with vibrant highly detailed colors
• isometric pixel-art 16-bit low-res blocky pixel art style 16-bit graphics detailed
• artistic pencil, pencil brush, artistic,
• cute funny smile. cartoon, 3D, immersive, quality, details, cinematic, atmosphere
• stained glass style vibrant beautiful translucent intricate detailed
• cute cartoon anime, kawaii romantic colors, immersive, photorealism
• fine art portrait from 18 century, museum masterpiece, floral background, pastel colors
• abstract expressionist painting energetic brushwork bold colors abstract forms expressive painting

Prompt: wooden dummy doll in a cardboard box city	/img/lens-studio/5-platform-solutions/immersive-ml-black and white chalk drawing
Prompt: knotted handmade doll out of fabric and yarn	Prompt: curly hair, volume, curls, waves, hairdo, tiny look
Prompt: fine art portrait from 18 century, museum masterpiece, floral background, pastel colors	Prompt: masterpiece, face portrait painting, impressionism, watercolor, in the style of Claude Monet
Prompt: neon lights, long exposure of light, luminous, radiant	Prompt: cinematic red light, bokeh, cartoon face, white particles in background

• What doesn’t work well: Short prompts (only 1-2 words). Examples of prompts that don’t generate good quality effects:
  • Blue pixelated art
    • Reason: This is not a detailed description, it’s too vague.
  • Hot dogs all over
    • Reason: The prompt is centered on a real object(s).
  • chinese new year paper lanterns, layered paper, red
    • Reason: This prompt is actually too specific.
  • Person wearing a hat
    • Reason: This is a realistic transformation which is out of the scope of current effect capabilities.

Important Notes About Using Immersive ML Effects

• Model training will take 8-10 hours to complete. You may close the window and come back to it later.
• The effects resulting from this feature are currently only compatible with powerful, high end devices such as the latest generation iPhone versions.