Multimodal Video Generation via Google's Gemini Omni
Text, image, and video in โ video out. One unified Gemini model handles four workflows through a Task selector โ text-to-video, image-to-video, reference-guided video (up to 3 references), and conversational editing of an existing clip. Runs on Google's servers via API.
๐ Requires a Google API key
What it does
This node connects your CNAPS Studio flows to Google's Gemini Omni โ a unified multimodal model that generates video from a mix of text, image, and video inputs. Pick a Task and the node adapts: Text to Video takes just a prompt, Image to Video takes a prompt plus a starting frame, Reference to Video takes a prompt plus up to three reference images (for subject, style, or product guidance), and Edit takes a prompt plus an existing video and returns a conversationally-edited version. All four modes share the same model, the same API call, and the same node โ the Task selector decides which input ports are actually used.
Problem it solves
- One node, four video workflows โ Text-to-video, image-to-video, reference-guided generation, and video editing all live in a single node; switch Task instead of swapping models.
- Conversational video editing โ Give Omni an existing clip plus a natural-language instruction and get an edited version back. No separate editing pipeline required.
- Reference-guided consistency โ Up to 3 reference images can steer subject, style, or product across a generated clip, useful for brand/character consistency in short-form content.
- Wire-once, switch-mode โ Unused input ports are dropped even if wired, so you can build one flow that supports multiple Tasks and just flip the Task setting per run.
- No local GPU needed โ Runs entirely on Google's infrastructure through the API; the studio just orchestrates the call.
Configuring the Node
Basic Settings
Model variant
Model | Best For |
omni-flash | The default (and currently only) Omni tier โ fast, cost-efficient multimodal video generation with conversational editing |
Task โ decides which input ports apply:
Task | Inputs used | Notes |
Text to Video | Text prompt only | Pure text-to-video generation |
Image to Video | Text prompt + Image 1 | Image 1 acts as the starting frame; video is animated forward from it |
Reference to Video | Text prompt + Image 1โ3 | Up to 3 reference images guide subject, style, or product across the generated clip |
Edit | Text prompt + input Video | Conversationally edits the input video (e.g., "change the sky to sunset", "swap the shirt for blue"). Aspect ratio setting is ignored โ output preserves the input video's aspect |
Unused ports are dropped even if wired. If Task = Text to Video and you have an image wired into the node, the image is ignored (not an error). This lets you build one flow that supports multiple Tasks and just flip the Task per run.Aspect ratio โ for generated output:
Options | Notes |
16:9, 9:16 | Applied to Text to Video, Image to Video, and Reference to Video. Ignored in Edit mode (output preserves the input video's aspect). |
System prompt / instruction for video generation or editing
- Your main instruction for what to generate or how to edit. Supports natural language including cinematic direction, camera angles, lighting, mood, audio cues, and โ in
Editmode โ specific edit instructions ("replace the background with a beach at sunset", "keep the character, change the outfit to a red dress").
Compliance & Provenance
Provider | Google |
Provider type | GPAI โ General-Purpose AI (GPAI) |
License | Google Cloud Terms |
EU AI Act risk class | Limited Risk |
Art. 50 transparency | Required โ outputs are marked. See AI Policy ยง2. |
Region availability | Available globally |
For more on how we classify models and mark outputs, see our AI Policy.
How Omni compares
CNAPS Studio has several video generation models. They differ in length, audio, and what you can feed in.
Model | Best for | Notes |
Cosmos3 Nano | Short clips from text or one image | 16B; ~8 s at 720p by default |
Wan2.2 TI2V 5B | Compact 720p text- or image-to-video | 5B dense model |
Helios Base | Long videos, up to ~60 s, highest Helios quality | 14B; 19.5 FPS on one H100 |
Helios Distilled | Long videos, fastest Helios variant | Same length limit as Base |
LTX-2.5 | Video with a generated soundtrack | 22B distilled |
LTX-2.5 Audio-to-Video | A talking video from one photo + a voice | Lip motion follows your audio |
LatentSync 1.6 | Re-sync an existing video to new speech | Dubbing and translation |
MiniMax H3 | Video from text, image, and reference inputs with stereo audio | 33B; 5โ15 s at 24 fps |
Omni | Text, image, and video in โ video out | External Gemini API |
Rough guidance: Cosmos3 Nano or Wan2.2 for short clips, Helios for anything longer than ~15 s, LTX-2.5 or MiniMax H3 when you need sound, and LTX-2.5 Audio-to-Video or LatentSync for talking heads. The Create a Video with Sound card uses Cosmos3 Nano. Scope notes come from each node's own documentation; the models have not been benchmarked against each other inside CNAPS Studio.
All of these run inside CNAPS Studio โ compare them side-by-side.
