Video Generation - Cosmos3 Nano

🛠

Text-to-Video and Image-to-Video — Powered by NVIDIA's Cosmos 3 World Model

Generate short cinematic video clips from a text description or a single image. Cosmos3-Nano is NVIDIA's lightweight 16B-parameter "world model" that has learned how the physical world looks and behaves. The studio exposes its video generation capability as a standalone feature first; other modalities (audio, action, reasoning) the model also supports may follow).

What it does

Cosmos3-Nano generates short, realistic video clips from either a text prompt ("a wide shot of a winter forest, snow falling, soft golden hour light…") or a single starting image plus a text description of what should happen next. The output is an MP4 video, by default ~7.9 seconds long at 1280×720 and 24 fps. Optionally, the model can also generate synchronized audio muxed into the video — useful for ambient sound, footsteps, environmental noise, etc. Cosmos3-Nano comes from NVIDIA's Cosmos 3 family of world foundation models, designed with Physical AI in mind (robotics, autonomous driving, industrial simulation), which makes it especially strong on realistic motion, scene dynamics, and camera behavior. "Nano" is the lightweight variant of the family (16B parameters); a larger Cosmos3-Super variant (64B) also exists with higher quality at heavier compute cost.

Problem it solves

  • Two generation modes in one model – Text-to-video for fully imagined scenes, image-to-video for animating a starting frame
  • Optional matching audio – Generate ambient sound or environmental audio that's synchronized with the generated video, in one pass
  • Physical-world realism – Trained for Physical AI use cases (robotics, autonomous driving, industrial scenes), so motion and scene dynamics tend to look more physically plausible than general-purpose video generators
  • Cinematic 720p output – Native 1280×720 generation at 24 fps, supports multiple aspect ratios (16:9, 4:3, 1:1, 3:4, 9:16)
  • Animate a single photo – Bring a still image to life with a short text instruction
  • Open-source and commercial-friendly – Released under OpenMDW 1.1, approved for both commercial and non-commercial use

Input/Output

  • Input:
    • Text to Video:
      • Plain-language description of the scene you want
      • Text up to 4,096 tokens
      • English works best
        • image
    • Image to Video:
      • One reference image (PNG / JPG / JPEG / WebP) at 256p, 480p, or 720p
      • Aspect ratios: 16:9, 4:3, 1:1, 3:4, 9:16
      • A text description of what should happen in the video starting from that image
  • Options:
    • Resolution (W × H) - Pick the ratio that matches the input image (when using image-to-video)
    • Frame count (must be a 4n + 1 value, e.g. 61, 121, 189, 241, 301) - default 189 frames (≈ 7.9s at 24 fps)
    • Output FPS (24 is the native frame rate) - 16, 24, 30
    • Sound generation (On / Off) - default Off
      • When On, describe the desired sound in the text prompt (e.g. "footsteps on gravel, wind in trees")
    • Diffusion steps (20 - 75, default 35) - More steps = higher quality, slower generation
    • Advanced Options:
      • CFG scale (default 6) - how strictly the output follows the prompt
      • Additional negative prompt (default empty) - extra description of what to avoid in the output. The model already ships with a built-in default negative prompt; this field is for anything you want to add on top
      • Random seed (-1 = random, default 1) - set a fixed integer to reproduce the exact same video
  • Output: Generated video
    • Format: MP4
    • Default size: 1280 × 720 at 24 fps
    • Default length: 189 frames (≈ 7.9 seconds), configurable 5 – 400 frames

Accuracy & Speed

Model size
16 billion parameters (the "Nano" variant of NVIDIA Cosmos 3)
Bigger sibling
Cosmos3-Super (64B) — higher quality, much heavier to run
Default video output
1280×720, 24 fps, 189 frames (≈ 7.9 seconds)
Output range
5 to 400 frames, multiple aspect ratios supported
Tested hardware
NVIDIA GB200 and H100
Recommended serving GPU
NVIDIA H200 (single-GPU, the documented vLLM-Omni configuration)
Multi-GPU scaling
Context parallelism (--ulysses-degree) or tensor parallelism (--tensor-parallel-size)
Lower-VRAM option
Layerwise offload (--enable-layerwise-offload) for smaller GPUs
Precision
BF16 — the only officially tested precision (FP16 / FP8 / FP4 not officially supported)
Strengths
Realistic motion and scene dynamics (Physical-AI training focus), optional synchronized audio, multi-aspect-ratio support

Technical Details

Architecture
Mixture-of-Transformers (MoT) — two complementary transformer towers in a unified model
Towers
(1) Autoregressive transformer for text; (2) Diffusion transformer for non-text (images, video, audio, actions). Video generation is produced by the diffusion tower through iterative denoising
Parameters
16B (Cosmos3-Nano)
Framework
NVIDIA Cosmos Framework
Default Inference Hyperparameters
num_inference_steps=35, guidance_scale=6.0, flow_shift=10.0, max_sequence_length=4096, num_frames=189, fps=24, size 1280×720
Prompt upsampling
Text prompts should be JSON-upsampled (separate utility provided in the cosmos-framework repo) for optimal quality; NVIDIA's text-to-video example uses Claude Opus 4.6 as the upsampler
Training data (generation)
767M image, 348M video, 139M audio, 8M action samples — drawn from public datasets (OpenImages, Coyo700M, YouTube Video, UMI), private datasets (Egocentric, Nexar, AgiBot, HOI), and large synthetic augmentation (HiDream-I1, Qwen-Image-2512)
Supported Hardware Microarchitectures
NVIDIA Ampere, Blackwell, Hopper
OS
Linux
Supported Frameworks
PyTorch, vLLM-Omni (Cosmos3 container), vLLM (with vllm-cosmos3 package), Hugging Face Diffusers (Cosmos3OmniPipeline)
Other capabilities of the underlying model
The same Cosmos3-Nano model also supports text reasoning over images and video (256K context), action-conditioned video generation for robotics, and action prediction from video — not exposed in the studio's video generation feature, but may follow in later releases
Release date
May 31, 2026
Paper
"Cosmos 3: Omnimodal World Models for Physical AI" — NVIDIA Cosmos Lab technical report

Compliance & Provenance

Provider
NVIDIA (open)
Provider type
Specialized
License
OpenMDW 1.1
EU AI Act risk class
Limited Risk
Art. 50 transparency
Required — outputs are marked. See AI Policy §2.
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source