🧠

Multimodal Language Models - Molmo2-8B

🛠

Open Vision-Language Model

Molmo2-8B understands a text prompt plus up to two images in a single pass, and is state-of-the-art among open multimodal models in its 8B size class on counting, captioning, and visual question answering.

What it does

Molmo2-8B is a multimodal AI from the Allen Institute for AI (Ai2) that reads images and answers questions or generates captions in natural language. Given a text prompt with one or two reference images, it can describe what it sees, answer questions about the content, compare two images, count objects, or extract structured information. The model is built on Qwen3-8B as the language backbone with SigLIP 2 as the vision encoder, trained on the publicly released Molmo2 data collection of highly curated image-text pairs. The CNAPS Studio integration runs the bf16 variant in ~17GB VRAM.

Problem it solves

  • Visual question answering (VQA) – Answer natural-language questions about image content with strong accuracy across everyday scenes, documents, charts, and product photos
  • Image captioning at scale – Generate descriptive, accurate captions for single images — useful for accessibility, content indexing, and dataset enrichment
  • Two-image comparison and analysis – Pass a primary image plus an optional second image to compare, contrast, or describe their relationship in one model call (e.g., before/after shots, A/B product comps)
  • Counting and enumeration – Best-in-class among open 8B-scale multimodal models at counting objects and listing distinct items
  • Cost-efficient open-weights deployment – At ~17GB VRAM in bf16, runs on a single mid-range GPU and is fully Apache 2.0 — no API fees, no rate limits, no data leaving your infrastructure

Input/Output

  • Input:
    • Required input:
      • Text prompt – Question, instruction, or caption request (e.g., "Describe this image", "What is the person doing?", "Compare these two images")
    • Optional inputs:
      • Image 1 – Primary reference image
      • Image 2 – Second reference image for comparison or multi-image questions
    • Note: All inputs accept standard image formats (PNG, JPEG). Prompts work with one or both images, or text-only if you want to query the model conversationally.
  • Advanced configuration:
Parameter
Range
Default
What it does
Max new tokens
1 ~ 4096
512
Maximum length of the model's response. Increase for long captions or detailed analyses; lower it for short answers and faster generation
Temperature
0 ~ 2
0.7
Controls randomness. Lower (0.2–0.5) for factual VQA and consistent captioning; higher (0.8–1.2) for varied, creative descriptions
Top-p (nucleus)
0 ~ 1
0.95
Restricts sampling to tokens whose cumulative probability adds up to this threshold. Lower = more focused; default 0.95 is a good general setting
Top-k
integer
50
Limits sampling to the top-K most likely tokens at each step. Works alongside Top-p; 50 is a balanced default
  • Output:
    • Text – Natural-language answer, caption, comparison, or structured response based on the prompt
      • image
      image

Accuracy & Speed

Benchmark Performance

Average Score on 15 Academic Video Benchmarks (from Ai2's technical report)

Model
Score
Molmo2-8B (this model)
63.1
Molmo2-4B
62.8
Eagle2.5-8B
60.7
Qwen3-VL-8B
59.5
Molmo2-7B
59.7
GLM-4.1V-9B
56.9
MiniCPM-V-4.5-8B
56.6
InternVL3.5-8B
54.1

Reference closed-source models (for context, not direct competitors in the open-weights class)

Model
Score
Gemini 2.5 Pro
71.2
GPT-5
70.6
Gemini 3 Pro
70.0
GPT-5 mini
65.0
Claude Sonnet 4.5
59.6

Molmo2-8B is the best-in-class open-weights model at the 8B scale on this aggregate, ahead of Qwen3-VL-8B, Eagle2.5-8B, GLM-4.1V-9B, MiniCPM-V-4.5-8B, and InternVL3.5-8B

Technical Details

Model
Molmo2-8B (allenai/Molmo2-8B)
Architecture
Vision-language model — Qwen3-8B language backbone + SigLIP 2 (siglip-so400m-patch14-384) vision encoder
Base model
Qwen3-8B-Base → fine-tuned to Qwen3-8B → Molmo2-8B
Parameter count
~8B
Precision
bf16
VRAM footprint
~17 GB
Modalities (CNAPS node)
Text + up to 2 images
Training data
Molmo2 data collection (publicly released — Molmo2-SynMultiImageQA, Molmo2-VideoPoint, Molmo2-MultiImagePoint, and others)
Supported frameworks
Hugging Face Transformers ≥ 4.57.1 (AutoModelForImageTextToText), Safetensors

Compliance & Provenance

Provider
Allen AI (open)
Provider type
GPAI — General-Purpose AI (GPAI)
License
Apache 2.0
EU AI Act risk class
Limited Risk
Art. 50 transparency
Required — outputs are marked. See AI Policy §2.
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source