🧩

Segmentation - SAM3.1 (Scene)

🛠

Language-Prompted Image Segmentation

Segment every instance of an open-vocabulary concept in an image using a short text phrase like "person" or "red car". SAM 3.1 is the Object Multiplex refresh of SAM 3 - same prompting interface and concept coverage, with improved detection scores over SAM 3.

What it does

SAM 3.1 (Scene) segments objects in images using text prompts (or visual prompts: points, boxes, masks). Point it at a photo, type a short phrase — "cat", "person wearing glasses", "yellow car" - and it automatically finds and masks all matching objects in the scene. Architecturally it's the next-generation drop-in upgrade to SAM 3: the prompting model and 4M+ concept vocabulary carry over, but the new Object Multiplex memory design improves detection scores on the SA-Co and VEval benchmarks over SAM 3 at comparable inference cost. Use this when you would have used SAM 3 - you'll get the same workflow with better accuracy.

Problem it solves

  • Better detection accuracy than SAM 3 – Object Multiplex's shared-memory design improves SA-Co / VEval scores over SAM 3 without changing how you prompt the model
  • Find specific objects with text – Describe what to find ("person", "cat in red", "car on the left") and segment all matching objects in one pass
  • Understand attributes – Distinguishes "person in white shirt" from "person in red shirt"
  • Understand spatial relationships – Interprets "left", "right", "background", "foreground"
  • Privacy filtering – Automatically blur faces, license plates, screens, or documents
  • Image annotation – Generate precise masks for every instance of a concept
  • Object counting – Identify and count all instances of a given type
  • Content moderation – Detect and mask sensitive content by description

Input/Output

  • Input: Images with text prompts (or visual prompts)
    • Format: PNG, JPEG, BMP, GIF
    • Content: Photos, screenshots, documents, surveillance footage, artwork
    • Prompts: Short text phrases ("person", "cat in red", "car on the left")
    • Visual prompts: Points, bounding boxes, or example masks
  • Options:
    • Merge to binary mask (checkbox) - when on, all per-instance masks are flattened into a single binary mask for the concept
  • Output: Segmentation masks with instance IDs
    • Masks: Binary or confidence masks for each matching object (or a single merged binary mask when "Merge to binary mask" is on)
    • Coordinates: Bounding boxes and exact object boundaries
    • Confidence scores: Model confidence per mask

Performance

Inference time
~10 – 30 seconds per class on typical hardware
Detection accuracy
Improved over SAM 3 on SA-Co / VEval benchmarks (Object Multiplex refresh)
Concept vocabulary
4M+ concepts (same as SAM 3)
Single-concept segmentation
State-of-the-art
Attribute-based segmentation
Excellent — separates similar objects by color, clothing, or appearance
Spatial relationship understanding
Very good — interprets "left", "right", "background", "foreground"
Object counting
Excellent - finds and counts all instances of a type
Privacy filtering
Excellent for faces, license plates, screens, documents

Technical Details

Architecture
Vision foundation model for promptable image segmentation
Family
SAM 3 (Segment Anything with Concepts) — SAM 3.1 is the Object Multiplex refresh
Key innovation in SAM 3.1
Object Multiplex — shared-memory approach that improves detection scores on SA-Co / VEval vs. SAM 3
Components
Vision Encoder → text-conditioned Detector (DETR-based) → Presence Head (distinguishes closely related text prompts, e.g. "white" vs. "red")
Inference Pipeline
Encode image → process text prompt → detector finds all matching objects → return masks and coordinates
Concept Coverage
4M+ unique concepts (vs. ~50K in previous benchmarks)
Training
Trained on the SAM 3 large-scale concept-segmentation dataset with hard negatives; auto-annotated data engine
Framework
PyTorch — checkpoints distributed via HF; runtime via the SAM 3 GitHub repo (no Transformers integration)

Compliance & Provenance

Provider
Meta (open)
Provider type
Specialized
License
Others
EU AI Act risk class
Minimal Risk
Art. 50 transparency
Not applicable
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source

  • Huggingface: https://huggingface.co/facebook/sam3.1
  • License: others
    • Important: You are not the target of Trade Controls and your use of SAM Materials must comply with Trade Controls. You agree not to use, or permit others to use, SAM Materials for any activities subject to the International Traffic in Arms Regulations (ITAR) or end uses prohibited by Trade Controls, including those related to military or warfare purposes, nuclear industries or applications, espionage, or the development or use of guns or illegal weapon.