🧩

Segmentation - SAM3 (Scene)

🛠

Segment Objects with Text Prompts

Automatically segment all objects matching a text description in images. Understands spatial and attribute relationships like "person in red" or "car on the left.”

What it does

SAM3 segments objects in images using text prompts. Point it at a photo and describe what you want to find—"cat", "person wearing glasses", "yellow car"—and it automatically finds and masks all matching objects. Can also segment using visual prompts (points, boxes, masks). Understands 4 million different concepts from short text descriptions.

Problem it solves

  • Find specific objects with text – Describe what to find and segment all matching objects instantly
  • Understand attributes – Segment "person in white shirt" differently from "person in red shirt"
  • Understand spatial relationships – Find "car on the left" vs "car on the right"
  • Privacy filtering – Automatically blur faces, license plates, screens, or documents
  • Image annotation – Generate precise masks for all instances of a concept
  • Object counting – Identify and count all instances of a specific type
  • Content moderation – Automatically detect and mask inappropriate content

Input/Output

  • Input: Images with text prompts (or visual prompts)
      • image
    1. Format: PNG, JPEG, BMP, GIF
    2. Content: Photos, screenshots, documents, surveillance footage, artwork
    3. Prompts: Text descriptions ("person", "cat in red", "car on the left")
    4. Visual prompts: Points, bounding boxes, or example masks
  • Output: Segmentation masks with instance IDs
      • image
    1. Masks: Binary or confidence masks for each matching object
    2. Coordinates: Bounding boxes and exact object boundaries
    3. Confidence scores: How confident the model is in each mask

Performance

  • Accuracy by Task Type:
    • Single concept segmentation: State-of-the-art performance (75-80% of human performance on SA-CO benchmark)
    • Attribute-based segmentation: Excellent (handles "white player" vs "red player" distinction)
    • Spatial relationship understanding: Very good (interprets "left", "right", "background", "foreground")
    • Object counting: Excellent (finds and counts all instances of a type)
    • Concept vocabulary: Supports 4M+ concepts (vs 50K in previous benchmarks)
  • Real-World Scenarios:
    • Privacy filtering: Excellent (faces, plates, screens, documents)
    • Object counting: Excellent (finds and counts all instances of a type)
    • Product tagging: Excellent (identifies products by description)
    • Attribute distinction: Excellent (separates similar objects by color or appearance)
    • Spatial understanding: Very good (understands relative positions)

Technical Details

Architecture
Vision foundation model for image segmentation
Components
• Vision Encoder: Processes images into feature representations • Detector: DETR-based detector for text-conditioned image segmentation • Presence Head: Distinguishes closely related text prompts (e.g., "white" vs "red")
Inference Pipeline
Encode image → Process text prompt → Detector finds all matching objects → Return masks and coordinates
Training
• Dataset: 4M unique concepts with hard negatives • Data engine: Automatically annotated to create largest concept segmentation dataset • Framework: PyTorch

Compliance & Provenance

Provider
Meta (open)
Provider type
Specialized
License
Others
EU AI Act risk class
Minimal Risk
Art. 50 transparency
Not applicable
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source

  • Github: https://github.com/facebookresearch/sam3
  • Huggingface: https://huggingface.co/facebook/sam3
  • License: others
    • Important: You are not the target of Trade Controls and your use of SAM Materials must comply with Trade Controls. You agree not to use, or permit others to use, SAM Materials for any activities subject to the International Traffic in Arms Regulations (ITAR) or end uses prohibited by Trade Controls, including those related to military or warfare purposes, nuclear industries or applications, espionage, or the development or use of guns or illegal weapon.