🗺️

Segmentation - RF-DETR Seg Medium

🛠

Find Objects and Cut Them Out, Pixel by Pixel

The "cutout" version of RF-DETR-Medium. Same idea - finds objects in a photo - but instead of just drawing a box around each one, it traces the exact outline of every object, pixel by pixel. Great for cutouts, background removal, compositing, or anything that needs precise object shapes rather than just rectangles

What it does

RF-DETR-Seg-Medium looks at a photo and, for every object it spots, returns four pieces of information: a label ("person", "car", "dog"), a box around the object, a confidence score, and - the new part - a pixel-precise mask showing the exact shape of the object. Where the regular detector says "there's a dog inside this rectangle," this one says "here are the exact pixels that belong to the dog." It even handles overlapping objects of the same kind - two cats on a couch get two separate masks. It recognizes the same 80 everyday categories as the detection sibling (people, animals, vehicles, household items, food, etc.). The "Medium" size is the balanced choice between accuracy and speed.

Problem it solves

  • Pixel-precise object shapes, not just boxes - Get the exact outline of every detected object - useful for background removal, cutouts, compositing, photo editing, and any analysis that needs object shape
  • Separates overlapping objects of the same kind - Two cats sitting together? You get two separate masks, not one blob
  • One model, all in one pass - Returns label + box + mask for every object in a single run - no need for a separate "detect first, then segment" pipeline
  • Fast enough for real-time use - Built for speed; the Medium size balances accuracy and latency
  • 80 everyday categories out of the box - Covers the most common things you'd want to segment
  • Strong base for custom training - Roboflow released this family to be fine-tuned on custom datasets
  • Easy to plug in - Works directly with Hugging Face Transformers, the standard AI library

Input/Output

  • Input: A regular color photo (PNG, JPEG, etc.)
    • image
  • Options:
    • Minimum confidence score to keep a detection (0.0 – 1.0, default 0.5)
      • Only detections with a confidence score at or above this threshold are returned
      • Higher = fewer, more confident objects; lower = more objects, including weaker guesses
  • Output: For every object the model finds:
      • image
      image
    1. Label – What it is (e.g., "person", "car", "dog")
    2. Confidence – How sure the model is, from 0.0 to 1.0 (only objects at or above the threshold above are returned)
    3. Mask – A pixel-precise outline of the object's exact shape
    4. Bounding box – Also returned, in case a rectangle is enough for your use case
    5. The result is structured as a single image-sized "label map" plus a list of per-object entries — so you can pull out each object's mask separately

Accuracy & Speed

Model size
35.7 million parameters — slightly bigger than the detection sibling (which doesn't need to draw masks)
Speed
Real-time on modern GPUs; the "Medium" pick on the speed-vs-accuracy spectrum
Where it sits in the family
RF-DETR-Seg comes in multiple sizes — Medium is the balanced default
How it compares to YOLO
At the same speed bracket, RF-DETR-Seg-Medium produces more accurate masks than similarly fast YOLO-seg models (e.g., +1.2 mask AP over YOLO26m-seg at slightly faster speed). The trade-off: RF-DETR uses more parameters, so it needs more memory than YOLO of the same "size letter"
What it recognizes
80 everyday object categories (the standard COCO list)
What you get per object
Label + confidence + box + pixel-precise mask, all in a single pass
Good for
Production segmentation where mask quality matters and you want better results than YOLO-seg at similar speed

Technical Details

Architecture
RF-DETR (segmentation variant) — DINOv2-with-registers ViT backbone + multi-scale projector + Deformable DETR decoder + instance-segmentation head
Backbone Detail
DINOv2-with-registers style ViT with RF-DETR's windowed / full attention alternation
Multi-scale Fusion
RF-DETR multi-scale projector (C2f-style blocks, LW-DETR lineage) aggregates multi-level backbone features before the decoder
Decoder
Deformable DETR-style decoder with multi-scale deformable cross-attention; mask prediction head added on top of box/class outputs
Object Queries
DETR-style queries with bipartite matching and auxiliary decoder losses
Training Losses
Box (L1 + GIoU), classification, and mask losses (focal / dice-style terms as configured), with auxiliary supervision on intermediate decoder layers
Training Tricks
Group DETR (parallel decoder copies for faster convergence)
Family Discovery
Weight-sharing Neural Architecture Search (NAS) over accuracy/latency knobs — Seg-Medium is one subnet on the frontier
Parameters
35.7M (F32 weights)
Training Data
COCO 2017 instance segmentation (80 thing categories, as reflected in config.id2label)
Supported Frameworks
Hugging Face Transformers (RfDetrForInstanceSegmentation), Safetensors
Paper
"RF-DETR: Neural Architecture Search for Real-Time Detection Transformers" — arXiv:2511.09554 (Robinson et al., Nov 2025)
Sibling model
RF-DETR-Medium (detection only) — same architecture without the

Compliance & Provenance

Provider
Roboflow (open)
Provider type
Specialized
License
Apache 2.0
EU AI Act risk class
Minimal Risk
Art. 50 transparency
Not applicable
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source