Pose Estimation - RF-DETR Keypoint

🛠

Real-Time Human Pose Estimation — 17 Keypoints Per Person, Rendered as an OpenPose-Style Skeleton

Real-time human pose estimation predicting 17 COCO keypoints per person (eyes, nose, ears, shoulders, elbows, wrists, hips, knees, ankles), rendered as an OpenPose-format skeleton map.

What it does

RF-DETR Keypoint looks at a photo and, for each person it finds, returns where the joints of that person are. Specifically, it predicts the 17 standard COCO keypoints: nose, left eye, right eye, left ear, right ear, left shoulder, right shoulder, left elbow, right elbow, left wrist, right wrist, left hip, right hip, left knee, right knee, left ankle, and right ankle. The node returns two things at the same time: a skeleton image (a black canvas with colored bones connecting the joints, in the well-known OpenPose visual style — red/orange arms, yellow/green legs, purple face) and a text list of every detection with pixel coordinates and confidence scores. This version is marked "Preview" — it's a work-in-progress addition to the RF-DETR lineup.

Problem it solves

  • See what pose someone is in – Get the exact location of every joint (eyes, shoulders, elbows, wrists, hips, knees, ankles) for every person in a photo
  • Drive character animation / motion transfer – The OpenPose-format skeleton map is the standard input for pose-guided image generation, ControlNet Pose, or motion transfer pipelines
  • Sports / fitness analytics – Measure body posture, form, movement — golf swing, yoga pose, running form, weightlifting stance
  • Health & rehab tracking – Monitor range of motion, symmetry, gait, posture over time
  • Photo/video composition – Understand where the people are and how they're arranged in a scene
  • Multi-person scenes – Returns a separate skeleton for each person in the image
  • Real-time performance – Same fast transformer backbone as RF-DETR detection/segmentation, so it's fast enough for live video

Input/Output

  • Input: A regular color photo (PNG / JPG / JPEG / WebP)
  • Options:
    • Minimum confidence score to keep a detected person (0.0 – 1.0, default 0.5)
      • Only people the model is at least this confident about are returned
      • Higher = fewer, more confident detections; lower = more people kept, including weaker guesses
  • Output: Two outputs, in parallel:
    • Skeleton image — a rendered picture on a black background showing every detected person as an OpenPose-format stick figure: colored bones connecting the 17 joints, with joint dots. Ready to feed into any downstream node that expects an OpenPose-format pose map (e.g., pose-guided image generation, ControlNet Pose)
    • Text list — one line per detected element:
      • Person line — person,(x1 y1 x2 y2),score — the person's bounding box and detection score
      • Keypoint lines — one per joint, <joint_name>,(x y),confidence — e.g. nose,(223.74 276.33),0.998, left_eye,(260.49 246.91),0.999, left_wrist,(410.11 522.4),0.87, etc.
      • Joints in order: nose, left_eye, right_eye, left_ear, right_ear, left_shoulder, right_shoulder, left_elbow, right_elbow, left_wrist, right_wrist, left_hip, right_hip, left_knee, right_knee, left_ankle, right_ankle
        • image

Performance

Accuracy & Speed

Speed profile
Real-time on modern GPUs — same fast transformer backbone as RF-DETR detection & segmentation
Keypoints per person
17 — the standard COCO keypoint layout
Rendering format
OpenPose (compatible with pose-guided generation, ControlNet Pose, and other downstream pose-consumers)
Multi-person support
Yes — returns a skeleton per detected person
Confidence threshold default
0.5 for filtering detected people
Family
Roboflow's RF-DETR (real-time transformer for detection, segmentation, and — in preview — keypoints)
Status
Preview — expect the model to be updated as it matures

Technical Details

Architecture
RF-DETR (keypoint variant) — DINOv2-with-registers ViT backbone + multi-scale projector + Deformable DETR decoder + keypoint prediction head
Backbone Detail
DINOv2-with-registers style ViT with RF-DETR's windowed / full attention alternation
Multi-scale Fusion
RF-DETR multi-scale projector (C2f-style blocks, LW-DETR lineage) aggregates multi-level backbone features before the decoder
Decoder
Deformable DETR-style decoder with multi-scale deformable cross-attention; extended with a keypoint prediction head on top of person detection
Keypoint Format
17 COCO keypoints — nose, L/R eye, L/R ear, L/R shoulder, L/R elbow, L/R wrist, L/R hip, L/R knee, L/R ankle
Rendering
OpenPose-format skeleton map (colored bones on black background — arms/legs/face color-coded per OpenPose convention)
Detection style
End-to-end set prediction (DETR-style queries, no anchors, no NMS) — one pass returns all persons + all keypoints
Framework
rfdetr open-source Python package (Roboflow), PyTorch
Related paper
"RF-DETR: Neural Architecture Search for Real-Time Detection Transformers" — arXiv:2511.09554 (Robinson et al., Nov 2025). ICLR 2026 accepted
Sibling models in the studio
RF-DETR-Medium (detection), RF-DETR-Seg-Medium (instance segmentation)
License
Apache 2.0 (per the open-source RF-DETR package)
Preview flag
This checkpoint is labeled a Preview — API and accuracy characteristics may change as the model matures

Compliance & Provenance

Provider
Roboflow (open)
Provider type
Specialized
License
Apache 2.0
EU AI Act risk class
Minimal Risk
Art. 50 transparency
Not applicable
Region availability
Available globally
Training data summary
Provider model card / training data

For more on how we classify models and mark outputs, see our AI Policy.

Model Source