Skip to content

Track Objects

Combine object detection with multi-object tracking to follow objects through video sequences, maintaining consistent IDs even through occlusions and fast motion.

What you'll learn:

  • Run tracking from the command line with a single command
  • Configure detection models and tracking algorithms
  • Visualize results with bounding boxes, IDs, and trajectories
  • Build custom tracking pipelines in Python

Install

Use the base install for tracking with your own detector. The detection extra adds inference-models for built-in detection.

pip install trackers

pip install "trackers[detection]"

For more options, see the install guide.

Watch: Building Real-Time Multi-Object Tracking with RF-DETR and Trackers

Quickstart

Read frames from video files, webcams, RTSP streams, or image directories. Each frame flows through detection to find objects, then through tracking to assign IDs.

Track objects with one command. Uses RF-DETR Nano and ByteTrack by default.

trackers track --source source.mp4 --output.video output.mp4

While trackers focuses on ID assignment, this example uses inference-models for detection and supervision for format conversion to demonstrate end-to-end usage.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker()

cap = cv2.VideoCapture("source.mp4")
while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

CLI Command Builder

Generate a production-ready trackers track command in seconds. Use interactive controls to tune core settings without memorizing flags.

Trackers

Trackers assign stable IDs to detections across frames, maintaining object identity through motion and occlusion.

Select a tracker with --tracker and tune its behavior with --tracker.* arguments.

trackers track \
    --source source.mp4 \
    --tracker bytetrack \
    --tracker.lost_track_buffer 60 \
    --tracker.min_consecutive_frames 5

CLI parameter names abbreviate the standard leading token: minimum_ becomes min_ and maximum_ becomes max_. Domain words such as threshold stay spelled out, and the Python constructor names are unchanged.

Customize the tracker by passing parameters to the constructor, then call update() each frame and reset() between videos.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker(
    lost_track_buffer=60,
    minimum_consecutive_frames=5,
)

cap = cv2.VideoCapture("source.mp4")
while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

Detectors

Trackers don't detect objects—they link detections across frames. A detection or segmentation model provides per-frame bounding boxes or masks that the tracker uses to assign and maintain IDs.

Configure detection with --detection.* arguments. Filter by confidence and class before tracking.

trackers track \
    --source source.mp4 \
    --detection.model rfdetr-medium \
    --detection.confidence 0.3 \
    --detection.device cuda \
    --filters.classes [person,car]

Trackers are modular—combine any detection library with any tracker. This example uses inference with RF-DETR.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker()

cap = cv2.VideoCapture("source.mp4")
while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame, confidence=0.3)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

Visualization

Visualization renders tracking results for debugging, demos, and qualitative evaluation.

Enable display and annotation options to see results in real time or in saved video.

trackers track \
    --source source.mp4 \
    --display \
    --show.labels \
    --show.confidence \
    --show.trajectories

Use supervision annotators to draw results on frames before saving or displaying.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker()
box_annotator = sv.BoxAnnotator()
label_annotator = sv.LabelAnnotator()

cap = cv2.VideoCapture("source.mp4")
while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

    frame = box_annotator.annotate(frame, detections)
    frame = label_annotator.annotate(frame, detections)

Source

trackers accepts video files, webcams, RTSP streams, and directories of images as input sources.

Pass videos, webcams, streams, or image directories as input.

trackers track --source source.mp4

Use opencv-python's VideoCapture to read frames from files, webcams, or streams.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker()

cap = cv2.VideoCapture(0)
while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

Output

Save tracking results as annotated video files or display them in real time.

Specify an output path to save annotated video.

trackers track --source source.mp4 --output.video output.mp4 --output.overwrite

Use opencv-python's VideoWriter to save annotated frames with full control over codec and frame rate.

import cv2

import supervision as sv
from inference import get_model
from trackers import ByteTrackTracker

model = get_model("rfdetr-nano")
tracker = ByteTrackTracker()
box_annotator = sv.BoxAnnotator()

cap = cv2.VideoCapture("source.mp4")
width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = cap.get(cv2.CAP_PROP_FPS)

fourcc = cv2.VideoWriter_fourcc(*"mp4v")
out = cv2.VideoWriter("output.mp4", fourcc, fps, (width, height))

while True:
    ret, frame = cap.read()
    if not ret:
        break

    result = model.infer(frame)[0]
    detections = sv.Detections.from_inference(result)
    detections = tracker.update(detections)

    frame = box_annotator.annotate(frame, detections)
    out.write(frame)

cap.release()
out.release()

Dynamic frame rate

By default, each update() assumes one frame at a steady rate. If your pipeline skips frames or has irregular timing, pass an optional monotonic timestamp (seconds) to tracker.update() so Kalman prediction and lost-track pruning match the real gap. Omit timestamp to keep fixed-rate behaviour.

See the Dynamic Frame Rate guide for when to enable it, frame_step semantics, edge cases, and a Python example.

CLI Reference

The commonly used arguments accepted by the trackers track command. --tracker.* is generated from the tracker registry, so it also carries parameters specific to one algorithm (cbiou's buffer ratios, botsort/mcbyte's camera motion compensation, mcbyte's mask pipeline) that this table does not enumerate. Run trackers track --help for the complete, always-current set, and trackers track --tracker.mask.help for the mcbyte mask sub-options.

Most --tracker.* flags default to null on the CLI — leaving a flag unset means the selected tracker's own default value is used, and that value differs per algorithm. See the tracker API reference for actual constructor defaults and the individual SORT, ByteTrack, OC-SORT, BoT-SORT, C-BIoU, and McByte pages for parameter guidance. Note that OC-SORT has no track_activation_threshold parameter, even though --tracker.track_activation_threshold is exposed globally on the CLI.

Argument Description Default
--source Input source. Accepts file paths (.mp4, .avi), device indices (0, 1), stream URLs (rtsp://), or image directories. —
--output.video Path for output video. If a directory is given, saves as output.mp4 inside it. none
--output.overwrite Allow overwriting existing output files. Without this flag, existing files cause an error. false
--output.mot_results Path to write MOT-format predictions. none
--detection.model Model identifier. Pretrained: rfdetr-nano, rfdetr-small, rfdetr-medium, rfdetr-large. Segmentation: rfdetr-seg-*. rfdetr-nano
--detection.confidence Minimum confidence threshold. Lower values increase recall but may add noise. 0.5
--detection.device Compute device. Options: auto, cpu, cuda, cuda:0, mps. auto
--detection.api_key Roboflow API key for custom hosted models. none
--detection.mot_file Path to a pre-computed MOT-format detector-output file. When set, this takes precedence over --detection.model, which is ignored. none
--filters.classes List of class names or IDs to track. Example: [person,car], [0,2], or the mixed [person,2]. all
--filters.track_ids List of track IDs to keep in the output. Example: [1,3,5]. all
--tracker Tracking algorithm. Options: bytetrack, sort, ocsort, botsort, cbiou, mcbyte. bytetrack
--tracker.frame_rate Video frame rate used to scale the lost track buffer to time-like behavior. Must be positive. null (tracker default)
--tracker.lost_track_buffer Frames to retain a track without detections. Higher values improve occlusion handling but risk ID drift. null (tracker default)
--tracker.track_activation_threshold Minimum confidence to start a new track. Lower values catch more objects but increase false positives. Not used by OC-SORT. null (tracker default)
--tracker.min_consecutive_frames Successful matched detections required before a track is confirmed. Whether a miss resets this count depends on the selected tracker. null (tracker default)
--tracker.min_iou_threshold Minimum IoU overlap to match a detection to an existing track. Higher values require tighter alignment. null (tracker default)
--tracker.min_iou_threshold_first_assoc Minimum score-fused geometric similarity for first-pass association. BoT-SORT and McByte use IoU; C-BIoU uses buffered IoU. McByte may further condition association with masks. null (tracker default)
--tracker.min_iou_threshold_second_assoc Minimum geometric similarity for second-pass association without detection-score fusion. BoT-SORT and McByte use IoU; C-BIoU uses buffered IoU. McByte may further condition association with masks. null (tracker default)
--tracker.min_iou_threshold_unconfirmed_assoc Minimum score-fused geometric similarity for matching unconfirmed tracks to remaining high-confidence detections. BoT-SORT and McByte use IoU; C-BIoU uses buffered IoU. McByte may further condition association with masks. null (tracker default)
--tracker.high_conf_det_threshold Confidence threshold splitting detections into high- and low-confidence sets for two-stage association. null (tracker default)
--tracker.direction_consistency_weight Weight for direction consistency in the association cost. Higher values prioritize angle alignment between motion and association direction. ocsort only. null (tracker default)
--tracker.delta_t Number of past frames used for velocity estimation. Higher values give more stable direction estimates during occlusion. ocsort only. null (tracker default)
--tracker.iou_variant IoU similarity metric for data association. Options: iou, giou, diou, ciou, biou. Applies to all trackers. iou
--tracker.enable_cmc Camera motion compensation toggle. botsort and mcbyte only. null (tracker default)
--tracker.cmc_method CMC method. Options: orb, sift, sparseOptFlow, ecc. botsort and mcbyte only. null (tracker default)
--tracker.cmc_downscale Downscale factor used inside CMC for speed/robustness. botsort and mcbyte only. null (tracker default)
--tracker.instant_first_frame_activation If true, tracks spawned on the very first frame receive a real track ID immediately. If false, they start unconfirmed (-1) and must survive min_consecutive_frames like any other track. null (tracker default)
--tracker.state_estimator_class State estimator used by newly created tracks, given as a dotted class path (e.g. trackers.utils.state_representations.XCYCWHStateEstimator). null (tracker default)
--tracker.enable_mask_manager Enable McByte's SAM + Cutie mask pipeline. mcbyte only; see --tracker.mask.* below. false
--tracker.mask.* Backend settings for the mcbyte mask pipeline — SAM/Cutie device and checkpoints. Run trackers track --tracker.mask.help for the full sub-option list. —
--display Opens a live preview window. Press q or ESC to quit. false
--show.boxes Draw bounding boxes around tracked objects. true
--show.masks Draw segmentation masks. Only available with rfdetr-seg-* models. false
--show.confidence Show detection confidence scores in labels. false
--show.labels Show class names in labels. false
--show.ids Show tracker IDs in labels. true
--show.trajectories Draw motion trails showing recent positions of each track. false