Skip to content

Benchmark Runner

Run a tracker over a complete benchmark test set — MOT17, DanceTrack, SportsMOT, or SoccerNet-tracking — and write one MOTChallenge-format result file per sequence, step by step.

What you'll learn:

  • Get detection/frame data in place, downloaded or self-supplied
  • Point the command at your detection and frame directories
  • Select one or more datasets to run
  • Read the output layout, including the MOT17 submission files

trackers benchmark is a command group, one subcommand per tracker with a benchmark harness. McByte is the only one available today, so every example below reads trackers benchmark mcbyte; the steps and the output layout are the same for any harness added later.

McByte specifics

trackers benchmark mcbyte always builds McByte's full SAM + Cutie mask pipeline (enable_mask_manager=True), so — unlike the default McByteTracker() construction described on the McByte page — both SAM and Cutie must be installed before running this command. See the optional heavyweight dependencies note for install steps.

Step 1 — Get Your Data

Dataset Detection format Layout note
mot17 xyxy Frame directories use the <sequence>-FRCNN suffix.
dancetrack xyxy —
sportsmot xyxy —
soccernet mot Detection filenames follow the SoccerNet naming convention.

mot17 and sportsmot can be fetched directly with trackers download — see Download Datasets. dancetrack and soccernet aren't downloadable via that command yet; supply your own detection_root/image_root directories for those two, pointed at in Step 2.

Each sequence is processed independently with a fresh tracker instance. If a sequence fails, the error is logged and the run continues with the remaining sequences.

Step 2 — Configure Dataset Roots

Every dataset needs a detection_root (one detection file per sequence) and an image_root (one frame directory per sequence). Neither has a built-in value — supply both per run, either through a --config file or inline as JSON with --dataset_roots.

Supply roots as JSON on the command line.

trackers benchmark mcbyte --dataset_roots='{"mot17": {"detection_root": "/data/dets", "image_root": "/data/frames"}}'

A config file is the readable spelling.

# run.yaml
dataset: [mot17]
dataset_roots:
  mot17:
    detection_root: /data/detections/MOT17/test
    image_root: /data/datasets/MOT17/test

trackers benchmark mcbyte --config run.yaml

detection_root holds one detection .txt file per sequence:

detections/MOT17/test/MOT17-01.txt
detections/dancetrack/test/dancetrack0003.txt
detections/sportsmot/test/v_-9kabh1K8UA_c008.txt
detections/SoccerNet_tracking_2022_test_set_dets/SNMOT-116__det.txt

image_root holds one directory per sequence, each with an img1 subdirectory of frames, for every dataset:

datasets/dancetrack/test/
    dancetrack0003/img1/
    dancetrack0009/img1/
    ...

MOT17, DanceTrack and SportsMOT detections use frame,x1,y1,x2,y2,confidence (XYXY). SoccerNet-tracking detections use the original ground-truth MOT layout, frame,id,left,top,width,height,confidence,... — the identity column is ignored, since identities are produced by the tracker.

Step 3 — Select Datasets

Pass --dataset as a list to choose which datasets to run. Omit it to run every dataset from Step 1's table.

trackers benchmark mcbyte --dataset=[mot17,soccernet] --device=cuda

A bare repeated --dataset overwrites the previous value — use --dataset+ to append instead:

trackers benchmark mcbyte --dataset+ mot17 --dataset+ soccernet

As with every trackers subcommand, hyphens and underscores are interchangeable (--cmc-downscale and --cmc_downscale are the same option), and every boolean flag has a --no_ negation, e.g. --no_enable_cmc.

Step 4 — Run and Read the Output

Results are written under --output_root, in one timestamped directory per run (<timestamp>__isolation or <timestamp>__no_isolation, reflecting --enable_isolated_mask_matching), with one subdirectory per dataset and a run.log capturing progress and failures.

outputs/mcbyte_benchmarks/
└── 20260807_120000__no_isolation/
    ├── run.log
    └── mot17/
        ├── raw/
        │   ├── MOT17-01.txt
        │   └── ...
        └── submission/
            ├── MOT17-01-FRCNN.txt
            ├── MOT17-01-SDP.txt
            ├── MOT17-01-DPM.txt
            └── ...

Each sequence result is first written to a .partial file and only replaces the final file on success; use --skip_existing to skip a sequence whose result file is already present, and --keep_partial_results to keep a failed sequence's .partial file instead of deleting it.

MOT17 submission files

The MOT17 evaluation server expects one result file per detector name (FRCNN, SDP, DPM). Since the tracker is detector-agnostic, trackers benchmark mcbyte duplicates each completed sequence's result across all three suffixes under mot17/submission/. Seven MOT17 test-sequence numbers (02, 04, 05, 09, 10, 11, 13) have no detection file in this benchmark's layout and are never attempted — for these, all three suffixes always get an empty placeholder file. The other seven (01, 03, 06, 07, 08, 12, 14) are attempted; if one of them fails, its raw result is missing and no placeholder is written for it — that sequence's submission files are simply absent, and a warning is logged instead.

trackers benchmark mcbyte writes raw per-sequence result files, not aggregate scores — it doesn't compute HOTA/IDF1/MOTA itself. Score the output with trackers eval (see Evaluate Trackers); published McByte numbers already appear on the Results page.

CLI Reference

All arguments accepted by trackers benchmark mcbyte.

Argument Description Default
--config Path to a configuration file. —
--print_config Print the configuration after applying all other arguments, then exit. —
--dataset Datasets to run, as a list: --dataset=[mot17,soccernet]. Repeat as --dataset+ to append instead of overwrite. all datasets
--dataset_roots Where each dataset's files live, keyed by the same names --dataset selects. Each entry holds a detection_root and an image_root; neither has a built-in value. —
--device Device for SAM + Cutie, e.g. cuda, cpu, or mps. auto resolves to CUDA when available, otherwise CPU; MPS is never auto-selected and must be requested explicitly. auto
--enable_isolated_mask_matching Match masks in isolation. Negate with --no_enable_isolated_mask_matching. false
--output_root Directory holding one timestamped run directory per run. outputs/mcbyte_benchmarks
--skip_existing Skip a sequence whose result file is already present. Negate with --no_skip_existing. false
--enable_cmc Compensate for camera motion. Negate with --no_enable_cmc. true
--cmc_method Camera-motion compensation method. Options: orb, sift, sparseOptFlow, ecc. sparseOptFlow
--cmc_downscale Frame downscale factor applied before compensation. 6
--keep_partial_results Keep the .partial file a failed sequence leaves behind instead of deleting it. Negate with --no_keep_partial_results. false