YOLO Object Detection

YOLO Object Detection runs a Python worker over the app’s live stream: it pulls the stream over HLS, runs ultralytics YOLO inference, and POSTs detections to a callback URL. It is the reference example of a worker plugin — the framework owns the worker’s whole lifecycle.

Plugin id	yolo
Category	processor
UI slot	player-overlay
Needs worker	Yes

The plugin stays fully decoupled from the core: it is only its manifest plus a pure worker.spawn(ctx) that returns { command, args, env, cwd }. Enabling the plugin (re)starts the worker; disabling stops it. Worker start/stop/status/logs are exposed over the API (see below).

Configuration

Field	Type	Default	Description
room	string	``(empty)	Required. HLS room name to analyze — /hls/<app>/<room>/index.m3u8.
model	select	nano	Model size → ultralytics weight. Options: nano (yolov8n, fastest/CPU-friendly), small (yolov8s), medium (yolov8m), large (yolov8l), xlarge (yolov8x, most accurate/GPU).
cuda	boolean	false	On uses the GPU (CUDA); off runs on CPU.
callbackUrl	string	`` (empty)	Required. Where the worker POSTs {app, room, ts, detections:[{class,conf,bbox}]} per frame.
confidence	number	0.35	Minimum detection confidence, 0–1.
fps	number	2	Frames per second to sample from the stream (lower = less CPU). Min 0.1, max 30.
classes	string	`` (empty)	Comma-separated COCO class names to keep (e.g. person,car,dog); empty detects all 80 classes.

room and callbackUrl are required, so the install cannot be enabled until both are set.

The worker

When enabled, the framework spawns the worker from the repo-root yolo-worker/ Python package, roughly:

python3 -m yolo_worker --app <app>

The spawn function maps the config into YOLO_* environment variables the worker reads (YOLO_ROOM, YOLO_MODEL, YOLO_DEVICE = cuda/cpu, YOLO_CONFIDENCE, YOLO_FPS, YOLO_CLASSES, YOLO_CALLBACK_URL, plus stream-source hints YOLO_HLS_DIR / YOLO_PUBLIC_BASE / YOLO_LIVEKIT_URL). Overridable env:

• PLUGIN_PYTHON — Python executable (default python3).
• YOLO_WORKER_DIR — path to the yolo-worker/ checkout (defaults to the copy at the repo root).

Compute requirements

Inference is heavy. Set cuda: true only on a host with a CUDA GPU — on a CPU-only host (e.g. stream01) leave it false. On CPU, keep model: nano and a low fps to stay within budget.

Enable and configure

Dashboard: open the app’s Plugins tab → Install YOLO Object Detection → Configure the room and callback URL (both required) plus model/confidence/fps → toggle Active to start the worker. Use the plugin’s Logs to watch the worker output.

API:

curl -X PATCH https://YOUR-DOMAIN/api/v1/apps/live/plugins/yolo \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
        "enabled": true,
        "config": {
          "room": "live",
          "model": "nano",
          "cuda": false,
          "callbackUrl": "https://example.com/hooks/yolo",
          "confidence": 0.4,
          "fps": 2,
          "classes": "person,car"
        }
      }'

Worker lifecycle and logs:

# Explicitly (re)start / stop the worker
curl -X POST https://YOUR-DOMAIN/api/v1/apps/live/plugins/yolo/worker/start \
  -H "Authorization: Bearer sk_..."
curl -X POST https://YOUR-DOMAIN/api/v1/apps/live/plugins/yolo/worker/stop \
  -H "Authorization: Bearer sk_..."

# Current worker state + recent logs
curl https://YOUR-DOMAIN/api/v1/apps/live/plugins/yolo/worker/status \
  -H "Authorization: Bearer sk_..."
curl "https://YOUR-DOMAIN/api/v1/apps/live/plugins/yolo/logs?limit=200" \
  -H "Authorization: Bearer sk_..."

Note

Although its UI slot is player-overlay, the detections themselves are delivered to your callbackUrl. The overlay/config sanitization on the public endpoint strips the callback URL, so anonymous players never see it.