Computer Vision: From Fundamentals to Applications

Contents

• Introduction and overview
• 0. Review: convolution, CNNs, boxes
• 1. Object detection: the two paradigms
• 2. Evaluation (mAP) and modern detectors
• 3. Segmentation: semantic, instance, panoptic, SAM
• 4. Data, benchmarks and annotation
• 5. Multi-object tracking
• 6. Applied 3D vision: stereo and LiDAR
• 7. Industrial case studies
• 8. Deployment: ONNX, quantization, edge
• 9. Ethics, robustness and limits
• Exercises
• Projects and assessment
• References

Introduction and overview

Audience: a CV engineer leaving benchmarks for field latency / robustness. Empirical first: high lab mAP, low nighttime recall; a tracker that swaps IDs at crossings — then we name detection, SAM, MOT, stereo/LiDAR, edge deploy.

The “systems” counterpart of foundational vision: full application pipeline.

What you will be able to do

• Compare the two paradigms of object detection and choose a detector under the application constraint.
• Rigorously evaluate a detector with mAP and read a public benchmark without being misled.
• Distinguish semantic, instance and panoptic segmentation, and use a generic segmentation model such as SAM.
• Implement multi-object tracking on a video sequence and measure it with MOTA and IDF1.
• Exploit stereo and LiDAR for applied 3D perception.
• Build and annotate a vision dataset, and measure its biases.
• Deploy a model under constraint: ONNX export, quantization, edge execution.
• Analyze the ethics, robustness and limits of an embedded vision system, with a per-subgroup evaluation.

Official objectives and syllabus (2026-2027 catalog)

Since this module is published by the brochure without a SynapseS course description of its own, the closest official reference in the 2026-2027 catalog is the description of CSC_52002_EP — “IA générative multimodale” (Multimodal Generative AI), a course unit of the (computer science) in-depth program of the LLGA track S2 with a vision focus.

Its official description presents multimodal generative AI — vision, audio, tactile information — as a paradigm shift that recomposes entire industries, from healthcare to automotive (tools such as Dall-E 3, Midjourney, GPT-Vision; autonomous driving).

The official course proposes to go “beyond CNNs and transformers” and to cover the advanced topics of multimodal generative AI: from image generation to that of videos, sounds and robot actions. Official prerequisites: not communicated (empty tab on the description).

Complement to the official syllabus — the themes structuring the brochure and these materials (object detection, segmentation and SAM, multi-object tracking, 3D vision with stereo and LiDAR, dataset construction, industrial case studies, ONNX/quantization/edge deployment, ethics and robustness) do not appear in the description of the CSC_52002_EP course unit: the correspondence above is partial and these materials keep the brochure framing (36 h, 4.5 ECTS).

Where this course fits in the program

Module of period 2 of M1 (36 h and 4.5 ECTS in the brochure), announced in the MSc&T brochure; not in the 2026–27 LLGA lists; covered here for reference — the home page places it on the AI-systems route. No dedicated SynapseS course description exists in the 2026-2027 catalog (period and assessment modalities: not communicated). It is the module that turns vision skills into deployment skills.

• Prerequisites — Image Analysis and Computer Vision (P1), whose applied extension it is: its statements of IoU, AP, NMS and rectified stereo are recalled here, not redone. Deep Learning (P1), or its P2 variant taken in parallel, for CNNs and training; the review section covers convolution, CNNs and bounding boxes.
• Downstream modules — the deployment chapter (ONNX, quantization, edge) directly prepares ML Ops and LLM Engineering Principles (M2), and the industrial case studies anticipate the careers targeted by the master’s program; vision systems also feed the multimodal pipelines of the Multimodal Generative AI module (same period), and the attacks of ch. 9 are treated in depth in Security and Robustness (M2).

Course outline

• Review: convolution, CNNs, boxes — the deep vision building blocks revisited before the application tasks.
• Object detection: the two paradigms — one-stage and two-stage detectors, and their speed-accuracy trade-offs.
• Evaluation (mAP) and modern detectors — measuring honestly first, then the panorama of recent architectures.
• Segmentation: semantic, instance, panoptic, SAM — the three granularities of segmentation and the era of foundation models.
• Data, benchmarks and annotation — building the dataset and the protocol before chasing the model.
• Multi-object tracking — following targets across a video, identities included.
• Applied 3D vision: stereo and LiDAR — recovering depth for embedded applications.
• Industrial case studies — real deployments, with their constraints and lessons learned.
• Deployment: ONNX, quantization, edge — fitting a model into an embedded compute budget.
• Ethics, robustness and limits — biases, attacks and failure cases: what a deployed system must own up to.
• Exercises, Projects and assessment and References — systematic training, project subjects with an exam proposal, and sources to go further.

Session plan

36 h in class = 12 sessions of 3 h. The volume is the 36 h announced for this module by the MSc&T brochure — an announced volume, with no SynapseS sheet yet (the module has no catalogue code of its own); no session-by-session programme is published, so the split below is our proposal for covering these notes in that time. Level 1 = foundations (reviews, definitions, first worked examples), 2 = core methods and systems; the level never goes down from one session to the next.

Self-study, outside the 36 h: none — every chapter is covered in class; the project (Ex. 13) is the personal work expected alongside.

How to study this module

• Redo the review section if the P1 vision course is no longer fresh: detection and segmentation reuse convolution and boxes without revisiting them.
• For every task, start from the metric before the model: understanding mAP or IoU first avoids 90 % of benchmark misinterpretations.
• Reproduce a small end-to-end experiment on a public dataset: training, evaluation, then export — that is the heart of the module.
• Concretely try ONNX export and quantization on a small model: the deployment chapter is understood with hands on the keyboard.
• Prepare questions for the industrial case studies: that is where constraints, trade-offs and real careers play out.
• Redo the exercises of each section before moving on to the next; they calibrate your expectations of what a deployed system can actually do.

0. Review: convolution, CNNs, boxes

Every task of this module — detecting, segmenting, tracking — outputs regions of an image and must compare them with annotated regions. This chapter recalls the deep-learning background assumed and the one tool used everywhere afterwards — the bounding box and its overlap measure, the IoU, stated in the P1 vision course — then proves the properties of the IoU that the metrics of chapter 2 rely on.

The P1 course “Image Analysis and Computer Vision” (CSC_51073_EP) laid the foundations: filtering, local features, projective geometry, reconstruction. The present module takes the systems and applications orientation: we build the complete pipeline of a vision product — from annotated data to a model deployed on constrained hardware — and study the building blocks industry actually uses (detectors, segmenters, trackers, 3D sensors, inference runtimes).

1. Object detection: the two paradigms

Concrete scene: an embedded detector misses small objects because it looks at a single scale. This chapter defines the detection task — finding an unknown number of objects and their boxes — and the two families of networks that solve it.

1.1. The two-stage family: propose then refine

Two-stage detectors separate the generation of candidates from their classification:

• R-CNN (2014): ~2000 candidate regions extracted by cluster-based segmentation (selective search), each crop goes through a CNN; slow (tens of seconds per image), but the proof of concept of the CNN in detection;
• Fast R-CNN (2015): the whole image passes through the CNN once, candidates are projected into the feature space (RoI pooling); only a light classifier is applied per candidate;
• Faster R-CNN (2015): candidates are produced by an integrated, end-to-end learned Region Proposal Network (RPN) — the canonical scheme;
• FPN (2017): feature pyramid; small objects are detected on the high-resolution maps, large ones on the low-resolution maps.

1.2. The single-stage family: everything in one pass

Single-stage detectors directly predict boxes and classes over a dense grid, in a single pass:

• YOLO (2016): the image is cut into a 7 \times 7 grid; each cell predicts 2 boxes + class probabilities; remarkable speed (45 images/s at the time), at the cost of coarser localization;
• SSD (2016): multi-scale predictions via feature maps of decreasing resolutions, anchors per cell;
• RetinaNet (2017): introduced the focal loss to correct the foreground/background imbalance (hundreds of thousands of easy anchors against a few objects), reconciling single-stage and two-stage accuracy.

The effect of the factor (1 - p_t)^{\gamma} is to divide the contribution of easy examples: a background anchor classified at p_t = 0.9 has its loss reduced by a factor 0.1^2 = 0.01; the rare hard examples (small p_t) keep an almost intact gradient.

For a positive anchor (y = 1, so p_t = p and \partial p_t / \partial z = p_t(1 - p_t)), the gradient

It reduces to the cross-entropy gradient -(1 - p_t) for \gamma = 0, \alpha = 1: the extra factor (1-p_t)^\gamma makes it vanish on well-classified anchors, so the magnitude concentrates on the hard ones.

1.3. Non-maximum suppression (NMS)

Dense detectors emit many redundant boxes around the same object. NMS, stated in Image Analysis and Computer Vision, Definition 9.2, keeps the highest-scoring box and removes the ones that overlap it too much; this section adds its vectorized code, its cost and its limits:

1. sort the N detections by decreasing score;
2. take the first box b, add it to the kept ones;
3. remove every box b' such that \mathrm{IoU}(b, b') > \tau (typically \tau = 0.5);
4. resume at step 2 with the remaining boxes, until exhaustion.

import numpy as np

def nms(boxes, scores, tau=0.5):
    """boxes: (N,4) [x1,y1,x2,y2]; scores: (N,); returns the kept indices."""
    x1, y1, x2, y2 = boxes[:, 0], boxes[:, 1], boxes[:, 2], boxes[:, 3]
    areas = np.maximum(0, x2 - x1) * np.maximum(0, y2 - y1)
    order = scores.argsort()[::-1]          # sort by decreasing score
    keep = []
    while order.size > 0:
        i = order[0]
        keep.append(i)
        xx1 = np.maximum(x1[i], x1[order[1:]])
        yy1 = np.maximum(y1[i], y1[order[1:]])
        xx2 = np.minimum(x2[i], x2[order[1:]])
        yy2 = np.minimum(y2[i], y2[order[1:]])
        inter = np.maximum(0, xx2 - xx1) * np.maximum(0, yy2 - yy1)
        iou = inter / (areas[i] + areas[order[1:]] - inter)
        order = order[1:][iou <= tau]       # remove the overlapping ones
    return np.array(keep)

2. Evaluation (mAP) and modern detectors

Two detectors can only be compared with a single number that accounts for misses, false alarms, localization quality and score ranking at once. This chapter recalls that number, the mAP, from the P1 vision course, works it by hand on a new list and adds what the COCO protocol specifies; it then reviews the detectors that improved it since 2017 — without anchors, without NMS, and the industrial YOLO line — and ends with the session-4 lab.

2.1. Precision, recall, AP and the COCO protocol

The example below repeats the P1 computation on a longer list, then the note gives what the COCO protocol adds.

2.2. Without anchors: anchor-free detectors

Anchors impose an arbitrary grid of hypotheses. Anchor-free detectors directly predict, per pixel of the feature map: (i) a probability of being an object center (centerness / center heatmap, cf. FCOS and CenterNet) and (ii) regressions of the distances to the four edges.

The decoding becomes: b = (x_c - l, \; y_c - t, \; x_c + r, \; y_c + b'), where (x_c, y_c) is the position of the central pixel and l, t, r, b' the regressed distances — all of it without any reference box.

Benefits: fewer hyperparameters (scales/ratios), no anchor/object imbalance, better on atypical shapes; cost: an NMS is still needed, and the center-to-ground-truth assignment requires zone rules (assignment by feature zone).

2.3. Detection as set prediction: DETR

2.4. The industrial YOLO ecosystem

The important point for the engineer: these versions are less scientific revolutions than training and tooling recipes (mosaic augmentation, dynamic assignment loss, multi-format export). Knowing how to read a detector’s changelog — which modules change, which data, which licenses (GPL, now AGPL-3.0, for the Ultralytics line, Apache-2.0 for others) — is part of the module’s skills.

2.5. Lab sheet: from the P1 detector to the COCO protocol (session 4)

About two hours, on a laptop CPU or a free GPU tier, in PyTorch and torchvision. It continues the detector lab of Image Analysis and Computer Vision, which fine-tuned a Faster R-CNN on Penn-Fudan and measured its AP@0.5 and mAP@[0.5:0.95], and it hands its detections to the tracking session (chapter 5).

1. Models. Two torchvision detectors pretrained on COCO: the two-stage fasterrcnn_resnet50_fpn (section 1.1) and the one-stage retinanet_resnet50_fpn (section 1.2) — or, for the class person, your fine-tuned detector of the P1 lab.
2. Data. 500 images of COCO val2017 that contain people, with their official annotations.
3. COCO protocol. Run COCOeval of pycocotools on both models for the class person: AP, AP@0.5, AP@0.75 and the small, medium and large breakdown of the COCO note in section 2.1. Check that small people are the weak spot, and say why with Example 0.3.
4. Error analysis. Sort 50 errors by hand, or all of them with the TIDE toolbox (Bolya et al., ECCV 2020): localization errors (IoU between 0.1 and 0.5 with a ground truth), duplicates (section 1.3), background detections and missed people. TIDE also gives the AP that fixing each type would gain.
5. Ablation. NMS threshold 0.5 against 0.7, score threshold 0.05 against 0.3: effect on the AP and on the number of boxes per image.
6. Hand-off to tracking. Run the better model on the training sequence MOT17-04 and save its boxes in the MOTChallenge text format (frame, identity −1, left, top, width, height, score): session 7 tracks them with SORT.
7. Report, one page: the two COCO tables, the error breakdown, and the latency per image of both models on your machine.

3. Segmentation: semantic, instance, panoptic, SAM

A box is a coarse answer when the application needs the exact outline of an object — a tumor, a road, a defect. Segmentation assigns a label to every pixel; this chapter distinguishes its three variants, the architectures and metrics of each, and the promptable foundation model SAM.

3.1. Three granularities

3.2. Semantic segmentation: from FCN to U-Net and DeepLab

• FCN (2015): fully convolutional classification — the fully-connected layers become 1×1 convolutions, and upsampling (transposed) layers restore the resolution; dense output, end-to-end training;
• U-Net (2015): symmetric encoder-decoder with skip connections that reinject the encoder’s high-resolution details into the decoder; designed for medical imaging with few annotated images, and still the domain’s reference;
• DeepLab (v1→v3+): dilated (atrous) convolutions: a 3 \times 3 kernel with dilation d sees a neighborhood of size 2d+1, enlarging the receptive field without losing resolution; the ASPP module (Atrous Spatial Pyramid Pooling) combines several dilations in parallel to capture several scales; v3+ adds a light decoder.

3.3. Instance segmentation: Mask R-CNN

Mask R-CNN (2017) extends Faster R-CNN: in parallel with the classification and box-regression heads, each region of interest receives a segmentation head that predicts a 28 \times 28 binary mask (per class). The decisive detail is RoI Align: the original RoI pooling quantized the cell coordinates (rounding), shifting the features by a few tenths of a pixel — negligible for classification, disastrous for aligning a mask.

RoI Align samples by bilinear interpolation at the real coordinates, without rounding. The authors report about 3 mask AP points gained with a ResNet-50-C4 backbone (26.9 → 30.2) and about 7 points with coarser stride-32 features — a relative gain of 10 % to 50 %, larger under stricter IoU thresholds — from a mere numerical-precision fix: a fine engineering lesson.

3.4. Panoptic: a single metric

3.5. SAM: promptable segmentation

The Segment Anything Model (Meta, 2023) changes paradigm: instead of training a model per task, SAM learns a general task — produce a valid mask for any prompt (point, box, coarse mask, text to come):

• architecture: a heavy image encoder (a vision transformer, Deep Learning, section 6.6; in depth in Advanced Deep Learning, ch. 4) computed once per image; a light prompt encoder; a light mask decoder, re-runnable in real time (~50 ms) for every prompt — the expensive-once / light-many-times separation is the “interactive” deployment pattern;
• data engine: an assisted-annotation loop (model suggests, human corrects) at first, then semi-automatic then automatic annotation — culminating in SA-1B: 11 million licensed images, 1.1 billion masks, 99 % automatic;
• usage: zero-shot mask extraction, serving as a base for fine-tuning (medical, satellite), or as an annotation assistant — SAM closes the loop there: the model manufactures its own data.

import numpy as np
from segment_anything import sam_model_registry, SamPredictor

sam = sam_model_registry["vit_b"](checkpoint="sam_vit_b_01ec64.pth")
predictor = SamPredictor(sam)
predictor.set_image(image)                       # H x W x 3 RGB uint8: heavy encoder, once
masks, scores, _ = predictor.predict(
    point_coords=np.array([[420, 310]]),         # one click (x, y)
    point_labels=np.array([1]),                  # 1 = foreground, 0 = background
    multimask_output=True)                       # three nested candidates
best = masks[np.argmax(scores)]                  # highest predicted IoU
masks, scores, _ = predictor.predict(            # a box removes the ambiguity
    box=np.array([380, 280, 470, 340]), multimask_output=False)

4. Data, benchmarks and annotation

The quality of a deployed vision system is bounded by its data long before its architecture. This chapter reviews the reference datasets, what annotation costs and how to reduce that cost, and the three evaluation pitfalls that make a model look better than it is.

4.1. The reference datasets

4.2. The cost of annotation

A COCO box costs from a few seconds to a minute; a polygon mask, much more; a 3D LiDAR box at 25 Hz over 20 objects, several minutes with dedicated tooling. Three industrial levers:

• assisted annotation: a pretrained model proposes, the human corrects — the SA-1B loop of chapter 3 is its large-scale version; a complementary lever is active learning, which sends to annotation only the examples on which the model is uncertain;
• synthetic data: procedural 3D rendering with domain randomization (random textures, lighting to force invariance); the P2 elective Image Synthesis: Theory and Practice (one of the Choose-1 options) covers rendering; sim2real remains the hard part;
• augmentation: geometric (cropping, rotation, mosaic) and photometric (color, blur, noise); caution: geometric transformations must transform the boxes/masks and the metadata (rotation → bounding boxes of the rotated masks).

4.3. Evaluation pitfalls

5. Multi-object tracking

A detector run frame by frame does not know that the pedestrian of frame 12 is the one of frame 11. Tracking adds this identity across time; this chapter presents the association methods that dominate in practice and the metrics that evaluate them.

5.1. SORT: Kalman + Hungarian matching

SORT (2016) is the minimal tracker:

1. each track carries a constant-velocity Kalman filter with state [u, v, s, r, \dot u, \dot v, \dot s]: box center (u, v), area s and aspect ratio r, the ratio being assumed constant; the detection observes [u, v, s, r], and the Kalman filter is recalled below;
2. prediction: the boxes are extrapolated to the current frame;
3. association: solve the bipartite coupling between predictions and detections with cost 1 - \mathrm{IoU} (the Hungarian algorithm of Refresher in Computer Science, section 7.7); pairs with an IoU below a minimum are rejected;
4. the formed pairs update the filters (correction); tracks left unmatched for T_{\text{max}} consecutive frames are destroyed; orphan detections create new tracks.

5.2. DeepSORT: adding appearance

In a crowd, two pedestrians who cross paths swap their boxes: IoU alone re-associates poorly. DeepSORT (2017) associates in two stages:

1. matching cascade, which gives priority to the tracks seen most recently. The cost combines the Mahalanobis distance between a detection and the Kalman prediction of the track, measured in the covariance of that prediction, and the cosine distance between appearance embeddings — a small ReID (re-identification) network learns a descriptor e_i \in \mathbb{R}^{128} invariant to pose/scale. Both distances also act as gates: a pair beyond either threshold is forbidden;
2. IoU matching (as in SORT) on what is left: the unconfirmed tracks and those lost for a single frame.

The “appearance memory” (a gallery of the last descriptors per track) resists long occlusions.

5.3. ByteTrack: don’t throw away low-score detections

ByteTrack (2022) starts from a simple observation: during a partial occlusion, the detector often emits a correct but low-confidence detection — which classical pipelines discard at the threshold. The algorithm sketch:

1. split the detections into high score (\geq \tau_{\mathrm{high}}, e.g. 0.5) and low score (\tau_{\mathrm{low}} \leq s < \tau_{\mathrm{high}});
2. first matching: active tracks vs high-score detections, Hungarian coupling on the IoU distance — or on an appearance distance when a ReID model is added;
3. second matching: tracks still orphaned vs low-score detections, coupling on IoU alone — the low-confidence boxes “carry” the track through the occlusion;
4. tracks still orphaned are put on hold then destroyed; orphan high-score detections create tracks (never the low-score ones).

Result: notable MOTA/IDF1 gains without any new detector — association is a lever as profitable as perception.

5.4. Measuring a tracker

6. Applied 3D vision: stereo and LiDAR

A robot or a vehicle needs to know how far objects are, which a single image does not tell. This chapter shows how depth is recovered from two cameras (stereo) or measured directly (LiDAR), how networks process the resulting point clouds, and how sensors are fused.

6.1. The pinhole model (reminder)

6.2. Stereo: disparity as a depth measurement

The depth uncertainty grows quadratically with distance — stereo is precise up close, coarse far away; increasing B or f improves range at the price of the visible field and harder matchings.

The classical stereo pipeline:

1. rectification;
2. cost volume — for every left pixel and every candidate disparity d, a matching cost (census, SSD of descriptors) — this is an H \times W \times D_{\max} volume;
3. aggregation and regularization: SGM (Hirschmüller, 2008) aggregates costs along several directions (per-path dynamic programming) with discontinuity penalties — the industrial standard (baked into GPUs!);
4. disparity map.

Deep methods (PSM-Net, Chang and Chen 2018, and successors) learn the cost volume: the learned descriptors of the left image and of the shifted right image are stacked for every candidate disparity into a 4D volume (height, width, disparity, features), regularized by 3D convolutions; a soft argmin over the disparities then gives a sub-pixel estimate.

6.3. LiDAR: point clouds

LiDAR emits pulses and measures time of flight: the output is a 3D point cloud (x, y, z, reflectivity), ~100k–2M points/s, accurate in distance (~2 cm), sparse and irregular — unstructured: grid convolutions do not apply directly. Families of methods:

• PointNet (2017): per point, a shared MLP; aggregation by global max; then an MLP. Permutation invariance comes from the max: the global descriptor does not depend on the order of the points — cf. the proposition below;
• Voxels / pillars: discretize space (VoxelNet) or only the ground plane (PointPillars: vertical columns), aggregate the point features there, then run a 2D/3D CNN on the resulting grid;
• Center detection: CenterPoint projects everything into bird’s eye view (BEV) and detects objects by their center (heatmap + size/rotation regression) — the anchor-free formulation of chapter 2, transposed to a 2D “top-down view”.

6.4. Multi-sensor fusion in bird’s eye view

Cameras provide semantics (classes, traffic lights, signs) but only indirect depth; LiDAR provides geometry but poor semantics. Modern architectures fuse in BEV: every sensor is brought into a common top-down frame where detection, tracking and trajectory prediction combine.

BEVFormer (2022): image features are “queried” in BEV space by spatial cross-attention (every BEV cell queries the pixels along its view pillar), with temporal attention toward the previous BEV map — tracking becomes implicit. The following table summarizes the sensor trade-offs:

7. Industrial case studies

The previous chapters provide building blocks; a product combines them under constraints of cost, cadence and regulation. This chapter walks through four application domains and, for each, the constraint that changes the engineering choices.

7.1. Industrial inspection (quality control)

Inspecting parts (scratches, cracks, contamination) on a production line: controlled lighting (strobe, dark background), imposed cadence (decisions per second), and — the major specificity — very few defect examples.

Hence the move to unsupervised methods: learn normality (autoencoders, feature distances; cf. Deep Learning, ch. 9 for autoencoders) and raise an alert on deviation; the MVTec AD dataset is the reference benchmark (15 categories: screws, textiles, wood, bottles…, with pixel-level defect masks).

Metrics: image-level detection (AUROC) and defect segmentation (pixel AUROC, PRO). Constraints: an error is asymmetrically costly (false rejection = yield loss; missed defect = product recall); the threshold is tuned on business cost, not on 0.5.

7.2. Autonomous driving

The typical perception pipeline of a SAE level 2+ to 4 vehicle:

1. multi-camera detection/segmentation (chapters 1–3);
2. camera/LiDAR/radar fusion in BEV (chapter 6);
3. tracking and prediction of agents (chapter 5 + trajectory models);
4. planning (out of scope — cf. Reinforcement Learning, ch. 1 for the MDP formalization);
5. system monitoring: redundancy, diagnostics, degraded mode.

Engineering lessons: corner cases dominate the cost (roadworks, flashing lights, overturned trailers); closed benchmarks (nuScenes) are saturated and correlate poorly with real kilometers; safety imposes latency guarantees (chapter 8) and traceability (version of the model, the dataset, the threshold).

7.3. Medical imaging

Organ/lesion segmentation (U-Net is still the backbone, chapter 3), cell counting, fracture detection. Specificities: rare and costly data (clinicians), DICOM formats (metadata, windowing), strict clinical validation, regulation (medical CE marking); the metric is per-structure Dice and inter-observer consistency is the reachable ceiling — a Dice above human agreement is suspicious; the models are assistance tools (CAD — computer-aided diagnosis), not decision makers.

7.4. Other domains (summary)

• retail: planograms (products on shelves), customer flows (anonymous counting);
• agriculture: weed detection for targeted spraying (drone, embedded);
• sports: player/ball tracking (chapter 5 at scale), tactical statistics;
• satellite: building/road detection, deforestation masks — with domain randomization and multispectral imaging as a bonus.

8. Deployment: ONNX, quantization, edge

A model trained on a workstation GPU must run within a few milliseconds on an embedded board with little memory. This chapter covers the steps that get it there — export, quantization, compression — and how to check a latency budget.

8.1. The deployment pipeline

From notebook to product, the canonical steps:

1. export: the trained model (PyTorch) is exported as a frozen graph — ONNX (Open Neural Network Exchange), a common format accepted by most runtimes;
2. graph optimization: operator fusion (conv + batchnorm → single conv; conv + activation), dead-layer elimination, kernel selection;
3. targeted compilation: TensorRT (NVIDIA), Core ML (Apple), TF Lite / LiteRT (mobile), OpenVINO (Intel) — these generate hardware-specialized engines;
4. calibration/quantization: move to INT8 (below);
5. measurement: P50/P99 latency, throughput, memory, energy — on the target hardware, never on the development workstation alone.

import torch

model = DetectorWrapper()          # PyTorch, eval mode
model.eval()
dummy = torch.randn(1, 3, 640, 640)

torch.onnx.export(
    model, dummy, "detector.onnx",
    input_names=["image"], output_names=["boxes", "scores", "labels"],
    dynamic_axes={"image": {0: "batch"},                # variable batch
                  "boxes": {0: "batch", 1: "num_boxes"}},  # variable N
    opset_version=17)              # recent opset for special ops

# runtime side: onnxruntime (CPU/GPU), TensorRT, Core ML...
import onnxruntime as ort
sess = ort.InferenceSession("detector.onnx", providers=["CPUExecutionProvider"])
boxes, scores, labels = sess.run(None, {"image": dummy.numpy()})

8.2. INT8 quantization

8.3. Compressing: pruning and distillation

• pruning: zero out the lowest-magnitude weights (unstructured pruning — no gain without a sparse library) or whole channels (structured — actually accelerates); follow the accuracy/size curve;
• knowledge distillation (Hinton et al., 2015): a small student imitates the output distribution of a large teacher — the loss combines the ground truth and the KL divergence with the teacher’s logits “softened” by a temperature T, with \lambda \in [0, 1] the weight of the teacher term: \mathcal{L}_{\mathrm{KD}} = (1 - \lambda)\, \mathrm{CE}(y, z_s) + \lambda\, T^2\, \mathrm{KL}\!\big(\mathrm{softmax}(z_t / T)\,\|\,\mathrm{softmax}(z_s / T)\big) — the convention of the site’s notation page, covered in detail in Advanced Deep Learning, Definition 3.2.

8.4. Latency budgets and profiling

8.5. Operations, drift and privacy

• monitoring: log score distributions, detection rates, “world” alerts (dirty camera, night); detect domain drift and plan retraining (the data loop of chapter 4);
• privacy: process on-device as much as possible (no image leaves the sensor); if a stream is sent: blurring/anonymization of faces and plates (…detected by… the models of chapter 3!), minimal retention;
• versioning: model + code + data + thresholds = a reproducible artifact — the methodology of Advanced Deep Learning, ch. 1 applies mutatis mutandis.

9. Ethics, robustness and limits

A deployed vision system can be fooled on purpose, can work worse for some groups of people, and can be used for surveillance. This chapter reviews these three risks and the limits that no current model overcomes.

9.1. Adversarial examples

Small imperceptible perturbations change the prediction. The simplest attack — FGSM — moves the input one step in the direction of the loss gradient: x_{\mathrm{adv}} = x + \varepsilon\, \mathrm{sign}\!\left(\nabla_x \mathcal{L}(\theta, x, y)\right), with \varepsilon on the order of 2–8/255.

Physical attacks exist too: stickers on a stop sign make a classifier read a speed limit (Eykholt et al., 2018), and a printed patch held by a person hides that person from a detector (Thys et al., 2019). Defenses range from adversarial training (min-max over perturbations) to bound-based certification.

FGSM, PGD and adversarial training are treated in Advanced Deep Learning, section 4.5 and in Deep Learning (P2 variant), section 7.1, certification and attacks on deployed systems in Security and Robustness (M2). The engineer mainly remembers: a model embedded in a product can be targeted, and robustness is tested (attack toolboxes) before deployment.

9.2. Bias and fairness of vision systems

• representation bias: mostly North American/European datasets → performance drops in other contexts (streets, clothing, objects). The “Gender Shades” audit (Buolamwini and Gebru, 2018) found that commercial gender-classification systems misclassified darker-skinned women far more often than lighter-skinned men (error rates up to about 35 %, against under 1 %). In practice: break the evaluation down by subgroup, as in Machine Learning, section 4.8 and Example 9.1 below;
• surveillance and biometrics: mass facial recognition is restricted/banned in several jurisdictions; the European AI regulation (AI Act, 2024) classifies remote biometric identification among high-risk or prohibited uses; anonymize before storage;
• consent: scraped data (LAION etc.) and image rights; prefer clearly licensed datasets and document provenance (data cards).

9.3. Fundamental limits to know

Exercises

1. ★ IoU and thresholds. Compute the IoU between g = (10, 10, 40, 60) and p = (20, 5, 55, 50); determine the maximum offset \delta in x of a prediction identical to g to stay above \tau = 0.75.
2. ★ NMS by hand. Five detections of the same class, scores 0.9; 0.85; 0.8; 0.75; 0.7, box 2 overlapping box 1 (IoU 0.6), box 3 overlapping box 1 (IoU 0.3) and box 5 overlapping box 3 (IoU 0.8). Give the boxes kept at \tau = 0.5 then at \tau = 0.7 with linear Soft-NMS (s \leftarrow s(1 - \mathrm{IoU})); comment on the “two objects stuck together” case.
3. ★ AP. With 4 ground truths and the sorted detections (TP, TP, FP, FP, TP, TP): draw the precision-recall curve, compute the all-points interpolated AP, then verify that moving the two FPs to the head of the list (FP, FP, TP, …) makes AP drop — why does the order of the scores matter?
4. ★ Focal loss. Show that for \gamma = 2, an easy anchor (p_t = 0.95) receives a modulating weight (1-p_t)^\gamma that is 196 times smaller than that of a hard anchor (p_t = 0.3); compute the ratio for plain cross-entropy and conclude.
5. ★ Hungarian coupling. Three tracks T_1, T_2, T_3, three detections D_1, D_2, D_3, costs 1-\mathrm{IoU}: T_1\!: (0.1,\ 0.8,\ 0.9), T_2\!: (0.9,\ 0.2,\ 0.7), T_3\!: (0.8,\ 0.9,\ 0.3). Solve the minimum-cost assignment by hand and compare with the greedy assignment — in which case do they differ?
6. ★ Stereo depth. Camera f = 700 px, baseline B = 0.5 m; compute the disparity of an object at 10 m, then the depth uncertainty if disparity is measured to ±0.25 px; at what distance does the uncertainty exceed 1 m?
7. ★ PQ. One image: 4 ground-truth segments, 3 predicted with matched IoUs 0.9; 0.8; 0.6 plus one FP; one FN. Compute SQ, RQ then PQ; redo with IoU 0.7 everywhere and comment on the decomposition.
8. ★ Quantization. A weight tensor has min −0.82 and max 0.55; give the scale and zero point (asymmetric 8-bit min-max scheme), quantize −0.1 and 0.3, dequantize, compute the relative error — why can a 99.9th percentile help?
9. ★★ PointNet. Replace the max by an average in proposition 6.4: does the invariance hold? Build two clouds with the same average but very different contents and conclude on the “selectivity” of the max.
10. ★★ Real-time budget. An application must detect at 25 fps on hardware where FP32 inference takes 58 ms and INT8 21 ms; preprocessing 5 ms, post-processing 4 ms. Does a single inference per frame fit? Propose two strategies (skip-frame + tracking; resolution reduction ÷1.4 with cost in s^2) and quantify their trade-offs.
11. ★★ Data choice. For a vehicle-counting system on a European road (day/night, rain), propose: a starting training set, a splitting protocol, an augmentation plan, a subgroup evaluation method, and two monitoring metrics in production.
12. ★★ MOTA versus IDF1. Two pedestrians A and B are visible in all 6 frames of a clip, so there are 12 ground-truth boxes. Compute MOTA and IDF1 (Definition 5.2) for two trackers, then say which one you would deploy to count the people entering a shop, and why.
    • Tracker 1 detects every box but swaps the two identities when the pedestrians cross at frame 4: its track 1 follows A then B, its track 2 follows B then A.
    • Tracker 2 follows A with one track throughout, follows B with track 2 on frames 1–3, misses B on frames 4–5 (occlusion) and restarts it as a new track 3 on frame 6.
13. ★★★ Project. Complete pipeline on a free set (a COCO subset or public drone images): fine-tuning of a modern YOLO, ONNX export, INT8 quantization, latency/mAP measurement before/after, then a one-page report in “industrial datasheet” format (error costs included, cf. example 7.1).

Projects and assessment

No official assessment is published for this module: it has no SynapseS sheet of its own, and the MSc&T brochure that announces it states no modality (fiche: “not displayed”). The subjects below are proposals of these notes, not official rules. If the module opens with a project-based assessment, Ex. 13 or one solo or group subject stands for the project; the written exam is only a proposal, for instance for a resit.

The projects exercise mainly O7 (end-to-end pipelines measured on the target hardware and reported as a datasheet), O8 (robustness, subgroup and provenance audits) and O6 (the perception side of multimodal systems), with O4 in the tracking-graph subject and O1 in training and quantization. Their detection and segmentation work can serve as the perception front end of the end-of-year project Y1.5 (scene-graph question answering), and their deployment measurements (ONNX, INT8, latency percentiles) carry over to the latency budget of Y2.3.

Solo projects

1. [Solo · ≈ 20 h] Edge detector datasheet under domain shift. How much accuracy does INT8 cost on night and rain images, and when is quantization-aware training worth it? The subject extends Ex. 13.
   • Work: fine-tune a small detector (YOLOv8n, or torchvision’s SSDlite320-MobileNetV3) on three classes; export it to ONNX; compare post-training static quantization in ONNX Runtime with quantization-aware training (Definition 8.1, Note PTQ vs QAT); set the decision threshold on costs as in Example 7.1.
   • Data: COCO 2017 for pretraining; the BDD100K driving dataset (Yu et al., CVPR 2020; free registration), whose images are tagged by time of day and weather, with day images for fine-tuning and night and rainy images as the shifted test splits.
   • Success criterion: mAP@[0.5:0.95] with its small, medium and large breakdown for FP32, PTQ and QAT on each split, and P50/P99 latency on a laptop CPU or a Jetson measured as in Example 8.2; the INT8 loss and the night gap explained with chapters 2 and 4.
   • Deliverables: code, a one-page industrial datasheet plus a 4-page report, a 10-minute talk; chapters 1, 2, 4, 7 and 8. The export, versioning and monitoring steps lead to ML Ops and LLM Engineering Principles (M2).
2. [Solo · ≈ 20 h] Paper study: stereo depth against LiDAR and the quadratic error law (Hirschmüller, TPAMI 2008; Chang and Chen, CVPR 2018). Does the depth error of semi-global matching and of PSM-Net grow like Z^2/(fB), as the key result of chapter 6 predicts, and which disparity error does each method imply?
   • Work: disparities from OpenCV’s semi-global matcher (StereoSGBM) and from the authors’ PSM-Net weights — the KITTI 2015 ones, fine-tuned on these training pairs and therefore optimistic here, and the Scene Flow ones as the cross-domain contrast; depth by Z = fB/d with the calibration files; median depth error per 5 m distance bin; a straight-line fit of the log error against the log distance.
   • Data: the 200 training pairs of KITTI Stereo 2015 (Menze and Geiger, CVPR 2015), whose ground-truth disparities come from accumulated LiDAR scans, with 3D models fitted to the moving cars; as a stretch, point crops of the KITTI 3D object benchmark to test the max-versus-mean claim of Ex. 9 with a small PointNet.
   • Success criterion: the fitted slope with a confidence interval, compared with 2; the implied disparity error of each method; their D1-all error rates (disparity wrong by more than 3 px and 5 %), with PSM-Net’s set beside the 2.32 % its paper reports on the KITTI 2015 test set — SGM’s 2008 paper predates KITTI.
   • Deliverables: code, a 4–6 page report, a 10-minute talk; chapter 6, with Example 6.3 as the starting point.

Group projects

1. [Group of 3 · ≈ 30 h each] From tracks to interaction graphs. Does a better association (a higher IDF1) give better short-term trajectory forecasts when the tracks feed a graph network? The subject links the Kalman–Hungarian pipeline of chapter 5 to the message passing of Graph Machine and Deep Learning, ch. 5.
   • Work: SORT written from section 5.1, against the public DeepSORT and ByteTrack code, all fed with the same detections; per frame, a graph whose nodes are the tracked people (Kalman position and velocity) and whose edges join people closer than a radius; a message-passing GNN that predicts the next 12 positions from the last 8, against the constant-velocity Kalman forecast.
   • Data: the MOT17 training sequences (Milan et al., “MOT16: A Benchmark for Multi-Object Tracking”, 2016) with their public detections; the ETH and UCY pedestrian sequences (Pellegrini et al., ICCV 2009; Lerner et al., Eurographics 2007) for forecasting in world coordinates.
   • Split: the trackers and their evaluation; the graph construction and the GNN; the forecasting study and the propagation of tracking errors.
   • Success criterion: MOTA, IDF1 and HOTA of the three trackers computed with the official TrackEval code, and their differences explained by the association rules of chapter 5; average and final displacement errors (ADE, FDE) of the GNN against constant velocity on ETH and UCY; the forecast error on tracker output plotted against IDF1.
   • Deliverables: code, an 8–10 page joint report, a 20-minute talk; chapters 1, 2 and 5.
2. [Group of 3 · ≈ 30 h each] SAM in the loop: an annotation engine for industrial defects. How much annotation time does a promptable segmenter save on defect masks, and do the saved seconds cost downstream accuracy?
   • Work: an annotation tool in which point or box prompts give SAM masks (Example 3.3) that the annotator corrects with a brush, every action timed; an active-learning loop that sends the most uncertain images next (section 4.2); a U-Net trained once on assisted masks and once on manual masks; PatchCore (Roth et al., CVPR 2022) as the label-free baseline of section 7.1.
   • Data: MVTec AD (Bergmann et al., CVPR 2019), three to five categories with their pixel-level masks; its training set holds no defect, so the defective test images are split in two halves.
   • Split: the tool and a timing study with at least two annotators; active learning and U-Net training; the anomaly baseline and the evaluation.
   • Success criterion: seconds and clicks per mask, and mask IoU and Dice against the ground truth, for manual and assisted annotation of the same 100 defects; pixel AUROC and PRO of both U-Nets and of PatchCore; the time saved weighed against the accuracy lost.
   • Deliverables: code, a joint report, a talk; chapters 3, 4 and 7.
3. [Group of 3 · ≈ 30 h each] Paper study: robustness and fairness audit of a deployed detector (Thys, Van Ranst and Goedemé, CVPR Workshops 2019). Can a printed patch hide a person from your detector, does the detector work equally well for every group, and what do the fixes cost after INT8?
   • Work: FGSM and PGD on the inputs (section 9.1); the person-hiding patch of the paper, optimized against your detector and compared with a random patch; per-group recall as in Example 9.1, by lighting, weather, scene type and box size; augmentation or adversarial training as mitigation; a model card (Mitchell et al., FAT* 2019).
   • Data: the INRIA Person dataset (Dalal and Triggs, CVPR 2005) used by the paper, available from public mirrors; BDD100K night and rain images; the detector of Ex. 13 or of solo subject 1.
   • Split: the attacks; the domain-shift and subgroup evaluation; the mitigation, the INT8 re-audit and the model card.
   • Success criterion: the fraction of people no longer detected at the operating threshold with the patch, with the random patch and with no patch; per-group recall with 95 % intervals; mAP, worst-group recall and latency before and after mitigation and INT8.
   • Deliverables: code, a joint report with the model card, a talk; chapters 2, 4, 8 and 9. It prepares Security and Robustness (M2) and applies the subgroup evaluation of Machine Learning, section 4.8.

Written exam (proposal)

Two hours, one handwritten A4 sheet allowed, a simple calculator allowed; four exercises.

1. Question (4 points). (a) Why does a one-stage detector over dense anchors need the focal loss, or another way to tame the background? (b) Why does DETR need neither anchors nor NMS? (c) What does COCO’s mAP@[0.5:0.95] reward that AP@0.5 does not, and why does COCO also report results by object size?
   • Expected: (a) about 10^5 anchors per image, almost all easy background: their summed cross-entropy dominates the loss and its gradient; the factor (1-p_t)^\gamma divides the loss of an anchor at p_t = 0.9 by 100 (Definition 1.4).
   • (b) DETR predicts a set of N queries matched one-to-one with the ground truths by the Hungarian algorithm: a duplicate is matched with “no object” and penalized, so the model learns not to emit duplicates (Definition 2.3).
   • (c) Averaging over ten IoU thresholds rewards precise localization; the size breakdown exposes small objects, where a few pixels of offset destroy the IoU (Example 0.3).
2. Question (5 points). Activations measured on a calibration set lie in [-1, 3]; they are quantized to signed 8 bits by the min-max affine scheme of Definition 8.1. (a) Give the scale s and the zero point z. (b) Quantize and dequantize 1 and 0.01; give the absolute and relative errors. (c) One calibration image produces an outlier activation of 30: give the new scale and the new error bound, and say what a 99.9th-percentile calibration changes.
   • Expected: (a) s = 4/255 \approx 0.01569, and z is set so that Q(-1) = -128: z = \mathrm{round}(-128 + 1/s) = \mathrm{round}(-64.25) = -64 , and indeed Q(3) = 191 - 64 = 127.
   • (b) For 1: 1/s = 63.75 rounds to 64, so Q = 0, dequantized to 64s \approx 1.0039; the absolute error is 0.0039, the relative error 0.4 %.
   • For 0.01: 0.01/s \approx 0.64 rounds to 1, so Q = -63, dequantized to s \approx 0.0157; the absolute error is 0.0057, the relative error 57 %. The bound s/2 is absolute, so small values suffer most.
   • (c) The scale becomes 31/255 \approx 0.1216 and the bound s/2 grows from 0.0078 to 0.061, almost 8 times, for every activation. Percentile calibration keeps s \approx 4/255 and clips the rare outlier to 3: one large error instead of a coarse grid for all.
3. Question (5 points). For a positive anchor (y = 1) with logit z and p = \sigma(z), prove the focal-loss gradient of chapter 1, \frac{\partial \mathrm{FL}}{\partial z} = -\alpha\,(1-p)^{\gamma}\big(1 - p - \gamma\, p \log p\big). Deduce the cross-entropy gradient for \gamma = 0, \alpha = 1. For \gamma = 2, compute the ratio of the gradient magnitudes at p = 0.3 and at p = 0.9, and compare it with the cross-entropy.
   • Expected: \sigma'(z) = p(1-p) and, from \mathrm{FL} = -\alpha(1-p)^\gamma \log p, \frac{\partial \mathrm{FL}}{\partial p} = -\alpha\Big(\frac{(1-p)^{\gamma}}{p} - \gamma\,(1-p)^{\gamma-1}\log p\Big).
   • Multiplying by p(1-p) gives the formula; at \gamma = 0, \alpha = 1 it reduces to -(1-p), the cross-entropy gradient.
   • With g(p) = (1-p)^2(1 - p - 2p\log p): g(0.3) \approx 0.697 and g(0.9) \approx 0.0029, a ratio of about 241, against 0.7/0.1 = 7 for the cross-entropy: the focal loss moves the gradient onto the hard anchors.
4. Question (6 points). For a predicted mask p and a ground truth g, write I = |p \cap g| and U = |p \cup g|, and let t = I/U be the IoU. (a) Prove that \mathrm{Dice} = 2t/(1+t), and that Dice is at least the IoU, with equality only when the IoU is 0 or 1. (b) A paper reports a Dice of 0.90: what IoU is that? (c) Two radiologists agree with each other at a Dice of 0.85 on the same lesions, and a model reaches 0.92 against one of them: what do you conclude (section 7.3)?
   • Expected: (a) |p| + |g| = I + U, so \mathrm{Dice} = 2I/(I + U) = 2t/(1+t), an increasing function of t (derivative 2/(1+t)^2 > 0): both metrics rank models identically. Moreover \mathrm{Dice} - t = \frac{t(1-t)}{1+t} \geq 0 , with equality only at t = 0 or t = 1.
   • (b) Inverting, t = D/(2 - D) = 0.9/1.1 \approx 0.818.
   • (c) The inter-observer agreement is the reachable ceiling: 0.92 above 0.85 suggests that the model fits one annotator’s habits, or a leak between training and test (section 4.3); evaluate against several annotators or their consensus.

Grading rubric (proposal)

References

• R. Szeliski, Computer Vision: Algorithms and Applications, 2nd ed., Springer, 2022 — the reference textbook, free online.
• O. Ronneberger, P. Fischer, T. Brox, U-Net: Convolutional Networks for Biomedical Image Segmentation, MICCAI 2015.
• S. Ren, K. He, R. Girshick, J. Sun, Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks, NeurIPS 2015.
• J. Redmon, S. Divvala, R. Girshick, A. Farhadi, You Only Look Once: Unified, Real-Time Object Detection, CVPR 2016.
• W. Liu et al., SSD: Single Shot MultiBox Detector, ECCV 2016.
• T.-Y. Lin, P. Goyal, R. Girshick, K. He, P. Dollár, Focal Loss for Dense Object Detection (RetinaNet), ICCV 2017; T.-Y. Lin et al., Feature Pyramid Networks for Object Detection, CVPR 2017.
• K. He, G. Gkioxari, P. Dollár, R. Girshick, Mask R-CNN, ICCV 2017.
• N. Bodla et al., Soft-NMS — Improving Object Detection with One Line of Code, ICCV 2017.
• N. Carion et al., End-to-End Object Detection with Transformers (DETR), ECCV 2020; X. Zhu et al., Deformable DETR, ICLR 2021.
• Z. Tian, C. Shen, H. Chen, T. He, FCOS: Fully Convolutional One-Stage Object Detection, ICCV 2019.
• A. Kirillov et al., Panoptic Segmentation, CVPR 2019.
• A. Kirillov et al., Segment Anything (SAM), ICCV 2023.
• T.-Y. Lin et al., Microsoft COCO: Common Objects in Context, ECCV 2014 — mAP protocol.
• A. Bewley, Z. Ge, L. Ott, F. Ramos, B. Upcroft, Simple Online and Realtime Tracking (SORT), ICIP 2016; N. Wojke, A. Bewley, D. Paulus, Simple Online and Realtime Tracking with a Deep Association Metric (DeepSORT), ICIP 2017.
• Y. Zhang et al., ByteTrack: Multi-Object Tracking by Associating Every Detection Box, ECCV 2022.
• H. Hirschmüller, “Stereo Processing by Semiglobal Matching and Mutual Information”, IEEE TPAMI 30(2), 2008 (SGM); J. Chang, Y. Chen, Pyramid Stereo Matching Network (PSM-Net), CVPR 2018.
• C. R. Qi et al., PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, CVPR 2017; Y. Zhou, O. Tuzel, VoxelNet, CVPR 2018; A. Lang et al., PointPillars, CVPR 2019.
• T. Yin, X. Zhou, P. Krähenbühl, Center-based 3D Object Detection and Tracking (CenterPoint), CVPR 2021; Z. Li et al., BEVFormer, ECCV 2022.
• H. Caesar et al., nuScenes: A Multimodal Dataset for Autonomous Driving, CVPR 2020; A. Geiger, P. Lenz, R. Urtasun, “Are we ready for Autonomous Driving? The KITTI Vision Benchmark Suite”, CVPR 2012.
• P. Bergmann, M. Fauser, D. Sattlegger, C. Steger, “MVTec AD — A Comprehensive Real-World Dataset for Unsupervised Anomaly Detection”, CVPR 2019 — dataset page.
• B. Jacob et al., Quantization and Training of Neural Networks for Integer-Arithmetic-Only Inference, CVPR 2018.
• G. Hinton, O. Vinyals, J. Dean, Distilling the Knowledge in a Neural Network, 2015.
• I. Goodfellow, J. Shlens, C. Szegedy, Explaining and Harnessing Adversarial Examples (FGSM), ICLR 2015.
• D. Bolya, S. Foley, J. Hays, J. Hoffman, TIDE: A General Toolbox for Identifying Object Detection Errors, ECCV 2020.
• A. Milan, L. Leal-Taixé, I. Reid, S. Roth, K. Schindler, MOT16: A Benchmark for Multi-Object Tracking, 2016; J. Luiten et al., HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking, IJCV 2021.
• M. Menze, A. Geiger, “Object Scene Flow for Autonomous Vehicles”, CVPR 2015 (KITTI Stereo 2015); F. Yu et al., BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning, CVPR 2020.
• K. Roth et al., Towards Total Recall in Industrial Anomaly Detection (PatchCore), CVPR 2022.
• K. Eykholt et al., Robust Physical-World Attacks on Deep Learning Visual Classification, CVPR 2018; S. Thys, W. Van Ranst, T. Goedemé, Fooling Automated Surveillance Cameras: Adversarial Patches to Attack Person Detection, CVPR Workshops 2019.
• J. Buolamwini, T. Gebru, Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification, FAT* 2018; M. Mitchell et al., Model Cards for Model Reporting, FAT* 2019.
• Lab tools: torchvision, ONNX and ONNX Runtime, TensorRT, Roboflow/supervision — the annotation and metrics ecosystem.
• Instructor note: course announced in the MSc&T brochure (period 2), not in the 2026–27 LLGA lists; no instructor or official syllabus is published — the references above constitute the standard foundation of the field.