Deep Learning and Signal Processing: Introduction and Industrial Applications

Contents

• Introduction and overview
• 0. Review: Fourier bases, Gaussian noise and the chain rule
• 1. Signals, sampling and filtering
• 2. Time–frequency and time–scale representations
• 3. The machine-learning toolbox: classification, clustering, optimisation
• 4. Deep learning: basics and main architectures
• 5. Use case: signal detection for cognitive radio
• 6. Use case: radar emitter identification
• 7. Use case: speaker recognition and few-shot learning
• 8. Use case: reconstruction of altered audio signals
• 9. Use case: predictive maintenance
• 10. Synthesis: what makes deep learning deep, and research in industry
• 11. Research directions and open problems
• Exercises
• Projects and assessment
• References

Introduction and overview

Audience: a student with an M1 in mathematics, computer science or physics who has trained a classifier before and wants to know when a neural network beats — or loses to — decades of signal-processing know-how on radio, radar, speech and vibration data.

Empirical first: on 256 samples of a narrowband transmitter at −15 dB, the textbook energy detector reaches an AUC of only 0.64, while a periodogram peak search and a logistic regression on sorted spectra both reach 0.82 (Code 5.9). A bearing whose raw vibration looks Gaussian (kurtosis 3.3) shows, once band-passed and demodulated, a line 28 times the background at the fault frequency of 90 Hz (Code 9.6). The course explains both numbers.

The course asks what deep learning brings to signal processing, one industrial use case at a time: spectrum sensing for cognitive radio, radar emitter identification, speaker recognition, restoration of damaged audio and predictive maintenance. Each case starts from the physics, builds the classical chain, then revisits it with learning. It shares its Fourier and statistical tools with Machine Learning for Time Series and Deep Learning, and prepares Audio Signal Processing: Time–Frequency Analysis and Detection Theory and Industrial Applications.

What you will be able to do

• Model a sensor signal in complex baseband, sample it without aliasing and choose DFT and STFT parameters from a resolution budget.
• Design and analyse linear filters, including the matched filter and radar pulse compression.
• Compute and interpret spectrograms, scalograms, mel spectra and MFCCs, and explain the time–frequency trade-off behind them.
• Formulate detection as a Neyman–Pearson test, size an energy detector and compute its SNR wall.
• Train, evaluate and compare classical classifiers and clustering algorithms on signal features, with leakage-free splits.
• Build and train 1D and 2D convolutional, recurrent and attention-based networks on raw signals and spectrograms in PyTorch.
• Deinterleave radar pulse trains with clustering and TOA-difference histograms, and explain RF fingerprinting.
• Build a speaker-recognition chain from GMM-UBM to deep embeddings, and enrol new classes with few-shot prototypes.
• Restore degraded audio with Wiener gains, autoregressive interpolation and learned masks.
• Detect bearing faults with envelope analysis, condition indicators and anomaly detection with a controlled false-alarm rate.
• Prove a depth-separation result and argue, case by case, when a classical pipeline should be preferred to a deep one.

Official objectives and syllabus (2026–2027)

The course description on master-mva.com, repeated on the Université Paris-Saclay page, sets the objective of understanding what deep-learning approaches can bring to signal-processing problems through several industrial use cases, for instance radar emitter identification, signal detection for cognitive radio, speaker recognition, the reconstruction of altered audio signals and predictive maintenance.

For each use case, the description announces that “classical” processing chains are studied, discussed and implemented in labs, then revisited with a “learning” approach. Each case begins with its operational context and a model of the physics of the problem, so that students acquire a culture of the application domains.

The theory part introduces signal-processing tools (time–frequency and time–scale representations, filtering) and machine-learning tools (supervised and unsupervised approaches, the main families of classification and clustering algorithms, optimisation, gradient descent, the basics of deep learning and the main network architectures).

Through these back-and-forths between theory and practice, the course tries to answer the question “what makes the depth of deep learning?”. The master-mva.com page adds that this industry-oriented course is also an occasion to discuss how research is organised in companies. The organisation is 12 h of lectures and 18 h of labs, taught in French; the Université Paris-Saclay page presents the course as module 1 of the “Industrie et Ingénierie” (I&I) track.

This page’s extensions beyond the syllabus are the proofs of the sampling, uncertainty, matched-filter, Neyman–Pearson and gradient-descent results, the scattering transform as a deep network without learning (ch. 10.2), and a self-study chapter on research directions (ch. 11).

Where this course fits in the programme

Official prerequisites: data structures, Python programming, linear algebra, differential calculus, probability and statistics at L3 level, and the basics of machine learning (the last item appears on master-mva.com only).

• Preparation — the refreshers Probability and Statistics, Linear Algebra, Analysis and Optimization and Scientific Python, NumPy and PyTorch. The catalogue’s calendar marks the refresher courses as mandatory for the I&I track.
• Related courses of semester 1 — Machine Learning for Time Series, Deep Learning, Introduction to Statistical Learning, Convex Optimization and Applications in Machine Learning and Theoretical Foundations of Deep Learning; within the I&I track, The Machine Intelligence of Images and Statistical Learning for Scientific Imaging and Physical Measurements: Geosciences.
• Courses that build on it (semester 2) — Audio Signal Processing: Time–Frequency Analysis, Audio Signal Analysis, Indexing and Transformations, Algorithms for Speech and Natural Language Processing, Detection Theory and Industrial Applications and Remote Sensing Data.
• Same track, semester 2 — Learning Physical Phenomena: Metamodelling and Uncertainty Quantification and Algorithms for Network Optimization and Management; the projects lead naturally to the Research Internship.

The catalogue’s validation rules require two I&I modules in the I&I track, and let students outside the track validate one I&I module, which then counts as Modelling.

Course outline

• Review: Fourier bases, Gaussian noise and the chain rule — the M1 tools used everywhere else, with the chi-square law of noise energy.
• Signals, sampling and filtering — complex baseband, the sampling theorem, the DFT, LTI filters and the matched filter.
• Time–frequency and time–scale representations — STFT and spectrogram, the uncertainty principle, wavelets, filter banks, mel spectrum and MFCC.
• The machine-learning toolbox — supervised and unsupervised learning, classifier families, clustering, metrics and gradient descent.
• Deep learning: basics and main architectures — backpropagation, CNN, RNN, autoencoders, attention, training practice and a first answer on depth.
• Signal detection for cognitive radio — Neyman–Pearson, the energy detector and its SNR wall, cyclostationary and learned detectors.
• Radar emitter identification — pulse descriptor words, deinterleaving, clustering and emitter fingerprinting.
• Speaker recognition and few-shot learning — the source–filter model, GMM-UBM, x-vectors, prototypes for new speakers.
• Reconstruction of altered audio signals — Wiener gains, autoregressive gap interpolation, sparse inpainting and learned masks.
• Predictive maintenance — bearing physics, envelope analysis, condition indicators, anomaly detection and remaining useful life.
• Synthesis: what makes deep learning deep, and research in industry — depth, invariance and stability, hybrid models, the industrial research cycle.
• Research directions and open problems — self-supervised signal models, open-set recognition, domain shift and model-based learning (self-study).

Session plan

30 h in class = 10 sessions of 3 h.

The volume is the official one of the Université Paris-Saclay programme table, 12 h of lectures and 18 h of labs, the split also announced on master-mva.com. The division into sessions is a proposal: each session pairs about 1 h 12 of lecture with 1 h 48 of lab, which reproduces the official 12 h + 18 h. The review of ch. 0 is read before the first session. Level 1 = foundations, 2 = core methods and use cases, 3 = advanced use cases and synthesis.

Self-study, outside the 30 h (about 7 h in total): ch. 0 Review, to read before session 1 (≈ 2 h) · ch. 11 Research directions and open problems (≈ 3 h) · Exercises 13–14, depth separation in practice and the SNR wall for learned detectors (≈ 2 h).

How to study this module

• Read ch. 0 before the first session if Fourier series or Gaussian vectors are rusty; the three refreshers cover the rest.
• Chapters 5 to 9 share one structure: operational context and physics, classical chain, learned chain, comparison. Run the classical code before the learned one, and write down what each assumes.
• Keep a lab notebook with one table per use case: data, classical result, learned result, cost, and what failed. It feeds the projects and ch. 10.
• Do the exercises listed in the session plan after each session; the ★ ones check the computations, the ★★ ones the proofs.
• All code runs on a laptop CPU in seconds with NumPy, SciPy, scikit-learn and PyTorch, on synthetic data; no download is needed.

0. Review: Fourier bases, Gaussian noise and the chain rule

Every later chapter projects signals on Fourier-type bases, models noise as Gaussian and differentiates compositions of functions. This review fixes the notation and proves the three facts used most often: the discrete Fourier basis is orthonormal, Gaussian vectors stay Gaussian under linear maps, and the energy of Gaussian noise follows a chi-square law.

1. Signals, sampling and filtering

A sensor delivers a continuous waveform; a computer stores a finite list of numbers. This chapter states when the list loses nothing (sampling), how to read its frequency content (DFT), and how to transform it linearly (filters), up to the matched filter that every radar and receiver uses.

1.1 Signals and complex baseband

Radio, radar and many vibration signals are narrowband: their energy sits in a band of width B around a carrier f_c with B \ll f_c. Sampling them directly with the low-pass theorem of section 1.2 would require more than 2 f_c samples per second; one samples instead the slowly varying envelope.

1.2 Sampling and aliasing

The Fourier transform of a finite-energy signal is X(f) = \int x(t)\, e^{-2i\pi f t}\, dt. A sampling period T_s defines the rate f_s = 1/T_s and the samples x[n] = x(n T_s).

1.3 The discrete Fourier transform

A tone whose frequency falls between two bins spreads its energy over all bins: this is spectral leakage, caused by the implicit rectangular window of the finite observation. Tapering windows trade a wider main lobe for lower side lobes. The highest side lobe sits at about -13 dB for the rectangular window, -31 dB for Hann and -43 dB for Hamming.

The code shows aliasing and designs a low-pass filter (ch. 1.5).

# Code 1.6 — aliasing, and a windowed-sinc low-pass filter
import numpy as np
from scipy import signal

fs = 10_000                          # sampling rate (Hz)
t = np.arange(0, 0.1, 1 / fs)        # 1000 samples
x = np.sin(2 * np.pi * 7_000 * t)    # a 7 kHz tone, above fs / 2
X = np.abs(np.fft.rfft(x))
f = np.fft.rfftfreq(len(x), 1 / fs)
print("apparent frequency:", f[np.argmax(X)], "Hz")

# 101-tap windowed-sinc low-pass filter, cut-off 1 kHz
h = signal.firwin(101, 1_000, fs=fs, window="hamming")
_, H = signal.freqz(h, worN=[500, 1_000, 2_000], fs=fs)
print("gain (dB) at 0.5, 1, 2 kHz:", np.round(20 * np.log10(np.abs(H)), 1))

The output is an apparent frequency of 3000 Hz, and gains of 0, -6 and -65 dB at 0.5, 1 and 2 kHz.

1.4 Linear time-invariant filters

1.5 Designing a FIR filter

The ideal low-pass filter of cut-off f_c has the impulse response h_d[n] = (2 f_c / f_s)\, \mathrm{sinc}(2 f_c n / f_s): infinite and non-causal. The window method keeps M central taps, multiplies them by a tapering window and delays the result by (M-1)/2 samples.

• Symmetric taps give a linear phase: every frequency is delayed by the same (M-1)/2 samples, so pulse shapes are preserved — essential for radar and for transients.
• The transition width shrinks like 1/M; the window sets the stop-band attenuation (about -53 dB for Hamming), as Code 1.6 shows.

1.6 The matched filter and pulse compression

A receiver looks for a known waveform s (a radar pulse, a pilot sequence) in white noise. Which linear filter makes the waveform stand out most?

A radar wants both a long pulse (energy, hence range) and a fine range resolution (a short pulse). Pulse compression reconciles them: transmit a long chirp of duration T sweeping a band B, then apply the matched filter. The compressed pulse has a width of about 1/B.

• Sampled at its bandwidth, the chirp has N = BT samples, so the matched filter gains 10 \log_{10}(BT) dB.
• The range resolution is c / (2B) instead of cT/2.
• With T = 50 µs and B = 4 MHz: BT = 200, a gain of 23 dB, and a resolution of 37.5 m instead of 7.5 km.

# Code 1.10 — pulse compression of a linear FM chirp by the matched filter
import numpy as np

rng = np.random.default_rng(0)
B, T = 4e6, 50e-6                     # 4 MHz sweep, 50 us pulse
fs = B                                # complex baseband sampled at the bandwidth
N = int(round(B * T))                 # N = BT = 200 samples
t = np.arange(N) / fs
s = np.exp(1j * np.pi * (B / T) * (t - T / 2) ** 2)   # linear FM chirp, |s[n]| = 1
y = (rng.normal(size=2000) + 1j * rng.normal(size=2000)) / np.sqrt(2)  # noise, power 1
y[700:700 + N] += 0.25 * s            # echo at -12 dB per sample, delay 700
z = np.abs(np.correlate(y, s, mode="valid"))   # matched filter (s is conjugated)
print("estimated delay:", np.argmax(z))
print("output SNR (dB): %.1f" % (10 * np.log10(z.max() ** 2 / np.mean(z[:600] ** 2))))
print("predicted: -12 + 10 log10(BT) = %.1f dB" % (-12 + 10 * np.log10(B * T)))

The echo, invisible sample by sample, is found at delay 700 with an output SNR of about 10 dB, close to the predicted 11 dB.

2. Time–frequency and time–scale representations

The DFT of a whole recording says which frequencies are present, not when. Speech, chirps, radar pulses and bearing impacts are non-stationary, so we need representations indexed by time and frequency. This chapter builds them, proves the limit on their resolution, and derives the speech features used in chapters 7 and 8.

2.1 Short-time Fourier transform and spectrogram

The STFT is redundant when H < L, and it can be inverted.

2.2 The uncertainty principle

Short windows localise events in time but blur frequencies; long windows do the opposite. The trade-off is a theorem, not a weakness of the STFT.

2.3 Wavelets and the scalogram

The STFT analyses all frequencies with the same window. Physical signals often call for a constant relative resolution instead: a 50 Hz hum needs fine frequency resolution, a 5 kHz impact needs fine time resolution.

• If \psi has centre frequency f_0 and bandwidth \Delta f, the wavelet dilated by a has centre f_0 / a and bandwidth \Delta f / a. The relative bandwidth is constant: a constant-Q analysis.
• The Morlet wavelet, a Gaussian-windowed complex exponential, is the usual choice for time–frequency maps: \psi(t) = \pi^{-1/4}\, e^{i \omega_0 t}\, e^{-t^2/2}, \qquad \omega_0 \approx 6 . Its mean is not exactly zero, but its spectrum at f = 0 is e^{-\omega_0^2/2} \approx 10^{-8} times its peak: it is admissible in practice.
• When C_\psi < \infty, x can be reconstructed from W_x (Calderón’s formula; see Mallat’s book in the References).

2.4 Discrete wavelets and filter banks

Sampling scales on a dyadic grid a = 2^j leads to a fast, non-redundant transform implemented by a cascade of two-channel filter banks.

Daubechies wavelets use longer filters, so that smooth pieces of a signal give small details; soft thresholding of the details at \sigma \sqrt{2 \log N} then denoises (Donoho and Johnstone, 1994).

2.5 Perceptual features: mel spectrum, cepstrum and MFCC

Speech and audio systems compress the spectrogram along a perceptual frequency axis and separate the source from the filter.

The logarithm in step 3 is not cosmetic.

# Code 2.9 — the time-frequency trade-off, and a Morlet scalogram
import numpy as np
from scipy import signal

fs = 8_000
t = np.arange(2 * fs) / fs                       # 2 s
x = signal.chirp(t, f0=100, t1=2, f1=3_000)      # linear chirp 100 Hz -> 3 kHz
x[8_000:8_008] += 5.0                            # a 1 ms click at t = 1 s

for L in (64, 1024):                             # short and long Hann windows
    f, tt, S = signal.stft(x, fs, window="hann", nperseg=L, noverlap=L // 2)
    P = np.abs(S) ** 2
    k = np.argmin(np.abs(tt - 0.5))              # frame closest to t = 0.5 s
    width = np.sum(P[:, k] > 0.5 * P[:, k].max()) * fs / L
    hf = P[f > 3_500].sum(axis=0)                # energy above the chirp band
    spread = np.sum(hf > 0.1 * hf.max()) * (L / 2) / fs * 1e3
    print(f"L={L:4d}: ridge {f[np.argmax(P[:, k])]:6.1f} Hz, -3 dB width {width:5.1f} Hz,"
          f" click spread over {spread:5.1f} ms")

def morlet_cwt(x, fs, freqs, w0=6.0):
    """CWT with an analytic Morlet wavelet, computed in the Fourier domain."""
    X = np.fft.fft(x)
    om = 2 * np.pi * np.fft.fftfreq(len(x), 1 / fs)    # angular frequencies
    W = np.empty((len(freqs), len(x)), complex)
    for i, fc in enumerate(freqs):
        a = w0 / (2 * np.pi * fc)                # scale whose centre frequency is fc
        psi_hat = np.pi ** -0.25 * np.exp(-0.5 * (a * om - w0) ** 2) * (om > 0)
        W[i] = np.fft.ifft(X * np.sqrt(a) * psi_hat)
    return W

freqs = np.geomspace(100, 3_500, 48)             # constant-Q grid of centre frequencies
W = np.abs(morlet_cwt(x, fs, freqs)) ** 2
print("scalogram ridge at t = 0.5 s: %.0f Hz (true 825 Hz)" % freqs[np.argmax(W[:, 4_000])])

With 64 samples, the ridge is 250 Hz wide and the click spreads over 4 ms. With 1 024 samples, the ridge is 70 Hz wide — the chirp moves by 185 Hz within one window — and the click spreads over 128 ms. The scalogram ridge lies at 832 Hz for a true 825 Hz.

3. The machine-learning toolbox: classification, clustering, optimisation

Before deep networks, a signal is usually summarised by a few hand-crafted features — energy, kurtosis, spectral centroid — and fed to a standard learner. This chapter states what such a learner estimates, reviews the main families of classifiers and clustering algorithms, and proves the convergence rate of gradient descent, the engine of everything in chapter 4.

3.1 Supervised learning and the Bayes rule

Data are pairs (x_i, y_i), drawn independently from an unknown distribution P on features x and labels y \in \{1, \dots, K\}. A classifier f is judged by its risk R(f) = P(f(x) \neq y), and learning minimises an empirical estimate of it over a class of functions.

Unsupervised learning has no labels: it looks for structure — clusters, low-dimensional subspaces, densities — in the x_i alone. Industrial data are often unlabelled or labelled only for rare events, so both settings appear in every use case.

3.2 The main families of classifiers

3.3 Unsupervised learning: clustering

Clustering partitions n points into groups. Three algorithms cover most industrial needs.

Hierarchical clustering starts from n singletons and repeatedly merges two clusters. Ward’s criterion merges the pair A, B that increases the k-means objective J the least; with sizes n_A, n_B and means \mu_A, \mu_B, this increase is \Delta J(A, B) = \frac{n_A\, n_B}{n_A + n_B}\, \|\mu_A - \mu_B\|^2 . The merges form a tree, the dendrogram; cutting it at a given height yields a partition for every number of clusters. Principal component analysis completes the toolbox.

3.4 Evaluation

3.5 Optimisation and gradient descent

Training minimises an empirical loss F(\theta). Gradient descent iterates \theta_{k+1} = \theta_k - \eta\, \nabla F(\theta_k) with a step size \eta.

Stochastic gradient descent replaces \nabla F by an unbiased estimate on a random mini-batch; momentum and Adam (Kingma and Ba, 2015) rescale the steps. On the non-convex losses of deep networks, guarantees only concern stationary points.

The code runs the classical chain on three kinds of noisy synthetic signals: tones, square waves and sparse impulses.

# Code 3.10 — hand-crafted features, two classifiers and k-means
import numpy as np
from scipy import stats
from sklearn.linear_model import LogisticRegression
from sklearn.ensemble import RandomForestClassifier
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, adjusted_rand_score

rng = np.random.default_rng(0)
fs, n = 1_000, 512

def make(label):
    t = np.arange(n) / fs
    f0 = rng.uniform(20, 120)
    if label == 0:                                   # tone
        x = np.sin(2 * np.pi * f0 * t + rng.uniform(0, 2 * np.pi))
    elif label == 1:                                 # square wave (odd harmonics)
        x = np.sign(np.sin(2 * np.pi * f0 * t))
    else:                                            # sparse impulses (a "fault")
        x = (rng.random(n) < 0.02) * rng.normal(0, 3, n)
    return x + rng.normal(0, 1.5, n)

def features(x):
    P = np.abs(np.fft.rfft(x)) ** 2
    f = np.fft.rfftfreq(n, 1 / fs)
    return [np.sqrt(np.mean(x ** 2)),               # RMS
            stats.kurtosis(x, fisher=False),        # kurtosis (3 for a Gaussian)
            (f * P).sum() / P.sum(),                # spectral centroid
            np.mean(np.diff(np.sign(x)) != 0)]      # zero-crossing rate

y = rng.integers(0, 3, 900)
F = np.array([features(make(c)) for c in y])
Ftr, Fte, ytr, yte = train_test_split(F, y, test_size=0.3, random_state=0)
scaler = StandardScaler().fit(Ftr)                  # fitted on the training set only
Ftr, Fte = scaler.transform(Ftr), scaler.transform(Fte)
for clf in (LogisticRegression(max_iter=1_000), RandomForestClassifier(random_state=0)):
    acc = accuracy_score(yte, clf.fit(Ftr, ytr).predict(Fte))
    print(f"{type(clf).__name__:24s} test accuracy {acc:.3f}")
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(scaler.transform(F))
print("k-means ARI (no labels used):", round(adjusted_rand_score(y, km.labels_), 3))

Expected output: test accuracies of 0.96 (logistic regression) and 0.93 (random forest), and an ARI of 0.75 for k-means. Chapter 4 asks whether a network finds such features alone.

4. Deep learning: basics and main architectures

A deep network learns the features and the classifier together, from raw samples or spectrograms. This chapter defines the building blocks, shows that a convolutional layer is a learnable filter bank, explains why recurrent networks forget, and ends with a first, proven answer to the course’s question: what does depth add?

4.1 Multilayer perceptrons

The theorem says nothing about the number of hidden units needed. Section 4.7 shows that this number can be exponentially larger for a shallow network than for a deep one.

4.2 Backpropagation

4.3 Convolutional networks: learnable filter banks

On spectrograms, 2D convolutions treat the time–frequency plane as an image.

4.4 Recurrent networks and long dependencies

A recurrent network reads a sequence x_1, \dots, x_T and updates a state h_t = \tanh(W h_{t-1} + U x_t + b). Training unrolls it in time and backpropagates through the unrolled graph.

4.5 Autoencoders, attention and transformers

4.6 Training in practice

The code below trains a small 1D CNN on the raw signals of Code 3.10, with no hand-crafted features.

# Code 4.11 — a 1D CNN learns its own filter bank on raw signals
import numpy as np
import torch
from torch import nn

torch.manual_seed(0)
torch.set_num_threads(1)
rng = np.random.default_rng(0)
n, fs = 512, 1_000

def batch(m):
    """Tone / square wave / sparse impulses in strong noise, random phase and pitch."""
    y = rng.integers(0, 3, m)
    t = np.arange(n) / fs
    f0 = rng.uniform(20, 120, (m, 1))
    ph = rng.uniform(0, 2 * np.pi, (m, 1))
    tone = np.sin(2 * np.pi * f0 * t + ph)
    square = np.sign(tone)
    imp = (rng.random((m, n)) < 0.02) * rng.normal(0, 3, (m, n))
    x = np.choose(y[:, None], [tone, square, imp]) + rng.normal(0, 1.5, (m, n))
    return torch.tensor(x[:, None, :], dtype=torch.float32), torch.tensor(y)

net = nn.Sequential(                       # a learnable filter bank, then invariance
    nn.Conv1d(1, 16, kernel_size=33, padding=16), nn.BatchNorm1d(16), nn.ReLU(), nn.MaxPool1d(4),
    nn.Conv1d(16, 32, kernel_size=9, padding=4), nn.BatchNorm1d(32), nn.ReLU(),
    nn.AdaptiveAvgPool1d(1), nn.Flatten(),  # global average pooling: shift invariance
    nn.Linear(32, 3))
opt = torch.optim.Adam(net.parameters(), lr=3e-3)
for step in range(250):
    x, y = batch(64)
    loss = nn.functional.cross_entropy(net(x), y)
    opt.zero_grad(); loss.backward(); opt.step()
x, y = batch(1_000)
net.eval()
with torch.no_grad():
    acc = (net(x).argmax(1) == y).float().mean().item()
print(f"test accuracy {acc:.3f}, parameters {sum(p.numel() for p in net.parameters())}")

With 5 400 parameters and 16 000 training signals, the network reaches a test accuracy of 0.91, below the 0.96 of the four features of Code 3.10: it had to discover its detectors from data alone. Exercise 8 asks whether longer training closes the gap.

4.7 What makes deep learning deep? A first answer

The universal approximation theorem allows one hidden layer; the question is how many units it needs. The following example, due to Telgarsky (2016), shows a function that a narrow deep network computes exactly and that every shallow network of sub-exponential width misses.

# Code 4.14 — counting the linear pieces of a deep, narrow ReLU network
import numpy as np

relu = lambda z: np.maximum(z, 0.0)
tent = lambda z: 2 * relu(z) - 4 * relu(z - 0.5)      # one layer with two ReLU units
x = np.linspace(0, 1, 2 ** 12 + 1)
y = x.copy()
for k in range(1, 7):
    y = tent(y)                                       # k hidden layers of width 2
    slopes = np.diff(y) / np.diff(x)
    pieces = 1 + np.count_nonzero(np.abs(np.diff(slopes)) > 1e-6)
    print(f"depth {k}: {pieces:3d} linear pieces with {2 * k:2d} ReLU units;"
          f" an exact one-hidden-layer copy needs at least {pieces - 1} units")

At depth 6, twelve units produce 64 pieces; an exact shallow copy needs 63 units. Composition reuses the same folds at every scale. Chapter 10 completes this answer with invariance and stability.

5. Use case: signal detection for cognitive radio

A cognitive radio (Mitola and Maguire, 1999; Haykin, 2005) transmits in licensed bands when, and only when, their owners are silent. It must decide from a few milliseconds of samples whether a band is occupied — a detection problem at negative SNR. This chapter solves it optimally under a physical model, finds the limit of that solution, and asks what learning adds.

5.1 Operational context and physical model

A secondary user senses a licensed band, transmits in a spectrum hole, and leaves when the primary user returns; IEEE 802.22, for instance, standardises such access in television bands. Sensing must work below 0 dB, because the secondary receiver may sit in a fade while a nearby primary receiver still needs protection (the hidden-terminal problem).

5.2 The Neyman–Pearson test

5.3 The energy detector and its SNR wall

The formula promises detection at any SNR if one waits long enough. Real receivers only know the noise power approximately, and this breaks the promise.

5.4 Feature detectors: exploiting the structure of the signal

Man-made signals are not Gaussian noise: they have carriers, symbol rates, pilots and cyclic prefixes. Cyclostationary detectors (Gardner, 1991) use the periodicity of their correlation.

The price is knowledge of the cyclic frequencies. Eigenvalue-based detectors (Zeng and Liang, 2009) use the sample covariance matrix instead.

5.5 Learned detectors

A learned detector is a classifier trained on labelled windows of IQ samples, spectra or spectrograms. O’Shea, Corgan and Clancy (2016) showed that a CNN on raw IQ samples recognises modulations at least as well as expert features; O’Shea, Roy and Clancy (2018) extended this to over-the-air recordings.

# Code 5.8 — energy detector: Monte Carlo against the Gaussian approximation
import numpy as np
from scipy.stats import norm

rng = np.random.default_rng(0)
N, pfa, snr_db = 1_000, 0.01, -10.0
g = 10 ** (snr_db / 10)                              # SNR as a ratio, 0.1
lam = N + np.sqrt(N) * norm.isf(pfa)                 # threshold (noise power 1)
pd_theory = norm.sf((norm.isf(pfa) - np.sqrt(N) * g) / (1 + g))

def cn(shape, power=1.0):                            # circular complex Gaussian samples
    return np.sqrt(power / 2) * (rng.normal(size=shape) + 1j * rng.normal(size=shape))

T0 = np.sum(np.abs(cn((5_000, N))) ** 2, axis=1)                      # H0: noise only
T1 = np.sum(np.abs(cn((5_000, N), g) + cn((5_000, N))) ** 2, axis=1)  # H1: Gaussian signal
print(f"threshold {lam:.1f}; P_FA {np.mean(T0 > lam):.3f} (target {pfa})")
print(f"P_D Monte Carlo {np.mean(T1 > lam):.3f}, Gaussian approximation {pd_theory:.3f}")
M = int(np.ceil(((norm.isf(pfa) - (1 + g) * norm.isf(0.9)) / g) ** 2))   # Example 5.4
lam_M = M + np.sqrt(M) * norm.isf(pfa)
T1 = np.sum(np.abs(cn((5_000, M), g) + cn((5_000, M))) ** 2, axis=1)
print(f"N = {M} samples for P_D = 0.9 at -10 dB; Monte Carlo P_D {np.mean(T1 > lam_M):.3f}")

The simulated false-alarm rate is 0.009 for a target of 0.01, and the detection rate is 0.787 against 0.776 predicted. With the 1 396 samples of Example 5.4, the simulated detection rate is 0.886 for a target of 0.9: the Gaussian approximation is accurate to about 0.015.

The second experiment observes a narrowband primary user, a tone of unknown frequency at -15 dB, over 256 samples. It compares the energy detector, the periodogram maximum (the generalised likelihood-ratio test, GLRT, for an unknown frequency) and a logistic regression on the 32 largest periodogram values, a shift-invariant input.

# Code 5.9 — classical versus learned detection of a narrowband signal
import numpy as np
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import roc_auc_score

rng = np.random.default_rng(1)
N, snr_db, m = 256, -15.0, 4_000
a = np.sqrt(10 ** (snr_db / 10))                     # tone amplitude, noise power 1

def draw(m):
    y = rng.integers(0, 2, m)
    n = np.arange(N)
    f = rng.uniform(0, 1, (m, 1))                    # unknown normalised frequency
    tone = a * np.exp(2j * np.pi * (f * n + rng.uniform(0, 1, (m, 1))))
    w = (rng.normal(size=(m, N)) + 1j * rng.normal(size=(m, N))) / np.sqrt(2)
    return w + y[:, None] * tone, y

def periodogram(x):
    P = np.abs(np.fft.fft(x, 4 * N, axis=1)) ** 2 / N   # zero-padded periodogram
    return np.sort(P, axis=1)[:, ::-1][:, :32]          # 32 largest values: shift-invariant

xtr, ytr = draw(m)
xte, yte = draw(m)
energy = np.sum(np.abs(xte) ** 2, axis=1)               # energy detector
glrt = np.max(np.abs(np.fft.fft(xte, 4 * N, axis=1)) ** 2, axis=1)   # max periodogram
clf = LogisticRegression(max_iter=2_000).fit(np.log(periodogram(xtr)), ytr)
learned = clf.decision_function(np.log(periodogram(xte)))
for name, s in [("energy", energy), ("max-periodogram", glrt), ("learned", learned)]:
    print(f"{name:16s} AUC {roc_auc_score(yte, s):.3f}")

The energy detector reaches an AUC of 0.64; the periodogram peak and the learned detector both reach 0.82. Physics (a model of the tone) and learning (with the right invariance) find the same structure; learning keeps its edge where no closed-form test exists.

6. Use case: radar emitter identification

An electronic-support receiver listens to the radars around it: it intercepts a stream of pulses from several emitters, interleaved in time, and must say how many radars there are, of which types, and possibly which individual transmitters. This chapter models the pulses, separates the trains with clustering and timing histograms, and discusses what learning adds.

6.1 Operational context and physics of pulsed radars

A pulsed radar emits pulses of width \tau every pulse repetition interval (PRI) T_r. The receiver measures each intercepted pulse and summarises it by a small vector.

Emitters are classified by their PRI pattern — constant, jittered, staggered, sliding or dwell-and-switch — by RF agility and by intrapulse modulation (Wiley, 2006).

6.2 Deinterleaving by timing: TOA-difference histograms

When several radars share a frequency band and a direction, the PDW parameters do not separate them; their timing does.

The PRI transform (Nishiguchi and Kobayashi, 2000) suppresses the sub-harmonics by giving each pair a phase: D(\tau) = \sum_{j > i} \delta\big(\tau - (t_j - t_i)\big)\, \exp\Big(2i\pi\, \frac{t_j}{t_j - t_i}\Big). For pairs at distance T of a train, the phase is constant modulo 2\pi and the terms add up. For pairs at distance 2T, it alternates by \pi with the parity of the pulse index and the terms cancel.

6.3 Deinterleaving by clustering the PDWs

When emitters differ in RF, PW or AOA, clustering the PDWs separates them. The number of emitters is unknown and spurious pulses abound, so DBSCAN (Definition 3.6) fits where k-means does not. The practical chain has two stages: cluster the PDWs, then run a difference histogram in each cluster to split look-alike emitters and estimate their PRIs.

6.4 Learning to identify emitters

Learning enters at three places.

• Emitter type from PDW sequences: a recurrent network or a transformer, with the TOAs as positions (Proposition 4.9), learns agile and staggered PRI patterns end to end.
• Intrapulse modulation: a CNN on the spectrogram or IQ samples of a pulse, as in ch. 5.5.
• Deinterleaving as sequence labelling, trained on simulated mixtures.

Operational data are scarce and often confidential, so training relies on simulators; robustness to emitters absent from the library is open (ch. 11).

6.5 Specific emitter identification

Two radars of the same model share their PRI and RF settings, yet their hardware differs.

CNNs on raw IQ samples learn the fingerprint directly (Merchant, Revay, Stantchev and Nousain, 2018). The pitfall is shortcut learning: a network may learn the session or the channel instead of the transmitter, so training and test captures must come from different days (Note 3.2).

The code below simulates three emitters observed for 100 ms with 10 % missed pulses. Emitters B and C share their RF and PW and differ by 2° in AOA, less than the AOA measurement noise.

# Code 6.6 — deinterleaving: DBSCAN on PDWs, then TOA-difference histograms
import numpy as np
from sklearn.cluster import DBSCAN
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import adjusted_rand_score

rng = np.random.default_rng(0)
# emitter: (PRI in us, RF in MHz, PW in us, AOA in deg); B and C look alike
emitters = {"A": (1_000, 9_400, 1.0, 30), "B": (1_300, 3_000, 5.0, 75), "C": (770, 3_000, 5.0, 77)}
toa, pdw, lab = [], [], []
for k, (pri, rf, pw, aoa) in enumerate(emitters.values()):
    t = rng.uniform(0, pri) + pri * np.arange(int(100_000 / pri))   # 100 ms of intercept
    t = t[rng.random(len(t)) > 0.1]                                   # 10 % missed pulses
    toa.append(t + rng.normal(0, 0.05, len(t)))                       # 50 ns TOA jitter
    pdw.append(np.c_[rf + rng.normal(0, 2, len(t)), pw + rng.normal(0, 0.05, len(t)),
                     aoa + rng.normal(0, 2, len(t))])
    lab.append(np.full(len(t), k))
toa, pdw, lab = np.concatenate(toa), np.vstack(pdw), np.concatenate(lab)

# step 1: cluster the pulse descriptor words on (RF, PW, AOA)
cl = DBSCAN(eps=0.3, min_samples=10).fit_predict(StandardScaler().fit_transform(pdw))
print("clusters:", len(set(cl) - {-1}), " ARI:", round(adjusted_rand_score(lab, cl), 3))

# step 2: inside each cluster, histogram of all TOA differences below 3 ms
for c in sorted(set(cl) - {-1}):
    t = np.sort(toa[cl == c])
    d = (t[None, :] - t[:, None]).ravel()
    h, e = np.histogram(d[(d > 0) & (d < 3_000)], bins=np.arange(97.5, 3_000, 5))
    pri = []
    for p in e[:-1][h > 0.4 * h.max()] + 2.5:        # strong peaks, in increasing order
        if all(abs(p / q - round(p / q)) > 0.05 for q in pri):   # drop multiples
            pri.append(round(float(p)))
    print(f"cluster {c}: {len(t)} pulses, PRIs found (us): {pri}")

DBSCAN finds two clusters (ARI 0.58): emitter A alone, and B and C merged. The difference histogram of the merged cluster then shows two fundamental PRIs, 770 µs and 1 300 µs, and the one of A shows 1 000 µs. Exercise 7 does the same computation by hand.

7. Use case: speaker recognition and few-shot learning

A voice carries its speaker’s anatomy and habits. Speaker recognition turns them into a score, from a statistical model of short-term spectra (GMM-UBM) to embeddings learned by deep networks. Enrolling a new speaker from a few seconds of speech is also the textbook case of few-shot learning, the last theme of the syllabus.

7.1 Tasks and metrics

7.2 Physics: the source–filter model

Voiced speech is a glottal pulse train at the pitch F_0 (roughly 85–180 Hz for adult men and 165–255 Hz for adult women) filtered by the vocal tract, whose resonances are the formants. Speaker identity lives in both: pitch statistics, and formant positions set by the length and shape of the tract.

Linear prediction estimates the vocal-tract filter as an all-pole model, s[n] \approx \sum_{k=1}^p a_k\, s[n - k].

7.3 The classical chain: GMM-UBM

Components that saw little enrolment data stay close to the UBM, which is why the method works with short enrolments. i-vectors (Dehak, Kenny, Dehak, Dumouchel and Ouellet, 2011) later compressed the adapted means into a low-dimensional vector scored by cosine similarity or PLDA.

7.4 Deep speaker embeddings

Statistics pooling ignores frame order and maps any length to a fixed size, like the global pooling of Proposition 4.5. ECAPA-TDNN (Desplanques, Thienpondt and Demuynck, 2020) adds channel attention and angular-margin training, benchmarked on VoxCeleb (Nagrani, Chung and Zisserman, 2017).

7.5 Few-shot learning

The code synthesises ten speakers with a source–filter model, scaling average vowel formants (Peterson and Barney, 1952) by a speaker-specific tract factor. It compares one GMM per speaker, trained on five utterances, with K-shot prototypes of a fixed embedding: the mean and standard deviation of the MFCC frames.

# Code 7.8 — speaker identification: GMM per speaker versus few-shot prototypes
import numpy as np
from scipy import signal
from scipy.fft import dct
from sklearn.mixture import GaussianMixture

rng = np.random.default_rng(0)
fs, S = 8_000, 10                                    # 8 kHz audio, 10 speakers

def mel_filterbank(n_fft, n_mels=24, fmax=4_000):
    mel = lambda f: 2595 * np.log10(1 + f / 700)
    imel = lambda m: 700 * (10 ** (m / 2595) - 1)
    edges = imel(np.linspace(0, mel(fmax), n_mels + 2))
    f = np.fft.rfftfreq(n_fft, 1 / fs)
    lo, c, hi = edges[:-2, None], edges[1:-1, None], edges[2:, None]
    return np.maximum(0, np.minimum((f - lo) / (c - lo), (hi - f) / (hi - c)))

FB = mel_filterbank(256)

def mfcc(x, n_ceps=13):
    _, _, Z = signal.stft(x, fs, nperseg=256, noverlap=128)      # 32 ms frames
    E = FB @ (np.abs(Z) ** 2) + 1e-10                             # mel energies
    return dct(np.log(E), axis=0, norm="ortho")[1:n_ceps].T      # frames x 12

# a speaker = mean pitch + vocal-tract length factor (scales all formants)
speakers = [(rng.uniform(90, 220), rng.uniform(0.85, 1.15)) for _ in range(S)]
vowels = [(730, 1090, 2440), (270, 2290, 3010), (300, 870, 2240), (530, 1840, 2480)]

def utterance(spk, dur=1.0):
    f0, alpha = speakers[spk]
    x = []
    for v in rng.permutation(4):                     # the four vowels, random order
        n = int(dur * fs / 4)
        f = f0 * (1 + 0.05 * rng.normal()) * (1 + 0.03 * np.sin(2 * np.pi * 3 * np.arange(n) / fs))
        e = np.diff(np.floor(np.cumsum(f) / fs), prepend=0)       # glottal pulse train
        for F in vowels[v]:                          # cascade of formant resonators
            r, th = np.exp(-np.pi * 80 / fs), 2 * np.pi * F * alpha / fs
            e = signal.lfilter([1 - r], [1, -2 * r * np.cos(th), r * r], e)
        x.append(e / e.std())
    x = np.concatenate(x)
    return x + 0.05 * rng.normal(size=x.size)

train = {s: np.vstack([mfcc(utterance(s)) for _ in range(5)]) for s in range(S)}
gmms = [GaussianMixture(8, covariance_type="diag", random_state=0).fit(train[s]) for s in range(S)]
tests = [(s, mfcc(utterance(s))) for s in range(S) for _ in range(10)]
acc = np.mean([s == np.argmax([g.score(X) for g in gmms]) for s, X in tests])
print(f"GMM identification accuracy: {acc:.2f}")

def embed(X):                                        # statistics pooling: mean and std
    return np.r_[X.mean(0), X.std(0)]

for K in (1, 5):                                     # K-shot prototypes (Definition 7.6)
    protos = np.array([np.mean([embed(mfcc(utterance(s))) for _ in range(K)], 0) for s in range(S)])
    pred = [np.argmin(((protos - embed(X)) ** 2).sum(1)) for _, X in tests]
    print(f"{K}-shot prototype accuracy: {np.mean([s == p for (s, _), p in zip(tests, pred)]):.2f}")

The GMMs reach an accuracy of 0.94, the 1-shot prototypes 0.94 and the 5-shot prototypes 0.99. Every utterance here contains the same four vowels; on real text-independent speech, fixed MFCC statistics lose to learned x-vectors.

8. Use case: reconstruction of altered audio signals

Old recordings crackle, packets get lost in voice calls, loud passages clip, and microphones pick up noise. Restoration inverts a known degradation with prior knowledge of what audio looks like; the prior is either a model (Wiener, autoregressive, sparse) or learned from clean examples.

8.1 Degradation models

8.2 Denoising: the Wiener gain

Spectral subtraction (Boll, 1979) estimates S_n on noise-only segments and S_x by \max(|Y|^2 - S_n, 0); it leaves “musical noise”, which the estimator of Ephraim and Malah (1984) suppresses by smoothing the SNR over time.

8.3 Filling gaps with an autoregressive model

Short gaps — a few milliseconds — are well predicted from both sides by linear prediction (Proposition 7.3).

For longer gaps and clipping, sparse priors take over: audio inpainting (Adler, Emiya, Jafari, Elad, Gribonval and Plumbley, 2012) seeks a few Gabor or DCT atoms matching the known samples; declipping also forces restored samples above the clipping level.

8.4 Learned restoration

A network learns the prior from clean audio: degrade clean examples with the model of Definition 8.1, and train it to undo the degradation, as a denoising autoencoder (Definition 4.7).

• Architectures: U-Nets on magnitude spectrograms (Jansson et al., 2017) and on waveforms (Wave-U-Net, Stoller, Ewert and Dixon, 2018); Demucs for real-time speech enhancement in the waveform domain (Défossez, Synnaeve and Adi, 2020).
• Phase: a mask on magnitudes keeps the noisy phase; Griffin and Lim (1984) estimate a consistent phase by alternating projections between the STFT and the signal domain.
• Metrics: SNR and scale-invariant SDR (Le Roux, Wisdom, Erdogan and Hershey, 2019); perceptual scores such as PESQ (ITU-T P.862) and STOI (Taal, Hendriks, Heusdens and Jensen, 2011).

# Code 8.5 — a 5 ms gap: linear versus AR interpolation; denoising: oracle mask versus spectral subtraction
import numpy as np
from scipy import signal
from scipy.linalg import solve_toeplitz

rng = np.random.default_rng(0)
fs = 8_000
t = np.arange(4_000) / fs
x = (np.sin(2 * np.pi * 440 * t) + 0.5 * np.sin(2 * np.pi * 660 * t + 1)
     + 0.3 * np.sin(2 * np.pi * 1_250 * t)) + 0.02 * rng.normal(size=t.size)

# --- gap interpolation with an AR(p) model (Proposition 8.3) ---------------------
gap = np.arange(2_000, 2_040)                         # 40 missing samples = 5 ms
known = np.setdiff1d(np.arange(x.size), gap)
p = 40
ctx = np.r_[x[1_500:2_000], x[2_040:2_540]]           # context used to fit the model
r = np.correlate(ctx, ctx, "full")[ctx.size - 1:][: p + 1] / ctx.size
a = solve_toeplitz(r[:p], r[1:p + 1])                 # normal equations (Proposition 7.3)
b = np.r_[1, -a]                                      # prediction-error filter
seg = np.arange(gap[0] - p, gap[-1] + p + 1)          # residuals touching the gap
B = np.zeros((seg.size - p, seg.size))
for i in range(seg.size - p):
    B[i, i:i + p + 1] = b[::-1]                       # e[n] = sum_k b_k x[n-k]
U = np.isin(seg, gap)
xu = np.linalg.lstsq(B[:, U], -B[:, ~U] @ x[seg[~U]], rcond=None)[0]
lin = np.interp(gap, known, x[known])
snr = lambda est: 10 * np.log10(np.sum(x[gap] ** 2) / np.sum((x[gap] - est) ** 2))
print(f"gap of 5 ms: linear interpolation {snr(lin):5.1f} dB, AR({p}) {snr(xu):5.1f} dB")

# --- denoising: ideal ratio mask (oracle) versus spectral subtraction -------------
noise = 0.5 * rng.normal(size=x.size)
y = x + noise
_, _, X = signal.stft(x, fs, nperseg=256)
_, _, N = signal.stft(noise, fs, nperseg=256)
_, _, Y = signal.stft(y, fs, nperseg=256)
Sn = np.mean(np.abs(N) ** 2, axis=1, keepdims=True)  # noise PSD (from a noise-only recording)
gains = {"ideal ratio mask": np.abs(X) ** 2 / (np.abs(X) ** 2 + np.abs(N) ** 2),
         "spectral subtraction": np.maximum(1 - Sn / np.abs(Y) ** 2, 0)}
out = lambda z: 10 * np.log10(np.sum(x ** 2) / np.sum((x - z[: x.size]) ** 2))
print(f"input SNR {out(y):5.1f} dB")
for name, G in gains.items():
    print(f"{name:21s} {out(signal.istft(G * Y, fs, nperseg=256)[1]):5.1f} dB")

On the 5 ms gap, linear interpolation scores -1 dB and AR(40) interpolation 29 dB. For denoising from an input SNR of 4 dB, spectral subtraction reaches 10 dB and the oracle mask 17 dB: the gap between the two is the room left for a learned mask estimator.

9. Use case: predictive maintenance

A rotating machine announces its failure weeks in advance, in its vibrations. Predictive maintenance listens to them to repair just in time — not too early, which wastes parts, and not too late, which breaks the machine. This chapter models the most common fault, a damaged rolling bearing, and builds the detection chain from physics to anomaly detection and learned prognostics.

9.1 Operational context

Failures are rare and machines differ, so the unsupervised setting — learn “healthy”, flag the rest — dominates.

9.2 Physics of a bearing fault

A rolling bearing has n balls of diameter d on a pitch circle of diameter D, with contact angle \varphi; the inner race turns at f_r and the outer race is fixed. A local defect produces an impact each time a ball rolls over it.

Each impact rings a structural resonance, typically several kilohertz, so the fault frequency appears as an amplitude modulation of that resonance, not as a spectral line of its own. Envelope analysis band-passes the resonance, takes its envelope and looks at the envelope’s spectrum.

9.3 Condition indicators

Root mean square, peak value and crest factor (peak over RMS) track energy; kurtosis tracks impulsiveness.

# Code 9.6 — envelope analysis of a simulated outer-race fault
import numpy as np
from scipy import signal, stats

rng = np.random.default_rng(3)
fs, dur = 20_000, 1.0
n_balls, fr, ratio = 9, 25.0, 0.2                    # 9 balls, 1500 rpm, d/D = 0.2
bpfo = n_balls * fr / 2 * (1 - ratio)                # 90 Hz (contact angle 0)
t = np.arange(int(fs * dur)) / fs

def vibration(faulty):
    x = 0.5 * np.sin(2 * np.pi * fr * t) + rng.normal(0, 1.0, t.size)   # shaft + noise
    if faulty:                                       # impacts at BPFO ringing at 3 kHz
        imp = np.zeros(t.size)
        times = np.arange(0, dur, 1 / bpfo) + rng.normal(0, 2e-4, int(np.ceil(dur * bpfo)))
        imp[(times[times < dur] * fs).astype(int)] = 3.0
        ring = np.exp(-800 * t[:200]) * np.sin(2 * np.pi * 3_000 * t[:200])
        x += np.convolve(imp, ring)[: t.size]
    return x

for faulty in (False, True):
    x = vibration(faulty)
    sos = signal.butter(4, [2_000, 4_000], btype="bandpass", fs=fs, output="sos")
    xb = signal.sosfiltfilt(sos, x)                  # the resonance band, 2-4 kHz
    env = np.abs(signal.hilbert(xb))                 # its envelope (Proposition 9.4)
    E = np.abs(np.fft.rfft(env - env.mean()))
    f = np.fft.rfftfreq(env.size, 1 / fs)
    band = (f > 10) & (f < 500)
    k = f[band][np.argmax(E[band])]
    print(f"faulty={faulty!s:5}: kurtosis raw {stats.kurtosis(x, fisher=False):4.2f},"
          f" band {stats.kurtosis(xb, fisher=False):4.2f}; envelope peak {k:5.1f} Hz,"
          f" {E[band].max() / np.median(E[band]):4.1f} x the median")

On the healthy signal, both kurtoses stay at 3 and the envelope spectrum has no line (its maximum, at 206 Hz, is 3 times the median). On the faulty one, the raw kurtosis is only 3.3, the band kurtosis 5.5, and the envelope spectrum shows a line at exactly 90 Hz, 28 times the median: the BPFO of Example 9.3.

9.4 Anomaly detection with a controlled false-alarm rate

Learn the distribution of indicators on healthy data, and raise an alarm when a new window is unlikely under it.

On raw spectra, the reconstruction error of an autoencoder trained on healthy data (Definition 4.7) plays the role of D^2; one-class SVMs (Schölkopf et al., 2001) and isolation forests (Liu, Ting and Zhou, 2008) are alternatives.

# Code 9.8 — a Mahalanobis health indicator on a slowly growing fault
import numpy as np
from scipy import signal, stats

rng = np.random.default_rng(0)
fs, n, bpfo = 20_000, 4_096, 90.0
sos = signal.butter(4, [2_000, 4_000], btype="bandpass", fs=fs, output="sos")
ring = np.exp(-800 * np.arange(200) / fs) * np.sin(2 * np.pi * 3_000 * np.arange(200) / fs)

def features(severity):
    """One 0.2 s window: noise + impacts of amplitude `severity` at BPFO."""
    x = rng.normal(size=n)
    imp = np.zeros(n)
    imp[(np.arange(0, n / fs, 1 / bpfo) * fs).astype(int)] = severity
    xb = signal.sosfiltfilt(sos, x + np.convolve(imp, ring)[:n])
    return [np.sqrt(np.mean(xb ** 2)), stats.kurtosis(xb, fisher=False),
            np.max(np.abs(xb)) / np.sqrt(np.mean(xb ** 2))]     # RMS, kurtosis, crest

H = np.array([features(0.0) for _ in range(300)])               # healthy reference
mu, P = H.mean(0), np.linalg.inv(np.cov(H.T))
d2 = lambda z: (z - mu) @ P @ (z - mu)                           # Mahalanobis distance
thr = stats.chi2.ppf(0.99, df=3)                                 # 1 % false alarms (Proposition 9.7)
print(f"threshold {thr:.2f}; false alarms on new healthy data:",
      np.mean([d2(features(0.0)) > thr for _ in range(300)]))
for day, sev in enumerate(np.linspace(0, 2, 9)):                 # a slowly growing fault
    print(f"day {day}: severity {sev:4.2f}, alarm rate",
          np.mean([d2(features(sev)) > thr for _ in range(50)]))

The false-alarm rate on new healthy windows is 0.02 for a target of 0.01, a symptom of non-Gaussian features. The alarm rate rises from 0.1 at severity 0.75 to 0.48 at 1.0 and 1.0 from severity 1.5: the indicator detects the fault before it dominates the noise.

9.5 Prognostics and learned models

Estimating the RUL is a regression on degradation trajectories. The C-MAPSS data set (Saxena, Goebel, Simon and Eklund, 2008) simulates run-to-failure trajectories of turbofan engines with 21 sensors. Its score penalises late predictions more than early ones, with d the predicted minus the true RUL: s(d) = e^{-d/13} - 1 \ \ (d < 0), \qquad s(d) = e^{d/10} - 1 \ \ (d \geq 0).

1D CNNs and LSTMs on sliding windows of the sensors are standard learned baselines. Deployment meets few run-to-failure examples, domain shift between machines and speeds, and the need to explain alarms. Physics helps: order tracking resamples the signal at constant shaft angle, and envelope features make good inputs. Lei and coauthors (2020) review the learned approaches.

10. Synthesis: what makes deep learning deep, and research in industry

Five use cases later, the course question can be answered from evidence: where did learning help, where did depth matter, and why? The chapter closes with a deep network that learns nothing — the scattering transform — and with how industrial research turns such ideas into products.

10.1 What the use cases showed

Three lessons recur. Learning wins when the physics leaves a gap that data can fill, and loses when data are few or shifted. Its best inputs are representations with the right invariance (sorted spectra, global pooling, statistics pooling). Depth multiplies the number of distinct regions a network can carve (Theorem 4.13) and composes invariances layer after layer.

10.2 A deep network without learning: the scattering transform

Why do spectrograms and mel features work so well as inputs? Because a small deformation of a signal — a slight change of speed or pitch — barely changes them, whereas it moves the high-frequency lines of a Fourier spectrum a lot. Mallat (2012) built a deep network with this stability by design: cascade wavelet filters, moduli and averaging, S x = \big(x * \phi,\ |x * \psi_{\lambda_1}| * \phi,\ \big||x * \psi_{\lambda_1}| * \psi_{\lambda_2}\big| * \phi,\ \dots\big).

# Code 10.2 — a 3 % dilation: Fourier modulus versus constant-Q averages
import numpy as np

fs, n = 16_000, 16_000
t = np.arange(n) / fs
harmonic = lambda f0: sum(np.sin(2 * np.pi * k * f0 * t) / k for k in range(1, 31))
x, x_warp = harmonic(200.0), harmonic(206.0)          # a 3 % dilation of the frequency axis

f = np.fft.rfftfreq(n, 1 / fs)
centres = np.geomspace(100, 7_000, 40)                # constant-Q bands (about 11 % wide)
bank = np.exp(-0.5 * (np.log(f[None, 1:] / centres[:, None]) / 0.05) ** 2)

def fourier(x):                                       # Fourier modulus: shift-invariant only
    return np.abs(np.fft.rfft(x))[1:]

def averaged(x):                                      # band energies = first-order scattering
    return np.sqrt(bank @ fourier(x) ** 2)

for name, phi in [("Fourier modulus", fourier), ("constant-Q averages", averaged)]:
    a, b = phi(x), phi(x_warp)
    print(f"{name:20s} relative change {np.linalg.norm(a - b) / np.linalg.norm(a):.2f}")

The Fourier modulus changes by 1.41 — its lines no longer overlap — while the constant-Q averages change by 0.30. Learned front-ends such as SincNet (Ravanelli and Bengio, 2018) and LEAF (Zeghidour, Teboul, de Chaumont Quitry and Tagliasacchi, 2021) start from such filter banks and let training adjust them. Algorithm unrolling goes the other way: it turns the iterations of a classical algorithm into network layers, as LISTA did for sparse coding (Gregor and LeCun, 2010).

10.3 How research is organised in industry

The syllabus announces a discussion of industrial research; the following points frame it.

• From use case to product. A need from an operational team becomes a research question, then a prototype on recorded data, then a demonstrator in real conditions, then an industrialised function. Technology readiness levels (TRL 1–9) name these stages.
• Data. Field data are costly, sometimes confidential, often unlabelled; simulators, data-collection campaigns and annotation are research tasks in their own right.
• Constraints. Embedded hardware limits memory, power and latency; safety-critical functions need guarantees, explanations and qualification; deployed models must be monitored for drift.
• Collaboration. Companies work with academic laboratories through joint projects, industrial PhDs (the French CIFRE scheme), internships and shared publications, while protecting know-how through patents.

11. Research directions and open problems

This chapter is for self-study. It surveys where research on learning for signals is heading, around the five use cases of the course, and lists open problems suitable for a project or an internship. The works cited are entry points, not a complete bibliography.

11.1 Self-supervised and foundation models for signals

Labels are the bottleneck of every use case. Self-supervised learning trains on unlabelled signals with a pretext task, then adapts to the target task with few labels.

• In speech, wav2vec 2.0 (Baevski, Zhou, Mohamed and Auli, 2020) masks latent frames and solves a contrastive task over quantised targets; HuBERT (Hsu and coauthors, 2021) predicts cluster labels of masked frames; WavLM (Chen and coauthors, 2022) adds denoising and reaches strong results on speaker verification among other tasks.
• For radio, radar and vibration, comparable pretrained models are much less mature. Which pretext tasks respect the physics — carrier offsets, multipath, operating speeds — is an open question.

11.2 Open-set recognition and novelty

An emitter library, a speaker list or a set of fault types is never complete. A closed-set classifier assigns every input to a known class, often with high confidence.

• Scheirer, de Rezende Rocha, Sapkota and Boult (2013) formalised open-set recognition; Bendale and Boult (2016) proposed OpenMax to reject unknown inputs from deep features.
• Calibration (Guo, Pleiss, Sun and Weinberger, 2017) and conformal prediction (Angelopoulos and Bates, 2023) give confidence statements with explicit guarantees, the kind that safety-critical users ask for.

Open problem: open-set emitter identification with guarantees on the rate of unknown emitters wrongly accepted.

11.3 Domain shift and simulation-to-real transfer

Most industrial training data come from simulators or from a few machines, and deployment happens elsewhere. Domain-adversarial training (Ganin and coauthors, 2016) learns features that a discriminator cannot attribute to a domain. For maintenance, transfer across machines, loads and speeds remains largely unsolved; benchmarks often split windows of the same recordings, which overstates performance (Note 3.2).

11.4 Model-based deep learning

Between hand-crafted chains and black boxes lies a spectrum of hybrids.

• Algorithm unrolling turns the iterations of a classical solver into trainable layers (Monga, Li and Eldar, 2021).
• Model-based deep learning (Shlezinger, Whang, Eldar and Dimakis, 2023) replaces only the parts of a classical algorithm that depend on an unknown model, keeping its structure and interpretability.
• Physics-informed networks (Raissi, Perdikaris and Karniadakis, 2019) add residuals of differential equations to the loss.

Open problem: when does a hybrid provably need fewer samples than a generic network for the same accuracy?

11.5 Generative priors for restoration

Diffusion models learn a prior over clean audio. DiffWave (Kong, Ping, Huang, Zhao and Catanzaro, 2021) generates waveforms with a diffusion model; diffusion posterior sampling (Chung, Kim, McCann, Klasky and Ye, 2023) combines such a prior with a measurement model to solve inverse problems, including inpainting. The Wiener and AR priors of ch. 8 become special cases of a learned prior plus a likelihood.

Open problem: restoration with guarantees of consistency with the observed samples and controlled hallucination for archival audio.

11.6 Theory of depth

Theorem 4.13 is a one-dimensional example. Eldan and Shamir (2016) proved a separation between one and two hidden layers for a radial function in high dimension; Poggio and coauthors (2017) showed that deep networks avoid the curse of dimensionality for compositional functions whose constituents depend on few variables. Mallat’s scattering theory (ch. 10.2) explains stability to deformations. Why trained networks find such structured solutions by gradient descent remains open (see Theoretical Foundations of Deep Learning).

11.7 Frugal and embedded learning

Sensors run on small processors and batteries. Integer quantisation (Jacob and coauthors, 2018) and pruning cut memory and energy, while certification asks for models whose behaviour can be bounded. Designing networks that are small, robust and certifiable at once is an active question for every use case of the course.

Exercises

1. ★ Aliasing and complex baseband. (a) A 9 kHz tone is sampled at 8 kHz. At which frequency does it appear? (b) A receiver captures a band of 20 MHz around 2.4 GHz. Compare the minimum rate of complex IQ sampling with the rate that Theorem 1.3 requires for real sampling of the raw band-pass signal.

   Solution: (a) the tone is folded by one sampling rate, 9 - 8 = 1 kHz. (b) Complex sampling needs 20 million complex samples per second (the bandwidth). Theorem 1.3 asks real sampling of the raw signal to exceed twice its highest frequency, 2 \times 2.41 GHz = 4.82 GHz: about 240 times more samples per second, or 120 times more real numbers, since an IQ sample holds two.

   Band-pass (under-)sampling of the real signal can go down to about 2B = 40 MHz when the band sits suitably between multiples of f_s/2, but it then needs a sharp analogue band-pass filter; this is why receivers prefer IQ demodulation.

2. ★ A resolution budget. Speech is sampled at 16 kHz. A Hann window has a main lobe four bins wide. Which window length separates two harmonics 100 Hz apart, and what hop gives 50 % overlap?

   Solution: two lines stay separated when they are at least half a main lobe apart, two bins. Hence 2 f_s / N \leq 100 Hz, so N \geq 320 samples (20 ms). With N = 512 (32 ms, a power of two), the hop is 256 samples, 16 ms; the bin spacing is 31.25 Hz.

3. ★ Pulse compression. A radar transmits a linear chirp of T = 20 µs sweeping B = 10 MHz. Compute the compression gain, the compressed range resolution, and the resolution without compression.

   Solution: BT = 200, a gain of 10 \log_{10} 200 = 23 dB (Theorem 1.9, with sampling at the bandwidth). Resolution with compression: c / (2B) = 15 m; without: c T / 2 = 3 km.

4. ★ Mel, cepstrum and formants. (a) Compute the mel values of 500 Hz and 4 kHz. (b) Where is the cepstral peak of a 200 Hz voice sampled at 16 kHz? (c) What are the first three formants of a uniform tube of 15 cm (Example 7.2)?

   Solution: (a) m(500) = 2595 \log_{10}(1.714) \approx 607 and m(4000) = 2595 \log_{10}(6.714) \approx 2146: the mel axis compresses high frequencies. (b) At the pitch period, 5 ms, that is quefrency index 80 (Proposition 2.8). (c) F_k = (2k - 1) \times 350 / 0.6: 583, 1 750 and 2 917 Hz.

5. ★ Metrics under imbalance. Among 2 000 windows, 40 precede a failure. A detector flags 60 windows, 30 of which precede a failure. Compute recall, precision, F1 and the false-alarm rate.

   Solution: recall 30/40 = 0.75; precision 30/60 = 0.50; F_1 = 2 \times 0.5 \times 0.75 / 1.25 = 0.60; false-alarm rate 30 / 1960 \approx 1.5 %. The accuracy, (30 + 1930)/2000 = 98 %, says nothing useful: “always healthy” also scores 98 % (Example 3.8).

6. ★ Sizing an energy detector (computational). For P_{FA} = 0.05, P_D = 0.95 and an SNR of -13 dB, compute the number of samples from Proposition 5.3, then check it by adapting Code 5.8.

   Solution: \gamma = 10^{-1.3} = 0.0501 and Q^{-1}(0.05) = 1.645 = -Q^{-1}(0.95), so N = \Big(\frac{1.645 + 1.0501 \times 1.645}{0.0501}\Big)^2 \approx 4\,527 . With N = 4527, pfa = 0.05, snr_db = -13 in Code 5.8 (and fewer trials, to save memory), the Monte Carlo detection rate is close to 0.95 and the false-alarm rate close to 0.05.

7. ★ Deinterleaving by hand. An interceptor records the TOAs (µs) 0, 200, 500, 900, 1000, 1500, 1600, 2000, 2300. (a) List the order-1 differences: do they reveal a PRI? (b) Count all pairwise differences up to 1 100 µs and find the PRIs. (c) Give the unambiguous range of each emitter.

   Solution: (a) 200, 300, 400, 100, 500, 100, 400, 300: no dominant value. (b) The counts are 4 at 500 µs, 3 at 700 µs, 3 at 1 000 µs, and at most 2 elsewhere. The value 1 000 is twice 500, so the PRIs are 500 µs (pulses 0, 500, 1000, 1500, 2000) and 700 µs (200, 900, 1600, 2300), as Proposition 6.4 predicts. (c) c T_r / 2: 75 km and 105 km.

8. ★★ Equivariance, receptive field and training (computational). (a) Show that a convolutional layer with stride 2 is equivariant to shifts by an even number of samples (an input shift by 2\tau shifts the output by \tau), but not to a shift by one sample. (b) Compute the receptive field of four layers with kernel size 5 and dilations 1, 2, 4, 8. (c) Train Code 4.11 for 1 000 steps instead of 250 and compare its accuracy with Code 3.10.

   Hints and expected result: (a) the output at index n reads the inputs from 2n; a shift by 2\tau shifts the output by \tau, while a shift by 1 changes the sampling phase and no output shift compensates. (b) R = 1 + 4 \times (1 + 2 + 4 + 8) = 61 samples (Proposition 4.5). (c) The gap to the hand-crafted features narrows: after 1 000 steps the CNN reaches about 0.94, against 0.96 for Code 3.10. Report both accuracies on test sets of the same size.

9. ★★ Prototypes and nearest neighbours (computational). (a) Prove that 1-shot prototype classification is nearest-neighbour classification, and that K-shot prototypes with squared Euclidean distance give a linear classifier (Proposition 7.7). (b) In Code 7.8, compute the accuracy for K = 1, \dots, 5, then replace the Euclidean distance by the cosine distance.

   Hints and expected result: (a) with one example per class, the prototype is that example; for the linear form, expand the square. (b) Accuracy is already high at K = 1 (0.91 to 0.99 depending on the random seed) and reaches 0.96 to 1.00 at K = 5: averaging reduces the variance of the prototype.

   With only 100 test utterances, the curve is not monotone and differences of a few points are within the noise of the draw; average over several seeds. Cosine distance ignores the norm of the embedding; here it stays within about 0.03 of the Euclidean accuracy.

10. ★★ The Wiener gain in numbers. (a) Show that the minimal mean squared error of Theorem 8.2 equals S_x S_n / (S_x + S_n), so that the output SNR is 1 + \xi. (b) Compute the gain and the SNR improvement for bins at 10 dB and at -5 dB.

    Solution: (a) Substituting G^\star in the error gives S_x S_n / (S_x + S_n); dividing S_x by it gives 1 + \xi.

    For (b): at 10 dB, the gain is 0.909, with an output SNR of 10 \log_{10} 11 = 10.4 dB, an improvement of 0.4 dB. At -5 dB, where \xi = 0.316: gain 0.240, output SNR 1.2 dB, an improvement of 6.2 dB. The Wiener gain helps most where the SNR is low, at the price of attenuating the signal there.

11. ★★ An inner-race fault (computational). A bearing has n = 8 balls, d/D = 0.25, contact angle 0, and turns at f_r = 30 Hz. (a) Compute BPFO and BPFI. (b) For impacts b\, g plus Gaussian noise w of variance \sigma^2, compute the kurtosis. (c) Modify Code 9.6 to simulate an inner-race fault whose impact amplitude is modulated by the shaft rotation, and identify the lines of the envelope spectrum.

    Solution: (a) \mathrm{BPFO} = 8 \times 15 \times 0.75 = 90 Hz and \mathrm{BPFI} = 8 \times 15 \times 1.25 = 150 Hz. (b) With b Bernoulli(p), g \sim \mathcal N(0, s^2), independent of w: K = \frac{3 p s^4 + 6 p s^2 \sigma^2 + 3 \sigma^4}{(p s^2 + \sigma^2)^2}, which is 3 for p = 1 and tends to 3/p when \sigma \to 0. (c) Multiply the impulse train at 150 Hz by 1 + 0.5 \cos(2\pi \cdot 30\, t): the envelope spectrum shows a line at 150 Hz with sidebands at 120 and 180 Hz, the signature of an inner-race defect passing through the load zone.

12. ★★ Gradient descent on least squares. Let F(\theta) = \frac12 \|A\theta - y\|^2, where the eigenvalues of A^\top A lie in [\mu, L] with \mu > 0. (a) Give the smoothness constant and the step of Theorem 3.9. (b) Show that the error of gradient descent obeys \theta_k - \theta^\star = \big(I - A^\top A / L\big)^k (\theta_0 - \theta^\star), and deduce a geometric rate. (c) With \mu = 1 and L = 100, how many iterations divide the error by 1 000?

    Solution: (a) The gradient A^\top(A\theta - y) is L-Lipschitz, so the step is 1/L. (b) Using A^\top A \theta^\star = A^\top y, the error obeys the linear recursion. The matrix has eigenvalues in [0, 1 - \mu/L], so the error contracts by 1 - \mu/L per step.

    For (c): one needs 0.99^k \leq 10^{-3}, that is k \geq \ln(1000) / (-\ln 0.99) \approx 688 iterations: the condition number L/\mu sets the speed.

13. ★★★ Depth separation in practice (computational). Fit \tau^5 (Theorem 4.13) on [0, 1] with (i) one-hidden-layer ReLU networks of widths 4, 8, 16, 64, and (ii) networks with 5 hidden layers of width 4, trained with Adam on a fine grid. Report the L^1 errors and compare with the bound of Theorem 4.13.

    Hints and expected result: with k = 5, the theorem guarantees an error of at least 1/16 for widths up to 2^3 - 2 = 6, and wide networks need about 31 units to fit exactly. The deep network can represent \tau^5 exactly, but gradient descent often fails to find that solution from a random start: expressivity and trainability are different questions. Discuss initialisation near the exact weights.

14. ★★★ Can a learned detector beat the SNR wall? Train a classifier (logistic regression on the periodogram, or a small CNN) to detect a white Gaussian signal when the noise power varies by ±1 dB between examples. Study its false-alarm rate on noise-only data at the worst noise level, as the SNR decreases towards and below -3.3 dB.

    Hints and expected result: with the notation of Proposition 5.5 and a signal of power P = \gamma \sigma_n^2, choose the noise power \rho \sigma_n^2 - P under \mathcal H_1.

    Below the wall it lies in the uncertainty interval, and the samples then have exactly the law of noise of power \rho\sigma_n^2 under \mathcal H_0. No test, learned or not, can then have P_D > P_{FA} for both configurations. With a BPSK signal the two laws differ, and a detector exploiting cyclostationarity (Definition 5.6) escapes the wall.

Projects and assessment

The Université Paris-Saclay page gives the assessment as “Evaluation Continue non Intégrale” (continuous assessment that is not exclusively continuous), without components or weights; master-mva.com displays no assessment. The proposals below are those of these notes, not official rules. They fit a continuous assessment built on the labs: a project that pairs a classical chain with a learned one, often by reproducing a published paper, reported in writing and orally, and a written exam.

Solo projects

1. [Solo · ≈ 20 h] Paper study: CNN modulation recognition (O’Shea, Corgan and Clancy, 2016). Does a CNN on raw IQ samples beat expert features at every SNR?
   • Work: generate a synthetic set of 6–8 modulations with random carrier offsets and noise (or use the public RadioML 2016.10A set), train the CNN of the paper, and compare with higher-order cumulants fed to gradient boosting.
   • Success criterion: reproduce the shape of the accuracy-versus-SNR curve (near chance at −20 dB, a plateau above 0 dB); confusion matrices at 0 and 18 dB.
   • Deliverables: code, a 4–6 page report, a 10-minute talk; chapters 4 and 5; graded with the rubric below.
2. [Solo · ≈ 20 h] The SNR wall, measured (Tandra and Sahai, 2008). Verify Proposition 5.5 and test which detectors escape it.
   • Work: Monte Carlo study with noise uncertainty of 0.5, 1 and 2 dB, for the energy detector, a cyclic-autocorrelation detector of a BPSK signal (Definition 5.6) and a learned detector.
   • Success criterion: measured walls within 0.5 dB of (\rho^2 - 1)/\rho; one figure of P_D against N at fixed P_{FA} per detector.
   • Deliverables: code, report, talk; chapters 0 and 5 and Exercise 14; graded with the rubric below.
3. [Solo · ≈ 20 h] Envelope analysis versus a 1D CNN on public bearing data. How much of the reported accuracy of learned diagnosis survives an honest split?
   • Data: the Case Western Reserve University bearing data, public, with the benchmark of Smith and Randall (2015) as the classical reference.
   • Work: envelope analysis with the band chosen by spectral kurtosis; a 1D CNN; splits by window, by record and by load.
   • Success criterion: a table of accuracies per split showing the effect of leakage (Note 3.2) and of the load change.
   • Deliverables: code, report, talk; chapters 3, 4 and 9; graded with the rubric below.

Group projects

1. [Group of 2–3 · ≈ 30 h each] A radar electronic-support chain on simulated pulses. When do learned deinterleavers beat histograms?
   • Work: a simulator of 5–20 emitters with constant, jittered and staggered PRIs, RF agility, missed and spurious pulses; DBSCAN plus the PRI transform (Nishiguchi and Kobayashi, 2000) against a transformer that labels pulses.
   • Split: one member builds the simulator and metrics, one the classical chain, one the learned chain.
   • Success criterion: ARI and emitter recall against pulse density; one regime where each approach wins.
   • Deliverables: code, an 8–10 page joint report, a 20-minute talk; chapters 3, 4 and 6; graded with the rubric below, plus an individual question during the talk.
2. [Group of 2–3 · ≈ 30 h each] Few-shot speaker verification on LibriSpeech. How many enrolment utterances does each method need?
   • Data: LibriSpeech (Panayotov, Chen, Povey and Khudanpur, 2015), public, with training and test speakers disjoint.
   • Work: GMM-UBM with MAP adaptation, a small x-vector-like network, and the same network trained on episodes with prototypes; enrolment with 1, 3 and 5 utterances.
   • Split: features and GMM-UBM; embedding networks; trial lists, EER and DET curves.
   • Success criterion: EER and DET curves for 1, 3 and 5 enrolment utterances and the three methods, on at least 40 test speakers, with bootstrap confidence intervals; the number of utterances each method needs to reach an EER of 5 %.
   • Deliverables: code, joint report, talk; chapters 2 and 7; graded with the rubric below, plus an individual question during the talk.
3. [Group of 2–3 · ≈ 30 h each] Paper study: audio inpainting (Adler et al., 2012; Janssen et al., 1986). Reproduce the reconstruction-quality-versus-gap-length curve, then add a learned inpainter.
   • Work: AR interpolation (Proposition 8.3), sparse inpainting with orthogonal matching pursuit in a DCT or Gabor dictionary, and a small convolutional network trained on artificially punctured public speech or music clips.
   • Split: one member per method, shared evaluation code.
   • Success criterion: SNR curves for gaps of 1 to 20 ms for the three methods, plus a short blind listening test.
   • Deliverables: code, joint report, talk; chapters 7.2 and 8; graded with the rubric below, plus an individual question during the talk.

Written exam (proposal)

Two hours, one handwritten A4 sheet and a simple calculator allowed, with a table of the Gaussian tail and of chi-square quantiles provided; four exercises.

1. Question (4 points). Explain the time–frequency trade-off of the STFT, state the uncertainty principle, and choose a representation for (a) bearing impacts ringing at 3 kHz and (b) the formants of speech. Expected: windowed DFTs and the bound \sigma_t \sigma_f \geq 1/(4\pi) (Theorem 2.3); (a) short windows or wavelets, which are fine in time at high frequency; (b) windows of 20–30 ms, or mel spectra.
2. Question (6 points). An energy detector uses N = 2000 samples with P_{FA} = 0.01, so that Q^{-1}(P_{FA}) = 2.326, at an SNR of -12 dB. Compute the threshold and P_D, then the SNR wall for a noise uncertainty of 0.5 dB.
   • Expected: the threshold is about 2104\, \sigma^2, since \sqrt{2000} = 44.7; with \gamma = 0.063, P_D = Q(-0.47) \approx 0.68 (Proposition 5.3).
   • With \rho = 1.122, the wall is (\rho^2 - 1)/\rho = 0.231, that is -6.4 dB (Proposition 5.5).
3. Question (5 points). Prove that among linear filters the matched filter maximises the output SNR, and give the SNR gain of a chirp with BT = 500 sampled at its bandwidth. Expected: output decomposition, noise variance \sigma^2 \|h\|^2, Cauchy–Schwarz with its equality case (Theorem 1.9); gain 10 \log_{10} 500 \approx 27 dB.
4. Question (5 points). A bearing has 10 balls, d/D = 0.15, contact angle 0, and turns at 20 Hz. Compute BPFO and BPFI, describe the envelope-analysis chain, and set a 1 % false-alarm threshold on a Mahalanobis score built from four Gaussian indicators. Expected: \mathrm{BPFO} = 10 \times 10 \times 0.85 = 85 Hz and \mathrm{BPFI} = 115 Hz (Proposition 9.2); band-pass around the resonance, analytic-signal envelope, spectrum; threshold at the 99 % quantile of \chi^2_4, 13.28 (Proposition 9.7).

Grading rubric (proposal)

References

Official references. The course pages on master-mva.com and on the Université Paris-Saclay site list no bibliography for this course (2026–2027). The works below are those cited on this page, together with standard textbooks and surveys for further reading, grouped by topic.

Textbooks

• S. Mallat. A Wavelet Tour of Signal Processing: The Sparse Way, 3rd edition. Academic Press, 2009.
• A. V. Oppenheim and R. W. Schafer. Discrete-Time Signal Processing, 3rd edition. Pearson, 2010.
• S. M. Kay. Fundamentals of Statistical Signal Processing, Volume II: Detection Theory. Prentice Hall, 1998.
• P. Flandrin. Time-Frequency/Time-Scale Analysis. Academic Press, 1999.
• L. R. Rabiner and R. W. Schafer. Theory and Applications of Digital Speech Processing. Pearson, 2011.
• M. A. Richards. Fundamentals of Radar Signal Processing, 2nd edition. McGraw-Hill, 2014.
• R. G. Wiley. ELINT: The Interception and Analysis of Radar Signals. Artech House, 2006.
• R. B. Randall. Vibration-Based Condition Monitoring: Industrial, Automotive and Aerospace Applications. Wiley, 2011.
• I. Goodfellow, Y. Bengio and A. Courville. Deep Learning. MIT Press, 2016.
• T. Hastie, R. Tibshirani and J. Friedman. The Elements of Statistical Learning, 2nd edition. Springer, 2009.
• C. M. Bishop. Pattern Recognition and Machine Learning. Springer, 2006.

Signal processing and representations

• D. Gabor. Theory of communication. Journal of the Institution of Electrical Engineers, Part III, 1946.
• J. W. Cooley and J. W. Tukey. An algorithm for the machine calculation of complex Fourier series. Mathematics of Computation, 1965.
• E. Bedrosian. A product theorem for Hilbert transforms. Proceedings of the IEEE, 1963.
• D. L. Donoho and I. M. Johnstone. Ideal spatial adaptation by wavelet shrinkage. Biometrika, 1994.
• S. B. Davis and P. Mermelstein. Comparison of parametric representations for monosyllabic word recognition in continuously spoken sentences. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1980.
• G. E. Peterson and H. L. Barney. Control methods used in a study of the vowels. Journal of the Acoustical Society of America, 1952.

Machine learning and deep learning

• A. P. Dempster, N. M. Laird and D. B. Rubin. Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society, Series B, 1977.
• M. Ester, H.-P. Kriegel, J. Sander and X. Xu. A density-based algorithm for discovering clusters in large spatial databases with noise. KDD, 1996.
• T. Cover and P. Hart. Nearest neighbor pattern classification. IEEE Transactions on Information Theory, 1967.
• D. P. Kingma and J. Ba. Adam: a method for stochastic optimization. ICLR, 2015.
• G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 1989.
• K. Hornik. Approximation capabilities of multilayer feedforward networks. Neural Networks, 1991.
• M. Leshno, V. Ya. Lin, A. Pinkus and S. Schocken. Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Networks, 1993.
• D. E. Rumelhart, G. E. Hinton and R. J. Williams. Learning representations by back-propagating errors. Nature, 1986.
• S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 1997.
• P. Baldi and K. Hornik. Neural networks and principal component analysis: learning from examples without local minima. Neural Networks, 1989.
• P. Vincent, H. Larochelle, Y. Bengio and P.-A. Manzagol. Extracting and composing robust features with denoising autoencoders. ICML, 2008.
• A. van den Oord, S. Dieleman, H. Zen, K. Simonyan, O. Vinyals, A. Graves, N. Kalchbrenner, A. Senior and K. Kavukcuoglu. WaveNet: a generative model for raw audio. arXiv:1609.03499, 2016.
• A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser and I. Polosukhin. Attention is all you need. NeurIPS, 2017.
• K. He, X. Zhang, S. Ren and J. Sun. Deep residual learning for image recognition. CVPR, 2016.
• D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk and Q. V. Le. SpecAugment: a simple data augmentation method for automatic speech recognition. Interspeech, 2019.
• M. Telgarsky. Benefits of depth in neural networks. COLT, 2016.

Cognitive radio and detection

• J. Neyman and E. S. Pearson. On the problem of the most efficient tests of statistical hypotheses. Philosophical Transactions of the Royal Society A, 1933.
• J. Mitola and G. Q. Maguire. Cognitive radio: making software radios more personal. IEEE Personal Communications, 1999.
• S. Haykin. Cognitive radio: brain-empowered wireless communications. IEEE Journal on Selected Areas in Communications, 2005.
• IEEE Std 802.22-2011. Cognitive wireless regional area networks (RAN): medium access control and physical layer specifications — policies and procedures for operation in the TV bands. IEEE, 2011.
• H. Urkowitz. Energy detection of unknown deterministic signals. Proceedings of the IEEE, 1967.
• R. Tandra and A. Sahai. SNR walls for signal detection. IEEE Journal of Selected Topics in Signal Processing, 2008.
• W. A. Gardner. Exploitation of spectral redundancy in cyclostationary signals. IEEE Signal Processing Magazine, 1991.
• Y. Zeng and Y.-C. Liang. Eigenvalue-based spectrum sensing algorithms for cognitive radio. IEEE Transactions on Communications, 2009.
• T. Yücek and H. Arslan. A survey of spectrum sensing algorithms for cognitive radio applications. IEEE Communications Surveys & Tutorials, 2009.
• T. J. O’Shea, J. Corgan and T. C. Clancy. Convolutional radio modulation recognition networks. Engineering Applications of Neural Networks (EANN), Springer, 2016.
• T. J. O’Shea, T. Roy and T. C. Clancy. Over-the-air deep learning based radio signal classification. IEEE Journal of Selected Topics in Signal Processing, 2018.

Radar emitter identification

• H. K. Mardia. New techniques for the deinterleaving of repetitive sequences. IEE Proceedings F (Radar and Signal Processing), 1989.
• D. J. Milojević and B. M. Popović. Improved algorithm for the deinterleaving of radar pulses. IEE Proceedings F (Radar and Signal Processing), 1992.
• K. Nishiguchi and M. Kobayashi. Improved algorithm for estimating pulse repetition intervals. IEEE Transactions on Aerospace and Electronic Systems, 2000.
• K. Merchant, S. Revay, G. Stantchev and B. Nousain. Deep learning for RF device fingerprinting in cognitive communication networks. IEEE Journal of Selected Topics in Signal Processing, 2018.

Speaker recognition and few-shot learning

• D. A. Reynolds, T. F. Quatieri and R. B. Dunn. Speaker verification using adapted Gaussian mixture models. Digital Signal Processing, 2000.
• N. Dehak, P. J. Kenny, R. Dehak, P. Dumouchel and P. Ouellet. Front-end factor analysis for speaker verification. IEEE Transactions on Audio, Speech, and Language Processing, 2011.
• D. Snyder, D. Garcia-Romero, G. Sell, D. Povey and S. Khudanpur. X-vectors: robust DNN embeddings for speaker recognition. ICASSP, 2018.
• B. Desplanques, J. Thienpondt and K. Demuynck. ECAPA-TDNN: emphasized channel attention, propagation and aggregation in TDNN based speaker verification. Interspeech, 2020.
• J. Deng, J. Guo, N. Xue and S. Zafeiriou. ArcFace: additive angular margin loss for deep face recognition. CVPR, 2019.
• A. Nagrani, J. S. Chung and A. Zisserman. VoxCeleb: a large-scale speaker identification dataset. Interspeech, 2017.
• V. Panayotov, G. Chen, D. Povey and S. Khudanpur. LibriSpeech: an ASR corpus based on public domain audio books. ICASSP, 2015.
• J. Snell, K. Swersky and R. Zemel. Prototypical networks for few-shot learning. NeurIPS, 2017.
• F. Schroff, D. Kalenichenko and J. Philbin. FaceNet: a unified embedding for face recognition and clustering. CVPR, 2015.
• C. Finn, P. Abbeel and S. Levine. Model-agnostic meta-learning for fast adaptation of deep networks. ICML, 2017.

Audio restoration

• S. Boll. Suppression of acoustic noise in speech using spectral subtraction. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1979.
• Y. Ephraim and D. Malah. Speech enhancement using a minimum mean-square error short-time spectral amplitude estimator. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1984.
• D. Griffin and J. Lim. Signal estimation from modified short-time Fourier transform. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1984.
• A. J. E. M. Janssen, R. N. J. Veldhuis and L. B. Vries. Adaptive interpolation of discrete-time signals that can be modeled as autoregressive processes. IEEE Transactions on Acoustics, Speech, and Signal Processing, 1986.
• A. Adler, V. Emiya, M. G. Jafari, M. Elad, R. Gribonval and M. D. Plumbley. Audio inpainting. IEEE Transactions on Audio, Speech, and Language Processing, 2012.
• A. Jansson, E. Humphrey, N. Montecchio, R. Bittner, A. Kumar and T. Weyde. Singing voice separation with deep U-Net convolutional networks. ISMIR, 2017.
• D. Stoller, S. Ewert and S. Dixon. Wave-U-Net: a multi-scale neural network for end-to-end audio source separation. ISMIR, 2018.
• A. Défossez, G. Synnaeve and Y. Adi. Real time speech enhancement in the waveform domain. Interspeech, 2020.
• J. Le Roux, S. Wisdom, H. Erdogan and J. R. Hershey. SDR — half-baked or well done? ICASSP, 2019.
• C. H. Taal, R. C. Hendriks, R. Heusdens and J. Jensen. An algorithm for intelligibility prediction of time–frequency weighted noisy speech. IEEE Transactions on Audio, Speech, and Language Processing, 2011.
• ITU-T Recommendation P.862. Perceptual evaluation of speech quality (PESQ). International Telecommunication Union, 2001.

Predictive maintenance

• J. Antoni. The spectral kurtosis: a useful tool for characterising non-stationary signals. Mechanical Systems and Signal Processing, 2006.
• R. B. Randall and J. Antoni. Rolling element bearing diagnostics — a tutorial. Mechanical Systems and Signal Processing, 2011.
• W. A. Smith and R. B. Randall. Rolling element bearing diagnostics using the Case Western Reserve University data: a benchmark study. Mechanical Systems and Signal Processing, 2015.
• A. Saxena, K. Goebel, D. Simon and N. Eklund. Damage propagation modeling for aircraft engine run-to-failure simulation. International Conference on Prognostics and Health Management, 2008.
• Y. Lei, B. Yang, X. Jiang, F. Jia, N. Li and A. K. Nandi. Applications of machine learning to machine fault diagnosis: a review and roadmap. Mechanical Systems and Signal Processing, 2020.
• B. Schölkopf, J. C. Platt, J. Shawe-Taylor, A. J. Smola and R. C. Williamson. Estimating the support of a high-dimensional distribution. Neural Computation, 2001.
• F. T. Liu, K. M. Ting and Z.-H. Zhou. Isolation forest. IEEE International Conference on Data Mining (ICDM), 2008.

Depth, invariance and hybrid models

• S. Mallat. Group invariant scattering. Communications on Pure and Applied Mathematics, 2012.
• J. Andén and S. Mallat. Deep scattering spectrum. IEEE Transactions on Signal Processing, 2014.
• M. Ravanelli and Y. Bengio. Speaker recognition from raw waveform with SincNet. IEEE Spoken Language Technology Workshop (SLT), 2018.
• N. Zeghidour, O. Teboul, F. de Chaumont Quitry and M. Tagliasacchi. LEAF: a learnable frontend for audio classification. ICLR, 2021.
• K. Gregor and Y. LeCun. Learning fast approximations of sparse coding. ICML, 2010.
• R. Eldan and O. Shamir. The power of depth for feedforward neural networks. COLT, 2016.
• T. Poggio, H. Mhaskar, L. Rosasco, B. Miranda and Q. Liao. Why and when can deep-but-not-shallow networks avoid the curse of dimensionality: a review. International Journal of Automation and Computing, 2017.

Research directions

• A. Baevski, Y. Zhou, A. Mohamed and M. Auli. wav2vec 2.0: a framework for self-supervised learning of speech representations. NeurIPS, 2020.
• W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov and A. Mohamed. HuBERT: self-supervised speech representation learning by masked prediction of hidden units. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2021.
• S. Chen and coauthors. WavLM: large-scale self-supervised pre-training for full stack speech processing. IEEE Journal of Selected Topics in Signal Processing, 2022.
• W. J. Scheirer, A. de Rezende Rocha, A. Sapkota and T. E. Boult. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013.
• A. Bendale and T. E. Boult. Towards open set deep networks. CVPR, 2016.
• C. Guo, G. Pleiss, Y. Sun and K. Q. Weinberger. On calibration of modern neural networks. ICML, 2017.
• A. N. Angelopoulos and S. Bates. Conformal prediction: a gentle introduction. Foundations and Trends in Machine Learning, 2023.
• Y. Ganin, E. Ustinova, H. Ajakan, P. Germain, H. Larochelle, F. Laviolette, M. Marchand and V. Lempitsky. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 2016.
• V. Monga, Y. Li and Y. C. Eldar. Algorithm unrolling: interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 2021.
• N. Shlezinger, J. Whang, Y. C. Eldar and A. G. Dimakis. Model-based deep learning. Proceedings of the IEEE, 2023.
• M. Raissi, P. Perdikaris and G. E. Karniadakis. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 2019.
• Z. Kong, W. Ping, J. Huang, K. Zhao and B. Catanzaro. DiffWave: a versatile diffusion model for audio synthesis. ICLR, 2021.
• H. Chung, J. Kim, M. T. McCann, M. L. Klasky and J. C. Ye. Diffusion posterior sampling for general noisy inverse problems. ICLR, 2023.
• B. Jacob, S. Kligys, B. Chen, M. Zhu, M. Tang, A. Howard, H. Adam and D. Kalenichenko. Quantization and training of neural networks for efficient integer-arithmetic-only inference. CVPR, 2018.

Software

• F. Pedregosa and coauthors. Scikit-learn: machine learning in Python. Journal of Machine Learning Research, 2011.
• A. Paszke and coauthors. PyTorch: an imperative style, high-performance deep learning library. NeurIPS, 2019.
• P. Virtanen and coauthors. SciPy 1.0: fundamental algorithms for scientific computing in Python. Nature Methods, 2020.