End-of-year projects

Contents

• What these projects are for
• Common rules
• Projects at a glance
• Learning theory route: kernel regime or feature learning?
• Vision and imaging route: from a noisy sensor to a 3D model
• Generative AI and LLM route: code generation that checks itself
• Track Santé: from brain MRI to shape statistics
• Track I&I: a surrogate with honest uncertainty
• Track RR: a reproducible edition of a classic paper
• Graphs, time series and extremes: anomalies in a sensor network
• Audio: separating and describing music
• Responsible learning: auditing a decision model
• Every course’s own projects

What these projects are for

The MVA year has two halves: about seven months of courses, then a research internship of at least four months. Most courses are assessed by a project, very often the study and reimplementation of a research paper, plus a written exam. Each project stays inside one course.

The projects below are integrative: each one needs three to five courses of the same route or track. They are sized for a group of two to four students over four to six weeks, for example during the second period or between the end of the courses and the start of the internship.

They rehearse what the internship asks for: a research question, a baseline, a reproducible protocol, a written report and a defence.

These are proposals of these notes, not official subjects. The official assessment of each course is quoted in the fiche of its page. The validation rules of the year are on the home page.

Common rules

Charter (week 0)

One page, written before any code:

• the question, in one sentence, and the hypothesis the team expects to confirm or refute;
• the baseline that the project must beat or explain, and the metric that decides;
• the data, its licence, and the split used for every number in the report;
• the role of each member and the courses each member brings;
• the two main risks and the fallback if they happen.

Reproducibility

• One repository with an environment file, fixed seeds, and one command that reproduces every figure of the report.
• Every number in the report comes with the seed or the confidence interval it was computed with.
• The practices of Fundamentals of Reproducible Research apply: version control from day one, assertions on invariants, a short README that a stranger can follow.

Deliverables

• A report of about eight pages in article format: introduction, method, experiments, limitations.
• The code and a short demo, in the style of an IPOL online demo when the project allows it.
• A fifteen-minute defence with questions.
• A one-page review of another team’s report, as in the reproducible research project: every student is both author and reviewer.

Rubric (proposal)

Projects at a glance

Each project can be done by a student who has not taken every course listed: the charter names which member brings which course, and the missing background is read from the corresponding page of these notes.

Learning theory route: kernel regime or feature learning?

Courses. Convex Optimization · Theoretical Foundations of Deep Learning · Kernel Methods · Large Random Matrices · optionally Sequential Learning.

Question. On which synthetic tasks does a trained two-layer network behave like kernel ridge regression with its neural tangent kernel, and on which does it learn features that the kernel cannot?

Plan.

• Week 1: data models where the answer is known, such as a single-index target in high dimension, and a random-features model as the baseline.
• Week 2: kernel ridge regression with the NTK and with the random-features kernel; the random-matrix prediction of their test error as a function of the sample-to-dimension ratio, including the double-descent peak.
• Weeks 3–4: train the network by gradient descent at several widths and initialisation scales; compare its test error and its learned first layer with the kernel predictions.
• Weeks 5–6: write up which regime holds where, and why.

Deliverables and success criterion. The predicted and measured test-error curves agree within the confidence band in the lazy regime. The report identifies at least one setting where the trained network clearly beats its kernel, and explains it.

Variants. Solo: only the kernel and random-matrix part. Group of three: add an online-learning view of the training dynamics.

Vision and imaging route: from a noisy sensor to a 3D model

Courses. Image Denoising · Mathematical Models for Digital Photography · Deep Learning for Image Restoration and Synthesis · 3D Computer Vision · Point Clouds and 3D Modelling.

Question. How much does each restoration step (noise model, denoising, demosaicking or super-resolution) change the accuracy of a stereo reconstruction downstream?

Plan.

• Week 1: simulate realistic sensor noise (Poisson–Gaussian, then demosaicking) on rectified stereo pairs with ground-truth disparity, such as the Middlebury stereo data sets.
• Weeks 2–3: compare classical denoisers (non-local means, a patch-based Bayesian denoiser) with a trained CNN denoiser, at equal noise levels.
• Week 4: compute disparities with a classical matcher and convert them to point clouds.
• Week 5: measure the reconstruction error against ground truth, per restoration pipeline.
• Week 6: report and demo.

Deliverables and success criterion. A table of disparity error and point-cloud error per pipeline, with seeds. The report states which image-quality metric best predicts the 3D error, and how far each pipeline is from the noise-free reconstruction.

Variants. Solo: the denoising-to-disparity part only. Group of four: add surface reconstruction from the point cloud and a perceptual study.

Generative AI and LLM route: code generation that checks itself

Courses. Foundations of Large Language Models · Large Language Models for Code and Proof · Reinforcement Learning · Training and Deploying Large-Scale Models.

Question. For a small open code model, how much does feedback from executing tests improve the pass rate, compared with simply sampling more candidates?

Plan.

• Week 1: a small open-weight code model, the HumanEval and MBPP benchmarks, and the unbiased pass@k estimator as the metric.
• Week 2: baselines: greedy decoding, temperature sampling with k candidates, and selection by test execution.
• Weeks 3–4: fine-tune with rewards from executing unit tests, using a policy-gradient method with a KL penalty to the initial model, on a training split disjoint from the benchmarks.
• Week 5: compute-matched comparison: the same budget in floating-point operations spent on sampling or on fine-tuning.
• Week 6: report, including contamination checks.

Deliverables and success criterion. pass@1 and pass@10 with confidence intervals, before and after fine-tuning, and a compute-matched comparison. The report says whether fine-tuning beats best-of-k sampling at equal compute.

Variants. Solo: the sampling and selection study only. Group of four: add a proof-assistant task where the checker is the proof kernel.

Track Santé: from brain MRI to shape statistics

Courses. Medical Image Analysis · Deep Learning for Medical Imaging · Geometric Data Analysis · Geometry and Shape Spaces · Biostatistics.

Question. Does the shape of a subcortical structure, beyond its volume, differ between groups of subjects in an open brain MRI data set, and how sensitive is the conclusion to the segmentation method?

Plan.

• Weeks 1–2: segment the structure with a classical atlas-based method and with a trained U-Net, on an open data set such as OASIS or a task of the Medical Segmentation Decathlon.
• Week 3: build surfaces or point sets from both segmentations and register them to a template.
• Week 4: shape statistics: a mean shape and principal modes of deformation in a shape space.
• Week 5: a group comparison with a proper statistical test and correction for age, and the same test on volumes alone.
• Week 6: report.

Deliverables and success criterion. Dice scores of both segmentations, the shape modes, and the p-values of the shape test and the volume test under both segmentations. The report states whether the conclusion survives a change of segmentation method.

Variants. Solo: one segmentation method and the volume-versus-shape test. As the paired project with a physician: the clinical question comes from the physician.

Track I&I: a surrogate with honest uncertainty

Courses. Learning Physical Phenomena · Statistical Learning for Geosciences · Machine Learning for Time Series · Bayesian Machine Learning.

Question. Can a cheap surrogate of a physical simulator answer an engineering question, with error bars that are actually calibrated?

Plan.

• Week 1: a simulator the team writes itself, for example a one-dimensional heat or advection–diffusion equation with uncertain coefficients, and a quantity of interest.
• Week 2: a design of experiments and three surrogates: Gaussian-process regression, a reduced-order model by proper orthogonal decomposition, and a small neural network.
• Week 3: uncertainty propagation: the distribution of the quantity of interest and its Sobol sensitivity indices.
• Week 4: calibration of the surrogates’ intervals on fresh simulations.
• Week 5: an inverse problem: estimate the coefficients from noisy observations of the solution over time.
• Week 6: report.

Deliverables and success criterion. Coverage of the 90 % intervals between 85 % and 95 % on fresh simulations, the Sobol indices with error bars, and the posterior of the inverse problem. The report gives the cost of each surrogate per evaluation.

Variants. Solo: one surrogate and the forward propagation. Group of three: add adaptive enrichment of the design.

Track RR: a reproducible edition of a classic paper

Courses. Fundamentals of Reproducible Research · Reproducible Research Project · Sub-Pixel Image Processing · Detection Theory.

Question. Does an independent reimplementation reproduce the published results of a classic image-processing method, and where does it differ?

Plan.

• Week 1: choose a paper whose method fits the courses, for example the LSD line segment detector of Grompone von Gioi, Jakubowicz, Morel and Randall (IEEE TPAMI, 2010), which rests on the a contrario detection theory of the Detection Theory course.
• Weeks 2–3: reimplement from the paper alone, with assertions on every invariant, then compare with the authors’ reference code.
• Week 4: reproduce the paper’s figures and test the claims on new images, including the false-detection guarantee under pure noise.
• Weeks 5–6: write the article in IPOL format, with pseudo-code that matches the code line by line, and a demo.

Deliverables and success criterion. A complete IPOL-style pre-print with code and demo. The number of false detections on noise images agrees with the theoretical bound, and every difference with the published results is explained.

Variants. Solo: reimplementation and reproduction without the demo. Group of three: two independent implementations compared with each other.

Graphs, time series and extremes: anomalies in a sensor network

Courses. Graphs in Machine Learning · Machine Learning for Time Series · Statistical Learning with Extreme Values · Sequential Learning · optionally ALTEGRAD.

Question. Does using the graph of sensors help detect rare traffic anomalies, compared with treating each sensor’s time series alone?

Plan.

• Week 1: a public road-traffic sensor data set with its sensor graph, such as METR-LA, and a set of injected anomalies with known times and locations.
• Week 2: per-sensor baselines: forecasting residuals and their thresholds.
• Week 3: graph methods: graph-signal smoothness, a graph Laplacian filter, and a small graph neural network forecaster.
• Week 4: thresholds from extreme-value theory (peaks over a threshold with a fitted generalised Pareto tail) instead of empirical quantiles.
• Week 5: an online version that adapts its threshold sequentially, with a bound on false alarms.
• Week 6: report.

Deliverables and success criterion. Detection rate and false-alarm rate at fixed budgets, per method, with intervals over injected anomalies. The report states whether the graph helps, and whether extreme-value thresholds control the false-alarm rate better than empirical quantiles.

Variants. Solo: one series model, one graph model and the extreme-value threshold. Group of three: the full online study.

Audio: separating and describing music

Courses. Audio Signal Processing: Time–Frequency Analysis · Audio Signal Analysis, Indexing and Transformations · Algorithms for Speech and NLP · Deep Learning.

Question. How far does a learned time–frequency mask improve on non-negative matrix factorisation for separating the voice from music, and does the separated voice transcribe better?

Plan.

• Week 1: the MUSDB18 data set and the signal-to-distortion ratio as the metric; short-time Fourier transform settings justified by the time–frequency uncertainty.
• Week 2: non-negative matrix factorisation with a supervised dictionary.
• Weeks 3–4: a small mask-estimation network on spectrograms; ideal masks as an upper bound.
• Week 5: run an open speech recogniser on the mixture, the separated voice and the clean voice, and compare word error rates on the lyrics.
• Week 6: report and listening demo.

Deliverables and success criterion. Signal-to-distortion ratios per method with intervals over tracks, the ideal-mask upper bound, and the word error rates. The report explains where the learned mask fails.

Variants. Solo: the separation part only. Group of three: add a music-indexing task on the separated stems.

Responsible learning: auditing a decision model

Courses. Responsible Machine Learning · Multimodal Explainable AI · Bayesian Machine Learning · Turing Seminar · with Regulating AI for the legal frame.

Question. Is a model trained on public census data fair, private and explainable enough for a stated use, and what does each fix cost in accuracy?

Plan.

• Week 1: the folktables income task (Ding, Hardt, Miller and Schmidt, NeurIPS 2021), a stated use case, and a gradient-boosting baseline.
• Week 2: group-fairness metrics, their incompatibilities, and one mitigation.
• Week 3: a differentially private version of the model and its accuracy–privacy curve.
• Week 4: explanations (Shapley values and counterfactuals) and a test of their stability across retrainings.
• Week 5: calibrated uncertainty with a Bayesian or conformal method, and an abstention rule.
• Week 6: an audit report written for a non-technical reader, with the regulatory classification of the use case.

Deliverables and success criterion. A table of accuracy, fairness gaps, privacy budget and explanation stability for each version of the model, and the audit report. The report states which fixes are compatible and which trade-off it recommends.

Variants. Solo: fairness and explanations only. Group of three: the full audit.

Every course’s own projects

Every scientific course page of these notes ends with a Projects and assessment section: solo and group subjects tied to that course alone, at least one of them the study of a research paper, a written-exam proposal with expected answers, and a rubric. Start there to prepare a project above, for example in Kernel Methods, Image Denoising or Biostatistics.

The research internship is the natural continuation of these projects: the same skills, at the scale of four to six months.