Optimization and Responsible AI for Sustainability

Contents

• Introduction and overview
• 0. Refresher: optimization, duality, complexity
• 1. Multi-objective optimization and the Pareto front
• 2. Optimizing under resource constraints
• 3. Model frugality: distillation, quantization, pruning
• 4. Sustainable decision support: linear and mixed integer programming
• 5. Case studies: data centers and electric vehicles
• 6. Measuring the footprint: energy, PUE, carbon accounting
• 7. Limits, rebound effect and greenwashing
• 8. Course research notes — ORAILIX (Vanier, Matin)
• Exercises
• Projects and assessment
• References

Introduction and overview

Audience: an engineer who ships models and must trade off performance, cost and carbon without greenwashing. Empirical first: every chapter starts from a situation that goes wrong in practice — a “green model” whose training is left off the balance sheet, a Pareto front ignored in favour of one magic KPI — and only then names the tool that repairs it: multi-objective optimization, linear and integer programming, carbon-aware scheduling, measurement protocols.

The module is the responsible-AI compass of the program: decisions backed by numbers, not by slogans.

What you will be able to do

• Formulate an AI problem as a multi-objective trade-off and explore the Pareto front.
• Integrate resource constraints — compute, energy, budget — into the very formulation of a problem.
• Make a model frugal through distillation and quantization.
• Model sustainable decision support in linear and integer linear programming.
• Quantify the emissions of an AI system with a defensible methodology.
• Analyze real case studies: data centers, electric vehicles.
• Identify rebound effects and greenwashing in a technical discourse, and dismantle it argument by argument.

Official objectives and syllabus

The course changed code and period. In the 2026–2027 catalogue, “Optimisation and Responsible AI” carries the code CSC_51076_EP (Programme d’approfondissement, Computer Science) and is taught in Period 1, with a weekly lecture on Wednesday 08:30–10:30 followed by a tutorial 10:45–12:45; Sonia Vanier is listed by École Polytechnique and Hossein Nick Zinat Matin on a Télécom Paris listing. The former code CSC_52073_EP (Period 2, 2025–2026) is no longer in the 2026–2027 LLGA 1 – S2 list, and the new code is not in the LLGA lists either: these notes are kept for reference.

The last sheet published under the former code (SynapseS 2025–2026) is minimal: no description, no learning objectives, no detailed syllabus, no prerequisites and no language are communicated there. The table below therefore reports the available official fields — most of them “not communicated”; the content of this module (Pareto front, resource constraints, frugality, ILP, greenwashing) extends the official title without matching any published official syllabus.

Where this course fits in the program

Notes kept for reference: formerly taught in Period 2 (CSC_52073_EP), the course moves to Period 1 in 2026–2027 (CSC_51076_EP) and is not in the LLGA lists; the last official sheet (2025–2026) lists no prerequisites. Chapter 0 rests on same-period material: Machine Learning, ch. 0 (convexity; 0.4 Lagrangian duality and KKT) and ch. 3 (the SVM dual), and Refresher in Computer Science, ch. 7 (NP-hardness). The distillation lab assumes the PyTorch training of CNNs and MLPs from Deep Learning (P1, ch. 3–4), and the scaling laws of ch. 1 are those of Emerging ML, ch. 7.

Related, later period: Optimization for AI (P2) turns the convexity and duality of chapter 0 into algorithms; Deep Learning (P2 variant) makes compute budgets a primary quantity; and ML Ops and LLM Engineering Principles (M2, ch. 10) places frugality and green ops at the heart of ML system deployment.

Course outline

• Refresher: optimization, duality, complexity — the optimization toolbox reloaded for use under constraints.
• Multi-objective optimization and the Pareto front — thinking in trade-offs rather than a single score, and representing the arbitrages.
• Optimizing under resource constraints — integrating energy, compute and cost into the problem, not as an afterthought.
• Model frugality: distillation, quantization — reducing a model’s footprint while preserving the essence of its performance.
• Sustainable decision support: LP and ILP — linear and integer linear programming for concrete, auditable decisions.
• Case studies: data centers, electric vehicles — real deployments where the trade-offs are genuinely quantified.
• Measuring the footprint: energy, PUE, carbon accounting — a measurement protocol, operational and embodied emissions, and honest error bars.
• Limits, rebound effect and greenwashing — what optimization does not solve, and the discourses that must be dismantled.
• Course research notes — ORAILIX (Vanier, Matin) — the research work of the ORAILIX team (LIX), read with the module’s tools.
• Exercises — formulations and trade-offs to defend with figures in hand.
• References — the methodological sources and the case studies.

Session plan

36 h in class (announced) = 9 sessions of 4 h. The 36 h are the programme value quoted for this course (the SynapseS sheets display no teaching hours), and they correspond to nine of the weekly 4 h slots of the 2026–2027 timetable (lecture on Wednesday 08:30–10:30, then tutorial 10:45–12:45); no session-by-session programme is published, so the split below is our proposal for covering these notes in that time. Level 1 = foundations (reviews, definitions, first worked examples), 2 = core methods, 3 = advanced or open questions; the level never goes down from one session to the next.

Self-study, outside the 36 h (about 2 h in total): ch. 8 Course research notes — ORAILIX (Vanier, Matin) (≈ 2 h).

How to study this module

• Return to chapter 0 whenever duality or complexity become fuzzy: the whole module uses them continuously.
• For every emission or consumption figure, ask for the methodology: scope, units, source — this is the first weapon against greenwashing.
• In the exercises, explicitly separate objectives and constraints before coding anything.
• Connect frugality (distillation, quantization) to the courses where the same levers return under other names, all in a later period: Deep Learning (P2 variant), ch. 5–6, and Large Language Models, 4.3.
• Play devil’s advocate systematically: for every proposed solution, look for the rebound effect it triggers.
• Study the cases (data centers, electric vehicles) like a professional: what data, what actors, what real trade-offs.

0. Refresher: optimization, duality, complexity

1. Multi-objective optimization and the Pareto front

A loss such as “error + λ · energy” looks like a sensible way to obtain a frugal model, but the single number it returns hides the trade-off that a decision maker needs to see. This chapter replaces the single score by the Pareto front: the set of solutions that cannot be improved on one objective without being worsened on another.

“Maximize accuracy”, “minimize energy”, “minimize latency”: three objectives with no common unit. Adding them naively — a “magic loss” — hides the trade-offs; the Pareto formalism makes them explicit and negotiable.

import numpy as np
import matplotlib.pyplot as plt

# example: two models (growing size), fictitious but realistic measurement
E = np.array([0.4, 1.1, 2.6, 5.0, 9.5, 18.0])      # kWh (measured at the wall)
A = np.array([71.2, 78.9, 84.1, 87.0, 88.3, 89.0]) # accuracy % on a validation set

# Pareto front (minimize E, maximize A): keep the non-dominated points
keep = [True]*len(E)
for i in range(len(E)):
    for j in range(len(E)):
        if E[j] <= E[i] and A[j] >= A[i] and (E[j], A[j]) != (E[i], A[i]):
            keep[i] = False
print("front points:", list(zip(E[keep], A[keep])))
# marginal gain of acc per kWh -- the "elbow" of the front
print("d.acc/dE:", np.diff(A)/np.diff(E))
plt.loglog(E, 100 - A, "o-"); plt.xlabel("E (kWh)"); plt.ylabel("error (%)")
plt.show()

2. Optimizing under resource constraints

This chapter puts resource limits inside the optimization problem instead of checking them afterwards. The resources are GPU hours, watts, megabytes of memory and — the module’s signature angle — carbon, which depends on when and where one computes. The electric grid is not uniformly green: its carbon intensity varies by a factor of up to 10 depending on the hour and the region, which leaves a large margin of manoeuvre to a scheduler.

import numpy as np
c = np.array([ 85, 72, 60, 55, 58,110,220,310,380,340,260,160,
              95, 70, 55, 45, 38, 42,120,280,420,470,390,240])  # gCO2/kWh, 0h..23h
c = np.concatenate([c, c])            # two identical days (simplification)
T, P, pue = 6, 0.4, 1.1

best = min(range(48 - T + 1), key=lambda s: c[s:s+T].sum())
worst = max(range(48 - T + 1), key=lambda s: c[s:s+T].sum())
emi = lambda s: P*pue*c[s:s+T].sum() / 1000               # kg CO2eq
print(f"best slot {best}h-{best+T}h: {emi(best):.2f} kg")
print(f"worst slot {worst}h-{worst+T}h: {emi(worst):.2f} kg")
print(f"relative gain: {1 - emi(best)/emi(worst):.0%}")
# expected: best 12h-18h, about 0.15 kg; worst 18h-24h, about 0.84 kg;
# a gain of about 82% (a factor 5.6) between the solar afternoon and the evening peak

3. Model frugality: distillation, quantization, pruning

The cheapest kg CO₂ is the one not emitted: reducing the model’s cost at equal quality is the first lever of sustainable AI. Three families, complementary, all “Pareto-active” (ch. 1).

import numpy as np
rng = np.random.default_rng(0)

def quantize_uniform(x, bits=8):
    xmin, xmax = x.min(), x.max()
    s = (xmax - xmin) / (2**bits - 1)
    q = np.clip(np.round((x - xmin)/s), 0, 2**bits - 1).astype(np.int64)
    return q, q*s + xmin                       # codes + dequantize

W = rng.normal(size=(64, 128)) * 0.03          # a trained layer (fictitious)
for bits in (16, 8, 4, 2):
    _, Wq = quantize_uniform(W, bits)
    err = np.abs(Wq - W).max()
    print(f"{bits:2d} bits: max error {err:.2e} (step/2 = "
          f"{((W.max()-W.min())/(2**bits-1))/2:.2e})")
# the error exactly follows the step/2 of proposition 3.3; at 4 bits it becomes
# perceptible -> that is where QAT (simulated fine-tuning) takes over.

4. Sustainable decision support: linear and mixed integer programming

The core business of operations research: turning a complex decision into a mathematical model that a solver drives to the optimum — not “nearly”. This is the toolbox of the module’s instructor (ch. 8) and the language of the case studies (ch. 5).

import pulp
# knapsack: maximize value under capacity, binary variables
v = {0: 6, 1: 5, 2: 5, 3: 1};  w = {0: 5, 1: 5, 2: 5, 3: 1};  C = 10
prob = pulp.LpProblem("knapsack", pulp.LpMaximize)
x = {i: pulp.LpVariable(f"x{i}", cat="Binary") for i in v}
prob += pulp.lpSum(v[i]*x[i] for i in v)
prob += pulp.lpSum(w[i]*x[i] for i in v) <= C
prob.solve(pulp.PULP_CBC_CMD(msg=0))
print([(i, x[i].value()) for i in v], "=", pulp.value(prob.objective))
# "sustainability" variant: two objectives (value, carbon) -> epsilon-constraint
# prob += carbon(x) <= budget; maximize value  (see exercise 8)

5. Case studies: data centers and electric vehicles

This chapter applies the LP and ILP toolbox of ch. 4 to two deployments where carbon, cost and physical capacity are traded off for real. In both cases the model fits on one page, and the dual variables carry the economic message.

import numpy as np
from scipy.optimize import linprog

# 3 vehicles, 8 hourly slots from 18h to 2h, transformer capacity 11 kW
T, K = 8, 11.0
c = np.array([300, 380, 320, 140, 60, 55, 52, 58])    # gCO2/kWh per slot
veh = [ (0, 8, 20.0, 7.0), (0, 8, 28.0, 7.0), (2, 8, 15.0, 3.5) ]  # (a, d, E, p): slots a <= t < d
n = len(veh) * T                                      # variable x[i, t] = idx i*T + t
cost = np.array([c[idx % T] for idx in range(n)])     # minimize sum c_t x_it

A_ub, b_ub = [], []                                   # capacity per slot
for t in range(T):
    row = [1.0 if idx % T == t else 0.0 for idx in range(n)]
    A_ub.append(row); b_ub.append(K)
A_eq, b_eq = [], []                                   # energy need
for i, (a, d, E, p) in enumerate(veh):
    row = [1.0 if (idx // T == i and a <= idx % T < d) else 0.0 for idx in range(n)]
    A_eq.append(row); b_eq.append(E)
bounds = [(0, p) for (a, d, E, p) in veh for _ in range(T)]
res = linprog(cost, A_ub=np.array(A_ub), b_ub=b_ub, A_eq=np.array(A_eq),
              b_eq=b_eq, bounds=bounds, method="highs")
x = res.x.reshape(len(veh), T)
print("optimal emissions (kg CO2eq):", round(res.fun / 1000, 2))
print(np.round(x, 1))   # the cars wait for the low-carbon slots
print(res.ineqlin.marginals)   # transformer prices (negative: scipy's sign convention)

6. Measuring the footprint: energy, PUE, carbon accounting

Every optimization of the previous chapters minimizes a number — kWh, gCO₂eq — that someone must first measure. This chapter turns that number into a protocol: what is measured, at which boundary, with which conversion factors and with what uncertainty, so that two claims can be compared and audited (ch. 7).

import numpy as np
rng = np.random.default_rng(0)
n = 100_000
E_it = 244.8 * rng.lognormal(0, 0.10, n)     # kWh, +-10% (counters, averaging)
pue  = 1.2   * rng.lognormal(0, 0.05, n)     # +-5%
ci   = 0.400 * rng.lognormal(0, 0.30, n)     # kgCO2eq/kWh, +-30% (annual average)
C = E_it * pue * ci
lo, med, hi = np.percentile(C, [5, 50, 95])
print(f"median {med:.0f} kg, 90% interval [{lo:.0f}, {hi:.0f}] kg")
print(f"relative spread {np.std(np.log(C)):.2f} vs quadrature "
      f"{np.sqrt(0.10**2 + 0.05**2 + 0.30**2):.2f}")   # prop. 6.4

7. Limits, rebound effect and greenwashing

The previous chapters optimize and measure; this one asks what those numbers can and cannot prove. It lists the classic pitfalls of carbon accounting, models the rebound effect that can cancel an efficiency gain, and ends with an audit grid for “green AI” claims.

8. Course research notes — ORAILIX (Vanier, Matin)

Team update: on the 2025–2026 sheet, the CSC_52073_EP course unit “Optimisation and Responsible AI” had two coordinator-instructors, Hossein Nick Zinat Matin and Sonia Vanier (occurrence coordinator: S. Vanier). In 2026–2027 the course is CSC_51076_EP, taught in Period 1; Sonia Vanier is listed by École Polytechnique and Hossein Nick Zinat Matin on a Télécom Paris listing. The works presented below are those of Sonia Vanier’s ORAILIX team.

Exercises

1. ★ Pareto front by hand. Objectives F_1(x)=(x-1)^2+1, F_2(x)=(x+1)^2+1, x \in [-3,3].

   1. Plot the attainable set in the (F_1,F_2) plane and the front.

   2. Show that x=-2 and x=2 are dominated.

   3. For w=(1,1), solve the scalarization and verify that it gives x=0.

   4. On the objectives F_1(x)=x, F_2(x)=1-x^2 of prop. 1.3, show that x=0.5 is Pareto-optimal but is not the minimizer of any weighted scalarization w>0.

   Solution: (c) F_w = 2x^2+4 minimal at 0; (d) F_w is concave in x, hence minimal at the endpoints, see prop. 1.3.

2. ★ Quantization error propagation. A two-layer network, \mathrm{Lip}_1 = 3, \mathrm{Lip}_2 = 2, 8-bit quantization over a weight domain [-0.5, 0.5].

   1. Compute the step with proposition 3.3 and check that \Delta_1 = \Delta_2 = 1/255 \approx 2^{-8}.

   2. Apply proposition 3.3 and bound the output error.

   3. Redo at 4 bits (\Delta = 1/15); for a stack of L identical layers with \mathrm{Lip} = 2, from what depth L does the bound exceed 0.1? Conclude on the necessity of QAT for int4 deep networks.

   Solution: (b) \frac{\Delta}{2}(\mathrm{Lip}_2 + 1) = \frac{3}{510} \approx 0.006 ; and for (c) the bound is \frac{\Delta}{2}(2^L - 1) = \frac{2^L - 1}{30}, which exceeds 0.1 as soon as 2^L > 4, i.e. from L = 3 on.

3. ★ Branch and bound by hand. Binary knapsack: capacity C = 10, items (v,w) \in \{(7,6), (5,5), (4,4), (2,1)\}. (a) Solve the greedy relaxation at the root. (b) Branch on the most fractional variable, draw the complete tree, indicate for each node: bound (relaxation), pruned or explored, and why. (c) Compare with the PuLP solution (ch. 4); (d) how many nodes if one branches on the least fractional variable? Conclusion on branching heuristics.

4. ★★ Chebyshev. Still on F_1(x)=x, F_2(x)=1-x^2, x \in [0,1] (ideal point z^\star=(0,0)).

   1. Verify that every point is Pareto-optimal.

   2. Show that g_w(x) = \max\{w_1 F_1(x),\, w_2 F_2(x)\} attains its minimum at the equilibrium w_1 x = w_2(1-x^2) (one grows, the other decreases).

   3. Deduce that every point of the front is produced by some w>0 — a complete contrast with prop. 1.3 — and compute w_1/w_2 to obtain x = 0.5.

   Solution: (c) at equilibrium w_1/w_2 = (1-x^2)/x = 3/2 for x=0.5.

5. ★★ Carbon scheduling. Take up example 2.3 with an additional constraint: deadline at +24 h. (a) Solve by hand with the exchange argument of theorem 2.2. (b) Write the dual of the “knapsack” \min \sum c_t x_t, \sum x_t = T, 0 \leq x_t \leq 1 and interpret the dual variable (price of an hour of delay). (c) What becomes the optimum if the job can be sequenced in pieces of one hour minimum? Hint: nothing changes — why?

6. ★★ Distillation lab. Teacher: a ~1M-parameter CNN on MNIST (98.5% expected); student: an MLP with ~20k. (a) Train the student alone (baseline ~97%). (b) Distill with def. 3.1 (T \in \{1,2,4\}, \lambda \in \{0.3, 0.7\}). (c) Measure inference energy per batch (protocol of ch. 6) and the number of parameters; plot the (energy, accuracy) point of the two models on the Pareto plot of example 1.6 (ch. 1). Partial solution (PyTorch):

   import torch, torch.nn as nn, torch.nn.functional as F
   def distill_loss(zs, zt, y, T=4.0, lam=0.7):   # def. 3.1
       kd = F.kl_div(F.log_softmax(zs/T, 1), F.softmax(zt/T, 1),
                     reduction="batchmean") * T*T
       return (1-lam)*F.cross_entropy(zs, y) + lam*kd
   # loop: zs = student(x); with torch.no_grad(): zt = teacher(x)

7. ★★ Framed measurement. With CodeCarbon (ch. 6), measure the energy of the same training (logistic regression on MNIST, 10 epochs) (a) on your machine, (b) with the offline tracker set to France (country_iso_code="FRA"), then Germany ("DEU"), then with a world-average intensity of about 475 gCO₂eq/kWh entered by hand. (c) Produce the protocol sheet of ch. 6: boundary, hardware, intensity source, uncertainty (prop. 6.4). (d) Sensitivity: replace the hourly intensity with the annual average and quantify the gap. Deliverable: 10 lines, two figures, one range.

8. ★★ Carbon-constrained knapsack (ε-constraint). Take the four items of exercise 3 (capacity 10, (v,w) \in \{(7,6), (5,5), (4,4), (2,1)\}) and give them carbon costs k = (5, 1, 2, 1).

   1. Solve the unconstrained problem and show that it has two optimal solutions of value 11.

   2. Add the constraint \sum_i k_i x_i \leq \varepsilon and solve it for \varepsilon = 0, 1, \dots, 7 (by enumeration, then with PuLP as in ch. 4).

   3. Draw the front in the (carbon, value) plane.

   4. Which points of the front does the weighted problem \max \sum_i (v_i - \lambda k_i) x_i find as \lambda \geq 0 varies?

   Solution (a). \{1,3\} (carbon 7) and \{2,3,4\} (carbon 4) both reach 11 — the same value for 43\% less carbon, a saving that a single-objective solver may or may not return.

   Solution (b–c). The front is (0,0), (1,5), (2,7), (3,9), (4,11).

   Solution (d). The four non-zero points (1,5), (2,7), (3,9), (4,11) lie on one line of slope 2, so all four tie at \lambda = 2.

   • For 0 < \lambda < 2 the weighted problem returns (4,11); at \lambda = 0 it may also return the dominated (7,11) of (a).
   • For 2 < \lambda < 5 it returns (1,5); at \lambda = 5 it ties (1,5) with (0,0).
   • For \lambda > 5 it returns (0,0).

   The points (2,7), (3,9) only appear as ties at \lambda = 2 — the \varepsilon-constraint finds them without ambiguity.

9. ★★ Rebound arithmetic. Use proposition 7.1.

   1. An efficiency gain k = 3 meets an elasticity \eta = 0.6: compute E/E_0 and the rebound fraction R.

   2. For which elasticity is the gain entirely cancelled?

   3. Energy is now only a share \varphi = 0.3 of the unit cost, so that p = p_0(1 - \varphi + \varphi/k): redo (a).

   4. Which policy instrument of ch. 1 makes the saving independent of the elasticity \eta?

   Solution (a–b). (a) E/E_0 = 3^{-0.4} \approx 0.64 and R \approx 0.47; (b) \eta = 1.

   Solution (c). p = 0.8\,p_0, Q/Q_0 = 0.8^{-0.6} \approx 1.14, E/E_0 \approx 1.14/3 \approx 0.38 and R \approx 0.07.

   Solution (d). An absolute cap, i.e. an \varepsilon-constraint on total energy or carbon.

10. ★★★ Audit. Find a public “green AI” claim (corporate report, lab blog). Apply grid 7.3 to it: scope, intensity, reference, reproducibility, real benefit; write a one-page audit note, with at least one quantified test redone by yourself (e.g. recompute the emission with the protocol sheet of ch. 6). Grading grid: technical accuracy, factuality (no unsourced claim), and nuance — distinguish what is false, imprecise, and merely incomplete.

Projects and assessment

Official assessment. The fiche can quote only the last published sheet (SynapseS 2025–2026, CSC_52073_EP): “Numerical grade out of 20 (literal conversion / American grade); resit allowed (max of the two grades); course unit earned if final grade ≥ 10”, with detailed modalities not communicated. No assessment is quoted for the 2026–2027 code CSC_51076_EP, and the unit is not in the LLGA lists, so no LLGA rule applies. Everything below is a proposal sized for a 36 h course: solo subjects of 15–20 h, group subjects of 25–30 h per member, a two-hour written exam and a rubric.

The projects exercise two master objectives above all: O7 (hands-on and industry practice — measured kWh, real solvers, real traces) and O8 (trustworthy and responsible AI — honest accounting, error bars, rebound and greenwashing). The LP and ILP subjects also train O1 (optimization and duality).

Three subjects reach other LLGA modules: the fine-tune subject adds O3 through Large Language Models, 4.3; the serving subject continues Deep Learning (P2 variant), ch. 5–6, and the green ops of ML Ops and LLM Engineering, ch. 10; the feeder subject adds O4 (graph-structured data).

Solo projects

1. [Solo · ≈ 20 h] Pareto front of a small-LLM fine-tune: accuracy, kWh and latency. Which LoRA rank, weight precision and model size are Pareto-rational for one task, and which of them can no weighted sum of the objectives ever select?
   • Implement: LoRA and QLoRA fine-tunes (Large Language Models, 4.3) of SmolLM2-135M and SmolLM2-360M, or Qwen2.5-0.5B, on a 10 000-sentence subset of SST-2 (GLUE) on a free-tier GPU, so that the 72 runs fit the budget; sweep the rank r \in \{4, 8, 16, 32\} and the precision of the frozen base (16-bit, int8, 4-bit NF4), with 3 seeds per configuration.
   • Measure: validation accuracy, training and inference energy with CodeCarbon under the protocol sheet of ch. 6, latency per query, and an uncertainty range for every energy figure (prop. 6.4).
   • Analyse: the non-dominated set; its supported points (on the convex hull of the front), the only ones a weighted sum can return (theorem 1.2, prop. 1.3), against the others; a selection by \varepsilon-constraint (def. 1.4) at a stated kWh budget; the training–inference break-even (def. 6.6) at 10^4 and 10^6 queries a day.
   • Deliverables: code, 4–6 page report with the protocol sheet, 10-minute talk. Chapters: ch. 1, 3, 6.
   • Success criterion: every point carries error bars from the seeds and from prop. 6.4; the report names the Pareto points that no weighting reaches (or shows there are none) and the differences that lie within noise (ch. 1, common mistake 2).
2. [Solo · ≈ 20 h] Distil, quantize, prune: a frugal text classifier with honest accounting. A paper study of Tang, Lu, Liu, Mou, Vechtomova and Lin, “Distilling task-specific knowledge from BERT into simple neural networks”, arXiv:1903.12136, 2019. How many parameters and joules per query does compression save on one task, and after how many queries does it repay the teacher?
   • Implement: a BERT-base teacher fine-tuned on SST-2; two students — the paper’s BiLSTM and a 4-layer compact BERT (Turc et al., 2019, google/bert_uncased_L-4_H-256_A-4) — distilled with def. 3.1 for T \in \{1, 2, 4, 8\} and \lambda \in \{0.3, 0.7, 0.9\}; then int8 post-training quantization and structured head pruning (def. 3.4).
   • Measure: accuracy, parameters, memory, throughput and energy per query on a CPU (RAPL through CodeCarbon) at batch sizes 1 and 64; test whether the memory-bound reasoning of example 3.5 predicts the int8 speedup at batch 1.
   • Account: the energy of the teacher’s fine-tuning and of every distillation run, discarded ones included (ch. 3, common mistake 3), and the number of queries that repays it (def. 6.6).
   • Deliverables: code, 4–6 page report, 10-minute talk. Chapters: ch. 1, 3, 6.
   • Success criterion: both students sit on the (energy per query, accuracy) plot of example 1.6 with error bars over 3 seeds; their compression ratios and accuracy gaps are compared with the paper, whose teacher is the larger BERT-large (335M parameters), and with the range of the ch. 3 table.
3. [Solo · ≈ 15 h] Auditing a published footprint: BLOOM. A paper study of Luccioni, Viguier and Ligozat, “Estimating the carbon footprint of BLOOM, a 176B parameter language model”, JMLR 24(253), 2023, which reports 24.7 t CO₂eq for the dynamic power of the final training and 50.5 t when every process, from equipment manufacturing to operation, is counted.
   • Recompute: each term of their total from the energy, PUE, intensity and embodied factors they report, with def. 6.1 and 6.2.
   • Stress-test: the same training on a gas-heavy grid and at a marginal intensity (ch. 7, pitfall 1), with the boundary choices of example 6.3 (GPU only, server, facility), and with an uncertainty range (prop. 6.4).
   • Audit: apply the grid of prop. 7.3 and the protocol sheet of ch. 6; add a rebound scenario for the deployment (prop. 7.1) over a stated range of elasticities.
   • Deliverables: a 3-page audit note, a notebook with every number, 10-minute talk. Chapters: ch. 6, 7.
   • Success criterion: every recomputed figure matches the paper up to rounding, or the gap is explained; each claim of the note is labelled false, imprecise or incomplete, as in exercise 10.

Group projects

1. [Group of 3 · ≈ 30 h each] Carbon-aware scheduler for a lab GPU cluster. How much carbon does a scheduler that knows tomorrow’s intensity forecast save on a real job mix without missing a deadline, and how much of the saving survives forecast errors? The design is compared with the production system of Radovanovic et al. (2022, references).
   • Data: job arrivals, durations and GPU counts from the Microsoft Philly traces (Jeon et al., “Analysis of large-scale multi-tenant GPU clusters for DNN training workloads”, USENIX ATC 2019; github.com/msr-fiddle/philly-traces), rescaled to a 32-GPU lab; hourly intensity for France from RTE’s éCO2mix and for a second zone from the Electricity Maps data portal.
   • Implement: the ILP of the ch. 2 note (durations, deadlines, precedence, GPU capacity) with PuLP and HiGHS on a rolling 48 h horizon; intensity forecasts by gradient boosting (Machine Learning, ch. 6); baselines: immediate launch and the cleanest-hours rule of theorem 2.2; the robust variant of case study 5.2 for forecast error.
   • Price: fix the integer choices, solve the LP that remains and read the dual of each hour’s capacity as a carbon price per GPU-hour shown to users (example 2.3, example 4.4).
   • Split of work: A: data and forecasts; B: the ILP, the solver and its optimality gaps; C: the robust variant, the dual prices and the evaluation.
   • Deliverables: repository, 6-page report, 15-minute talk. Chapters: ch. 2, 4, 5, 6.
   • Success criterion: on held-out weeks, the saving against immediate launch is reported with real forecast errors and next to a clairvoyant schedule; no deadline is missed; every ILP is solved to an optimality gap below 1 %.
2. [Group of 3 · ≈ 30 h each] Coordinated EV charging on a distribution feeder. Extend case study 5.2 from one transformer to a feeder graph: which lines are the bottlenecks, what is an upgrade worth in kg CO₂eq, and what do uncertain departures cost?
   • Data: charging sessions (arrival, departure, energy delivered) from ACN-Data (Lee, Li and Low, “ACN-Data: analysis and applications of an open EV charging dataset”, ACM e-Energy 2019); the IEEE 33-bus radial test feeder (Baran and Wu, 1989) as the graph, with vehicles attached to its nodes; hourly intensity from éCO2mix or Electricity Maps.
   • Implement: the LP of case study 5.2 with one capacity per line of the feeder (active power only: the flow on a line is the sum of the loads downstream of it); a two-stage stochastic variant whose scenarios resample departures from ACN-Data, with unmet energy as a penalized recourse; the fast-charging ILP variant (x \leq Mz, fixed cost \kappa z).
   • Price: line and transformer upgrades from the duals (example 5.3), each checked by re-solving with one more kW; the schedule under average and under marginal intensity (ch. 7, pitfall 1).
   • Split of work: A: data, scenarios and the feeder graph; B: the LP and ILP models; C: duals, sensitivity analysis and the report.
   • Deliverables: repository, 6-page report, 15-minute talk. Chapters: ch. 4, 5, 7.
   • Success criterion: every need is met in every scenario or its shortfall is priced; the saving against uncontrolled charging (full power on arrival) is reported for both intensity conventions; any dual price that disagrees with its re-solve is explained by a change of active constraints.
3. [Group of 2 · ≈ 25 h each] Joules per token: serving a small LLM frugally. How do weight precision and batch size move the energy per generated token, the latency and the quality of a small open LLM, and does the memory-bandwidth bound of example 3.5 predict the measurements?
   • Implement: generation with Qwen2.5-1.5B-Instruct (or SmolLM2-1.7B-Instruct) in 16-bit, int8 and 4-bit weights with Hugging Face Transformers or vLLM on one free-tier GPU, batch sizes 1 to 32; energy from NVML through CodeCarbon, sampled during decoding only.
   • Compare: the measured tokens per second at batch 1 with the bound bandwidth ÷ bytes of weights (example 3.5; Deep Learning (P2 variant), def. 5.1, and the roofline of its ch. 6); quality on a fixed 200-question subset of GSM8K.
   • Analyse: the three-objective front (joules per token, P99 latency, accuracy); the break-even of def. 6.6 for a stated traffic; the green-ops reporting of ML Ops and LLM Engineering, ch. 10.
   • Split of work: A: serving, quantization and latency; B: energy measurement, uncertainty and quality evaluation.
   • Deliverables: repository, 5-page report, 15-minute talk. Chapters: ch. 1, 3, 6.
   • Success criterion: every energy figure follows the protocol sheet of ch. 6 with a range; the batch-1 throughput lies within a factor 2 of the bandwidth bound, or the gap is explained (dequantization kernels, KV cache); the front says which precision wins at each latency budget.

Written exam (proposal)

Two hours, one handwritten A4 sheet allowed, a basic calculator allowed; four independent questions on the proved results of the course.

1. Question (5 points). Weighted scalarization and the hollow front (ch. 1).

   • 1. Let w \neq 0 be non-negative, and let x_w minimize F_w = \sum_i w_i F_i. Prove that x_w is weakly Pareto-optimal, and Pareto-optimal if w > 0.
   • 2. Give a two-solution example where w = (1, 0) admits a dominated minimizer.
   • 3. For F_1(x) = x and F_2(x) = 1 - x^2 on [0, 1], which points do weighted sums with w > 0 return? Which scalarization returns x = 0.5?

   Expected:

   • 1. Theorem 1.2. If x' dominates x_w, every weighted term weakly decreases and, for w > 0, the term of the strictly better objective decreases strictly: F_w(x') < F_w(x_w). If x' is strictly better on every objective, any positive weight, which exists since w \neq 0, gives the same strict inequality.
   • 2. F(a) = (0, 1) and F(b) = (0, 0): both minimize F_1, and a is dominated by b.
   • 3. F_w is concave in x, so its minimum over [0,1] is at an endpoint: weighted sums return only x = 0 and x = 1 (prop. 1.3).
   • The \varepsilon-constraint \min\{x : 1 - x^2 \leq \varepsilon\} returns x = \sqrt{1-\varepsilon}, hence x = 0.5 for \varepsilon = 0.75 (example 1.5). Chebyshev also reaches it, with w_1/w_2 = 3/2 (exercise 4).

2. Question (6 points). A deferrable job of 3 hours at constant power P = 0.5 kW can run in any of the next six hours, of intensities c = (300, 120, 80, 60, 200, 400) gCO₂eq/kWh; it may be split into pieces of any length.

   • 1. Give the minimal emission, the emission of an immediate launch and the relative gain (theorem 2.2).
   • 2. The PUE is 1.2; the relative uncertainties are 10\% on the IT energy, 5\% on the PUE and 20\% on the intensities. Report the optimal footprint with a one-standard-deviation range.
   • 3. Write the LP of (a) with 0 \leq x_t \leq 1 and \sum_t x_t = T. What does one more hour of work cost when 3 < T < 4? Which values can the multiplier of the equality constraint take at T = 3?

   Expected:

   • 1. The three cleanest hours are 60, 80 and 120 g/kWh, so the optimum is 0.5 \times 260 = 130 g; launching now costs 0.5 \times 500 = 250 g; the gain is 48\%.
   • 2. With the PUE, 130 \times 1.2 = 156 g; prop. 6.4 gives r_C = \sqrt{0.01 + 0.0025 + 0.04} \approx 0.23, so about 156 g \pm 23\%, roughly 120 to 192 g.
   • 3. Minimize P \sum_t c_t x_t. The extra work goes to the fourth cleanest hour (200 g/kWh) and costs 0.5 \times 200 = 100 g per hour of work: the multiplier is P c_{(4)}.
   • At T = 3 the LP is degenerate: every value of [P c_{(3)}, P c_{(4)}] = [60, 100] g per hour is an optimal multiplier — the left and right derivatives of the optimal value.

3. Question (5 points). Uniform quantization (ch. 3).

   • 1. A value x \in [x_{\min}, x_{\max}] is coded on b bits with step \Delta = (x_{\max} - x_{\min})/(2^b - 1) and rebuilt from the nearest level. Prove that |\hat x - x| \leq \Delta/2.
   • 2. A network has L layers with Lipschitz constants \mathrm{Lip}_\ell, and quantizing layer \ell shifts its output by at most \Delta_\ell/2. Prove the output bound of prop. 3.3 by induction.
   • 3. Three layers with \mathrm{Lip} = 2 and weights in [-1, 1]: evaluate the bound at 4 and at 8 bits.

   Expected:

   • 1. The levels are \Delta apart and cover the range, so the nearest one is at most half a cell away; clipping only brings values back inside.
   • 2. With \varepsilon_\ell the error after layer \ell, \varepsilon_{\ell+1} \leq \mathrm{Lip}_{\ell+1}\,\varepsilon_\ell + \Delta_{\ell+1}/2, \qquad \varepsilon_1 = \Delta_1/2, and unrolling multiplies each \Delta_\ell/2 by the constants of the later layers.
   • 3. The bound is (\Delta/2)(4 + 2 + 1) = 7\Delta/2: with \Delta = 2/15 at 4 bits it is 7/15 \approx 0.47, with \Delta = 2/255 at 8 bits it is 7/255 \approx 0.027 — 17 times smaller, which is why QAT takes over at 4 bits.

4. Question (4 points). Rebound (ch. 7). Demand follows Q = A p^{-\eta}, the unit cost is proportional to the energy per unit of service, and an efficiency gain divides that energy by k > 1.

   • 1. Prove that total energy becomes E_0 k^{\eta - 1}.
   • 2. For k = 2 and \eta = 0.5, compute the expected saving, the realized saving and the rebound fraction R.
   • 3. Which constraint of ch. 1 guarantees a saving whatever \eta, and why?

   Expected:

   • 1. E = A (p_0/k)^{-\eta} (e_0/k) = E_0 k^{\eta} k^{-1}, as in prop. 7.1.
   • 2. Expected 1 - 1/2 = 50\%; realized 1 - 2^{-0.5} \approx 29.3\%; R = 1 - 0.293/0.5 \approx 0.41.
   • 3. An absolute cap on total energy or carbon, i.e. an \varepsilon-constraint: it bounds the quantity that matters, whereas efficiency only bounds the energy per unit of service.

Total: 5 + 6 + 5 + 4 = 20 points.

Grading rubric (proposal)

References

• S. Boyd, L. Vandenberghe, Convex Optimization, Cambridge Univ. Press, 2004 — ch. 4 (multi-objective, scalarization).
• K. Miettinen, Nonlinear Multiobjective Optimization, Kluwer, 1999 — the Pareto/ε-constraints/Chebyshev reference.
• L. A. Wolsey, Integer Programming, 2nd ed., Wiley, 2020 — ILP, formulations, branch and bound.
• D. Bertsimas, M. Sim, The Price of Robustness, Operations Research 52(1), 2004 — budgeted uncertainty sets (robust variant of case study 5.2).
• A. Radovanovic et al. (Google), Carbon-Aware Computing for Datacenters, IEEE Trans. Power Systems, 2022 — the industrial deployment of ch. 2 and 5.
• E. Strubell, A. Ganesh, A. McCallum, Energy and Policy Considerations for Deep Learning in NLP, ACL 2019 — the paper that launched the debate.
• D. Patterson et al., Carbon Emissions and Large Neural Network Training, 2021 — measurement, PUE, regions (the methodological corrective).
• D. Patterson et al., The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink, IEEE Computer, 2022 — the share of inference in ML energy (ch. 6).
• M. Horowitz, Computing’s energy problem (and what we can do about it), ISSCC 2014 — the energy of memory accesses versus arithmetic (ch. 3).
• A. Lacoste et al., Quantifying the Carbon Emissions of Machine Learning, 2019 — the ML CO2 Impact tool.
• P. Henderson, J. Hu, J. Romoff, E. Brunskill, D. Jurafsky, J. Pineau, Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learning, Journal of Machine Learning Research 21(248), 2020 — the protocol of ch. 6.
• A. S. Luccioni, S. Viguier, A.-L. Ligozat, Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Model, Journal of Machine Learning Research 24(253), 2023 — a full life-cycle estimate (solo project 3).
• R. Schwartz et al., Green AI, Communications of the ACM, 2020 — efficiency as a research metric.
• G. Hinton, O. Vinyals, J. Dean, Distilling the Knowledge in a Neural Network, NeurIPS 2014 (workshop), 2015.
• V. Sanh, L. Debut, J. Chaumond, T. Wolf, DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter, 2019 — distillation on a broad benchmark (ch. 3).
• R. Tang, Y. Lu, L. Liu, L. Mou, O. Vechtomova, J. Lin, Distilling Task-Specific Knowledge from BERT into Simple Neural Networks, 2019 — distillation on one task (ch. 3, solo project 2).
• S. Mukherjee, A. H. Awadallah, XtremeDistil: Multi-stage Distillation for Massive Multilingual Models, ACL 2020 — the distillation range of ch. 3.
• S. Han, H. Mao, W. J. Dally, Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding, ICLR 2016.
• B. Jacob et al., Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference, CVPR 2018.
• CodeCarbon and Electricity Maps — measurement and hourly intensity data (ch. 6).
• PuLP / HiGHS — the solvers used in the labs (ch. 4).
• Research group (LIX, École Polytechnique): C. Gicquel, S. Vanier et al., Optimal deployment of virtual network functions for securing telecommunication networks against distributed denial of service attacks: a robust optimization approach, Computers & Operations Research, 2022; ORAILIX team (LIX) — operations research × learning, “Optimization for Mobility” chair.