Image Synthesis: Theory and Practice

Contents

• Introduction and overview
• 0. Background: probability, integrals, linear ODEs
• 1. Radiometry: radiance
• 2. Cameras: the pinhole model
• 3. Rasterization versus ray tracing
• 4. The rendering equation
• 5. BRDFs and material models
• 6. Spatial structures and shadows: intersection, BVHs, visibility queries
• 7. Monte Carlo path tracing
• 8. Noise, denoising, tone mapping
• 9. Real time: GPU, hybrid, upscaling
• 10. Neural rendering: NeRF and 3D Gaussians
• 11. Research of the instructor
• Exercises
• Projects and assessment
• References
• A complete path tracer in Python

Introduction and overview

Audience: an image or GPU engineer who wants to understand a path tracer from end to end before meeting NeRF. Empirical first: each topic starts from a failure — a black image because radiance was ill-defined, noise that never decreases because the estimator is biased — and only then names the concept that explains it: the rendering equation, BRDFs, Monte Carlo estimators, BVHs.

The module goes from the physics of light to neural rendering, with a companion path tracer in Python.

What you will be able to do

• Handle radiometric quantities, first and foremost radiance.
• State the rendering equation and explain why solving it exactly is out of reach.
• Choose a material model (BRDF) according to the intended effect, and drive its parameters with filtered, perspective-correct textures.
• Model a camera with the pinhole model and connect pixels and rays.
• Compare rasterization and ray tracing, their trade-offs and their hybrids.
• Compute hard and soft shadows by shadow rays and shadow maps, and accelerate ray queries with BVHs.
• Write a Monte Carlo path tracer, and know the classical approximations of global illumination (ambient occlusion, radiosity, irradiance caching, photon mapping).
• Handle noise, denoising and tone mapping to go from physical values to a displayable image.
• Program the GPU pipeline with vertex and fragment shaders, and situate NeRF and 3D Gaussians in the rendering landscape and the challenges of real time.

Official objectives and syllabus (2026-2027 catalog)

The official objective of the course unit is to make students able to develop complete interactive 3D rendering systems, and to give a detailed understanding of the simulation of light transport and of the formation of digital images from a 3D scene.

The official description covers the digital models of shape, appearance, lighting and sensors of a scene, the rendering equation and the standard models of illumination, shading and reflectance, the rendering algorithms (projective rasterization, ray tracing), real time, GPU programming and hierarchical spatial structures, with an opening towards global illumination.

The course has a strong practical dimension: the models and algorithms are implemented throughout the term (officially in C++ with the OpenGL API, technical notions reviewed in class; this material uses a Python path tracer as its guiding thread).

Official prerequisites (stated for the engineering degree and the exchange programmes): algorithms and data structures, calculus, programming (C or C++), notions of linear algebra. Official teaching methods: lectures, computer-lab sessions and a project; keywords: 3D computer graphics, image synthesis, rendering.

The sheet’s detailed programme lists ten themes, in the order of the table below, which maps each of them, and each element of the description, to the section that teaches it.

Where this course fits in the program

An option of period 2 of M1 that no route of the home page preselects, in the P2 “Choose 1” group: a student takes this course or one of its five siblings, among them Multimodal Generative AI and Deep Learning (P2 variant).

The Monte Carlo background — law of large numbers, central limit theorem, variance of a mean — comes from Refresher in Statistics, ch. 4; ch. 0 recalls it and ch. 7 builds the estimators. Probability Theory for ML: Monte Carlo and Generative Models goes further but is not in the 2026–2027 LLGA list, so it is optional reading.

Two P1 options help: Digital Representation and Analysis of Shapes for 3D surfaces, Signal Processing for filtering. Since Deep Learning is taken once, in P1 or P2, a student of this course took it in P1. The learned denoisers of our ch. 8 and the neural fields of our ch. 10 rely on its backpropagation (Deep Learning, ch. 2), CNNs (ch. 4) and recurrent networks (ch. 5).

Official prerequisites: algorithmics and data structures, calculus, programming in C or C++, notions of linear algebra — the module’s Python path tracer is their application ground, the official implementation of the course being in C++/OpenGL.

Multimodal Generative AI is an alternative to this course, not a sequel. Its §8.4 only sketches NeRF: a student who chose Multimodal instead can read our neural-rendering chapter as a complement. The camera and light models of ch. 1–2 meet the vision side in Image Analysis and Computer Vision, ch. 1. Downstream, the M2 course Analysis and Deep Learning on Geometric Data processes triangle meshes like ours rather than rendering them, and rendered views of 3D products feed the Y2.3 transverse project.

Course outline

• Background: probability, integrals, linear ODEs — the mathematics of rendering, put back in hand.
• Radiometry: radiance — the physical quantities that describe light, and the one that will carry everything.
• Cameras: the pinhole model — forming an image: projection, pixels and rays.
• Rasterization versus ray tracing — the two great families of rendering algorithms and their trade-offs.
• The rendering equation — the integral formulation that grounds the whole rest of the module.
• BRDFs and material models — how a surface responds to light, from diffuse to specular, and textures that vary it over the surface.
• Spatial structures and shadows — making ray-scene intersection computations affordable, and answering the visibility queries that make shadows.
• Monte Carlo path tracing — solving the rendering equation by random sampling.
• Noise, denoising, tone mapping — going from noisy physical values to a displayable final image.
• Real time: GPU, hybrid, upscaling — shaders, the strategies that bring realism closer to interactive budgets, and the approximations of global illumination.
• Neural rendering: NeRF and 3D Gaussians — when networks learn the scene and its rendering.
• Research of the instructor — the research papers of Tamy Boubekeur, read with the tools of the module.
• A complete path tracer in Python — the companion code of the course, to read, run and extend.
• Exercises — radiometric computations and guided extensions of the path tracer.
• Projects and assessment — solo and group subjects (paper studies included), a written-exam proposal and a rubric.
• References — the reference books and papers of rendering.

Session plan

36 h in class = 12 sessions of 3 h. The volume is the one of the SynapseS 2026–2027 sheet (36 h in person: 18 h of lectures and 18 h of small-class sessions, i.e. about 1 h 30 of each per session).

The sheet’s detailed programme lists ten themes without dates; the plan follows their order — introduction (session 1), rendering algorithms and visibility (2), appearance and textures (4), shadows and spatial structures (5), global illumination (6–7), GPU (9), global illumination approximation (10), radiance fields and neural rendering (11–12).

Two placements are ours: the rendering equation (session 3), which opens the global-illumination theme, comes before appearance because the BRDF is defined through it; denoising and tone mapping (session 8), which the sheet does not list, come right after path tracing, which produces the noise. To follow this order, the spatial-structures chapter (ch. 6) now precedes the path-tracing chapter (ch. 7). The split into sessions is our proposal.

Level 1 = foundations (radiometry, cameras, visibility), 2 = core rendering methods, 3 = recent research (neural rendering); the level never goes down from one session to the next.

Self-study, outside the 36 h (about 2 h in total): ch. 11 Research of the instructor (≈ 2 h). The final project (Ex. 10 or one of the project subjects, implementation of a research paper) is carried out outside class time, as the official assessment states.

How to study this module

• Run the Python path tracer from the first chapters on: every concept of the course has its line of code there.
• Recode the building blocks in small steps — ray, sphere, then direct lighting — rather than attacking everything at once.
• Draw the integrands: the rendering equation is understood on a sketch of light bouncing around.
• Connect the noise in your images to the limit theorems of Refresher in Statistics, ch. 4: variance, sampling, convergence.
• Test each idea on minimal scenes — one sphere, two lights — before scaling up in complexity.
• Keep the full pipeline in mind: radiometry, rendering, denoising, tone mapping — an upstream error is paid downstream.

0. Background: probability, integrals, linear ODEs

1. Radiometry: radiance

A render that looks pretty but is not physical breaks as soon as the lighting changes. This chapter fixes the physical quantities of light, and singles out the one that a renderer must compute: radiance.

2. Cameras: the pinhole model

An image is a set of radiance measurements, one per pixel. This chapter models the device that makes these measurements — the camera — and gives the two maps a renderer needs: from a 3D point to a pixel (projection), and from a pixel to a ray (ray generation).

3. Rasterization versus ray tracing

Once the camera is fixed, a renderer must decide which surface each pixel sees. This chapter compares the two ways of doing it — looping over the objects (rasterization) or over the pixels (ray tracing) — and the arithmetic that makes the first one so fast.

4. The rendering equation

Radiance is the quantity to compute; this chapter writes the equation it satisfies at every surface point. The equation is recursive — the light leaving a point depends on the light leaving all the others — and its solution is a series of bounces.

5. BRDFs and material models

The rendering equation contains one function that describes the material: the BRDF. This chapter defines it, states the two physical constraints it must satisfy, and presents the models used in practice, from the matte Lambertian surface to microfacets; it ends with textures, which make the parameters of these models vary over the surface.

Textures: mapping, perspective-correct interpolation, filtering

A BRDF with constant parameters gives plastic-looking objects. Real surfaces vary from point to point — the albedo of wood grain, the roughness of a scratched metal, the small-scale relief of a brick wall — and this variation is stored in textures: images whose texels are read during shading to set the parameters of the material model (albedo, roughness, metalness, normal perturbation).

6. Spatial structures and shadows: intersection, BVHs, visibility queries

A ray tracer (ch. 3) — and the path tracer of ch. 7 even more — spends most of its time asking which triangle a ray hits first, or whether anything blocks the segment towards a light. This chapter makes these queries cheap: a fast ray-triangle test, then a hierarchy of boxes that discards most of the scene without testing it; it ends with shadows, the visibility queries between a point and a light, in ray tracing and in rasterization.

Shadows: visibility between a point and a light

A shadow is a visibility question: is the light source visible from the shading point? In the rendering equation it appears as the visibility factor V(x \leftrightarrow y) \in \{0, 1\} of the surface form (ch. 4). Ray tracing answers it with one more ray; rasterization needs a second rendering pass from the light. Both are queries against the scene, answered by the same structures as primary visibility: a shadow ray traverses the BVH of definition 6.2 like any other ray.

Shadow mapping (Williams 1978), the rasterization answer:

1. render the scene from the light and store, per texel, the depth of the closest surface — the shadow map;
2. while shading from the camera, transform each point into the light’s view and compare its depth z with the stored z_{\text{map}}: the point is in shadow if z > z_{\text{map}} + b.

The bias b prevents shadow acne (a surface shadowing itself through rounding and the finite resolution of the map), but too large a bias detaches shadows from their casters (peter-panning).

The finite resolution produces jagged edges, reduced by percentage-closer filtering (PCF: average the binary test over neighbouring texels, which also fakes a penumbra) and by cascaded shadow maps (several maps covering increasing distances from the camera); ch. 9 places these techniques in the real-time budget.

7. Monte Carlo path tracing

The bounce series of ch. 4 is an integral over light paths of unbounded dimension, out of reach of deterministic quadrature. This chapter estimates it by random sampling, proves that the estimator is unbiased, and shows how the choice of the sampling density controls the noise.

8. Noise, denoising, tone mapping

A Monte Carlo render is noisy, and its values are physical radiances that no screen can display directly. This chapter covers the two last steps of the pipeline: removing the noise at a controlled bias, and mapping high-dynamic-range radiance to displayable pixel values.

9. Real time: GPU, hybrid, upscaling

Film rendering spends minutes per image; a game has about 16 ms. This chapter shows how the same equations are solved under that budget: a hardware rasterization pipeline programmed with shaders, a few rays where they matter most, learned reconstruction to fill in the rest, and the classical approximations of global illumination.

From model to pixels: the model-view-projection chain

The vertex shader of the next section receives three matrices, model, view and projection. They extend the pinhole camera of ch. 2 to the 4 \times 4 homogeneous form the GPU expects, and add a depth that the z-buffer (definition 3.3) can compare.

Chaining P, the divide and the viewport gives back the intrinsics of definition 2.1: with the depth d = -z, u = (H/2)\, s\, x/d + W/2 since W/a = H, and v = H/2 - (H/2)\, s\, y/d. So f_x = f_y = \tfrac{H}{2}\cot(\varphi/2), \qquad (c_x, c_y) = (W/2,\ H/2), with the -f_y of the y-up convention.

The third row of P adds a depth z_{\text{ndc}} = -A - B/z, where A and B are the two nonzero entries of that row: an affine function of 1/z, which proposition 5.6 shows is affine on screen, so the z-buffer interpolates it linearly with no perspective correction.

Normals transform by the inverse transpose of the 3 \times 3 block of M, as in the vertex shader below: if a tangent \mathbf{t} satisfies \mathbf{t}\cdot\mathbf{n} = 0, then (M\mathbf{t}) \cdot (M^{-\top}\mathbf{n}) = \mathbf{t}^\top M^\top M^{-\top} \mathbf{n} = \mathbf{t}\cdot\mathbf{n} = 0 .

Programming the GPU: shaders

The programmable stages of the pipeline of definition 9.1 are small programs, shaders, written in a C-like language (GLSL for OpenGL, the API of the official course; HLSL, WGSL or SPIR-V elsewhere).

The application uploads the geometry (vertex buffers), the textures and the uniform parameters (matrices, light positions), then issues draw calls; the GPU runs the vertex shader once per vertex and the fragment shader once per covered pixel sample, on thousands of threads in parallel.

The pair below implements the transformation chain of definition 9.3, the perspective-correct interpolation of proposition 5.6 (done by the hardware for every out/in variable), a texture lookup and the Blinn-Phong model of definition 5.3.

// vertex shader
#version 330 core
layout(location = 0) in vec3 position;
layout(location = 1) in vec3 normal;
layout(location = 2) in vec2 uv;
uniform mat4 model, view, projection;
out vec3 fragPos; out vec3 fragNormal; out vec2 fragUV;
void main() {
    vec4 world = model * vec4(position, 1.0);
    fragPos = world.xyz;
    fragNormal = mat3(transpose(inverse(model))) * normal;   // normals transform by the inverse transpose
    fragUV = uv;
    gl_Position = projection * view * world;                 // clip space; the divide by w is done by the GPU
}

// fragment shader
#version 330 core
in vec3 fragPos; in vec3 fragNormal; in vec2 fragUV;
uniform sampler2D albedoMap;                                 // mipmapped texture (example 5.7)
uniform vec3 lightPos, lightColor, camPos;
uniform float ks, shininess;
out vec4 color;
void main() {
    vec3 n = normalize(fragNormal);
    vec3 l = normalize(lightPos - fragPos);
    vec3 v = normalize(camPos - fragPos);
    vec3 h = normalize(l + v);                               // half vector (Blinn-Phong)
    vec3 albedo = texture(albedoMap, fragUV).rgb;            // filtered texture lookup
    float d2 = dot(lightPos - fragPos, lightPos - fragPos);
    vec3 E = lightColor * max(dot(n, l), 0.0) / d2;          // irradiance from a point light
    vec3 fr = albedo / 3.14159265 + ks * (shininess + 8.0) / (8.0 * 3.14159265) * pow(max(dot(n, h), 0.0), shininess);
    color = vec4(fr * E, 1.0);                               // linear radiance; tone map and gamma at the end (ch. 8)
}

The factor (n + 8)/(8\pi) is the usual approximate energy normalization of the Blinn-Phong lobe, the analogue of the (n + 2)/(2\pi) of Phong (definition 5.3). Shadows are added by sampling a shadow map in the fragment shader (ch. 6); compute shaders run general parallel kernels outside the graphics pipeline — denoisers, BVH builds, the image-space passes of the next section.

Global illumination approximations

Path tracing (ch. 7) solves the rendering equation without bias but needs many samples. Long before real-time path tracing, and still today inside most engines, global illumination is approximated: indirect light is smooth over diffuse surfaces, so it can be computed sparsely, precomputed, or replaced by a cheaper quantity — trading bias for speed, as definition 8.1 anticipated.

The other classical approximations, from the most precomputed to the most dynamic:

• Radiosity (Goral et al. 1984): for purely diffuse scenes, discretize the surfaces into patches of uniform radiosity B_i and solve the linear system B_i = E_i + \rho_i\sum_j F_{ij}B_j, where the form factor F_{ij} is the fraction of the energy leaving patch i that reaches patch j — the rendering equation of ch. 4 projected onto piecewise constant functions; its solution is view-independent and can be baked into light maps.
• Irradiance caching (Ward et al. 1988): compute the indirect diffuse irradiance accurately at sparse points, chosen where it varies quickly (near geometry), and interpolate elsewhere; light probes and irradiance volumes store precomputed incident light, often as a few spherical-harmonic coefficients, on a grid through the scene.
• Photon mapping (Jensen 1996): first trace photons from the lights and store their hits; then estimate reflected radiance by density estimation over the k nearest photons, L_r(x, \omega_o) \approx \sum_p f_r(x, \omega_p \to \omega_o)\,\Delta\Phi_p/(\pi r^2), with r the radius containing them. The estimate is biased (blurred over the radius r) but consistent as the number of photons grows, and it captures caustics that path tracing from the eye finds very hard.
• Instant radiosity / virtual point lights (Keller 1997): turn the photon hits into many small point lights and render with shadow maps; voxel cone tracing and screen-space reflections are their real-time descendants, now competing with the hybrid ray tracing of definition 9.2.

10. Neural rendering: NeRF and 3D Gaussians

The previous chapters render a scene that is given; this chapter recovers a scene from photographs, by making the renderer differentiable and fitting the scene to the images. Both NeRF and 3D Gaussians rest on the same volume rendering equation, whose discretization is alpha compositing.

11. Research of the instructor

This chapter, read outside the class hours, presents the research of the instructor and links each work to the chapters of the module: the same question — which representation of the scene for which rendering budget — runs through all of them.

Exercises

1. ★ Guided radiance. (a) Take up the proof of Theorem 1.2 and redo it with a sketch with explicit dimensions (dA_1 = 1 cm², dA_2 = 4 cm², r = 0.5 m, incidences 30° and 0°): verify L_1 = L_2 numerically. (b) Deduce the irradiance of a point 1 m then 3 m from an isotropic 10 W point source — explain the 1/r^2 without invoking radiance.

2. ★ Bounce series. Two planes facing each other with albedo \rho = 0.9, one emissive (L_e = 1). (a) Compute the equilibrium radiance (geometric series). (b) How many bounces to capture 99% of the energy? 99.9%? (c) What is the link with the default path depth (3–4) of real-time engines?

3. ★ Fresnel. Plot Schlick’s F(\theta) for F_0 = 0.04 and for gold (1.00,\,0.71,\,0.29) over [0°, 90°]; recover the values of Example 5.4; explain why wet grass looks shiny (dielectric water layer over a diffuse surface).

4. ★ Guided camera. (a) Implement pixel ⟶ ray (Definition 2.1) and check Example 2.3. (b) For f_x = 1000 px and a 1920×1080 image, compute the horizontal and vertical fields of view. (c) Add a thin lens (sample a disk of radius r in the lens plane, aim at the focal point) and show that the circle of confusion grows linearly with r.

5. ★ Guided Monte Carlo. (a) Re-derive the variance computation of Theorem 7.3 (uniform sampling: \pi^2/3). (b) Implement the two estimators of \int_{\Omega^+}\cos\theta\, d\omega and plot the error as a function of N (log scale); check the -1/2 slopes. (c) Generalize: for f(\theta) = \cos^k\theta, what is the zero-variance density? and a realistic approximate density?

6. ★★ Guided Lambert. (a) Redo the proof of Theorem 5.2 in detail (computation of \int_{\Omega^+} \cos\theta\, d\omega = \pi).

   (b) Check the energy conservation of Phong with k_d = 0.7, k_s = 0.3, n = 30 (numerical integration over the hemisphere); what happens for n = 1?

   (c) Sample the GGX BRDF by the normal distribution method (draw \mathbf{h} according to D(\mathbf{h})\cos\theta_h, bounce) — check unbiasedness by comparison with a reference integral.

7. ★★ Rasterization. (a) Rasterize by hand the triangle of Example 3.4 over the grid of pixels with integer centers — list of covered pixels and barycentrics (edge functions of Example 3.4, top-left rule for pixels on an edge). (b) Build an example with two triangles where painter’s sorting fails and the z-buffer does not. (c) Prove that the “top-left” rule avoids holes and double counting between adjacent triangles.

8. ★★ BVH/SAH. (a) Build by hand the median BVH of 8 triangles in [0,1]^3 (2 levels); count the boxes traversed by a diagonal ray. (b) Implement the SAH split (12 candidates per axis) and compare the total traversal cost with (a) on a scene of 1000 random triangles. (c) What becomes the BVH if all the triangles move? (refit versus rebuild — measure both.)

9. ★★ Volume rendering. (a) Check Example 10.5 against the exact integral (piecewise-constant densities, analytic integration of \int T \sigma c). (b) Implement alpha compositing over 64 randomly drawn samples (NeRF quadrature): does the bias vanish on average? (c) Simulate a “3D Gaussian” (\sigma(x) = \sigma_0 e^{-\|x - c\|^2/2s^2}) and render a cloud — why must the number of samples per ray grow with 1/s?

10. ★★★ Project — extend the mini tracer. (a) Mirror/glass materials (reflection + Snell refraction, Fresnel scheme); (b) explicit Russian roulette (continuation probability q = \rho, survivors divided by q) and measurement: average depth, per-pixel variance; (c) depth of field; (d) guided bilateral denoising with the buffers (normal, depth) and equal-time comparison against more spp. Conclusion: what budget for what quality?

Projects and assessment

The fiche above gives the official assessment: “Final project (implementation of a research paper with report), carried out outside class time; resit by a similar project; no written or oral exam; course unit earned if final grade ≥ 10”. These modalities are stated for the engineering curriculum; the LLGA-specific ones are not communicated. The subjects below are proposals of these notes, not official rules: each paper study can serve as the official paper-implementation project, and the written exam below is a self-test, since the sheet announces no exam.

The projects exercise mainly O7 (renderers written from the equations and measured against references at equal time) and O6 (images and 3D scenes as a modality: neural fields, rendered views for multimodal retrieval, scenes written from text). O1 enters with the learned denoiser and O3 with the language model that writes scenes.

Solo projects

1. [Solo · ≈ 25 h] Paper study: multiple importance sampling (Veach and Guibas, 1995). Can the companion path tracer reproduce the classic figure of glossy plates lit by spherical lights, where BRDF sampling fails on small lights and light sampling fails on sharp lobes?
   • Work: extend the companion code with Cook–Torrance materials (GGX, Smith, Schlick: definition 5.3, example 5.4), spherical area lights sampled by NEE, MIS with the balance and power heuristics (proposition 7.5), Russian roulette (definition 7.4) and a BVH (definition 6.2).
   • Paper: E. Veach and L. Guibas, “Optimally Combining Sampling Techniques for Monte Carlo Rendering”, SIGGRAPH 1995; the scene is rebuilt from its description (four plates of decreasing roughness, four lights of increasing size), or taken from the “Veach, MIS” scene of Benedikt Bitterli’s public Rendering Resources, whose Mitsuba version also gives the reference image.
   • Success criterion: against a 4096 spp reference and at equal sample count, the MIS image has the lowest whole-image RMSE, well below both single strategies. A per-plate RMSE table shows where each strategy fails, and that the power heuristic beats the balance heuristic where one strategy is already nearly perfect, the case that motivates it in the paper. Every BRDF passes a white-furnace test (albedo at most 1) and a reciprocity test on random direction pairs.
   • Deliverables: code, a 4–6 page report with RMSE-versus-time curves, a 10-minute talk; chapters 5–7, Exercises 5, 6 and 10.
2. [Solo · ≈ 20 h] A real-time renderer with shadows and ambient occlusion. How close does a 16 ms rasterizer come to a path-traced image, and which pass costs what?
   • Work: a C++/OpenGL renderer — the language of the official labs — or WebGL2, with the model-view-projection chain (definition 9.3), GGX or Blinn-Phong shading with mipmapped textures (ch. 5), shadow maps with slope-scaled bias, PCF and two cascades (ch. 6), and SSAO (definition 9.5).
   • Data: the Crytek Sponza scene of the McGuire Computer Graphics Archive (free, textured, about 260 000 triangles); a 4096 spp reference rendered with Blender Cycles or Mitsuba 3.
   • Success criterion: 60 Hz at 1080p on a laptop GPU, with per-pass times from GPU timer queries; no visible shadow acne or peter-panning at three light angles; SSAO compared with ray-traced AO (example 9.6) and the frame with the reference (PSNR, difference images).
   • Deliverables: code, report, talk; chapters 2, 5, 6 and 9.
3. [Solo · ≈ 20 h] Rendering the ABO catalogue for text-to-product retrieval. Do views rendered with the physically based materials of chapter 5 retrieve products from their titles better than the catalogue photos, and how many views are needed?
   • Data: Amazon Berkeley Objects, ABO (Collins et al., CVPR 2022): about 8 000 artist-made 3D models in glTF 2.0, whose metallic-roughness materials use the GGX distribution, a Smith masking term and Schlick’s Fresnel (definition 5.3, example 5.4); licence CC BY-NC 4.0.
   • Work: a renderer (pyrender, or the tracer of solo project 1) with fixed cameras (ch. 2) and an environment light; text-to-image retrieval with OpenCLIP on 1, 4 and 12 views per object, against the catalogue images (CLIP: Multimodal Generative AI, ch. 5).
   • Success criterion: recall@10 on 500 held-out product titles, with bootstrap confidence intervals. The renders are the rendered-view tower of the Y2.3 transverse project.
   • Deliverables: code, report, talk; chapters 2, 5 and 9.

Group projects

1. [Group of 3 · ≈ 30 h each] Paper study: a kernel-predicting denoiser (Bako et al., 2017). At equal time, does a small learned denoiser beat both the à-trous filter of chapter 8 and simply tracing more samples?
   • Paper: S. Bako, T. Vogels, B. McWilliams, M. Meyer, J. Novák, A. Harvill, P. Sen, T. DeRose and F. Rousselle, “Kernel-Predicting Convolutional Networks for Denoising Monte Carlo Renderings”, SIGGRAPH 2017.
   • Data: about 20 scenes from Benedikt Bitterli’s public “Rendering Resources” (Mitsuba format), rendered with Mitsuba 3: 1–16 spp inputs, normal, depth and albedo buffers, 4096 spp references, and a camera fly-through for temporal tests.
   • Work: a U-Net that predicts a 21 × 21 kernel per pixel (Deep Learning, ch. 4), trained on albedo-demodulated radiance (definition 8.2); baselines: the edge-avoiding à-trous filter (Dammertz et al., HPG 2010) and Open Image Denoise as an upper reference.
   • Split: data generation; network and training; evaluation (PSNR, SSIM, temporal flicker).
   • Success criterion: PSNR-versus-time curves at equal wall-clock time on held-out scenes; the network beats à-trous at 4 spp, and its gain over more spp is measured.
   • Deliverables: code, an 8–10 page joint report, a 20-minute talk; chapters 7–8.
2. [Group of 3 · ≈ 30 h each] Paper study: NeRF versus 3D Gaussian Splatting (Mildenhall et al., 2020; Kerbl et al., 2023). On the same calibrated views, what does each representation buy in quality, training time and rendering speed?
   • Data: the NeRF-Synthetic “Blender” scenes released with the NeRF paper (800 × 800 views with known poses), plus views of one scene rendered by the group’s own path tracer, with poses written from definition 2.1.
   • Work: a minimal NeRF in PyTorch (positional encoding, stratified sampling, alpha compositing and the photometric loss of definition 10.3) and a minimal 3DGS (projected covariances, depth-sorted compositing, densification; the gsplat rasterizer is allowed for speed).
   • Split: NeRF; 3DGS; data, poses and evaluation (PSNR, SSIM, LPIPS, frames per second).
   • Success criterion: at least 25 dB mean PSNR on the held-out views of lego at 400 × 400 for both methods; one table of PSNR, training time, memory and frame rate on the same free-tier GPU, explained with theorem 10.2.
   • Deliverables: code, joint report, talk; chapters 2, 3 and 10. It fits the official paper-implementation assessment and uses the multi-view geometry of Image Analysis and Computer Vision, ch. 6.
3. [Group of 4 · ≈ 30 h each] Language to scene: an LLM writes the scene, the renderer checks it. Can a language model turn a prompt into a scene that a physically based renderer draws faithfully, and can the image be checked against the prompt?
   • Work: an open instruction-tuned LLM outputs a JSON scene graph (objects, GGX parameters, lights, camera) under a schema; a rasterizer or the path tracer renders it (ch. 2–5, 9). Relations are checked exactly on the scene graph, and a CLIP score or a vision-language model’s answer on the image is compared with these exact checks; failures drive up to three repair rounds.
   • Data: 200 prompts written by the team in the style of CLEVR (Johnson et al., CVPR 2017): counts, colours, materials, “left of”, “behind”.
   • Split: prompting and schema; renderer; image-side checks; repair loop and benchmark release.
   • Success criterion: the fraction of prompts whose scene satisfies every stated relation, before and after repair; the agreement rate between image-side and exact checks.
   • Deliverables: code, joint report, talk; chapters 2–5 and 9. It connects to Introduction to Text Mining and NLP, ch. 7 and to the scene-graph capstone Y1.5.

Written exam (proposal)

Two hours, one handwritten A4 sheet and a simple calculator allowed; four exercises: a course question, two computations and a short proof.

1. Question (4 points). Write the rendering equation and name each term. Explain why its bounce series converges, what a path tracer with Russian roulette estimates, and why photon mapping is biased but consistent.
   • Expected: definition 4.1 with emission, incident radiance, BRDF and cosine. If the albedo \rho < 1, the transport operator is a contraction, so L = \sum_k T^k L_e with a truncation error in \rho^{k+1} (theorem 4.2).
   • Path tracing averages unbiased estimates of this whole series (theorem 7.2); Russian roulette keeps them unbiased, since q \cdot X/q = X in expectation, while paths have unbounded length (definition 7.4).
   • Photon mapping averages photon power over a disc of radius r: a blur, hence a bias. As photons are added and r shrinks, bias and variance both vanish: consistent.
2. Question (6 points). A triangle has pixel coordinates a = (0, 0), b = (4, 0), c = (0, 4), camera depths (Z_a, Z_b, Z_c) = (1, 1, 3) and texture coordinates (u_a, u_b, u_c) = (0, 0, 1).
   • 1. Is the pixel centre p = (1, 1) inside? Give its barycentrics.
   • 2. Compute the perspective-correct u at p and compare with naive interpolation.
   • 3. Update the three edge functions incrementally to p + (1, 0).
   • Expected (a): the edge functions at p are (E_{ab}, E_{bc}, E_{ca}) = (4, 8, 4), all positive: inside. Their values at the opposite vertices all equal 16, so \lambda = (1/2,\ 1/4,\ 1/4).
   • (b): 1/Z = 1/2 + 1/4 + 1/12 = 5/6, so Z = 1.2; u/Z = (1/4)(1/3) = 1/12, hence u = 0.1, against the naive 0.25 (proposition 5.6).
   • (c): subtract b_y - a_y = 0, c_y - b_y = 4 and a_y - c_y = -4 (theorem 3.2): (4,\ 4,\ 8), which direct evaluation at (2, 1) confirms.
3. Question (5 points). Two strategies with densities p_1, p_2 draw one sample each. Prove that the MIS estimator with balance weights is unbiased, give the contribution of one sample, and explain why it avoids counting a light twice with next event estimation.
   • Expected: the estimator is F = w_1 f/p_1 at X_1 plus w_2 f/p_2 at X_2. Its expectation is \int w_1 f + \int w_2 f = \int f, because the weights sum to 1 where f \neq 0 and vanish where their density does (proposition 7.5).
   • With w_i = p_i/(p_1 + p_2), each sample contributes f(X)/(p_1(X) + p_2(X)), whichever strategy drew it.
   • Adding the two unweighted estimates has expectation 2\int f: the light is counted twice. The weights split the integral between the strategies instead.
4. Question (5 points). A ray crosses three intervals of length \delta = 1 with densities \sigma = (\ln 2,\ 0,\ \ln 4), colours c = (0.2,\ 0.9,\ 1.0) and a black background.
   • 1. Compute the \alpha_i, the transmittances and the composited colour \hat{C}.
   • 2. For an observed pixel C = 0.6 and the loss \mathcal{L} = (\hat{C} - C)^2, give \partial\mathcal{L}/\partial c_i for each i and interpret the zero.
   • Expected (a): \alpha = (0.5,\ 0,\ 0.75) and T = (1,\ 0.5,\ 0.5), so \hat{C} = 0.1 + 0 + 0.375 = 0.475; the background is seen through a transmittance of 0.125 (theorem 10.2).
   • (b): \mathcal{L} = 0.125^2 \approx 0.0156, and \frac{\partial\mathcal{L}}{\partial c_i} = 2(\hat{C} - C)\, T_i \alpha_i = (-0.125,\ 0,\ -0.094).
   • The empty interval (\alpha_2 = 0) gets no colour gradient: NeRF learns colours only where it has put density (definition 10.3).

Grading rubric (proposal)

References

• J. Kajiya, The Rendering Equation, SIGGRAPH 1986 — the founding paper; T. Whitted, An Improved Illumination Model for Shaded Display, CACM 1980.
• E. Veach, Robust Monte Carlo Methods for Light Transport Simulation, Stanford PhD thesis, 1997 (MIS, MLT — the MC reference of chapter 7); E. Veach, L. Guibas, Optimally Combining Sampling Techniques for Monte Carlo Rendering, SIGGRAPH 1995 (the balance heuristic of proposition 7.5).
• M. Pharr, W. Jakob, G. Humphreys, Physically Based Rendering: From Theory to Implementation (pbrt, 4th ed. online); tutorials: Ray Tracing in One Weekend; Scratchapixel.
• B. Phong, Illumination for Computer Generated Pictures, CACM 1975; R. Cook, K. Torrance, A Reflectance Model for Computer Graphics, SIGGRAPH 1981; C. Schlick, An Inexpensive BRDF Model for Physically-based Rendering, CGF 1994; B. Walter, S. Marschner, H. Li, K. Torrance, Microfacet Models for Refraction through Rough Surfaces, EGSR 2007 (GGX and its Smith factor); B. Burley, Physically-Based Shading at Disney, SIGGRAPH Course 2012.
• T. Möller, B. Trumbore, Fast, Minimum Storage Ray-Triangle Intersection, JGT 1997; I. Wald, On fast Construction of SAH-based Bounding Volume Hierarchies, EGSR 2007.
• B. Mildenhall, P. Srinivasan, M. Tancik, J. Barron, R. Ramamoorthi, N. Ng, NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis, ECCV 2020; B. Kerbl, G. Kopanas, T. Leimkühler, G. Drettakis, 3D Gaussian Splatting for Real-Time Radiance Field Rendering, SIGGRAPH 2023.
• A. Tewari et al., State of the Art on Neural Rendering, Computer Graphics Forum (EG STAR), 2020 — the survey of chapter 10 (followed by Advances in Neural Rendering, 2021).
• C. Chaitanya, A. Kaplanyan, C. Schied, M. Salvi, A. Lefohn, D. Nowrouzezahrai, T. Aila, Interactive Reconstruction of Monte Carlo Image Sequences Using a Recurrent Denoising Autoencoder, SIGGRAPH 2017; Open Image Denoise for the labs.
• T. Boubekeur, M. Alexa, Phong Tessellation, TOG 2008; J.-M. Thiery, E. Guy, T. Boubekeur, Sphere-Meshes: Shape Approximation using Spherical Quadric Error Metrics, TOG 2013; T. Groueix, S. Chaudhuri, D. Ceylan, et al., AtlasNet: A Papier-Mâché Approach to Learning 3D Surface Generation, CVPR 2018; tutorial CreativeAI.

A complete path tracer in Python

The program below (~70 lines, NumPy only) implements chapters 1 (radiance), 2 (camera), 4–5 (the bounces of the rendering equation, Lambertian materials), 6 (brute-force ray-sphere intersection, shadow rays), 7 (cosine sampling, next event estimation towards a point light, a fixed depth of 4 bounces instead of Russian roulette) and 8 (stochastic antialiasing, gamma); its bounces are the terms of the series of Theorem 4.2. Reduce W, H, SPP to test; at full resolution, expect ~1 minute (interpreter): each pixel is an independent MC estimator — parallelization is trivial.

import numpy as np

W, H, SPP = 240, 180, 8
def norm(v): return v / np.linalg.norm(v)

# scene: spheres (center, radius, albedo, albedo rho)
S = [(( 0.0, -100.0, -1.0), 100.0, (0.85, 0.80, 0.75), 0.6),   # ground
     (( 0.0,    1.0, -5.0),   1.0, (0.75, 0.25, 0.20), 0.7),
     ((-2.3,    0.7, -4.2),   0.7, (0.25, 0.70, 0.30), 0.7),
     (( 2.3,    0.7, -4.2),   0.7, (0.25, 0.35, 0.90), 0.7)]
Lp  = np.array([6.0, 9.0, 1.0])      # point light
I   = np.array([180.0, 170.0, 150.0])
SKY = np.array([0.45, 0.60, 0.85])   # sky radiance

def intersect(o, d):
    tbest, ibest = 1e9, -1           # first intersection (naive ch. 6)
    for i, (c, r, _, _) in enumerate(S):
        oc = o - c; b = oc @ d; disc = b*b - (oc @ oc - r*r)
        if disc > 0:
            t = -b - np.sqrt(disc)
            if 1e-3 < t < tbest: tbest, ibest = t, i
    return tbest, ibest

def trace(o, d):
    col, att = np.zeros(3), np.ones(3)          # L = sum of T^k Le (th. 4.2)
    for depth in range(4):                      # fixed depth, no Russian roulette (Ex. 10b)
        t, i = intersect(o, d)
        if i < 0:
            return col + att * SKY * max(0.1, d[1])
        c, r, albedo, rho = S[i]
        p = o + t*d; n = norm(p - c)
        # next event estimation: point light + shadow ray (ch. 7)
        Lv = Lp - p; dist = np.linalg.norm(Lv); L = Lv / dist
        if intersect(p + 1e-3*n, L)[0] > dist:
            col += att * rho * albedo / np.pi * I / dist**2 * max(0.0, n @ L)
        # diffuse bounce: cosine sampling (p = cos/pi, th. 7.3)
        g = np.random.normal(size=3)            # uniform direction on the sphere
        d = norm(n + g / np.linalg.norm(g))
        o, att = p + 1e-3*n, att * rho * albedo
    return col

img = np.zeros((H, W, 3))
for y in range(H):
    for x in range(W):
        acc = np.zeros(3)
        for _ in range(SPP):                    # stochastic AA (ch. 8)
            u = (x + np.random.rand()) / W  - 0.5
            v = 0.5 - (y + np.random.rand()) / H
            acc += trace(np.array([0.0, 1.2, 0.0]), norm([u, v, -1.0]))
        img[y, x] = acc / SPP

img = np.clip(img, 0, None) ** (1.0 / 2.2)      # gamma tone map (ch. 8)
img = img / img.max()                           # exposure normalization
with open("render.ppm", "wb") as f:
    f.write(f"P6\n{W} {H}\n255\n".encode())
    f.write((img * 255).astype(np.uint8).tobytes())