Max Slater

Monte Carlo Crash Course

Sat, 02 Aug 2025 00:00:00 +0000

Quasi-Monte Carlo

We’ve learned how to define and apply Monte Carlo integration—fundamentally, it’s the only tool we need. In the remaining chapters, we’ll explore ways to reduce variance and successfully sample difficult distributions.

Variance & Correlation
Stratified Sampling
Adaptive Sampling
Latin Hypercube
Quasi-Monte Carlo
Low-Discrepancy Sequences

Variance & Correlation

In chapter two, we determined that the variance of a Monte Carlo estimator is inversely proportional to its sample count. Empirically, we confirmed that our integrators’ expected error scaled with $$\frac{1}{\sqrt{N}}$$ in any dimension.

Monte Carlo Crash Course

Sat, 19 Apr 2025 00:00:00 +0000

Case Study: Rendering

So far, we’ve explored Monte Carlo methods using simple examples, like sampling the unit disk and sphere. Now, we’ll apply Monte Carlo to a more realistic task: simulating light traveling through a scene, or rendering.

Direct Lighting
Indirect Lighting
Specular Reflection
Challenges

Direct Lighting

To keep our focus on Monte Carlo methods, we’ll use a simplified model of light transport in two dimensions.¹ In particular, we will only define one quantity: radiance. Denoted as $$\mathcal{L}_i(\mathbf{x},\theta)$$, radiance measures the amount of light arriving at a point $$\mathbf{x}$$ from an incoming direction $$\theta$$.

Monte Carlo Crash Course

Sat, 12 Apr 2025 00:00:00 +0000

Sampling

In the previous chapter, we assumed that we can uniformly randomly sample our domain. However, it’s not obvious how to actually do so—in fact, how can a deterministic computer even generate random numbers?¹

Pseudo-Random Numbers
Uniform Rejection Sampling
Non-Uniform Rejection Sampling
Inversion Sampling
Changes of Coordinates

Pseudo-Random Numbers

Fortunately, Monte Carlo methods don’t need truly random numbers. Instead, we can use a pseudo-random number generator, or PRNG. Given a random initial state (the seed), a PRNG produces a deterministic stream of numbers that look uniformly random:²

Monte Carlo Crash Course

Sat, 05 Apr 2025 00:00:00 +0000

Exponentially Better Integration

By introducing randomness, Monte Carlo methods let us integrate high-dimensional functions exponentially faster than traditional algorithms.

Integration
Quadrature
The Curse of Dimensionality
Monte Carlo Integration
Why Monte Carlo?
Bias and Consistency
Non-Uniform Sampling

Integration

Many interesting quantities can be computed by evaluating an integral. For example:

Monte Carlo Crash Course

Sat, 29 Mar 2025 00:00:00 +0000

Continuous Probability

Prerequisites: Discrete Probability, Multivariable Calculus

This chapter is a condensed review of continuous probability. If you’re comfortable working with continuous random variables, you may want to skip it.

Random Variables
Probability Mass vs. Probability Density
Cumulative Distributions
Joint Distributions
Dependence
Expectation
Probability Bounds
Variance
Covariance
Dirac Delta Distributions

Random Variables

In the discrete world, a random variable represents a random process with countably many outcomes. The classic example is flipping a fair coin: let’s say $$X$$ evaluates to $$1$$ if our coin comes up heads, otherwise $$0$$. Because there are only two possibilities, $$X$$ is a discrete random variable.

Oxidizing C++

Sat, 06 Jan 2024 00:00:00 +0000

I recently rewrote the C++20 infrastructure I use for personal projects, so I decided to snapshot its design in the style of another recent article. The library (rpp) is primarily inspired by Rust, and may be found on GitHub.

I use rpp as an STL replacement, and I’m currently developing a real-time path tracer with it. rpp attempts to provide the following features, roughly in priority order:

Fast Compilation

Including rpp should not increase compilation time by more than 250ms. It must not pull in any STL or system headers—they’re what’s really slow, not templates.

Functions are Vectors

Sat, 29 Jul 2023 00:00:00 +0000

Conceptualizing functions as infinite-dimensional vectors lets us apply the tools of linear algebra to a vast landscape of new problems, from image and geometry processing to curve fitting and machine learning.

Prerequisites: introductory linear algebra, introductory calculus, introductory differential equations.

This article received an honorable mention in 3Blue1Brown’s Summer of Math Exposition 3!

Functions as Vectors
Applications

Functions as Vectors

Vectors are often first introduced as lists of real numbers—i.e. the familiar notation we use for points, directions, and more.

Optimizing Open Addressing

Sun, 08 Jan 2023 00:00:00 +0000

Your default hash table should be open-addressed, using Robin Hood linear probing with backward-shift deletion. When prioritizing deterministic performance over memory efficiency, two-way chaining is also a good choice.

Code for this article may be found on GitHub.

Open Addressing vs. Separate Chaining
Benchmark Setup
Discussion
Unrolling, Prefetching, and SIMD
Benchmark Data

Open Addressing vs. Separate Chaining

Most people first encounter hash tables implemented using separate chaining, a model simple to understand and analyze mathematically. In separate chaining, a hash function is used to map each key to one of $$K$$ buckets. Each bucket holds a linked list, so to retrieve a key, one simply traverses its corresponding bucket.

Spherical Integration

Sun, 08 Jan 2023 00:00:00 +0000

Or, where does that $$\sin\theta$$ come from?

Integrating functions over spheres is a ubiquitous task in graphics—and a common source of confusion for beginners. In particular, understanding why integration in spherical coordinates requires multiplying by $$\sin\theta$$ takes some thought.

The Confusion

So, we want to integrate a function $$f$$ over the unit sphere. For simplicity, let’s assume $$f = 1$$. Integrating $$1$$ over any surface computes the area of that surface: for a unit sphere, we should end up with $$4\pi$$.

Exploring Neural Graphics Primitives

Sun, 27 Nov 2022 00:00:00 +0000

Neural fields have quickly become an interesting and useful application of machine learning to computer graphics. The fundamental idea is quite straightforward: we can use neural models to represent all sorts of digital signals, including images, light fields, signed distance functions, and volumes.

Neural Networks
Compression
A Basic Network
Activations
Input Encodings
- Positional Encoding
- Instant Neural Graphics Primitives
More Applications
- SDFs
- NeRFs
References

Neural Networks

This post assumes a basic knowledge of neural networks, but let’s briefly recap how they work.

Differentiable Programming from Scratch

Sun, 31 Jul 2022 00:00:00 +0000

Differentiable programming has become a hot research topic, and not only due to the popularity of machine learning frameworks like TensorFlow, PyTorch, and JAX. Many fields apart from machine learning are finding differentiable programming to be a useful tool for solving optimization problems. In computer graphics, differentiable rendering, differentiable physics, and neural representations are all gaining popularity.

This article received an honorable mention in 3Blue1Brown’s Summer of Math Exposition 2!

Prerequisites
- Differentiation
- Optimization
Differentiating Code
- Numerical
- Symbolic
Automatic Differentiation
- Forward Mode
- Backward Mode
De-blurring an Image
Further Reading

Prerequisites

Differentiation

It all starts with the definition you learn in calculus class:

Exponentially Better Rotations

Fri, 15 Apr 2022 00:00:00 +0000

If you’ve done any 3D programming, you’ve likely encountered the zoo of techniques and representations used when working with 3D rotations. Some of them are better than others, depending on the situation.

Based on CMU 15-462 course materials by Keenan Crane.

Representations
The Exponential and Logarithmic Maps
Further Reading

Representations

Rotation Matrices

Linear-algebra-wise, the most straightforward representation is an orthonormal 3x3 matrix (with positive determinant). The three columns of a rotation matrix specify where the x, y, and z axes end up after the rotation.

Graphics Blogroll

Mon, 04 Apr 2022 00:00:00 +0000

In no particular order, my list of cool computer graphics (or adjacent) blogs:

Amit Patel
- Interactive algorithm visualizations
- Highlight: Introduction to A*
Inigo Quilez
- 3D SDF rendering and shadertoy
- Highlight: 3D SDF Index
Fabian Giesen
- Graphics programming, compression, comp arch, optimization
- Highlight: A trip through the Graphics Pipeline 2011
Bart Wronski
- Image/signal processing, real-time rendering, linear algebra
- Highlight: Why are video game graphics still a challenge?
Nicky Case
- ‘Explorable’ real-world models, mental health
- Highlight: Parable of the Polygons
Bartosz Ciechanowski
- Highly detailed interactive explainers
- Highlight: Cameras and Lenses
Alan Zucconi
- Misc. game dev
- Highlight: Introduction to Procedural Animations
Marc Ten Bosch
- Higher-dimensional physics, math
- Highlight: Let’s remove Quaternions from every 3D Engine
Alan Wolfe
- Image/signal processing, real-time graphics, math
- Highlight: Fast Voronoi Diagrams and Distance Fields with Jump Flooding
Fabien Sanglard
- Retro game dev, graphics programming, game engines
- Highlight: Deciphering the Postcard Sized Pathtracer
Matt Pettineo
- Graphics programming, real-time rendering
- Highlight: Attack of the Depth Buffer
Aras Pranckevičius
- Misc. graphics, optimization, compilers
- Highlight: “Modern” C++ Lamentations
Ben Golus
- Graphics programming, shaders
- Highlight: The Quest for Very Wide Outlines
Kostas Anagnostou
- Graphics programming, rendering
- Highlight: Raytraced Global Illumination Denoising
Adrian Courrèges
- Real-time rendering
- Highlight: DOOM 2016 Graphics Study
Raph Levien
- 2D rendering, GPU programming
- Highlight: Swapchains and Frame Pacing
Matt Pharr
- Rendering, optimization
- Highlight: Rendering Moana with pbrt
János Turánszki
- Real-time rendering, game engines
- Highlight: Voxel-based Global Illumination
Arseny Kapoulkine
- Real-time rendering, optimization
- Highlight: Writing an efficient Vulkan renderer
Daniel Holden
- Animation, geometry, game dev
- Highlight: Game Developer’s Spring-Roll-Call
Dennis Gustafsson
- Real-time rendering, game dev, GPU programming
- Highlight: Bokeh depth of field in a single pass
Mykhailo Moroz
- Computational physics, rendering and other random stuff
- Highlight: Visualizing General Relativity
Jeremy Ong
- Computer graphics, C++
- Highlight: Graphics Pipelines for Young Bloods

Inactive

These blogs are very infrequently updated or haven’t been updated in years.

Hamming Hats

Tue, 29 Mar 2022 00:00:00 +0000

I first encountered the Hat Problem in CMU 15-251.

The Game

Consider a 31-person game in which each player is assigned a red or blue hat, determined by an independent coin flip. The players can see everyone else’s hat color, but not their own.

Once the game begins, no communication is allowed between the players. After looking at the other hats, all players must simultaneously either announce the color of their hat or abstain from guessing. If at least one player correctly announces their hat color, and no players incorrectly announce theirs, the group wins.

A Compiler Bug

Sun, 26 Dec 2021 00:00:00 +0000

While I was working on Dawn, I ran into a curious bug in the Visual Studio 2019 C++ compiler. I reported it to the bug tracker, where it was confirmed to be an interference analysis issue. It was eventually fixed nearly a year later in 2020. Today, let’s investigate what the issue really was.

The problem arose upon implementing a Perlin noise type for procedural solid texturing. My type just so happened to include a 4-kilobyte array of pre-initialized random data, and this precise size caused writes to the data to interfere with a preceding struct member. The resulting bug ended up bricking the output of my path tracer—but only with optimizations enabled!

Exile: Voxel Rendering Pipeline

Sun, 26 Aug 2018 00:00:00 +0000

No, Exile is not technically a “voxel” engine. A real voxel engine unifies objects, textures, and more into colored voxel data, rendering them via raymarching/marching cubes/dual contouring/etc., and can target realism. Instead, Exile is a “voxel” engine in that it’s a traditional 3D engine that happens to focus on representing and drawing textured cubes (like Minecraft and related games). Some have worked around this linguistic confusion by christening the technique “Swedish cubes,” but for the purposes of this post, please assume that voxels imply textured cubes.

Exile: Reflection

Tue, 14 Aug 2018 00:00:00 +0000

Reflection, specifically type introspection, is an immensely useful feature provided by many modern languages. By enabling a way to inspect the properties of types—for example, the types of the members of a data structure—introspection allows one to write much more general and effective generic code, enforce interfaces/contracts, and more.

Though most obviously found in dynamic languages, introspection is just as powerful in compiled, statically typed languages such as Go and Jai. Unfortunately, C++ does not support run-time reflection as a language feature, and template-based compile-time reflection is neither standardized, elegant, nor robust. Standards proposals have been put forward to add reflection features, but an implementation is still rather far off. Further, libraries aiming to provide run-time reflection typically require laborious setup for each type you wish to make reflect-able.

Exile: Hot Reloading

Wed, 25 Jul 2018 00:00:00 +0000

In game development, one of the main appeals of implementing game logic in a managed language (e.g. Lua) is the convenient ability to make changes to the source, reload the logic on the fly, and test the changes without restarting the game. A similar technique is also often used to refresh shader programs, asset files, and the like.

However, supporting a secondary scripting language—or using one from the start—is not the only way to implement such a system. While more complicated, implementing the ability to reload code on a binary level allows one to make potentially deep changes to the entire source and have them appear instantly. Further, any language that can export C ABI compatible functions can be dynamically loaded in this manner, using good old-fashioned dynamic link libraries/shared objects.