Open Jos Stam’s The Art of Fluid Animation to the core solver and you’ll find one time step built from four moves: add forces, diffuse, advect, project. Advection is particle tracing, which is the ODE integration you already know from 2.6 Vector Fields, Flow Lines & Streamlines. The other moves are different. Diffuse and project both come down to solving an equation built from a single operator, the Laplacian . This lesson closes the vector calculus module by putting together everything so far. You take the gradient (2.3 Gradient & Directional Derivatives), then the divergence (2.7 Divergence: Sources & Sinks), and the composition, , turns out to be the most important operator in mathematical physics. Then we use the identities from 2.8 Curl & Vorticity to prove the theorem that makes incompressible fluid solvers possible: every velocity field splits cleanly into a part that compresses and a part that swirls.
Transcript
- This is the last lesson of the vector calculus module, and it is the one where the whole module pays off inside a fluid solver.
- Here is one time step of Jos Stam's stable fluids solver: add forces, diffuse, advect, and project.
- Two of those four steps, diffuse and project, are really the same job: solving an equation built from one operator, the Laplacian. Let's understand that operator properly.
- The Laplacian of a function f is the divergence of its gradient. Take the gradient, which is a vector field, then measure how much that field spreads out. In coordinates it is f x x plus f y y.
- Picture a hill. Its gradient points uphill, so near the summit every arrow points inward. Arrows converging means negative divergence. At a peak, the Laplacian is negative.
- Flip it into a valley and the gradient points outward, away from the bottom. That's a source, so the Laplacian is positive. The Laplacian measures how much a point sits below its surroundings.
- Here is the cleanest way to see it. Take a point on a curve and its two neighbors, a distance h to each side. Average the neighbors: that's the middle of the dashed chord.
- At a peak, the neighbor average sits below the center. The gap, in red, is the second derivative times h squared over two. That comes straight from Taylor's theorem.
- Shrink h and the gap shrinks like h squared, but the ratio settles down to the second derivative. A second derivative is a comparison between a point and its neighborhood.
- In two dimensions, add the x and y versions. On a grid with spacing h, each point has four neighbors: east, west, north and south.
- The five point Laplacian adds up the four neighbors, subtracts four times the center, and divides by h squared. Rearranged, it is four over h squared, times the neighbor average minus the center.
- With these numbers and h equal to one: eighteen minus eight is ten. The neighbors average four and a half, well above the center value of two, so this point is a little valley and the Laplacian is positive.
- Now the payoff. The heat equation says u t equals kappa times the Laplacian: every temperature drifts toward the average of its neighbors. Here is a noisy grid, with the top edge held hot.
- Repeatedly replace each interior value by the average of its four neighbors. This is called Jacobi relaxation. Watch the rough noise vanish in a handful of sweeps.
- The smooth, large scale part takes far longer. Hundreds of sweeps later the grid settles into a steady state that no longer changes.
- In that steady state, every value equals the average of its neighbors. The Laplacian is zero everywhere inside. A function with zero Laplacian is called harmonic.
- Harmonic functions are as smooth and boring as possible. The value at a point equals its average over any circle around it, so there can be no peaks or pits inside: the extremes are always on the boundary.
- Jacobi relaxation kills wiggles fast but crawls on smooth errors, needing roughly N squared sweeps on an N by N grid. That one fact is why Stam's solver does twenty sweeps and hopes, and why serious codes use multigrid.
- So: the Laplacian is the divergence of the gradient, and on a grid it is just neighbor average minus center. Next, we use it to split any velocity field into two pieces, the secret behind Stam's projection step.
The Laplacian: divergence of the gradient
For a scalar field or , the Laplacian is
The two notations and mean the same thing. Physicists write (“del squared”) as a reminder that it is . Mathematicians and numerical analysts usually write . The derivation is one line: , and the divergence of a field is , so . The Laplacian takes in a scalar field and gives back a scalar field.
What the sign means. The gradient points uphill. Near the top of a hill every gradient arrow points inward, toward the summit. Arrows converging on a point means negative divergence (a sink, in the language of 2.7 Divergence: Sources & Sinks). So at a peak, . In a valley the gradient points outward, away from the bottom, so . On a tilted plane the gradient is constant and .
The divergence theorem from 2.10 Green, Stokes & the Divergence Theorem makes this precise. For a small disk around a point,
The right side is the net “uphill-ness” pointing out of the disk. If the boundary circle is, on average, higher than the centre, the gradient points outward across it and the integral is positive.
Worked example: computing Laplacians
Compute for (a) , (b) , (c) at the origin, (d) .
(a) and , so everywhere. The paraboloid is a bowl: every point sits below the average of its surroundings.
(b) and , so . This saddle curves up in and down in by exactly the same amount, and the two cancel.
(c) and . By symmetry , so . At the origin , a peak. For the Laplacian turns positive: out on the skirts, the surface curves upward like a bowl.
(d) and , so . The Laplacian just multiplies this function by , so is an eigenfunction of with eigenvalue . Hold on to that idea: in 8.3 Fourier Series and 9.3 Solving the Heat Equation: Separation of Variables, sines and cosines solve the heat equation precisely because the Laplacian only rescales them.
The Laplacian as “neighbourhood average minus centre”
The sign rule hides an exact statement. Let be the average of over a circle of radius centred at a point . Expand in a Taylor series around (2.4 Taylor Series in One and Several Variables). The point at angle on the circle is , so
Now average over . The averages of , and are all zero, and and each average to . Everything except the curvature terms drops out:
(The terms also average to zero, by odd symmetry, which is why the error is .) In 3D the same computation over a sphere gives .
This is why the Laplacian shows up wherever things even out. The heat equation , derived in 9.2 The Heat (Diffusion) Equation, says: a point that is colder than its neighbourhood average warms up, and one that is hotter cools down, at a rate proportional to the difference.
The 5-point discrete Laplacian
On a grid you don’t have derivatives, only values. Taylor series turns the averaging picture into a formula. In 1D, with spacing ,
Adding the two expansions cancels every odd term:
Do this once in and once in , and add. Writing for the centre value and for its east, west, north and south neighbours, the 5-point Laplacian is
The second form is the averaging picture again, now with the four nearest neighbours standing in for the circle. As a picture, the stencil of weights is
The error is : halve the grid spacing and the error drops by a factor of 4. This is a second-order method, in the sense of 4.1 Local Error, Global Error & Order. The full theory of finite-difference stencils is in 10.1 Finite Differences: Discretizing Derivatives.
Worked example: applying the stencil
(a) On a grid with , a cell has value 2 and its neighbours are 3 (east), 5 (west), 4 (north) and 6 (south). Find . (b) Show that the stencil is exact for for every . (c) Apply it to .
(a) . The neighbour average, 4.5, is well above the centre value 2, so this point is a small valley.
(b) In : , so the part contributes . The part contributes another 2, giving exactly. The error term involves the fourth derivatives, which are zero for a quadratic.
(c) Using ,
using . Since , the factor is . That is the exact eigenvalue plus an error, just as promised. Sines are eigenvectors of the discrete Laplacian too, which is the starting point of 10.3 Von Neumann Stability Analysis stability analysis.
Harmonic functions
A function with is called harmonic. You have already met several:
- linear functions ;
- and (saddles);
- and ;
- in 2D and in 3D, away from the origin.
In 2D, the real and imaginary parts of any complex-analytic function are harmonic: and come from , and comes from . That’s the engine behind classical airfoil theory (12.1 Airfoils, Circulation & Lift).
Harmonic functions are as smooth and featureless as a function can be, and two properties make that precise.
The first follows from our expansion. Differentiating and applying the divergence theorem to the disk gives , which is zero when is harmonic. So doesn’t depend on and equals its limit . The second follows from the first: an interior maximum would be the average of values around it, all of which are smaller. That’s impossible.
Physically, harmonic functions are steady states. They give the temperature in a plate once it has stopped changing ( in the heat equation), the shape of a soap film stretched over a slightly bent wire, the electrostatic potential in empty space, and the velocity potential of an ideal fluid in 11.7 Potential Flow & Stream Functions. Solving with given boundary values is the Laplace equation, the subject of 9.6 Laplace's Equation: Rectangles & Disks.
The discrete picture and Jacobi relaxation
On a grid, "" says . Every value is the average of its four neighbours. That suggests an algorithm. Hold the boundary values fixed, start from any guess inside, and repeatedly replace every interior value by the average of its neighbours, computed from the old values:
This is Jacobi relaxation. Notice that it is the same thing as , which is a forward-Euler step of the heat equation with the largest stable time step. Relaxation is just diffusion run until nothing changes any more.
Worked example: four unknowns by hand
A grid of points has its two top-middle boundary points held at and every other boundary point at . Find the four interior values of the discrete harmonic function, then run Jacobi from zero.
By left–right symmetry the two top interior values are equal (call them ) and so are the two bottom ones (call them ). The top-left interior point has neighbours (above), (left), (right) and (below):
The bottom-left interior point has neighbours (above), (left), (right) and (below):
Substituting gives , so and . Check the mean value property at : . ✓
Jacobi from all zeros gives:
| sweep | ||
|---|---|---|
| 0 | 0 | 0 |
| 1 | ||
| 2 | ||
| 3 | ||
| ∞ |
It converges, but each sweep only lets information travel one cell. As the extreme values must, the maximum and minimum sit on the boundary.
Things to try in the lab:
- Press Relax on the smooth blobs, then load rough noise and relax again. The noise is gone in a few sweeps, while the big blobs linger for hundreds. Averaging kills short wavelengths fast and long wavelengths slowly.
- Tick Show Δu. Peaks of show up blue () and valleys red. Relax and watch fade toward zero everywhere.
- Set the boundary to top edge +1 or left +1, right −1 and relax until “max |h²Δu|” is tiny. You now have a harmonic function. Hover over cells: the probe shows each value equals its neighbour average. Check that the interior range never exceeds the edge range (maximum principle).
- Load two held points, or paint your own holds, and relax. This is the electrostatic potential of two charged electrodes.
- Push α above 1. The update overshoots the average, and a checkerboard pattern explodes. This is the explicit heat-equation stability limit , which you’ll analyse in 10.2 Explicit Methods for the Heat Equation.
Compute the Laplacian of each function and decide whether it is harmonic (where defined).
- , for
- in 3D
- in 3D, for
Show worked solution
- and , so . Harmonic. (It is the real part of .)
- and , so . Harmonic.
- , so . By symmetry . The sum is , so it is harmonic away from the origin. (This is the potential of a 2D point source.)
- Each second partial is , so . Not harmonic: every point sits below its spherical average.
- and . Adding the three directions gives . Harmonic for . (This is the Newtonian and Coulomb potential.)
- Let . Compute the average of over the circle of radius centred at , and confirm that it equals for every .
- Repeat for . Show that the average is , and explain why this matches exactly, with no higher-order terms.
Show worked solution
- Parametrize the circle as , . Then . Averaging over , the and terms vanish and and both average to , so . ✓ This holds for every .
- In the same way, . Since , the formula gives . The match is exact because the remainder terms in the Taylor expansion involve third and higher derivatives, which all vanish for a quadratic.
Implement it: the stencil and Jacobi relaxation
The vector Laplacian
The Laplacian of a vector field, in Cartesian coordinates, is taken one component at a time:
This is the operator in the viscous term of Navier–Stokes: each velocity component diffuses like a temperature. That’s why Stam’s code simply calls diffuse once for u and once for v.
Key vector identities
Every identity below comes from writing out components and using one fact: mixed partial derivatives commute (Clairaut’s theorem, from 2.1 Functions of Several Variables & Partial Derivatives). Here are scalar fields and vector fields, all smooth.
| # | Identity | In words |
|---|---|---|
| 1 | gradients never swirl | |
| 2 | curls never pile up | |
| 3 | definition of the Laplacian | |
| 4 | “curl curl = grad div − Laplacian” | |
| 5 | product rule for divergence | |
| 6 | product rule for curl | |
| 7 | $(\vec u\cdot\nabla)\vec u = \nabla\big(\tfrac12 | \vec u |
Proof of 1. The -component of is , and the other two components work the same way.
Proof of 2. With ,
because the terms cancel in pairs. In 2D the same facts read: the scalar curl of a gradient, , is zero, and any field of the form built from a stream function automatically has zero divergence.
Identity 7 matters later: it lets you write the Euler equations in terms of vorticity and derive Bernoulli’s principle (11.3 Momentum: Pressure & the Euler Equations, 11.6 Vorticity Dynamics & Circulation). Identity 4 has a nice consequence for incompressible flow. If , then with , so viscosity acts by diffusing vorticity.
Worked example: checking curl curl = grad div − Δ
Verify identity 4 for .
Left side. First compute the curl:
Call that . Then take its curl:
Right side. , so . Componentwise, . The difference is . ✓
- Use identity 5 to show .
- Integrate over a region and apply the divergence theorem to get Green’s first identity
- Suppose in and on . Take and conclude that . Why does this show the Poisson problem , with given on the boundary, has at most one solution?
Show worked solution
- Apply identity 5 with : .
- Integrate both sides over . The divergence theorem turns the left side into .
- With , the boundary term vanishes because there, and . That leaves . The integrand is non-negative, so everywhere, so is constant, and since it is on the boundary, . If and both solve the Poisson problem, then satisfies with on the boundary, so . (With Neumann data instead, the same argument gives uniqueness up to an added constant, which is exactly the situation for pressure.)
Transcript
- Every incompressible fluid solver hides one beautiful theorem at its core. Let's find it.
- After a solver adds forces and advects, its velocity field is a mess. Somewhere fluid is piling up or being created from nothing. Here, the left blob spreads out like a source, and the right blob spins.
- Real water can't do the first thing. An incompressible flow must have zero divergence everywhere. The spinning is fine. So we want to remove the spreading part, and only that part.
- Two identities make this possible. First: the curl of a gradient is always zero. A pure slope field has no swirl.
- Second: the divergence of a curl is always zero. A pure swirl field never piles up fluid.
- Both come from the same fact. Write them out and every term is a mixed partial derivative minus the same partial taken in the other order, which is zero.
- The Helmholtz Hodge decomposition says: any velocity field w splits, uniquely, into the gradient of a scalar potential phi, plus a field u with zero divergence.
- The gradient part has no curl, so it carries all of the spreading. The divergence free part carries all of the swirl. Throw away the gradient, and what's left is a legal incompressible flow.
- How do we find phi? Take the divergence of both sides. The divergence of u is zero by design, and the divergence of a gradient is the Laplacian.
- So phi solves a Poisson equation: Laplacian of phi equals the divergence of w. Solve it, subtract the gradient, and you are done.
- Even better, the two pieces are perpendicular, in the sense that the integral of their dot product is zero. Integrate by parts once and the divergence of u kills it.
- So energies add like Pythagoras. Projection is an honest orthogonal projection: it removes exactly the compressive energy and never adds any.
- In Fourier space the whole thing becomes linear algebra you already know. Each wave in the field has a wave vector k and an amplitude vector w hat.
- The part along k is compression, the gradient piece. The part perpendicular to k is pure swirl. Projection just deletes the parallel component, one wave at a time. This is exactly Stam's FFT fluid solver.
- Now open Stam's project routine. It computes the divergence, solves a Poisson equation with twenty Gauss Seidel sweeps, and subtracts the gradient. That is Helmholtz Hodge, line for line.
- And the potential it solves for is the pressure. Pressure is whatever push is needed, at every point, to keep the fluid from compressing.
- Recap: split the field, solve one Poisson equation for the potential, keep the divergence free part. That completes our vector calculus toolkit. Next module, we start solving differential equations themselves.
The Helmholtz–Hodge decomposition
Here is the theorem this module has been building toward.
By identities 1 and 2, the pieces have complementary jobs. , so all the divergence lives in the gradient part. And , so all the curl lives in the divergence-free part. (On regions with holes, or with other boundary conditions, a third piece can appear: a field that is both curl-free and divergence-free, i.e. a gradient of a harmonic function. On a periodic box it is just a constant, as a quiz below explores.)
How to compute it. Take the divergence of :
That is a Poisson equation for . Its boundary condition comes from dotting with : . Solve it, and then set . The whole decomposition costs one Laplacian solve.
Why it’s unique, and why the pieces are orthogonal. Integrate over using identity 5 with and :
So the two pieces are orthogonal in the sense of the integral inner product, and energies add like Pythagoras:
Uniqueness follows: if , then is both a gradient and divergence-free with zero normal component. Being orthogonal to itself, it is zero.
Worked example: a decomposition by hand
On the periodic square , decompose .
The field points straight up or down everywhere, so it is not obvious that it contains either a source or a vortex. Compute its divergence:
Solve . We saw in the first worked example that , so works. It is periodic, and it is unique up to a constant. Then
Checks. . ✓ The 2D curl of is . ✓ Orthogonality: , and over the square each term integrates to , so the integral is . ✓ Energies: , and , which add up correctly. ✓
So a purely vertical field is secretly half a checkerboard of sources and sinks (the cells where has extrema) and half a checkerboard of counter-rotating vortices. Load w = (0, 2 sin x cos y) in the lab below and you’ll see exactly this split.
Things to try:
- With source + vortex loaded, colour by divergence: the red blob appears in and , but is blank. Switch to curl: now is blank and carries it all.
- Read the energy line: equals the sum of the two parts, and to machine precision.
- Choose Gauss–Seidel sweeps and set the slider to Stam’s default of 20. The ”% left” divergence is far from zero, and the energies no longer add up. Drag the slider to 400 and watch it converge.
- Pick empty and stir. Even pushing in a single straight line creates both parts: fluid shoved forward has to go somewhere, so the divergence-free part curls around into a vortex pair. That’s the classic smoke-ring shape you get in Stam’s demo.
Stam’s projection step is Helmholtz–Hodge
The incompressible Navier–Stokes equations (11.4 Viscosity & the Navier–Stokes Equations) are
There’s no equation for the pressure . It is whatever it needs to be to keep . Stam’s splitting handles this by first ignoring the pressure. Starting from a divergence-free , he adds forces, advects and diffuses to get an intermediate field that is generally not divergence-free. Then he restores incompressibility by subtracting a gradient:
Compare with Helmholtz–Hodge. This is the decomposition, with . Taking the divergence gives the pressure Poisson equation . The pressure is the potential of the compressive part of the velocity, the push needed at each point to cancel it. Stam’s project routine does exactly this, written here in NumPy for a periodic grid:
def project(u, v, h, sweeps=20):
div = (np.roll(u, -1, 1) - np.roll(u, 1, 1) + np.roll(v, -1, 0) - np.roll(v, 1, 0)) / (2 * h) # 1. divergence
p = np.zeros_like(div)
for _ in range(sweeps): # 2. relax Δp = div
p = (np.roll(p, -1, 1) + np.roll(p, 1, 1) + np.roll(p, -1, 0) + np.roll(p, 1, 0) - h * h * div) / 4
u -= (np.roll(p, -1, 1) - np.roll(p, 1, 1)) / (2 * h) # 3. subtract the gradient
v -= (np.roll(p, -1, 0) - np.roll(p, 1, 0)) / (2 * h)
return u, v
Line for line: compute the divergence, relax a Poisson equation with the 5-point Laplacian (Stam uses Gauss–Seidel in place, written as Jacobi here), subtract the gradient. (Stam’s code folds the factors of and into , and handles walls with the boundary condition instead of periodicity. His compact 5-point Laplacian also doesn’t exactly match the wide central-difference divergence and gradient, so even a fully converged solve leaves a little grid-scale divergence. The lab and the cells below use a matched stencil.) Everything in this lesson shows up here:
- The Laplacian and its averaging form are the relaxation loop.
- Jacobi/Gauss–Seidel slowness is why 20 sweeps leave visible divergence behind, so smoke slowly loses or gains volume. Production solvers use preconditioned conjugate gradients or multigrid.
- Curl of gradient = 0 guarantees that subtracting leaves the vorticity, the swirls you want to see, untouched.
- Orthogonality guarantees that projection only ever removes kinetic energy. It can’t make the simulation blow up, which is half of why Stam’s method is “stable”.
Implement it: the projection
First, see the cost of Stam’s 20 sweeps for yourself. This cell projects a messy field with a relaxation Poisson solver and reports how much divergence is left after each number of sweeps. (It uses a stencil that reaches two cells, matched to the central-difference divergence, so that the converged answer is exactly divergence-free on the grid.)
Now write the exact version, Stam’s FFT projection.
On the periodic square , let .
- Find and in the Helmholtz–Hodge decomposition .
- Check that and that and have the same curl.
- Verify the energy identity .
- If were Stam’s intermediate velocity, with density and time step , what pressure would the projection compute?
Show worked solution
- . Solve : since , take (plus any constant). Then and . The constant wind and the shear both end up in .
- . ✓ The scalar curl is for , and for . ✓ (And has curl .)
- Over the square, , and , and every cross term (like or ) integrates to zero. So . Also and . Then . ✓
- , so plus an arbitrary constant. The compression wave is removed by a pressure that is high where fluid is piling up. At , is decreasing in : fluid arrives faster than it leaves, so the convergence is greatest there, and that is exactly where peaks.
A single Fourier mode of a 2D velocity field has wave vector and amplitude .
- Compute the projection matrix and the divergence-free amplitude .
- Check that , and that .
- Explain in one sentence why applying the projection twice in a solver does nothing the second time, and why that’s a good sanity test for your own
project()code.
Show worked solution
- and , so . Then . The removed (gradient) part is , parallel to as it should be.
- . ✓ For : . ✓
- Once a field is divergence-free, its gradient part is zero, so projecting again removes nothing (). If running your
project()twice changes the field noticeably, your Poisson solve hasn’t converged or your discrete divergence and gradient don’t match.
Key Takeaways
- The Laplacian is the divergence of the gradient. It is negative at peaks and positive in valleys.
- Precisely, is proportional to (neighbourhood average − centre value). That’s why it drives diffusion, heat flow and viscosity: things relax toward their local average.
- The 5-point stencil is the same idea on a grid, and it is second-order accurate by Taylor’s theorem.
- Harmonic functions () equal their own averages: the mean value property, no interior extremes. Jacobi relaxation finds them by repeated averaging, quickly for wiggles and slowly for smooth errors.
- The vector Laplacian acts componentwise in Cartesian coordinates, and .
- curl grad = 0 and div curl = 0 (consecutive steps in grad → curl → div) both follow from equal mixed partials, and they hold exactly for matched discrete operators.
- Helmholtz–Hodge: every field splits uniquely and orthogonally as with . Find by solving .
- Stam’s projection step is exactly this decomposition, with pressure as the potential. One Poisson solve per time step makes the flow incompressible, keeps the vorticity and never adds energy.