You already know how to solve differential equations, in the sense that matters for a simulator. You have written Euler’s method and three of its variants, you have written RK4, and you have written RK45 with step-size control. You hand each of them a function and a starting value and they march forward. What none of those programs ever told you is what you are actually marching toward: what object is the solver approximating, when is there exactly one, and how would you recognize it if someone handed you a formula? This lesson builds that missing theory. It is the first lesson of the module on first-order ODEs, and its main habit is one you will use in every lesson after it: put the analytic solution next to the numerical one. Your solvers are the experiment; the theory tells you what the experiment should show.
The reason this matters for the fluid simulator you want to build is simple. Every law that governs air and water is a statement about a rate of change: how fast momentum, mass, heat or concentration changes at a point. We met the language for rates of change in space in 2.1 Functions of Several Variables & Partial Derivatives. Here we start with the simplest case, rates of change in time of a single number, and get the vocabulary straight: ODE, PDE, order, linear, solution, family, initial value problem.
Transcript
- You have written Euler, R K four and R K forty-five. Each one marches forward, but toward what? Today we name the thing your solvers are approximating.
- Almost no law of nature tells you what a quantity is. It tells you how fast the quantity is changing, given the situation it is in. Force sets the rate of change of velocity. The gap between a cup and the room sets how fast it cools. The number of rabbits sets how fast the rabbits multiply.
- Notice what every one of these has in common. The unknown is not a number. It is a whole function of time, and the equation ties that function to its own derivative. That is the definition of a differential equation.
- And this is the fluids connection. The equation that governs every flow you want to simulate is Newton's second law written per unit volume: density times the acceleration of a blob of fluid, equal to the sum of the forces on it, pressure, viscosity and gravity. It is the same idea as F equals m d v d t.
- If the unknown depends on one variable, the equation has only ordinary derivatives, and we call it an O D E. A cup of coffee has one temperature. If the unknown depends on several variables, there are partial derivatives, and it is a P D E. A rod has a temperature at every point along it. We spend this module on O D Es first, because they are the engine inside a P D E solver.
- The order of an equation is the order of the highest derivative in it, shown here in amber. One prime, order one. Two primes, order two. An equation of order n leaves n free constants, and needs n pieces of starting data. That is why your solvers carry a state vector as long as the order.
- Next, linear versus nonlinear. An equation is linear when the unknown and its derivatives appear only to the first power, multiplied by functions of t, and added up. These two, in green, are linear. The coefficients can be as wild as you like.
- These three, in red, are nonlinear. The unknown is squared, or multiplied by its own derivative, or sits inside a sine. This matters, because linear equations can be built from simple pieces by superposition, and nonlinear ones cannot. In the Navier Stokes equation, the one nonlinear term is the advection term, and it is the source of turbulence.
- How do you check that a function really is a solution? You substitute. Here is the equation y prime equals two y, and a candidate, C times e to the two t.
- Differentiate the candidate, and the left side is two C e to the two t. Put the candidate into the right side, and it is two C e to the two t again. The two sides match for every t and every C, so it is a solution.
- Now a function that fails. Try y equals t squared in the same equation. The left side is two t, the right side is two t squared.
- They agree at t equals zero, and again at t equals one. Two lucky points. But at t equals two, the left side is four and the right side is eight. A solution has to make the equation true for every t on the interval, not at a couple of points.
- Here is the visual heart of the lesson. The equation y prime equals y, the growth law with rate constant one, is solved by y equals C e to the t. Watch what happens as we turn the dial on C.
- Make C bigger, and the curve climbs faster. Take C to zero, and it flattens onto the axis, which is a solution too. Make C negative, and the whole curve flips below the axis. Every one of these curves is a solution of the very same equation.
- So the equation does not have one solution. It has a whole one-parameter family, filling the plane, with one free constant. Differentiating kills constants, so undoing a derivative cannot recover them. That is where the constant comes from, and an equation of order n leaves n of them.
- A family of solutions is not yet a prediction. To say what a quantity will actually do, you must say where it starts. An initial condition names one point, t zero, y zero, in the plane.
- Of all the curves in the family, exactly one passes through that point. Fix the point, and you fix the constant, C equals y zero times e to the minus t zero, about point nine one. That is an initial value problem: an equation plus a starting point, and it picks out a single curve.
- Now the habit this whole module is built on: put the analytic solution next to a numerical one. That amber curve is the exact solution of the initial value problem. Your Euler solver, with a step of one half, takes these red steps.
- Every red step lands below the curve, and the gap grows. By t equals two, Euler is off by about one point seven. Run R K four at the very same step size, in green, and the points sit on the curve, with an error of about three thousandths.
- To recap. A differential equation relates an unknown function to its own derivatives. Its general solution is a whole family of curves, and an initial condition picks out one. Euler, R K four and R K forty-five are all ways of computing that one curve.
- Next, we turn the equation itself into a picture, with direction fields, and watch Euler's method walk along them.
Laws of nature are statements about rates of change
Almost nothing in physics tells you directly what a quantity is. It tells you how the quantity changes, given the situation it is currently in. That is a much easier thing to know. You cannot write down, from first principles, the temperature of a cup of coffee at 3:17 pm. But you can say how fast it is cooling right now, given how much hotter it is than the room. Each law below is of that kind.
Newton’s second law. A body of mass with position and velocity obeys
The force depends on the situation (a spring gives , gravity gives ) and the equation turns it into a rule for how the velocity changes. Newton’s own statement in the Principia says, in words, that a change of motion is proportional to the force impressed. Reading “motion” as momentum gives in modern notation, a statement about the rate of change of momentum. Keep that form in mind; it comes back below.
Newton’s law of cooling. An object at temperature in surroundings at temperature loses heat at a rate proportional to the temperature difference:
The sign says: hotter than the room means (cooling), colder than the room means (warming). The proportionality is an assumption, a good one for small temperature differences.
Exponential growth. If every individual in a population reproduces at the same per-capita rate , then the total growth rate is proportional to the number of individuals:
Radioactive decay. Each nucleus decays independently, with a fixed probability per unit time , so the expected rate of loss is proportional to how many are left:
Drag. A skydiver of mass falling with velocity is pulled down by gravity and slowed by a drag force. For modest speeds, take the drag to be proportional to the speed, :
Notice what these have in common. The unknown is a function (, , , ), and the equation relates the function to its own derivative. That is the definition we want.
Worked example: the drag law reads off a speed limit. Take kg, m/s², kg/s. Dividing by :
You can learn a lot without solving anything. When the right side is , so the skydiver accelerates at . As grows the right side shrinks, and it hits zero at m/s. At that speed , so the velocity stops changing. That is the terminal velocity, and we found it by asking where the rate of change vanishes. We will return to this model with a full solution in the problems below.
ODEs, PDEs, and order
All five laws above have one independent variable, time. A differential equation whose unknown depends on one independent variable, so that it involves only ordinary derivatives, is an ordinary differential equation (ODE). If the unknown depends on several variables and the equation contains partial derivatives, it is a partial differential equation (PDE).
The classic first PDE is the heat equation. Let be the temperature at position along a rod at time :
It says that, at each position, the temperature rises where the profile curves upward and falls where it curves downward. You can read that sentence off the subscripts because of 2.1 Functions of Several Variables & Partial Derivatives. The cup of coffee has a single temperature and obeys an ODE; the rod has a temperature at every point and obeys a PDE.
Why do we spend a whole module on ODEs before touching PDEs? Two reasons. First, the tools (families of solutions, initial conditions, linearity, stability) are the same, and they are easier to see with one variable. Second, and this is the bridge to your simulator: when you put a grid of points along the rod and replace with the stencil , you get
which is a system of coupled ODEs, one per grid cell. This is the method of lines, and it is how a time-stepping PDE solver is organized: discretize space, and hand the resulting ODEs to a time integrator like RK4. We build that in 10.2 Explicit Methods for the Heat Equation. So an ODE solver is the engine inside a PDE solver, and understanding ODEs is not a detour.
Order
The order of a differential equation is the order of the highest derivative that appears.
| Equation | Order | Why |
|---|---|---|
| 1 | only | |
| 2 | highest derivative is | |
| 2 | is the highest derivative; the exponent on is a power, not an order | |
| 3 | highest derivative is |
Order tells you how much information it takes to pin down a solution. Integrating once introduces one constant of integration, so an order- equation leaves free constants and needs pieces of starting data: , then , and so on up to . You have already met this in code. A second-order equation such as becomes a first-order system by naming the derivative, :
Your solvers take a state vector, and the length of that vector is the order. Whenever you simulated something with second derivatives (a spring, a pendulum, a falling body under gravity), the state was position and velocity, because the order was two. We will treat such systems properly in 7.1 Systems & State Space; the rest of this module stays with first-order equations .
Linear or nonlinear?
Within ODEs, the most important split is between linear and nonlinear equations. An ODE is linear if it can be written in the form
where the coefficients and the right side may be any functions of , but the unknown and its derivatives appear only to the first power, multiplied by functions of , and added together. Anything else is nonlinear: or its derivatives squared, multiplied by each other, or inside a function like or .
Classification drill, worked.
| Equation | Order | Linear? | Reason |
|---|---|---|---|
| 1 | linear | and appear to the first power | |
| 1 | linear | is a coefficient and is the forcing term , neither involves | |
| 2 | linear | constant coefficients, forcing | |
| 1 | nonlinear | ||
| 1 | nonlinear | the unknown multiplies its own derivative | |
| 2 | nonlinear | the unknown sits inside a (this is the pendulum) | |
| 1 | nonlinear | expands to |
Why care? Because linear equations obey the principle of superposition. For a homogeneous linear equation (right side ), if and are solutions then so is for any constants. Here is the first-order proof. Suppose and . Then
That one line is why linear problems are solvable by building blocks (eigenvectors, sines and cosines, Laplace transforms), and why most of this course is about linear equations and about approximating nonlinear ones by linear ones.
Superposition fails for nonlinear equations. The function solves , because . But take : its derivative is , while its square is . These are not equal, so is not a solution. Notice also what does: it goes to infinity at , even though the equation contains nothing singular. Nonlinear equations can blow up in finite time, and their solutions need not exist for all . Linear equations with nice coefficients never do that.
What does it mean to be a solution?
A solution of a differential equation on an interval is a function with enough derivatives that, when you substitute it for the unknown, the equation becomes a true statement for every in . Verifying a solution is therefore pure substitution: compute the derivatives of the candidate, plug them in, and simplify until both sides match.
Example 1. Is a solution of , for any constant ? Compute . The right side is . They match for every and every , so yes.
Example 2. Is a solution of ? Here , and . Equal, so yes, for every .
Example 3: a function that is not a solution. Try in . The left side is , the right side is . They agree at (both ) and at (both ), but at the left side is and the right is . Agreeing at one or two points is not enough: the equation has to hold identically on the whole interval.
Example 4: a second-order equation. Check that solves for any constants and . We have and , so . ✓ Two free constants for a second-order equation, as promised by the order.
The same test works when you cannot do the algebra by hand, which is nearly always the case for a candidate that came out of a computer. Define the residual of a candidate for the equation as
A true solution has for every . For a candidate that is wrong, the residual measures how wrong. For example, in has , which is never zero. You can estimate with a central difference , exactly the stencil from 2.1 Functions of Several Variables & Partial Derivatives. You will build this check yourself at the end of the lesson.
General solutions: a whole family of curves
Look again at Example 1. The function is a solution for every value of . The equation does not have one solution; it has a whole one-parameter family. Is that all of them, or could there be others? For we can prove it. Multiply the equation by and rearrange:
A function whose derivative is zero everywhere on an interval is constant, so , that is, . Every solution of is in the family and nothing else is. We call the general solution. Choosing a particular value for gives a particular solution.
The simplest first-order equations show where the constant comes from. For , with no on the right at all, integration gives . The equation has solutions : all the parabolas, shifted up and down. Every one of them has the same slope at the same value of , which is exactly what the equation demands. That is the whole story of the constant. Differentiating kills constants, so undoing a derivative can’t recover them, and each level of integration leaves one free.
An order- equation has an -parameter family, so the equation has , with two constants. One warning, to be developed in 3.3 Separable Equations: for nonlinear equations a single formula with one constant may miss some solutions. The logistic equation has the family , but the constant solution belongs to none of its members (it appears only in the limit ).
The explorer below shows this directly. Pick an equation and the plot shows its slope field (the little arrows, drawn as short segments) and a family of exact solutions, one curve for each choice of the constant. Click anywhere to choose a starting point; the amber curve through it is the solution of the IVP described in the next section. The red dots are a numerical solution you can compare against it.
Try these:
- On y′ = k y, drag the starting point up and down. The amber curve slides through the family, and the readout shows the constant changing. The slope arrows do not change at all: the equation is fixed, only the choice of solution moves.
- Switch to cooling. Every curve, whatever its start, bends toward the horizontal line , the surrounding temperature. This is the equilibrium, where .
- With the Euler method at on the growth equation, start above the axis. The red dots fall below the exact curve and drift further away. Now switch to RK4 at the same . The dots sit on the curve to within the width of the line. Decrease and watch Euler close the gap, slowly.
- Switch to the nonlinear y′ = y² with a positive start. The exact curve shoots to infinity at the finite time . With Euler at from the exact solution is gone at , yet the dots keep going, climbing past the top of the plot: the solver does not know about the blow-up. The readout reports that the numerical solution only becomes unbounded around , long after the true solution ceased to exist.
- On logistic, click a point above and a point below and see what the two curves do, then click between and .
For each equation, state whether it is an ODE or a PDE, its order, and whether it is linear or nonlinear.
Show worked solution
- ODE, order 1, linear (forcing term ).
- ODE, order 2, nonlinear (the unknown is inside a sine).
- PDE (unknown , partial derivatives), order 1, linear. This is the advection equation, a profile moving at speed .
- PDE, order 2 (highest derivative is ), nonlinear because of the product . This is Burgers’ equation, a one-dimensional cousin of Navier–Stokes with the same nonlinear advection term.
- ODE, order 2, linear. The coefficients and depend on the independent variable, which is allowed.
Initial value problems: choosing one curve
A family of solutions is not yet a prediction. If you want to know what a quantity will do, you need to say where it starts. An initial value problem (IVP) is a differential equation together with starting data:
Geometrically, the initial condition names a point in the plane, and the solution of the IVP is the one curve of the family that passes through that point. Algebraically it fixes the constant. In the sim above, this is the amber curve and the value of in the readout.
Worked example: finding . Solve with . The general solution is . Impose the condition:
So . Check: ✓ and ✓. Note that is not the same as ; they coincide only when .
Worked example: a second-order IVP. Solve with and . Two constants need two conditions. From we get , so , and , so :
Check: ✓, so ✓.
Does every IVP have a solution, and is it unique? Usually yes, when is reasonably well behaved: through each point of the plane passes exactly one solution curve, and curves never cross. But “usually” hides interesting exceptions, and they matter for numerical work. We study them carefully in 3.7 Existence & Uniqueness.
Let us do that. Take the IVP , . From Example 2, the general solution is , and gives , so the exact solution is . The cell runs Euler and RK4 at three step sizes and compares both to it at .
Run it and read the two ratios in the output. Each time is halved, Euler’s error shrinks by a factor of about to (close to the of a first-order method) and RK4’s by a factor of about (close to the of a fourth-order method), the orders you know from your own solvers. What the theory adds is that the thing they are converging to is a specific curve you can write down.
The modeling cycle
So far we have been handed the equations. Real work starts one step earlier. Modeling is a loop with four steps:
- Assumptions. Decide what matters and what you will ignore. Write it down in words.
- Equation. Translate the assumptions into a differential equation and its initial condition.
- Solve. Find the solution, analytically if possible, numerically otherwise.
- Validate. Compare the prediction with measurements. If they disagree, the problem is almost never the algebra; it is an assumption. Go back to step 1.
Worked example: a cooling cup. Assumptions: the coffee is well mixed (one temperature ), the room stays at C, and heat loss is proportional to the temperature difference. Equation:
Solve: the general solution is (Example 1 with a shift), and gives . The constant is not known from first principles, so we calibrate it with one measurement. Suppose the coffee reads C after minutes. Then , so and per minute. Now the model makes predictions it can be tested on:
- At min: C.
- The time to cool to C: gives , so min.
Validate: put a thermometer in the cup. If it reads about C at ten minutes, the model is good. If it reads much higher, then some assumption is failing: perhaps evaporation cools the hot coffee faster at first, so the effective falls as the cup cools and the single calibrated overpredicts the cooling, or the room is warming. The algebra is not what went wrong; an assumption is. Notice that the same analytic formula serves both as the answer and as the way to check a numerical run with a step size of your choice.
Worked example: when the assumptions run out. The growth law with per year has , which doubles every years, forever. A real population cannot do that: the assumption “every individual reproduces at the same rate regardless of crowding” breaks down at large . The validation step flags it, and we revise the assumption: the per-capita rate falls as the population approaches a carrying capacity . That gives , the logistic equation, which is nonlinear, and which you can explore in the sim. We study it in 3.8 Autonomous Equations, Equilibria & Stability. Each trip around the cycle replaces a too-simple assumption with a better one.
- Show that is a solution of .
- The general solution of is . Find the solution with .
Show worked solution
- . Then . ✓ (The terms cancel, and .)
- gives , so . Check: , and ✓, and ✓.
Check that solves for all constants. Then find the solution with and .
Show worked solution
and . So
The initial conditions give two equations: and . Subtracting the second from the first: , so and . The solution is . Check: ✓, and ✓.
Carbon-14 decays by with half-life years.
- Find .
- What fraction of the carbon-14 remains after years?
- After how many years is only left?
Show worked solution
The general solution is . The half-life is defined by , so and .
- per year.
- , about .
- gives years.
The skydiver from earlier obeys with .
- Verify that solves the IVP.
- How long does it take to reach of terminal velocity?
- Replace the drag by a quadratic law, (with ). Is this equation linear? What is its terminal velocity?
Show worked solution
- . Also . Equal ✓. And ✓.
- gives , so s. The number s is the time constant: after one time constant the speed is m/s, about of terminal.
- It is nonlinear because of the term. Terminal velocity is where : , so . Quadratic drag is the realistic law at higher speeds; its dependence on the fluid density is the topic of 3.5 Modeling with First-Order ODEs.
Exercise: check a solution and classify an equation
Now build the two tools of this lesson in code. You will write a function that decides numerically whether a candidate satisfies , using the residual and a central difference. Then you will write a classifier that determines the order and linearity of an ODE it is given only as a Python function.
For the classifier, we describe an ODE in residual form , passed in as a function F(t, y, yp, ypp). For example, is F = lambda t, y, yp, ypp: yp - y**2. Probing a function numerically works because of two facts from this lesson. The order is the highest derivative that actually depends on, so perturb one slot at a time and see whether changes. The ODE is linear exactly when is an affine function of for fixed , which means for all choices of , and weight . That is superposition in disguise, and testing it at random points is enough to catch every nonlinearity in this lesson.
The classifier is the check you do by eye in the drill, turned into an algorithm: “does the unknown appear to the first power only?” becomes “does pass the superposition test?” Be a little suspicious of any numerical classifier: random probing can only fail to find a counterexample. For the functions in this lesson that is enough, but a function that is nonlinear only in some tiny region could slip through. That is a real limitation, and the reason computer-algebra systems classify ODEs symbolically.
The residual idea also lets you see the superposition failure from earlier numerically. The next cell tests scaling a solution, on a linear and on a nonlinear equation.
Where this goes next
We now have the vocabulary: an ODE relates a function to its rates of change, the order says how many constants and initial conditions it takes, linearity decides whether superposition works, a solution makes the equation hold identically, and an initial condition picks one member out of the family. We have not yet said how to find solutions in general, other than guessing. In 3.2 Direction Fields & Euler's Method Revisited we turn the slope-field picture of this lesson into a method: sketching solution curves without solving, and reading Euler’s method as “follow the arrows”. Then 3.3 Separable Equations and 3.4 Linear First-Order Equations & Integrating Factors give the two systematic techniques that solve most first-order equations, and 3.7 Existence & Uniqueness answers the question “can I trust that an IVP has exactly one answer?”. Behind all of it, 4.1 Local Error, Global Error & Order returns to the solvers you already wrote and explains their error in terms of what you have learned here.
Key Takeaways
- A differential equation relates an unknown function to its own derivatives. Physical laws (Newton’s second law, cooling, growth, decay, drag, Navier–Stokes) are statements about rates of change, which is easier to know than the quantity itself.
- An ODE has one independent variable; a PDE has several and partial derivatives (the heat equation ). Discretizing space turns a PDE into a large system of ODEs (the method of lines), so ODE solvers are the engine of PDE solvers.
- The order is the highest derivative present. An order- equation has free constants and needs initial conditions, which is also the length of the state vector in your solver.
- An ODE is linear if the unknown and its derivatives enter only to the first power, with coefficients depending on alone. Linear equations obey superposition; nonlinear ones do not and can blow up in finite time (as does). The advection term in Navier–Stokes is the nonlinearity.
- A solution makes the equation true identically on an interval, not just at a point. Check by substituting, or numerically by the residual , which is blind to the initial condition and is not the same as the error.
- The general solution is a family of curves with one constant per order (for it is , and that is every solution). Nonlinear equations can have solutions outside a one-formula family.
- An initial value problem adds and picks out one curve through that point. It is exactly the input of an ODE solver, so whenever a closed-form solution exists you can put it beside the numerical one and measure the error.
- The modeling cycle is assumptions, equation, solve, validate. When validation fails, suspect an assumption before suspecting the algebra or the solver.