Flow Lab

Syllabus / Module 4: Numerical Methods for ODEs — The Theory

Lesson 4.1

Local Error, Global Error & Order

Taylor-series analysis of one-step methods: truncation error, accumulation into global error, order of accuracy and convergence plots.

Written byClaude Sonnet

🌊
Why this matters for simulation: A '2nd-order accurate' fluid solver halves its error 4× when you halve the grid spacing — verifying this is how solvers are tested.

By the end of this lesson you can…

  • Distinguish the local truncation error of one step from the global error of a whole run, and say why the second is not the sum of the first.
  • Explain from Taylor matching why a method of order p has one-step error proportional to h^(p+1) and global error proportional to h^p.
  • Read the order of a method from a log-log convergence plot and from the ratio of errors at h and h/2.
  • State the Lipschitz error bound in plain terms and say what it does and does not predict.
  • Explain the round-off floor and why higher-order methods hit it at larger step sizes.
  • Explain how consistency and stability together give convergence, using a consistent scheme that diverges.
On this page
·
  1. Two different errors
  2. Where the order comes from
  3. From local to global: the error is carried, not just added
  4. The Lipschitz bound, in plain terms
  5. Order on a log-log plot
  6. The other end: round-off and the floor
  7. Consistency, stability, convergence
  8. Exercise: measure the order yourself
  9. Problems
  10. What comes next
  11. Key Takeaways

You have written Euler, three of its variants, RK4 and RK45, and you have seen that RK4 beats Euler by a mile. Nothing in your code said why, or by how much, or when the advantage disappears. This lesson supplies the vocabulary and the measurements: local error, global error and order. It also gives the test that every serious fluid or flight code is checked with: halve the step and see whether the error shrinks by the factor the theory promises.

Earlier lessons supply the tools: Taylor expansions (2.4 Taylor Series in One and Several Variables), the first look at Euler’s error in 3.2 Direction Fields & Euler's Method Revisited, and the Lipschitz constant from 3.7 Existence & Uniqueness. The story: what one step costs, how steps add up, and what happens as hh goes to zero on a real computer.

▶ Video One step versus a whole run, why order is read off a log-log slope, the round-off floor, and a scheme that is consistent but divergescaptions only
Transcript
  1. RK4 beats Euler by a mile. Why, and by how much? Two errors answer that. The local error is the miss of one step that starts exactly on the true curve. The global error is the miss at the end of a whole run, whose later steps start off the curve.
  2. Test problem: y prime equals minus two t y, exact solution e to the minus t squared. Start on the curve at t equals one, take one step of 0.05, and compare with the exact value. The misses run from 9.5 times ten to the minus four for Euler to 3.3 times ten to the minus eight for RK4.
  3. Halve the step to 0.025 and the misses shrink by 4.1, 8.1 and 33.2: near two squared, two cubed and two to the fifth. A method of order p has one-step error proportional to h to the p plus one.
  4. Now run Euler to t equals one. Each step plants a local error d k, and the equation carries it forward. The slope of f in y is minus two t, so a gap planted at t k shrinks by e to the minus T squared minus t k squared. Call that G k: it runs from 0.37 up to one.
  5. Weight each local error by G k and add: at h equals 0.05 that gives 6.459 times ten to the minus three, against an actual 6.505, within about one percent. Unweighted it is 1.896 times ten to the minus two, nearly three times too big: the equation damps errors, early ones most. Global error is the local errors filtered by the equation.
  6. Now the global error at t equals one, halving h from 0.1 four times, on log-log axes. Each method falls on a straight line, parallel to its dashed reference with slope one, two or four.
  7. The order is log base two of E at h over E at h over two: Euler's drifts from 1.09 to 1.01, Heun's from 1.96 to 2.00, RK4's sits at four. Local error h to the p plus one becomes global error h to the p: about one over h steps, one power lost.
  8. Smaller h does not always help. Round the stored state to a 32-bit float after every step. Each rounding is a fresh error up to six times ten to the minus eight, whatever h, and halving h doubles their number. For RK4 the dashed float32 curve leaves the solid float64 one at h equal 3.2 times ten to the minus two, then wanders.
  9. Euler and Heun do the same, later: float32 leaves Euler at 1.8 times ten to the minus five and Heun at 3.2 times ten to the minus four. Higher order, larger h: its truncation error falls so fast it sinks below round-off early.
  10. At the smallest steps we ran, RK4 reaches 8.3 times ten to the minus sixteen in float64 but only 5.6 times ten to the minus seven in float32. Past the floor, refining hurts: stop where the error stops falling.
  11. Now a trap. Put the exact solution into this two-step scheme: what is left, over h to the fourth, heads to one sixth. Local error h to the fourth: third order, better on paper than Heun. It is consistent.
  12. Run it on y prime equals minus y. At t equals one the error is 7.045 for h equals 0.1, 4.652 million for 0.05, 2.883 times ten to the nineteen for 0.025: finer is worse, and not round-off, since this is 80-digit arithmetic. One root is near one, the true solution; the other, near minus five, multiplies any error by about five every step. Consistent, not stable: no convergence.
  13. Recap: local error is one step from the curve; global error is those errors filtered by the equation. Order p: local h to the p plus one, global h to the p, read from a log-log slope. Round-off makes the error U-shaped, with the floor at larger h for higher order. Consistency plus stability gives convergence. Next: the Euler family.

Two different errors

Write one step of a method as a map: yn+1=Φh(tn,yn)y_{n+1} = \Phi_h(t_n, y_n). For Euler, Φh(t,y)=y+hf(t,y)\Phi_h(t, y) = y + h f(t, y). Two questions can be asked about its accuracy, and they have different answers.

Local truncation error (LTE): start the step on the exact solution, so yn=y(tn)y_n = y(t_n), take one step, and measure the miss:

dn+1=y(tn+h)−Φh(tn,y(tn)).d_{n+1} = y(t_n + h) - \Phi_h\big(t_n, y(t_n)\big).

Global error: run the whole computation from t0t_0 to TT and compare with the exact solution there, E=y(T)−yNE = y(T) - y_N.

The local error is a clean idea and a slightly fictional one: a step that begins exactly on the true curve happens only once, at the start. Every later step starts from a numerical point already off the curve, the equation does something to that earlier error, and the new step adds its own on top. So the global error is not the sum of the local errors, and the rest of this lesson is the story of the difference.

Where the order comes from

The cleanest laboratory is y′=λyy' = \lambda y. Put z=hλz = h\lambda. One step multiplies yy by a polynomial in zz:

Euler: 1+z,Heun: 1+z+z22,RK4: 1+z+z22+z36+z424.\text{Euler: } 1 + z, \qquad \text{Heun: } 1 + z + \tfrac{z^2}{2}, \qquad \text{RK4: } 1 + z + \tfrac{z^2}{2} + \tfrac{z^3}{6} + \tfrac{z^4}{24}.

(For Heun: k1=λyk_1 = \lambda y, k2=λy(1+z)k_2 = \lambda y (1 + z), so y+h2(k1+k2)=y (1+z+z22)y + \tfrac h2 (k_1 + k_2) = y\,(1 + z + \tfrac{z^2}{2}); RK4 is the same bookkeeping with four stages.) The exact step multiplies by ez=1+z+z22+z36+z424+z5120+⋯e^z = 1 + z + \tfrac{z^2}{2} + \tfrac{z^3}{6} + \tfrac{z^4}{24} + \tfrac{z^5}{120} + \cdots. Each method copies the Taylor series of eze^z and stops. The first term it leaves out is the local error:

d=y(t) zp+1(p+1)!+⋯ ,p=1,2,4 for Euler, Heun, RK4.d = y(t)\,\frac{z^{p+1}}{(p+1)!} + \cdots, \qquad p = 1, 2, 4 \text{ for Euler, Heun, RK4.}

A method has order pp when it reproduces the first pp Taylor terms of the true step. The mismatch is then proportional to hp+1h^{p+1}. The same holds for nonlinear ff, where the matching uses the multivariable chain rule (2.4 Taylor Series in One and Several Variables, and 4.3 Runge–Kutta Methods & Butcher Tableaux for the general family). For Heun and RK4 the leading constant on a general ff is a combination of derivatives, so only the power of hh carries over from the linear case; for Euler the constant does too: d≈h22y′′d \approx \tfrac{h^2}{2}y''.

We watch this on a running example:

y′=−2ty,y(0)=1,y(t)=e−t2.y' = -2ty, \qquad y(0) = 1, \qquad y(t) = e^{-t^2}.

The cell below prints four labelled blocks, A to D, which we read in turn. Block A is the one-step error at t=1t = 1 for three methods started on the exact curve. For Euler, y′′=(4t2−2) yy'' = (4t^2 - 2)\,y equals 2/e=0.73582/e = 0.7358 at t=1t = 1, so the Taylor estimate h22y′′\tfrac{h^2}{2}y'' is 9.2×10−49.2\times 10^{-4} at h=0.05h = 0.05.

🐍 PythonLocal error, global error, error accumulation, and a scheme that diverges
⌘/Ctrl + Enter

Block A: the measured Euler error at h=0.05h = 0.05 is 9.484×10−49.484\times10^{-4}, against the Taylor estimate 9.2×10−49.2\times10^{-4}. Halving hh divides the one-step error by 4.14.1, 8.18.1 and 33.233.2 for Euler, Heun and RK4, which is 2p+1=4,8,322^{p+1} = 4, 8, 32 up to the next Taylor term’s small correction. That is local error O(hp+1)O(h^{p+1}) made visible.

From local to global: the error is carried, not just added

Block B runs the whole computation, with h=0.1h = 0.1 halved four times. Euler’s global error at t=1t = 1 goes from 1.383×10−21.383\times10^{-2} to 7.720×10−47.720\times10^{-4}, Heun’s from 1.174×10−31.174\times10^{-3} to 4.781×10−64.781\times10^{-6}, and RK4’s from 1.625×10−61.625\times10^{-6} to 2.497×10−112.497\times10^{-11}. The printed orders log⁡2(Eh/Eh/2)\log_2(E_h/E_{h/2}) are 1.09,1.04,1.02,1.011.09, 1.04, 1.02, 1.01 for Euler, 1.96,1.99,1.99,2.001.96, 1.99, 1.99, 2.00 for Heun and 3.99,4.00,4.00,4.003.99, 4.00, 4.00, 4.00 for RK4. So local error O(hp+1)O(h^{p+1}) becomes global error O(hp)O(h^p): one power of hh is lost.

Why one power? Taking N=T/hN = T/h steps adds about 1/h1/h local errors of size Chp+1C h^{p+1}, and 1h⋅hp+1=hp\tfrac1h \cdot h^{p+1} = h^p. But adding them naively is wrong: the equation carries each error from where it was planted to the final time. A step at tkt_k plants dkd_k; the solution through the wrong point is a neighbour of the true one, and the gap δ\delta obeys the variational equation, the linearization of the ODE (2.2 Linearization, the Chain Rule & the Jacobian): δ′=fy δ\delta' = f_y\,\delta. Here fy=−2tf_y = -2t, so

δ(T)=δ(tk) e−(T2−tk2).\delta(T) = \delta(t_k)\,e^{-(T^2 - t_k^2)}.

Call that survival factor GkG_k (it is the sensitivity of y(T)y(T) to a nudge at tkt_k). Then, to first order,

E≈∑k=1NGk dk.E \approx \sum_{k=1}^{N} G_k\, d_k.

The global error is the sum of local errors, each multiplied by the factor the ODE applies between that step and the end. Block C tests this on Euler. At h=0.05h = 0.05 the actual error is 6.505×10−36.505\times10^{-3}. The plain sum of local errors gives 1.896×10−21.896\times10^{-2}, nearly three times too large because this equation damps errors (every GkG_k is below 11). With the factors the sum is 6.459×10−36.459\times10^{-3}, within one percent; at h=0.1h = 0.1 and 0.0250.025 the weighted sums are 1.367×10−21.367\times10^{-2} and 3.145×10−33.145\times10^{-3} against actual errors of 1.383×10−21.383\times10^{-2} and 3.157×10−33.157\times10^{-3}.

✅ Concept check

A one-step method has one-step error 4.0e-6 at h = 0.1 and 1.25e-7 at h = 0.05, both started on the exact solution. What is the method's order p (the exponent in global error proportional to h^p)?

The Lipschitz bound, in plain terms

The variational picture needs the exact solution’s sensitivity. Often all we know is a speed limit: from 3.7 Existence & Uniqueness, if ∣∂f/∂y∣≤L|\partial f/\partial y| \le L, two solutions separate at most at rate LL, so one Euler step stretches a gap ee to at most (1+hL) e(1 + hL)\,e. Split the new global error into a fresh local error plus the old error carried forward: y(tn+1)−yn+1=[y(tn+1)−Φh(y(tn))]+[Φh(y(tn))−Φh(yn)]y(t_{n+1}) - y_{n+1} = \big[y(t_{n+1}) - \Phi_h(y(t_n))\big] + \big[\Phi_h(y(t_n)) - \Phi_h(y_n)\big]. The first bracket is at most dˉ\bar d and the second at most (1+hL)(1+hL) times the old error, so the global error ene_n obeys

en+1≤(1+hL) en+dˉ.e_{n+1} \le (1 + hL)\,e_n + \bar d.

Unrolling it (the sum of a geometric series, and then (1+hL)N≤eLhN=eLT(1+hL)^N \le e^{LhN} = e^{LT}):

eN≤dˉ (1+hL)N−1hL≤dˉhL(eLT−1).e_N \le \bar d\,\frac{(1+hL)^N - 1}{hL} \le \frac{\bar d}{hL}\left(e^{LT} - 1\right).

For Euler, dˉ≤M2h2\bar d \le \tfrac{M}{2}h^2 with M=max⁡∣y′′∣M = \max|y''|, which gives the standard bound

∣E∣≤Mh2L(eLT−1).|E| \le \frac{M h}{2L}\left(e^{LT} - 1\right).

For a method of order pp replace the numerator by Chp+1C h^{p+1} and the result is CL(eLT−1) hp\tfrac{C}{L}(e^{LT}-1)\,h^p. Three things to take from it:

  1. The power of hh is right. One factor of hh cancels the N=T/hN = T/h terms.
  2. The constant is exponential in LTLT, because solutions can separate that fast. That is why long runs of sensitive systems cannot be trusted at any step size.
  3. It is a bound, not an estimate. For y′=−2tyy' = -2ty on [0,1][0,1] we can take M=2M = 2 (the maximum of ∣y′′∣|y''| is at t=0t = 0) and L=2L = 2. At h=0.05h = 0.05 the bound is 0.15970.1597 against an actual 0.00650.0065, a factor of 24.624.6: the bound assumes the worst case and this equation forgives errors. For the growing twin y′=+2tyy' = +2ty the bound is 1.30251.3025 against an actual 0.20760.2076, only 6.36.3 times too big. (Both printed in block C.)

Order on a log-log plot

If E≈ChpE \approx C h^p, then log⁡E=log⁡C+p log⁡h\log E = \log C + p\,\log h: on a log-log plot a method of order pp is a straight line of slope pp, and CC only moves it up or down. That is why convergence studies are plotted that way, and why the order from two runs is

p≈log⁡2E(h)E(h/2).p \approx \log_2\frac{E(h)}{E(h/2)}.

The lab below runs the three methods on three equations (final time T=1T = 1 for y′=−2tyy' = -2ty, T=3T = 3 for the logistic equation y′=y(1−y)y' = y(1-y) from y(0)=0.1y(0) = 0.1, T=6T = 6 for the oscillator x′′=−xx'' = -x), sweeps hh over a grid with four points per decade, and fits a line by least squares to the float64 points inside a shaded window.

◆ Interactive Convergence lab: choose the equation and method, move the fit window, read the slope

Loading simulation…

Things to try (all numbers refer to the default equation y′=−2tyy' = -2ty):

  1. With the default window h=10−2h = 10^{-2} to 10−110^{-1} (five grid points) the fitted slopes are 1.051.05 for Euler, 1.981.98 for Heun and 4.004.00 for RK4. Euler reads 1.051.05 because h=0.1h = 0.1 is not yet small: the next error term, proportional to h2h^2, still adds a little. With the window at 10−410^{-4} to 10−310^{-3} it reads 1.001.00.
  2. Move the window to the right, h=10−1h = 10^{-1} to 11. Euler’s slope becomes 1.621.62, Heun’s 2.142.14 and RK4’s 3.903.90: outside the asymptotic range the slope is not the order.
  3. For RK4, put the window at 10−2.2510^{-2.25} to 10−3.2510^{-3.25}: the slope reads 4.224.22, because the float64 errors there (about 10−1310^{-13} to 10−1510^{-15}) brush double precision’s own floor.

The other end: round-off and the floor

So far smaller hh always helps. On a computer it stops helping, because each step also makes an arithmetic error: rounding the stored state by a relative amount up to ε\varepsilon is a second kind of local error, of size about ε ∣y∣\varepsilon\,|y| and independent of hh. Errors from 1/h1/h steps pile up, so the round-off part of the global error is somewhere between about εT/h\varepsilon\sqrt{T/h} (if the roundings are random and partly cancel) and ε T/h\varepsilon\,T/h (if they all push the same way), and either way it grows as hh shrinks. Take the aligned worst case as a simple model and add the two contributions:

E(h)≈C hp⏟truncation, falls  +  Ah⏟round-off, rises.E(h) \approx \underbrace{C\,h^{p}}_{\text{truncation, falls}} \;+\; \underbrace{\frac{A}{h}}_{\text{round-off, rises}}.

The sum has a minimum: a U-shaped curve, the ODE twin of the finite-difference picture in 2.4 Taylor Series in One and Several Variables.

Double-precision round-off (ε≈10−16\varepsilon \approx 10^{-16}) needs enormous step counts to show in an ODE solver. So the lab reruns every sweep with the state stored in single precision: after each step every component is rounded to the nearest 32-bit float (relative size at most 2−24≈6×10−82^{-24} \approx 6\times10^{-8}) while the arithmetic inside the step stays in double. That models storage round-off; it is not a bit-exact emulation of a GPU. The red curve is whatever that rounding produces. It is deterministic but irregular, so no slope is fitted to it.

◆ Interactive Same lab, RK4: watch the red float32 curve peel off the straight blue line (untick it to hide it)

Loading simulation…

On y′=−2tyy' = -2ty with the default settings, the red float32 curve leaves the blue float64 curve (error more than twice as large) at:

methodfloat32 leaves the line at h≈h \approxat the smallest grid hh: float64 errorfloat32 error
Euler1.8×10−51.8\times10^{-5}3.9×10−73.9\times10^{-7} (at h=3.2×10−6h = 3.2\times10^{-6})2.6×10−42.6\times10^{-4}
Heun3.2×10−43.2\times10^{-4}1.2×10−101.2\times10^{-10} (at h=3.2×10−5h = 3.2\times10^{-5})2.4×10−62.4\times10^{-6}
RK43.2×10−23.2\times10^{-2}8.3×10−168.3\times10^{-16} (at h=3.2×10−4h = 3.2\times10^{-4})5.6×10−75.6\times10^{-7}

The higher the order, the larger the hh at which the floor appears. In the model above, truncation ChpC h^p falls so quickly for RK4 that it is below the round-off level by h≈0.03h \approx 0.03, while Euler’s falls so slowly that it only meets that level near h≈10−5h \approx 10^{-5}, with 3×1053\times10^5 steps. Setting the two terms equal gives a crossing at h∗∼(A/C)1/(p+1)h_\ast \sim (A/C)^{1/(p+1)}, which increases with pp; the table is the measured version. Note the last two columns for RK4: past its crossing, refining buys nothing in float32 (the error at the smallest step is 5.6×10−75.6\times10^{-7}, while the float64 run reaches 8.3×10−168.3\times10^{-16}), and the float64 curve itself flattens near 10−1510^{-15} for the smallest steps, which is double precision’s own floor on this problem.

✅ Concept check

In the lab (y' = -2ty, float32 storage) RK4 at h = 3.2e-4 has error 5.6e-7, while the float64 error there is 8.3e-16. Which is the best action to get a more accurate answer from the float32 solver?

Consistency, stability, convergence

Put the pieces into three words.

  • A scheme is consistent if its local error shrinks faster than hh: d/h→0d/h \to 0 as h→0h \to 0 (order p≥1p \ge 1). It says the scheme is aiming at the right equation.
  • A scheme is stable (in the sense that matters here, called zero-stability for multistep methods) if errors planted at each step are not magnified without limit as h→0h \to 0: the growth factor stays bounded, like eLTe^{LT} above.
  • A scheme converges if the global error goes to zero as h→0h \to 0.

The idea, and it is only an idea here, not a proof: consistency plus stability gives convergence, at the order of the consistency. The proof has the shape of the bound above: global error is at most (bounded growth factor) times (sum of local errors). For one-step methods on Lipschitz problems stability is free, so consistent means convergent. The same pattern is the Dahlquist equivalence theorem for multistep methods and the Lax–Richtmyer equivalence theorem for linear PDE schemes on a grid (standard textbook results, stated without proof).

Why bother with the stability half? Because consistency alone can be useless. Here is a two-step scheme, one that uses the two previous values (see 4.6 Multistep Methods):

yn+1=−4 yn+5 yn−1+h (4fn+2fn−1).y_{n+1} = -4\,y_n + 5\,y_{n-1} + h\,\big(4 f_n + 2 f_{n-1}\big).

Put the exact solution ete^t into the formula and measure what is left over (the residual). Block D prints residual/h4=0.1634, 0.1650, 0.1658/h^4 = 0.1634,\ 0.1650,\ 0.1658 at h=0.1,0.05,0.025h = 0.1, 0.05, 0.025, heading to 1/61/6: the terms through h3h^3 cancel and the left-over is h46 y(4)\tfrac{h^4}{6}\,y^{(4)}. Local error O(h4)O(h^4): this is a third-order consistent scheme, better on paper than Heun. Block D then runs it on y′=−yy' = -y with exact starting values and reports the error at t=1t = 1.

At h=0.1,0.05,0.025h = 0.1, 0.05, 0.025 the error is 7.0457.045, 4.652×1064.652\times10^{6} and 2.883×10192.883\times10^{19}.

The error grows as hh shrinks: no convergence. The recurrence runs in 80-digit decimal arithmetic, so this is not round-off; the seed is the scheme’s own truncation error, of size about h4h^4.

The mechanism is in the recurrence. On y′=−yy' = -y it reads ξ2+(4+4h) ξ−(5−2h)=0\xi^2 + (4 + 4h)\,\xi - (5 - 2h) = 0 for the growth factor ξ\xi per step. As h→0h \to 0 this becomes (ξ−1)(ξ+5)=0(\xi - 1)(\xi + 5) = 0, with roots 11 and −5-5. The root near 11 is the real solution (0.9050.905, 0.9510.951, 0.9750.975 at h=0.1,0.05,0.025h = 0.1, 0.05, 0.025). The root near −5-5 is a parasitic solution (−5.305-5.305, −5.151-5.151, −5.075-5.075): any error component that lands in it is multiplied by about 55 at every step, and the truncation error of the first steps puts a little there. After N=1/hN = 1/h steps the damage is about 5N5^{N} times a small seed, which grows faster as hh shrinks. The growth factor is not bounded as h→0h \to 0: exactly the stability property that fails.

This differs from the stability you met with y′=−10yy' = -10y in 3.2 Direction Fields & Euler's Method Revisited, where hh is fixed and Euler blows up once hh exceeds 0.20.2. That is absolute stability, a statement about one particular hh (4.2 The Euler Family: Forward, Backward, Heun & Midpoint, 4.5 Stability Regions & Stiffness). Zero-stability is the statement about h→0h \to 0. A good method needs both.

✅ Concept check

A scheme has local error proportional to h^4, yet its error at a fixed final time is larger for h = 0.025 than for h = 0.05. Which diagnosis fits best?

Exercise: measure the order yourself

Do what a solver developer does on day one: measure the order from error ratios. The equation is the logistic equation y′=y(1−y)y' = y(1 - y), y(0)=0.1y(0) = 0.1, exact solution 1/(1+9e−t)1/(1 + 9e^{-t}), run to T=3T = 3. Implement the global error at TT, the observed order from a list of halving step sizes, and the local exponent from one-step errors started on the exact curve.

🧪 Coding exerciseExercise: measure the order of Euler, Heun and RK4 from error ratios
⌘/Ctrl + Enter

Once it works, notice that the local exponents are one more than the global orders (about 2,3,52, 3, 5 against 1,2,41, 2, 4).

Problems

✏️ Pen & paperRefining two error sources at once●○○

A fluid solver has error E(Δt,Δx)≈0.8 Δt+5 Δx2E(\Delta t, \Delta x) \approx 0.8\,\Delta t + 5\,\Delta x^2 (made-up constants, but the structure is typical: first order in time, second order in space). Start from Δt=Δx=0.1\Delta t = \Delta x = 0.1.

  1. Compute EE at (0.1,0.1)(0.1, 0.1), (0.05,0.05)(0.05, 0.05) and (0.025,0.025)(0.025, 0.025), and the two error ratios. What order would you read off, and what order do you expect in the limit?
  2. Instead halve only Δx\Delta x from the start: compute E(0.1,0.05)E(0.1, 0.05) and its ratio to E(0.1,0.1)E(0.1, 0.1). What does that say about refining only the space grid?
Show worked solution
  1. E(0.1,0.1)=0.08+0.05=0.13E(0.1, 0.1) = 0.08 + 0.05 = 0.13; E(0.05,0.05)=0.04+0.0125=0.0525E(0.05, 0.05) = 0.04 + 0.0125 = 0.0525; E(0.025,0.025)=0.02+0.003125=0.023125E(0.025, 0.025) = 0.02 + 0.003125 = 0.023125. The ratios are 0.13/0.0525=2.4760.13/0.0525 = 2.476 and 0.0525/0.023125=2.2700.0525/0.023125 = 2.270. The observed orders are log⁡22.476=1.31\log_2 2.476 = 1.31 and log⁡22.270=1.18\log_2 2.270 = 1.18. In the limit the time term dominates (it falls like Δt\Delta t, the space term like Δt2\Delta t^2), so the ratio tends to 22 and the observed order to 11, not 22.
  2. E(0.1,0.05)=0.08+0.0125=0.0925E(0.1, 0.05) = 0.08 + 0.0125 = 0.0925, a ratio of 0.13/0.0925=1.4050.13/0.0925 = 1.405. The time term 0.080.08 is untouched, so refining space alone can never get below it. You must refine the dominant term.
✏️ Pen & paperThe Lipschitz bound on exponential growth●●○

For y′=yy' = y, y(0)=1y(0) = 1 on [0,T][0, T], Euler gives yN=(1+h)T/hy_N = (1 + h)^{T/h}.

  1. Show that the actual error is about hT2 eT\tfrac{hT}{2}\,e^{T} for small hh.
  2. Find LL and MM and write the bound Mh2L(eLT−1)\dfrac{Mh}{2L}\left(e^{LT} - 1\right).
  3. Find the limit of bound over actual as h→0h \to 0; evaluate it for T=3T = 3.
Show worked solution
  1. ln⁡((1+h)T/h)=Th(h−h22+⋯ )=T−Th2+O(h2)\ln\big((1+h)^{T/h}\big) = \tfrac{T}{h}\left(h - \tfrac{h^2}{2} + \cdots\right) = T - \tfrac{Th}{2} + O(h^2), so yN=eT e−Th/2+O(h2)=eT(1−Th2+⋯ )y_N = e^{T}\,e^{-Th/2 + O(h^2)} = e^T\left(1 - \tfrac{Th}{2} + \cdots\right). The error is eT−yN≈Th2 eTe^T - y_N \approx \tfrac{Th}{2}\,e^T.
  2. fy=1f_y = 1, so L=1L = 1. The solution is ete^t, so y′′=ety'' = e^t and M=eTM = e^{T}. The bound is eTh2(eT−1)\dfrac{e^T h}{2}\left(e^{T} - 1\right).
  3. Dividing, bound over actual tends to (eTh/2)(eT−1)(Th/2) eT=eT−1T\dfrac{(e^T h/2)(e^T - 1)}{(Th/2)\,e^T} = \dfrac{e^T - 1}{T}. For T=3T = 3 that is (e3−1)/3=6.36(e^3 - 1)/3 = 6.36. The bound is only a single-digit factor too large when errors really grow (compare 24.624.6 for the decaying equation in the lesson).
✏️ Pen & paperWhere is the bottom of the U?●●●

Model the total error of a method of order pp as E(h)=C hp+A/hE(h) = C\,h^{p} + A/h, with AA the accumulated round-off per unit step.

  1. Find the minimizing step h∗h_\ast and the minimum error E(h∗)E(h_\ast) in terms of pp, CC and AA.
  2. Take C=0.5C = 0.5 and A=2×10−8A = 2\times10^{-8}. Evaluate h∗h_\ast and E(h∗)E(h_\ast) for p=1p = 1 and p=4p = 4, and compare.
Show worked solution
  1. E′(h)=pC hp−1−A/h2=0E'(h) = pC\,h^{p-1} - A/h^2 = 0 gives h∗p+1=ApCh_\ast^{p+1} = \dfrac{A}{pC}, so h∗=(ApC)1/(p+1)h_\ast = \left(\dfrac{A}{pC}\right)^{1/(p+1)}. At that point A/h∗=pC h∗pA/h_\ast = pC\,h_\ast^{p}, hence E(h∗)=Ch∗p+pCh∗p=(1+p) C h∗pE(h_\ast) = C h_\ast^{p} + pC h_\ast^{p} = (1 + p)\,C\,h_\ast^{p}.
  2. For p=1p = 1: h∗2=2×10−8/0.5=4×10−8h_\ast^2 = 2\times10^{-8}/0.5 = 4\times10^{-8}, so h∗=2×10−4h_\ast = 2\times10^{-4} and E=2⋅0.5⋅2×10−4=2×10−4E = 2 \cdot 0.5 \cdot 2\times10^{-4} = 2\times10^{-4}. For p=4p = 4: h∗5=2×10−8/(4⋅0.5)=10−8h_\ast^5 = 2\times10^{-8}/(4 \cdot 0.5) = 10^{-8}, so h∗=10−1.6=0.0251h_\ast = 10^{-1.6} = 0.0251 and E=5⋅0.5⋅h∗4=2.5×3.98×10−7=9.95×10−7E = 5 \cdot 0.5 \cdot h_\ast^4 = 2.5 \times 3.98\times10^{-7} = 9.95\times10^{-7}. The fourth-order method has its best step about 126126 times larger (cheaper) and its best error about 200200 times smaller. The constants are invented, but the direction matches the lab: the higher the order, the larger h∗h_\ast.

What comes next

Next: 4.2 The Euler Family: Forward, Backward, Heun & Midpoint (implicit variants and absolute stability), 4.3 Runge–Kutta Methods & Butcher Tableaux (order conditions) and 4.4 Adaptive Step Size & RK45 (estimate dkd_k on the fly and choose hh to keep it at a target).

Key Takeaways

  • Local truncation error is the miss of one step started on the exact solution; global error is what a whole run produces. It is not the sum of the local errors: the ODE multiplies each one by a growth factor on its way to the end.
  • A method of order pp matches the first pp Taylor terms of the true step: local error proportional to hp+1h^{p+1}, global error to hph^p, because about 1/h1/h steps add up.
  • On a log-log plot error against hh is a line of slope pp; the order is log⁡2(E(h)/E(h/2))\log_2\big(E(h)/E(h/2)\big), valid only inside the asymptotic range.
  • The Lipschitz bound CL(eLT−1) hp\tfrac{C}{L}(e^{LT}-1)\,h^p gets the power of hh right and the constant too large (by 24.624.6 for a decaying and 6.36.3 for a growing equation here); eLTe^{LT} is why long runs of sensitive systems cannot be trusted at any hh.
  • Round-off adds an error that grows as hh shrinks, so total error is U-shaped; higher order moves the bottom to a larger hh.
  • Consistency plus stability gives convergence; a consistent third-order two-step scheme with a parasitic root near −5-5 diverges as h→0h \to 0.
  • In a refinement study, refine every error source together and expect the lowest order present to dominate once hh is small.
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.

Where this leads

These later lessons build directly on this one: