Flow Lab

Syllabus / Module 4: Numerical Methods for ODEs — The Theory

Lesson 4.4

Adaptive Step Size & RK45

Embedded Runge–Kutta pairs (Fehlberg, Dormand–Prince), error estimation, step-size control, tolerances, dense output.

Written byClaude Sonnet

🌊
Why this matters for simulation: Adaptive time stepping in CFD picks dt from the CFL number — a physics-based version of the same idea.

By the end of this lesson you can…

  • Explain why the difference of two embedded Runge–Kutta results estimates the local error of the lower-order one, while the step uses the higher-order one.
  • Verify the order conditions and FSAL property of Bogacki–Shampine 3(2) and Dormand–Prince 5(4) by computer.
  • Derive the controller h_new = h·safety·(tol/err)^(1/(q+1)) and explain safety, step-ratio limits and accept/reject.
  • State what rtol and atol control and count evaluations the way solve_ivp does.
  • Show on an eccentric orbit that the step follows the problem, and that a local tolerance promises neither a global bound nor a cure for stiffness.
On this page
·
  1. Two answers from one set of stages
  2. Why the difference is an error estimate
  3. Three pairs, and FSAL
  4. The controller
  5. rtol and atol
  6. Worked example: an eccentric orbit
  7. Dense output
  8. What the tolerance does not promise
  9. Exercise: an adaptive Bogacki–Shampine integrator
  10. Problems
  11. Key Takeaways

You have coded RK45 and called solve_ivp. This lesson is the why: why subtracting two answers estimates an error, where the controller’s exponent comes from, and what rtol and atol measure. A spacecraft on an eccentric orbit crawls far from the planet and whips around the close approach; a fixed step small enough for the close approach wastes nearly every evaluation of ff on the quiet part.

It builds on 4.1 Local Error, Global Error & Order (local and global error, order) and 4.3 Runge–Kutta Methods & Butcher Tableaux (tableaux and order conditions, not rederived here). Its limit is 4.5 Stability Regions & Stiffness.

▶ Video Two answers from one set of stages, the error estimate, the step-size controller, rtol and atol, and an eccentric orbit where the step shrinks at the close approachcaptions only
Transcript
  1. An embedded pair gives two answers from one set of stages: the same k's, two weight vectors. On y prime equals y times one minus y, the slopes 0.09 and 0.123975 give Euler 0.145 and Heun 0.1535. Exact is 0.1548. The second answer costs no extra evaluation.
  2. Why is the difference an error estimate? The two results share every order condition up to the lower order, so the difference is the lower method's own local error. Estimate over true error tends to one as h shrinks, here 0.4 to 0.025: Heun Euler 0.8888, 0.9702, 0.9924. Bogacki Shampine's 3.87 at h 0.4 is not yet asymptotic.
  3. At h one half the estimate is 0.00849375. Euler's true error is 0.00982810; Heun's is only 0.00133435. The solver steps with the better answer, Heun's, and uses the difference to judge the worse one: local extrapolation.
  4. Three pairs: orders two and one, three and two, five and four; b minus b hat first acts at order 2, 3, 5. Bogacki Shampine and Dormand Prince are FSAL: the last row of A equals b, so the last stage is the next step's first. Dormand Prince needs 6 new evaluations a step, not 7.
  5. Assume the scaled error is C h to the q plus one, q the lower order. Meeting the tolerance gives h times safety 0.9 times err to the minus one over q plus one, the factor held between 0.2 and 10. Scipy uses q 4 for RK45, 2 for RK23. Accept when err is below one.
  6. Worked step, Heun Euler, h one half, rtol ten to the minus 3, atol ten to the minus 6. Estimate over scale is error norm 54.978: rejected. Raw factor 0.1214 is under the floor, so the factor is 0.2. Norm 3.2255: rejected, h 0.0501. Norm 0.8513: accepted.
  7. The scale is atol plus rtol times the larger absolute value. The norm is a root mean square: scaled errors three and four give 3.5355, not four. Relative tolerance rules above y equal atol over rtol, here 0.001; below, atol is the floor.
  8. Kepler, e 0.9, rtol ten to the minus 6: 63 steps accepted, 21 rejected, 506 evaluations. The first step, 0.00392, grows by exactly ten; the largest, 0.4540, is at t 1.063; the smallest, 0.004467, at pi: ratio 101.6. 30 of 63 steps have r below a half; all 21 rejected trials begin on the approach.
  9. Fixed RK4, same evaluations. At rtol ten to the minus 6, Dormand Prince is off by 1.752 times ten to the minus 5; RK4 with 126 steps is 3.982 off. At minus 3: 0.1045 against 12.21; at minus 8: 8.969 times ten to the minus 8 against 1.265. To match the first, RK4 needs at most 2300 steps, 9200 evaluations, 18.2 times as many.
  10. Dense output: one Dormand Prince step of 0.454 from the exact state at t 1.0631. End of step error 9.232 times ten to the minus 8. Midpoint: Hermite cubic 2.488 times ten to the minus 5, scipy's interpolant 2.044 times ten to the minus 7.
  11. A local tolerance is not a global bound. After one orbit at rtol ten to the minus 6 the error is 17.5 times rtol; after ten orbits, 628 times, 36 times the one-orbit error for ten times the time. I have not derived the growth rate.
  12. Nor does it cure stiffness. For y prime equals minus 1000 times y minus cos t, RK45 takes 912 steps, 6350 evaluations, error 4.0 times ten to the minus 4, below the tolerance: accuracy did not pick the step. The Dormand Prince stability limit is z equals minus 3.3066; median accepted h lambda, 3.2593. Radau takes 15 steps, BDF 55.
  13. Recap: two answers from one set of stages; the difference estimates the lower method's error; the step keeps the better one. A tolerance is local, and does not cure stiffness. Next: stability and stiffness.

Two answers from one set of stages

An embedded pair computes one set of stages kik_i (one cc, one AA) and combines them with two weight vectors:

yn+1=yn+h∑ibiki,y^n+1=yn+h∑ib^iki.y_{n+1} = y_n + h\sum_i b_i k_i, \qquad \hat y_{n+1} = y_n + h\sum_i \hat b_i k_i .

The second answer costs no extra evaluations of ff. Let y^\hat y have order qq and yy order q+1q+1; the pair is written (q+1)(q)(q+1)(q). Heun–Euler 2(1) has a second bottom row:

011b1212b^10\begin{array}{c|cc} 0 & & \\ 1 & 1 & \\ \hline b & \tfrac12 & \tfrac12 \\ \hat b & 1 & 0 \end{array}

Why the difference is an error estimate

Measure errors as exact minus numerical, for a step started on the exact solution through (tn,yn)(t_n, y_n). The order-qq result has local error Chq+1+O(hq+2)C h^{q+1} + O(h^{q+2}), the order-(q+1)(q+1) result O(hq+2)O(h^{q+2}). Subtract:

yn+1−y^n+1=(y−y^n+1)−(y−yn+1)=Chq+1+O(hq+2)=(y(tn+1)−y^n+1) (1+O(h)).y_{n+1} - \hat y_{n+1} = \big(y - \hat y_{n+1}\big) - \big(y - y_{n+1}\big) = C h^{q+1} + O(h^{q+2}) = \big(y(t_{n+1}) - \hat y_{n+1}\big)\,\big(1 + O(h)\big).

The difference is the local error of the lower method, to leading order. In weights it is h∑ieikih\sum_i e_i k_i with e=b−b^e = b - \hat b. In Taylor series the coefficient of each elementary differential is ∑ieiΦi(τ)\sum_i e_i\Phi_i(\tau), the difference of the two methods’ order conditions for the rooted tree τ\tau (4.3 Runge–Kutta Methods & Butcher Tableaux). Both meet them through order qq, so ee first acts on trees with q+1q+1 nodes. Block A confirms it in exact fractions: the first nonzero order of ee is 2,3,52, 3, 5 for Heun–Euler, Bogacki–Shampine and Dormand–Prince, i.e. q+1q+1 for q=1,2,4q = 1, 2, 4.

Block B measures it on one step of y′=y(1−y)y' = y(1-y), y(0)=0.1y(0) = 0.1:

(estimate) / (lower method’s true error)h=0.4h = 0.4h=0.1h = 0.1h=0.025h = 0.025
Heun–Euler0.88880.88880.97020.97020.99240.9924
Bogacki–Shampine3.86783.86781.26541.26541.05791.0579
Dormand–Prince0.81010.81010.96450.96450.99210.9921
  1. The estimate is only asymptotic. The ratio tends to 11 (Heun–Euler’s distance from 11 is 0.111,0.030,0.0080.111, 0.030, 0.008, shrinking like hh), but at h=0.4h = 0.4 Bogacki–Shampine overstates by 3.93.9. Hence the safety margins below.
  2. The step uses the better answer. Block B also prints (true error of yn+1y_{n+1}) / (true error of y^\hat y): at h=0.025h = 0.025 it is 0.0080.008, 0.0580.058, 0.0080.008, so the estimate is pessimistic about the value used. This is local extrapolation, done by scipy in both RK23 and RK45. At h=0.4h = 0.4 it fails for Bogacki–Shampine, whose higher-order value is worse (ratio 2.8682.868): the asymptotics have not started.

The estimate is also local in a stricter sense: it refers to the solution through the current point, not the curve you started on (last section).

Three pairs, and FSAL

Bogacki–Shampine 3(2) (the pair behind RK23):

01212340341291349b2913490b^724141318\begin{array}{c|cccc} 0 & & & & \\ \tfrac12 & \tfrac12 & & & \\ \tfrac34 & 0 & \tfrac34 & & \\ 1 & \tfrac29 & \tfrac13 & \tfrac49 & \\ \hline b & \tfrac29 & \tfrac13 & \tfrac49 & 0 \\ \hat b & \tfrac7{24} & \tfrac14 & \tfrac13 & \tfrac18 \end{array}

Dormand–Prince 5(4) has seven stages, c=(0,15,310,45,89,1,1)c = (0, \tfrac15, \tfrac3{10}, \tfrac45, \tfrac89, 1, 1), and its full tableau is in block A. I wrote it from memory and checked it in exact fractions: bb meets every order condition through order 5 (the 1,1,2,4,91, 1, 2, 4, 9 rooted trees), b^\hat b through order 4 and not 5, the row sums equal cc, and the arrays equal scipy’s RK45. Block A checks the other two pairs too.

FSAL, first same as last. For Bogacki–Shampine and Dormand–Prince the last row of AA equals bb (with bs=0b_s = 0); block A prints True. So the last stage sits at (tn+1, yn+1)(t_{n+1},\ y_{n+1}), exactly where the next step’s k1k_1 is needed: 66 new evaluations per step for Dormand–Prince instead of 77, 33 instead of 44 for Bogacki–Shampine. It needs the step to use the higher-order weights, so it depends on local extrapolation. Heun–Euler is not FSAL (False): stage 2 sits at the Euler point, the accepted point is Heun’s.

The controller

Suppose the scaled estimate behaves like err(h)≈Chq+1\text{err}(h) \approx C h^{q+1}, with CC about the same on the next step. The step that would give exactly the tolerance solves Chnewq+1=tolC h_\text{new}^{q+1} = \text{tol}:

hnew=h⋅safety⋅(tolerr)1/(q+1).h_\text{new} = h\cdot \text{safety}\cdot\Big(\frac{\text{tol}}{\text{err}}\Big)^{1/(q+1)} .

Here qq is the order of the lower method, because that is what err measures: scipy uses q=4q = 4 for RK45 and q=2q = 2 for RK23 (block C prints error_estimator_order), exponents −15-\tfrac15 and −13-\tfrac13, although the steps are taken with orders 5 and 3. In scipy err is already divided by the tolerance, so tol=1\text{tol} = 1 and a step is accepted when err is below 1.

  • Safety 0.90.9. Aim below the tolerance, since CC drifts. The predicted error after the change is 0.9q+10.9^{q+1}: 0.810.81, 0.7290.729, 0.59050.5905 for q=1,2,4q = 1, 2, 4 (block B).
  • Ratio limits. The factor stays in [0.2,10][0.2, 10] (MIN_FACTOR, MAX_FACTOR) because the estimate is asymptotic; after a rejection the next accepted step may not grow.
  • Reject. If err is not below 1, retry from the same yny_n with the smaller hh, reusing k1k_1.

Worked example (Heun–Euler). y′=y(1−y)y' = y(1-y), y(0)=0.1y(0) = 0.1, h=0.5h = 0.5, rtol=10−3\text{rtol} = 10^{-3}, atol=10−6\text{atol} = 10^{-6}. Stages k1=0.09k_1 = 0.09, k2=f(0.5, 0.145)=0.123975k_2 = f(0.5,\ 0.145) = 0.123975; Heun gives 0.153493750.15349375, Euler 0.1450.145, exact 0.154828100.15482810. The estimate 0.008493750.00849375 compares with Euler’s true error 0.009828100.00982810 (Heun’s: 0.001334350.00133435). The scale is 10−6+10−3max⁡(0.1, 0.1535)=1.5449×10−410^{-6} + 10^{-3}\max(0.1,\ 0.1535) = 1.5449\times10^{-4}, so the error norm is 54.97854.978: reject. The raw factor 0.9⋅54.978−1/2=0.12140.9\cdot 54.978^{-1/2} = 0.1214 is below 0.20.2, so h=0.1h = 0.1. Its error norm is 3.22553.2255: reject, factor 0.9⋅3.2255−1/20.9\cdot3.2255^{-1/2}, giving h=0.0501h = 0.0501 with error norm 0.85130.8513: accept.

The first step has no history, so scipy guesses from y0y_0 and f0f_0 (a heuristic of Hairer, Nørsett and Wanner): h0=0.01∥y0∥/∥f0∥h_0 = 0.01\lVert y_0\rVert/\lVert f_0\rVert in scaled norms, an Euler probe (one extra evaluation) for how fast ff changes, then the exponent 1/(q+1)1/(q+1) again. On the orbit below it starts at 0.003920.00392 and the controller multiplies it by the maximum 1010.

rtol and atol

The error vector is divided by a scale per component and reduced to one number:

si=atol+rtol⋅max⁡(∣yn,i∣, ∣yn+1,i∣),err=1N∑i(e^i/si)2,e^=h∑ieiki.s_i = \text{atol} + \text{rtol}\cdot\max\big(\lvert y_{n,i}\rvert,\ \lvert y_{n+1,i}\rvert\big), \qquad \text{err} = \sqrt{\tfrac1N\textstyle\sum_i \big(\hat e_i/s_i\big)^2}, \qquad \hat e = h\sum_i e_i k_i .

scipy uses this root mean square over the NN components (block C: the norm of (3,4)(3, 4) is 3.53553.5355, not 44), with defaults rtol=10−3\text{rtol} = 10^{-3}, atol=10−6\text{atol} = 10^{-6}. Where ∣y∣\lvert y\rvert is large, rtol sets correct digits per step; where a component passes near zero rtol alone would demand impossible accuracy, and atol is the floor. The crossover is ∣y∣=atol/rtol\lvert y\rvert = \text{atol}/\text{rtol}. Two cautions: the RMS lets one component exceed its own scale by up to N\sqrt N (Problem 3); and one atol serves positions and velocities alike (the orbit below uses 10−910^{-9} because every component crosses zero).

✅ Concept check

A Dormand-Prince step returns y_hi (order 5) and y_lo (order 4). Which statement about what the solver does is correct?

Worked example: an eccentric orbit

A Kepler orbit x¨=−x/r3\ddot{\mathbf x} = -\mathbf x/r^3, semi-major axis 11, period 2π2\pi, eccentricity e=0.9e = 0.9, starting at apoapsis x=(−1.9, 0)\mathbf x = (-1.9,\ 0). The close approach is at t=πt = \pi: r=0.1r = 0.1, speed (1+e)/(1−e)=4.3589\sqrt{(1+e)/(1-e)} = 4.3589, so a passage lasts about r/v=0.0229r/v = 0.0229. The exact position comes from Kepler’s equation M=E−esin⁡EM = E - e\sin E (Newton’s method, machine precision), so every error below is a true distance. Block D integrates one period with Dormand–Prince (atol=10−9\text{atol} = 10^{-9}) and with fixed-step RK4 given the same evaluations (N=⌊nfev/4⌋N = \lfloor \text{nfev}/4\rfloor steps):

rtolDP accepted / rejected / nfevDP final errorRK4 stepsRK4 final error
10−310^{-3}24/6/18224 / 6 / 1821.045×10−11.045\times10^{-1}454512.2112.21
10−610^{-6}63/21/50663 / 21 / 5061.752×10−51.752\times10^{-5}1261263.9823.982
10−810^{-8}135/11/878135 / 11 / 8788.969×10−88.969\times10^{-8}2192191.2651.265

At equal cost the fixed-step body has left the orbit: an error of 3.9823.982 exceeds the apoapsis distance 1.91.9, and its step 0.04990.0499 is about twice the passage time. To match 1.752×10−51.752\times10^{-5} RK4 needs about 23002300 steps, 92009200 evaluations (a scan in steps of 100100), 18.218.2 times as many. At rtol=10−6\text{rtol} = 10^{-6} the largest step is 0.45400.4540 (at t=1.063t = 1.063), the smallest after the first 0.004470.00447 at t=3.1416t = 3.1416, a ratio of 101.6101.6; 3030 of 6363 accepted steps have r<0.5r \lt 0.5. All 2121 rejected trials begin between t=1.517t = 1.517 and 3.1373.137, on the approach. A plausible mechanism, read off the pattern and not proved: going in, the error constant CC rises fast, so a step sized from the last error is too long; going out, CC falls.

◆ Interactive Adaptive stepping on a Kepler orbit: markers, h against t, and the true error

Loading simulation…

Things to try:

  1. Defaults: accepted 6363, rejected 2121, nfev 506506, final error 1.752×10−51.752\times10^{-5}; RK4 ends 3.9823.982 away.
  2. Bogacki–Shampine: accepted 567567, rejected 1818, nfev 17571757.
  3. rtol 10−310^{-3}: final error 0.10450.1045.
  4. Orbits 1010: watch the third plot climb.

Dense output

Animation frames, t_eval and event detection (a zero crossing inside a step) need values between step ends. Dense output is a per-step interpolant built from stages already computed, so it costs no extra evaluations; it should be about as accurate as the step. Block F takes one Dormand–Prince step of h=0.454h = 0.454 from the exact state at t=1.0631t = 1.0631: end-of-step error 9.232×10−89.232\times10^{-8}. At the midpoint, the cubic Hermite interpolant from yy and y′y' at both ends has error 2.488×10−52.488\times10^{-5}, about 270270 times larger; scipy’s own continuous extension has 2.044×10−72.044\times10^{-7}. Fitting h=0.454h = 0.454 and 0.20.2, the errors scale as h5.68h^{5.68} (step end), h4.26h^{4.26} (Hermite), h5.34h^{5.34} (scipy), so the cubic falls behind more as hh shrinks (ratio 861861 at h=0.2h = 0.2).

🐍 PythonPairs verified, the estimate, the controller, scipy, Kepler, limits and dense output
⌘/Ctrl + Enter

What the tolerance does not promise

Global error is not the tolerance. Each step keeps its local error below the tolerance, but the dynamics carry local errors forward (4.1 Local Error, Global Error & Order). Block E1 repeats the orbit at rtol=10−6\text{rtol} = 10^{-6}:

orbitsacceptedrejectednfevfinal errorerror / rtol
11636321215065061.752×10−51.752\times10^{-5}17.517.5
221241244545101610164.188×10−54.188\times10^{-5}41.941.9
55305305111111249824981.800×10−41.800\times10^{-4}180180
1010607607225225499449946.282×10−46.282\times10^{-4}628628

Every step met its tolerance, yet after ten orbits the position is off by 628628 times rtol, growing faster than the number of orbits (3636-fold for 1010-fold time; I did not derive the rate). For a global bound, rerun with a tighter rtol and compare.

Adaptive steps do not cure stiffness. Take y′=−1000 (y−cos⁡t)y' = -1000\,(y - \cos t), y(0)=0y(0) = 0 on [0,3][0, 3], default tolerances: cos⁡t\cos t plus a transient dying like e−1000te^{-1000t}. Block E2: RK45 takes 912912 steps and 63506350 evaluations with final error 4.0×10−44.0\times10^{-4}, below the tolerance, so accuracy did not pick the step. Stability did: ∣R(z)∣=1\lvert R(z)\rvert = 1 on the real axis at z=−3.3066z = -3.3066 for Dormand–Prince, so h≲3.3066/1000h \lesssim 3.3066/1000. The median step after the transient has h∣λ∣=3.2593h\lvert\lambda\rvert = 3.2593, and all 145145 rejected trials after t=0.1t = 0.1 had h∣λ∣≥3.3535h\lvert\lambda\rvert \ge 3.3535: the controller rides the stability boundary and bounces off it. Implicit Radau and BDF take 1515 and 5555 steps (107107, 121121 evaluations); why they escape the bound is 4.5 Stability Regions & Stiffness.

✅ Concept check

solve_ivp with method RK45 tries h = 0.05 and gets scaled error norm 3.2e-3, accepted, with no rejection just before. Using safety 0.9, the exponent from the estimator order scipy uses for RK45, and limits 0.2 and 10, what is the next step?

✅ Concept check

A solve_ivp run with RK45 reports 130 accepted steps and you know 20 trial steps were rejected. scipy counts f(t0, y0), the first-step probe, and 6 new evaluations per attempt (FSAL). What is nfev?

Exercise: an adaptive Bogacki–Shampine integrator

Implement the step, error norm, step factor and driver. The tests compare your step count, evaluation count and final state with scipy’s RK23 from the same first step.

🧪 Coding exerciseExercise: adaptive Bogacki-Shampine 3(2) with a step-size controller
⌘/Ctrl + Enter

Problems

✏️ Pen & paperOne Heun–Euler step with a tolerance●○○

For y′=−2yy' = -2y, y(0)=1y(0) = 1, try h=0.4h = 0.4 with Heun–Euler, rtol=0.1\text{rtol} = 0.1, atol=0\text{atol} = 0, safety 0.90.9, exponent −12-\tfrac12.

  1. Compute the Euler value, the Heun value and the estimate.
  2. Find the error norm. Accepted? If not, what is the next hh?
  3. Compare the estimate with the true errors (e−0.8=0.449329e^{-0.8} = 0.449329).
  4. Try the new hh.
Show worked solution
  1. k1=−2k_1 = -2, Euler 0.20.2; k2=−0.4k_2 = -0.4, Heun 1+0.2(−2.4)=0.521 + 0.2(-2.4) = 0.52. Estimate 0.320.32.
  2. Scale 0.1⋅max⁡(1,0.52)=0.10.1\cdot\max(1, 0.52) = 0.1, norm 3.2≥13.2 \ge 1: reject. Factor 0.9⋅3.2−1/2=0.50310.9\cdot3.2^{-1/2} = 0.5031, so h=0.2012h = 0.2012.
  3. True errors: Euler 0.2493290.249329, Heun −0.070671-0.070671. The estimate overstates Euler’s error by 28%28\% and is 4.54.5 times Heun’s.
  4. h=0.20125h = 0.20125: z=−0.40249z = -0.40249, estimate z2/2=0.0810z^2/2 = 0.0810, norm 0.81=0.920.81 = 0.9^2: accepted, matching the predicted 0.9q+10.9^{q+1} exactly because the estimate is exactly quadratic in hh.
✏️ Pen & paperWhy Bogacki–Shampine's estimate scales as h cubed●●○

Use the Bogacki–Shampine tableau above.

  1. Compute ∑bici\sum b_ic_i, ∑b^ici\sum\hat b_ic_i, ∑bici2\sum b_ic_i^2, ∑b^ici2\sum\hat b_ic_i^2.
  2. Find AcAc and compute ∑bi(Ac)i\sum b_i(Ac)_i, ∑b^i(Ac)i\sum\hat b_i(Ac)_i.
  3. Give the orders of bb and b^\hat b, the first condition where e=b−b^e = b - \hat b is nonzero, and the controller’s exponent.
  4. For error norm 88, what factor does this pair give, and Dormand–Prince?
Show worked solution
  1. ∑bc=16+13=12\sum bc = \tfrac16+\tfrac13 = \tfrac12, ∑b^c=18+14+18=12\sum\hat bc = \tfrac18+\tfrac14+\tfrac18 = \tfrac12. ∑bc2=112+14=13\sum bc^2 = \tfrac1{12}+\tfrac14 = \tfrac13, ∑b^c2=116+316+18=38\sum\hat bc^2 = \tfrac1{16}+\tfrac3{16}+\tfrac18 = \tfrac38.
  2. Rows give 00, a21c1=0a_{21}c_1 = 0, 34⋅12=38\tfrac34\cdot\tfrac12 = \tfrac38, 13⋅12+49⋅34=12\tfrac13\cdot\tfrac12+\tfrac49\cdot\tfrac34 = \tfrac12: Ac=(0,0,38,12)Ac = (0, 0, \tfrac38, \tfrac12). Then ∑b(Ac)=49⋅38=16\sum b(Ac) = \tfrac49\cdot\tfrac38 = \tfrac16 and ∑b^(Ac)=18+116=316\sum\hat b(Ac) = \tfrac18+\tfrac1{16} = \tfrac3{16}.
  3. bb meets all four conditions through order 3 (1,12,13,161, \tfrac12, \tfrac13, \tfrac16): third order. b^\hat b misses both order-3 conditions (38\tfrac38 for 13\tfrac13, 316\tfrac3{16} for 16\tfrac16): second order. ee is first nonzero at order 3 (∑ec2=−124\sum ec^2 = -\tfrac1{24}, ∑e(Ac)=−148\sum e(Ac) = -\tfrac1{48}), so the estimate is O(h3)O(h^3), q=2q = 2, exponent −13-\tfrac13.
  4. 0.9⋅8−1/3=0.450.9\cdot8^{-1/3} = 0.45. Dormand–Prince: 0.9⋅8−1/5=0.59380.9\cdot8^{-1/5} = 0.5938, a milder cut because its error falls faster with hh.
✏️ Pen & paperRMS against max, and what atol is doing●●○

A step from y=(50, 0.002)y = (50,\ 0.002) to (50.4, 0.0021)(50.4,\ 0.0021) has estimate e^=(0.02, 4×10−6)\hat e = (0.02,\ 4\times10^{-6}); rtol=10−3\text{rtol} = 10^{-3}, atol=10−6\text{atol} = 10^{-6}.

  1. Compute the scales and scaled errors.
  2. Compute the RMS and max norms. Accepted under each?
  3. Show that RMS below 11 forces max⁡i∣e^i/si∣<N\max_i\lvert\hat e_i/s_i\rvert \lt \sqrt N. What does that allow for the four-component orbit?
  4. What happens to the second component if atol=0\text{atol} = 0?
Show worked solution
  1. s1=10−6+0.0504=0.050401s_1 = 10^{-6}+0.0504 = 0.050401, s2=10−6+2.1×10−6=3.1×10−6s_2 = 10^{-6}+2.1\times10^{-6} = 3.1\times10^{-6}. Scaled errors 0.39680.3968 and 1.29031.2903.
  2. RMS (0.39682+1.29032)/2=0.9546\sqrt{(0.3968^2+1.2903^2)/2} = 0.9546: accepted. Max 1.29031.2903: rejected. The small component is 29%29\% over its tolerance and scipy accepts.
  3. Every term of ∑i(e^i/si)2=N⋅RMS2<N\sum_i(\hat e_i/s_i)^2 = N\cdot\text{RMS}^2 \lt N is nonnegative, so (e^j/sj)2<N(\hat e_j/s_j)^2 \lt N for each jj. For N=4N = 4 one component may be up to 22 times over its scale (2=1.414\sqrt2 = 1.414 for N=2N = 2, consistent with 1.29031.2903).
  4. Its scale becomes 2.1×10−62.1\times10^{-6} and its scaled error 1.9051.905: the RMS exceeds 11, rejected. As a component nears zero its scale vanishes and the step shrinks without limit; atol is the floor.

Key Takeaways

  • An embedded pair gets a second answer free; h∑(bi−b^i)kih\sum(b_i-\hat b_i)k_i is, to leading order, the local error of the lower-order result, because bb and b^\hat b share their order conditions through qq.
  • The solver steps with the higher-order result (local extrapolation); the estimate is only asymptotic.
  • From err≈Chq+1\text{err}\approx Ch^{q+1}: hnew=0.9 h (tol/err)1/(q+1)h_\text{new} = 0.9\,h\,(\text{tol}/\text{err})^{1/(q+1)}, q=4q = 4 for RK45 and 22 for RK23, factor in [0.2,10][0.2, 10], no growth right after a rejection; a rejected step retries from the same yny_n.
  • The scale is atol+rtolmax⁡(∣y∣,∣ynew∣)\text{atol}+\text{rtol}\max(\lvert y\rvert,\lvert y_\text{new}\rvert), the norm an RMS over components (one can be up to N\sqrt N over); defaults 10−310^{-3}, 10−610^{-6}.
  • Bogacki–Shampine and Dormand–Prince are FSAL: nfev =2+6(accepted+rejected)= 2 + 6(\text{accepted}+\text{rejected}) for RK45.
  • On the e=0.9e = 0.9 orbit the step spans a factor of about 100100; 506506 evaluations gave 1.75×10−51.75\times10^{-5}, while fixed RK4 at that cost gave 3.983.98 and needed 1818 times as many to match.
  • Dense output must match the step’s accuracy: a cubic Hermite was 270270 times worse.
  • The tolerance is local: after ten orbits the error was 628628 times rtol. And it cannot fix stiffness: RK45 on y′=−1000(y−cos⁡t)y'=-1000(y-\cos t) rode the stability limit, 912912 steps against 1515 for Radau.
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.