Flow Lab

Syllabus / Module 7: Systems of ODEs & Dynamical Systems

Lesson 7.5

The Matrix Exponential & Fundamental Matrices

e^{At} as the solution operator, computing it, fundamental matrices, and connections to numerical time-stepping.

Written byClaude Sonnet

🌊
Why this matters for simulation: Exponential integrators solve the stiff linear part of a fluid equation exactly.

By the end of this lesson you can…

  • Build a fundamental matrix $\Phi(t)$ from independent solutions and turn it into the flow map $e^{At} = \Phi(t)\Phi(0)^{-1}$.
  • State and check $e^{A(t+s)} = e^{At}e^{As}$, $(e^{At})^{-1} = e^{-At}$ and $\tfrac{d}{dt}e^{At} = Ae^{At}$, and give a $2\times2$ example where $e^{(A+B)t} \neq e^{At}e^{Bt}$.
  • Compute $e^{At}$ for a $2\times2$ matrix by diagonalization, by the nilpotent series, by a Jordan block (deriving the $t$ in the corner) and by the Cayley–Hamilton closed form.
  • Show that $e^{[[0,-\omega],[\omega,0]]t}$ is a rotation by $\omega t$, and use $\det e^{At} = e^{\operatorname{tr}(A)t}$ to say how a flow changes areas.
  • Predict when a truncated series or an eigendecomposition loses accuracy, and say what `scipy.linalg.expm` does instead.
  • Write the exponential Euler step and explain why it takes steps at which explicit Euler and RK4 blow up on a stiff linear part.
On this page
·
  1. Fundamental matrices and the flow map
  2. The series, and how many terms it needs
  3. Three routes by hand
  4. The rotation generator and Euler’s formula
  5. Liouville: what the flow does to area
  6. Computing it well
  7. Exponential integrators
  8. Computing it
  9. Concept checks
  10. Problems
  11. Key Takeaways

7.4 Complex & Repeated Eigenvalues ended with a promise: for a defective matrix the solution carries a factor tt, and the claim eJt=eλt(1t01)e^{Jt} = e^{\lambda t}\begin{pmatrix}1&t\\0&1\end{pmatrix} would be derived later. This is later. The derivation is a special case of something bigger. Every case of 7.2 Linear Systems & the Eigenvalue Method and 7.3 Phase Portraits & the Trace–Determinant Plane is one formula, x(t)=eAtx0\mathbf x(t) = e^{At}\mathbf x_0, and eAte^{At} is a single matrix that moves every starting point at once. 1.8 The Matrix Exponential: From Hops to Flows defined it and diagonalized it; here we make it the working tool: three ways to compute it, its determinant, how it fails numerically, and the one idea from it that fluid and flight codes use directly, the exponential integrator.

▶ Video A blob of points carried by e^(At): the series, three routes to compute it, the area law, and a stiff step that is exactcaptions only
Transcript
  1. Every solution is x of t equals e to the A t times x zero. Solutions side by side form a fundamental matrix phi, and e to the A t is phi of t times phi of zero inverse: the flow map, moving a whole blob at once. At t equals 1: 0.60042, 0.23254, minus 0.46509, minus 0.09721.
  2. Flows compose, but the exponential of a sum is the product only for commuting matrices. For P with a one above the diagonal and Q with a one below, e to the P times e to the Q is 2, 1, 1, 1, yet e to the P plus Q differs by 0.54308.
  3. The series defines it. For the rotation generator at t equals 1, the top left entry after 2, 4, 6, 8 terms is 0.5, 0.54167, 0.54028, 0.54030, near cosine 1. But the terms needed grow with t: 14, 29, 44, 72, 127 for t equals 1, 5, 10, 20, 40.
  4. Three routes. If A is V lambda V inverse, the series sums to V e to the lambda t V inverse. If N squared is zero it stops: e to the N t is I plus N t, here 2, minus 2, 0.5, 0 at t equals 0.5, a shear.
  5. A defective matrix is lambda I plus N, with N squared zero, and lambda I commutes with everything. So e to the A t is e to the lambda t times I plus N t: the t in the corner is the first term. The earlier example at 0.5: 0.18394, 0.18394, minus 0.18394, 0.55182.
  6. For any two by two, Cayley-Hamilton gives a closed form with cosh, cos or t. For the rotation generator it is a rotation by omega t: Euler's formula, with J playing i. At t equals 1.5, that is 0.07074, minus 0.99749. The spiral adds a factor e to the minus 0.2 t.
  7. Liouville: the determinant of e to the A t is e to the trace times t, since the determinant of phi has derivative T times itself. So the flow scales areas by e to the T t: the saddle by 2.71828, the spiral 0.67032, the node 0.04979.
  8. The incompressible saddle has trace zero. At t equals 1 the disk becomes an ellipse with singular values 9.35647 and 0.10688, yet the area ratio is 1.00000. At t equals 4 it is a sliver 7663.87 long, area still 1. Euler's step gains area, 1.04.
  9. For diagonal minus 20, minus 1, the true first entry is 2.0612 times 10 to the minus 9, but 150 terms give 5.6219 times 10 to the minus 9: the largest term is 4.3 times 10 to the 7, so rounding costs about 4 times 10 to the minus 9. expm is right.
  10. The eigenvector route fails differently. For this nearly defective matrix, V has condition number 2 times 10 to the 12, so V e to the lambda V inverse is off by 2 times 10 to the minus 5 on an entry of 3678.8; expm by 9 times 10 to the minus 13.
  11. Now the exponential integrator. For x prime equals A x plus N, freeze N over a step. The exact solution is e to the A h times x, plus A inverse times e to the A h minus I times N. The linear part is exact, so only N limits h.
  12. A stiff test, diagonal minus 100, minus 1. Euler is stable only below h of 0.02, RK4 below 0.027853. At h equals 0.1 the errors are 3.5 times 10 to the 9 and 4.4 times 10 to the 24; exponential Euler's is 4 times 10 to the minus 16, but only first order.
  13. In fluid codes diffusion is diagonal in Fourier modes, so e to the lambda k h is cheap; from memory, ETD methods use this. Stam instead treats diffusion implicitly. An exact incompressible flow keeps area, and a solver should try to. Next: nonhomogeneous systems.

Fundamental matrices and the flow map

Take A=(01−2−3)A = \begin{pmatrix}0&1\\-2&-3\end{pmatrix}. Its eigenpairs are −1-1 with (1,−1)(1,-1) and −2-2 with (1,−2)(1,-2) (check: A(1,−2)=(−2,4)A(1,-2) = (-2, 4)). Put the two independent solutions side by side as columns:

Φ(t)=(e−te−2t−e−t−2e−2t).\Phi(t) = \begin{pmatrix} e^{-t} & e^{-2t}\\ -e^{-t} & -2e^{-2t}\end{pmatrix}.

This is a fundamental matrix: each column solves x′=Ax\mathbf x' = A\mathbf x, so Φ′=AΦ\Phi' = A\Phi, and the columns are independent at every tt (they are at t=0t = 0, and by uniqueness stay so). Any solution is Φ(t)c\Phi(t)\mathbf c, and matching x0\mathbf x_0 at t=0t = 0 forces c=Φ(0)−1x0\mathbf c = \Phi(0)^{-1}\mathbf x_0. So

x(t)=eAtx0,eAt=Φ(t)Φ(0)−1.\mathbf x(t) = e^{At}\mathbf x_0, \qquad e^{At} = \Phi(t)\Phi(0)^{-1}.

Here Φ(0)−1=(21−1−1)\Phi(0)^{-1} = \begin{pmatrix}2&1\\-1&-1\end{pmatrix}, giving eAt=(2e−t−e−2te−t−e−2t−2e−t+2e−2t−e−t+2e−2t)e^{At} = \begin{pmatrix}2e^{-t} - e^{-2t} & e^{-t} - e^{-2t}\\ -2e^{-t} + 2e^{-2t} & -e^{-t} + 2e^{-2t}\end{pmatrix}, which at t=1t = 1 is (0.600420.23254−0.46509−0.09721)\begin{pmatrix}0.60042&0.23254\\-0.46509&-0.09721\end{pmatrix} (block A matches it to expm). The matrix is independent of which fundamental matrix you started from: ΦC\Phi C is another one and CC cancels. It is the solution operator or flow map: feed it any point and it returns where that point is at time tt. It is exactly the object a time-stepper approximates, one step at a time, with t=ht = h.

Properties, each from the equation Φ′=AΦ\Phi' = A\Phi or from uniqueness:

  • ddteAt=AeAt\frac{d}{dt}e^{At} = Ae^{At} (eAte^{At} is a fundamental matrix with value II at 00).
  • eA(t+s)=eAteAse^{A(t+s)} = e^{At}e^{As}: flowing for ss then for tt is flowing for t+st + s (both sides solve the equation in tt with the same value at t=0t = 0). In particular (eAt)−1=e−At(e^{At})^{-1} = e^{-At}, so the flow can be run backwards and eAte^{At} is never singular.
  • Block A confirms them for this AA: the semigroup gap and eAe−A−Ie^Ae^{-A} - I are at rounding level (about 10−1410^{-14}), and the derivative check differs by 1.6×10−101.6\times10^{-10}, the finite-difference error.
  • e(A+B)t=eAteBte^{(A+B)t} = e^{At}e^{Bt} needs AB=BAAB = BA.

The honest caveat. Take P=(0100)P = \begin{pmatrix}0&1\\0&0\end{pmatrix} and Q=(0010)Q = \begin{pmatrix}0&0\\1&0\end{pmatrix}, with PQ−QP=diag(1,−1)≠0PQ - QP = \mathrm{diag}(1, -1) \neq 0. Then eP=(1101)e^P = \begin{pmatrix}1&1\\0&1\end{pmatrix}, eQ=(1011)e^Q = \begin{pmatrix}1&0\\1&1\end{pmatrix}, so ePeQ=(2111)e^Pe^Q = \begin{pmatrix}2&1\\1&1\end{pmatrix}, while P+Q=(0110)P + Q = \begin{pmatrix}0&1\\1&0\end{pmatrix} has eigenvalues ±1\pm1 and eP+Q=(cosh⁡1sinh⁡1sinh⁡1cosh⁡1)=(1.543081.175201.175201.54308)e^{P+Q} = \begin{pmatrix}\cosh1&\sinh1\\\sinh1&\cosh1\end{pmatrix} = \begin{pmatrix}1.54308&1.17520\\1.17520&1.54308\end{pmatrix}. The largest entry gap is 0.543080.54308 (block A). This is the splitting error of 1.8 The Matrix Exponential: From Hops to Flows, made concrete.

The series, and how many terms it needs

Define eM=∑k≥0Mk/k!e^{M} = \sum_{k\ge0}M^k/k! with M=AtM = At. For the rotation generator J=(0−110)J = \begin{pmatrix}0&-1\\1&0\end{pmatrix} at t=1t = 1 the answer is (cos⁡1−sin⁡1sin⁡1cos⁡1)\begin{pmatrix}\cos1&-\sin1\\\sin1&\cos1\end{pmatrix} with cos⁡1=0.54030\cos 1 = 0.54030. The (1,1)(1,1) entry of the partial sum after K=2,4,6,8K = 2, 4, 6, 8 terms is 0.5, 0.54167, 0.54028, 0.540300.5,\ 0.54167,\ 0.54028,\ 0.54030, with largest entry error 0.159, 8.1×10−3, 2.0×10−4, 2.7×10−60.159,\ 8.1\times10^{-3},\ 2.0\times10^{-4},\ 2.7\times10^{-6} (block B): fast, because the terms 1/k!1/k! collapse. The collapse starts only after k≈∥Mt∥k \approx \lVert Mt\rVert, because tk/k!t^k/k! first grows until k=tk = t. Block B counts the terms until tk/k!<10−10t^k/k! \lt 10^{-10} for JtJ t with ∥Jt∥=t\lVert Jt \rVert = t:

tt15102040
terms needed14294472127
largest term126.042755.74.3×1074.3\times10^{7}1.5×10161.5\times10^{16}
error of the sum7.6×10−137.6\times10^{-13}3.4×10−123.4\times10^{-12}7.7×10−127.7\times10^{-12}6.9×10−96.9\times10^{-9}2.92.9

The cost grows linearly in ∥At∥\lVert At\rVert, but the accuracy does not survive. Every term is rounded to about 10−1610^{-16} of its own size, and the sum is a difference of huge terms of both signs, so the error is about 10−1610^{-16} times the largest term: at t=40t = 40 the largest is 1.5×10161.5\times10^{16} and the “answer” is wrong by 2.92.9, on a rotation whose entries are at most 11. The series is a definition and a proof tool, not an algorithm for large ∥At∥\lVert At\rVert. The sim’s readout shows the same effect live: the gap between its two series-based routes grows as you raise tt.

Three routes by hand

Route 1: diagonalizable. If A=VΛV−1A = V\Lambda V^{-1} then Ak=VΛkV−1A^k = V\Lambda^kV^{-1} and the series sums to eAt=VeΛtV−1e^{At} = Ve^{\Lambda t}V^{-1}, with eΛt=diag(eλit)e^{\Lambda t} = \mathrm{diag}(e^{\lambda_it}). For the matrix above V=(11−1−2)V = \begin{pmatrix}1&1\\-1&-2\end{pmatrix}, which is Φ(0)\Phi(0), so this is the formula Φ(t)Φ(0)−1\Phi(t)\Phi(0)^{-1}: block C reproduces eAte^{At} at t=1t = 1 to rounding.

Route 2: nilpotent. If N2=0N^2 = 0 the series stops: eNt=I+Nte^{Nt} = I + Nt. Example: N=(2−41−2)N = \begin{pmatrix}2&-4\\1&-2\end{pmatrix} has N2=(4−4−8+82−2−4+4)=0N^2 = \begin{pmatrix}4-4 & -8+8\\2-2&-4+4\end{pmatrix} = 0, and e0.5N=I+0.5N=(2−20.50)e^{0.5N} = I + 0.5N = \begin{pmatrix}2&-2\\0.5&0\end{pmatrix} (block C).

Route 3: Jordan block. A defective AA with double eigenvalue λ\lambda is λI+N\lambda I + N with N=A−λIN = A - \lambda I, and Cayley–Hamilton gives N2=0N^2 = 0 (as in the previous lesson). The matrix λI\lambda I commutes with everything, in particular with NN, so the caveat above does not bite: e(λI+N)t=eλIteNte^{(\lambda I + N)t} = e^{\lambda It}e^{Nt}, and the series for eλIte^{\lambda It} is eλtIe^{\lambda t}I. Hence

eAt=eλt(I+Nt),for J=(λ10λ):eJt=eλt(1t01).e^{At} = e^{\lambda t}(I + Nt), \qquad \text{for } J = \begin{pmatrix}\lambda&1\\0&\lambda\end{pmatrix}:\quad e^{Jt} = e^{\lambda t}\begin{pmatrix}1&t\\0&1\end{pmatrix}.

There is the tt in the corner: it is the first term of the nilpotent series, nothing more. For the previous lesson’s A=(−31−1−1)A = \begin{pmatrix}-3&1\\-1&-1\end{pmatrix}, λ=−2\lambda = -2 and N=A+2I=(−11−11)N = A + 2I = \begin{pmatrix}-1&1\\-1&1\end{pmatrix}, so eAt=e−2t(1−tt−t1+t)e^{At} = e^{-2t}\begin{pmatrix}1-t&t\\-t&1+t\end{pmatrix}. At t=0.5t = 0.5 this is e−1(0.50.5−0.51.5)=(0.183940.18394−0.183940.55182)e^{-1}\begin{pmatrix}0.5&0.5\\-0.5&1.5\end{pmatrix} = \begin{pmatrix}0.18394&0.18394\\-0.18394&0.55182\end{pmatrix} (block C; the sim’s defective preset shows the same entries), and its second column applied to (0,1)(0,1) is e−2t(t,1+t)e^{-2t}(t, 1+t), the solution found there.

Route 4: a closed form for every 2×22\times2. Put m=T/2m = T/2 and B=A−mIB = A - mI. Then tr⁡B=0\operatorname{tr}B = 0, so Cayley–Hamilton for BB reads B2=−det⁡B I=qIB^2 = -\det B\,I = qI with q=m2−Dq = m^2 - D (using det⁡B=D−m2\det B = D - m^2). Even powers of BB are multiples of II, odd powers multiples of BB, and summing the series for eBte^{Bt} gives

eAt=emt[c I+s B],(c,s)={(cosh⁡rt, sinh⁡(rt)/r)q=r2>0(cos⁡rt, sin⁡(rt)/r)q=−r2<0(1, t)q=0e^{At} = e^{mt}\Big[c\,I + s\,B\Big],\quad (c, s) = \begin{cases}(\cosh rt,\ \sinh(rt)/r) & q = r^2 \gt 0\\ (\cos rt,\ \sin(rt)/r) & q = -r^2 \lt 0\\ (1,\ t) & q = 0\end{cases}

Real distinct, complex and repeated eigenvalues are the three lines; q=0q = 0 is the nilpotent route (with N=BN = B), and the star node A=λIA = \lambda I has B=0B = 0. Block C checks the formula against expm at t=1.3t = 1.3 for six matrices, including all three cases, and the largest gap is below 10−1210^{-12}. This is the formula the exercise below asks you to code.

The rotation generator and Euler’s formula

For Jω=(0−ωω0)J_\omega = \begin{pmatrix}0&-\omega\\\omega&0\end{pmatrix}: T=0T = 0, D=ω2D = \omega^2, so m=0m = 0, q=−ω2q = -\omega^2 and the closed form gives

eJωt=cos⁡(ωt) I+sin⁡ωtωJω=(cos⁡ωt−sin⁡ωtsin⁡ωtcos⁡ωt).e^{J_\omega t} = \cos(\omega t)\,I + \frac{\sin\omega t}{\omega}J_\omega = \begin{pmatrix}\cos\omega t&-\sin\omega t\\\sin\omega t&\cos\omega t\end{pmatrix}.

Directly from the series: Jω2=−ω2IJ_\omega^2 = -\omega^2I, so the even terms give cos⁡\cos and the odd terms sin⁡\sin, which is Euler’s formula eiθ=cos⁡θ+isin⁡θe^{i\theta} = \cos\theta + i\sin\theta with ii played by J1J_1 (J12=−IJ_1^2 = -I). This is the center of 7.3 Phase Portraits & the Trace–Determinant Plane: closed circles, speed ω\omega. For ω=1\omega = 1, t=1.5t = 1.5 block D prints (0.07074−0.997490.997490.07074)\begin{pmatrix}0.07074&-0.99749\\0.99749&0.07074\end{pmatrix} and the sim’s center preset shows the same matrix. The spiral (−0.2−11−0.2)=−0.2I+J1\begin{pmatrix}-0.2&-1\\1&-0.2\end{pmatrix} = -0.2I + J_1 is e−0.2te^{-0.2t} times that rotation (a scalar commutes), which at t=1t = 1 has entries 0.442360.44236 and 0.688940.68894 in the sim.

Liouville: what the flow does to area

Take w(t)=det⁡Φ(t)w(t) = \det\Phi(t) with columns x1,x2\mathbf x_1, \mathbf x_2. The derivative of a determinant is the sum of the determinants with one column differentiated: w′=det⁡[Ax1,x2]+det⁡[x1,Ax2]w' = \det[A\mathbf x_1,\mathbf x_2] + \det[\mathbf x_1, A\mathbf x_2]. For 2×22\times2 matrices this sum equals (a11+a22)det⁡[x1 x2](a_{11} + a_{22})\det[\mathbf x_1\ \mathbf x_2] (expand with x1=(p,q)\mathbf x_1 = (p, q), x2=(r,s)\mathbf x_2 = (r, s): the terms in bb and cc cancel and aa and dd each multiply ps−qrps - qr). So w′=Tww' = Tw and, with Φ(0)=I\Phi(0) = I,

det⁡eAt=eTt,T=tr⁡A.\det e^{At} = e^{T t},\qquad T = \operatorname{tr}A .

(For a diagonalizable AA it is also ∏eλit=e(∑λi)t\prod e^{\lambda_it} = e^{(\sum\lambda_i)t}; the argument above is the general one, and the n×nn\times n version is Liouville’s formula, from memory.) Geometrically, eAte^{At} maps a region of area A\mathcal A to one of area eTtAe^{Tt}\mathcal A: a flow with T=0T = 0 preserves area, however much it stretches. That is the incompressibility of 2.7 Divergence: Sources & Sinks: T=∇⋅uT = \nabla\cdot\mathbf u for a linear velocity field, the T=0T = 0 line of the phase-portrait plane. The sim carries a unit disk to its image, computes the area of the image with the shoelace formula, and compares it with det⁡eAt\det e^{At} and with eTte^{Tt} (a linear map scales every polygon’s area by exactly det⁡\det, so the three agree to rounding).

◆ Interactive Flow a blob by e^(At): matrix presets, a t slider, area against e^(Tt), singular values and three routes to e^(At)

Loading simulation…

What to try. (1) Incompressible saddle (122−1)\begin{pmatrix}1&2\\2&-1\end{pmatrix} at t=1t = 1: the disk becomes an ellipse with singular values 9.356479.35647 and 0.106880.10688 (e±5e^{\pm\sqrt5}, since AA is symmetric) and the area ratio is 1.000001.00000. At t=4t = 4 the lengths are 7663.877663.87 and 0.000130.00013, yet the area ratio is still 1.000001.00000: a thin sliver with the same area, which is how a strained flow smears a blob of dye into a filament. (2) Saddle (1120)\begin{pmatrix}1&1\\2&0\end{pmatrix}, T=1T = 1: at t=1t = 1 the ratio is 2.71828=e2.71828 = e, with singular values 7.751647.75164 and 0.350670.35067. (3) Spiral (T=−0.4T = -0.4): ratio 0.670320.67032 at t=1t = 1, singular values both 0.818730.81873 (a rotation, no shape change). (4) Shear (0100)\begin{pmatrix}0&1\\0&0\end{pmatrix}: area 1.000001.00000, singular values 1.618031.61803 and 0.618030.61803, a shear that is incompressible and not a rotation. (5) Node (−210−1)\begin{pmatrix}-2&1\\0&-1\end{pmatrix}, T=−3T = -3: ratio 0.04979=e−30.04979 = e^{-3}.

A link to the time-steppers. The truncations of ehAe^{hA} do not respect the area law exactly. For the center (01−40)\begin{pmatrix}0&1\\-4&0\end{pmatrix} at h=0.1h = 0.1 block D prints det⁡(I+hA)=1.04\det(I + hA) = 1.04 for Euler (the area gain seen in 7.3 Phase Portraits & the Trace–Determinant Plane), 0.999999120.99999912 for the RK4 step (the Taylor polynomial of degree 44) and 11 for ehAe^{hA}. RK4 loses 9×10−79\times10^{-7} of area per step, which is why it is so good and still not exactly conservative.

Computing it well

scipy.linalg.expm uses a Padé approximant combined with scaling and squaring (from memory, following Higham and Al-Mohy–Higham): choose ss so that ∥At∥/2s\lVert A t\rVert/2^s is small, approximate eAt/2se^{At/2^s} by a rational function, and square the result ss times. The sim does a simple version (halve until the norm is below 0.50.5, a 1616-term Taylor series, square back). Scaling avoids the huge terms of the table: eM=(eM/2)2e^{M} = (e^{M/2})^2 needs a series for a small matrix only.

Where the plain series fails. Take M=diag(−20,−1)M = \mathrm{diag}(-20, -1), so e11M=e−20=2.0612×10−9e^{M}_{11} = e^{-20} = 2.0612\times10^{-9}. The series with 150150 terms gives 5.6219×10−95.6219\times10^{-9}, wrong by a factor 2.72.7, because its largest term is 2020/20!=4.3×10720^{20}/20! = 4.3\times10^{7} and rounding of 10−1610^{-16} of that is 4×10−94\times10^{-9}. For diag(−30,−1)\mathrm{diag}(-30,-1) it returns −3.07×10−5-3.07\times10^{-5} for a quantity that is 9.4×10−149.4\times10^{-14}: negative and a billion times too large. expm gets both right (block E). Neither error is large in absolute terms, but for a stiff decaying mode they are the wrong sign or size.

Where the eigenvector route fails. Take A=(−11040−1+10−8)A = \begin{pmatrix}-1&10^4\\0&-1+10^{-8}\end{pmatrix}, eigenvalues −1-1 and −1+10−8-1 + 10^{-8}, a nearly defective non-normal matrix. The eigenvector matrix has condition number about 2×10122\times10^{12}, and VeΛV−1Ve^{\Lambda}V^{-1} is off by 2.0×10−52.0\times10^{-5} on an entry of size 3678.83678.8 (exact value 104e−1(e10−8−1)/10−810^4e^{-1}(e^{10^{-8}} - 1)/10^{-8}, block E), while expm is off by 9×10−139\times10^{-13}. About eight of sixteen digits are gone (platform dependent), the same ill-conditioning the previous lesson measured as eigenvalues merge. For non-normal matrices, prefer expm.

Exponential integrators

For x′=Ax+N(x)\mathbf x' = A\mathbf x + \mathbf N(\mathbf x) with a stiff linear part AA, freeze N\mathbf N at its current value Nn=N(xn)\mathbf N_n = \mathbf N(\mathbf x_n) over one step. The solution of x′=Ax+Nn\mathbf x' = A\mathbf x + \mathbf N_n with x(0)=xn\mathbf x(0) = \mathbf x_n is

xn+1=eAhxn+A−1(eAh−I)Nn,\mathbf x_{n+1} = e^{Ah}\mathbf x_n + A^{-1}\big(e^{Ah} - I\big)\mathbf N_n ,

the exponential Euler step (check: Axn+1+Nn=AeAhxn+eAhNn=ddhxn+1A\mathbf x_{n+1} + \mathbf N_n = Ae^{Ah}\mathbf x_n + e^{Ah}\mathbf N_n = \frac{d}{dh}\mathbf x_{n+1}). The linear part is advanced exactly for any hh, so hh is limited by how fast N\mathbf N changes, not by the fastest eigenvalue. The general formula for a time-varying forcing is Duhamel’s, in 7.6 Nonhomogeneous Linear Systems.

A stiff test. A=diag(−100,−1)A = \mathrm{diag}(-100, -1), constant N=(100,2)\mathbf N = (100, 2), x(0)=0\mathbf x(0) = 0, exact solution (1−e−100t, 2(1−e−t))(1 - e^{-100t},\ 2(1 - e^{-t})), solved to t=1t = 1. Explicit Euler is stable only for h<2/100=0.02h \lt 2/100 = 0.02 and RK4 for h<2.7853/100=0.027853h \lt 2.7853/100 = 0.027853 (4.5 Stability Regions & Stiffness). Block F gives the largest error at t=1t = 1:

hhexplicit EulerRK4exponential Euler
0.10.13.5×1093.5\times10^{9}4.4×10244.4\times10^{24}4×10−164\times10^{-16}
1/35=0.028571/35 = 0.028572.6×1092.6\times10^{9}43.543.500
1/36=0.027781/36 = 0.027789.9×1089.9\times10^{8}0.660.661.6×10−151.6\times10^{-15}

Just above RK4’s limit (row 2) the fast mode grows; just below (row 3) it decays, slowly (0.660.66 is the damped remnant of an error that started at 11). Exponential Euler is exact to rounding at all three, because the forcing is constant. With a state-dependent N=(100,−x22)\mathbf N = (100, -x_2^2) and x2(0)=1x_2(0) = 1 it is only first order: its error at t=1t = 1 is 1.45×10−2,7.0×10−3,3.4×10−3,1.7×10−31.45\times10^{-2}, 7.0\times10^{-3}, 3.4\times10^{-3}, 1.7\times10^{-3} for h=0.1,0.05,0.025,0.0125h = 0.1, 0.05, 0.025, 0.0125 (ratios about 22), while RK4 diverges at h=0.1h = 0.1 and 0.050.05 (4.4×10244.4\times10^{24}, 5.5×10225.5\times10^{22}) but reaches 3.0×10−83.0\times10^{-8} at h=0.025h = 0.025. So exponential Euler wins on stability, not on accuracy per step; higher-order exponential methods exist (from memory).

Computing it

🐍 PythonLab: properties, the series, three routes, rotation and determinant, where the series and eig fail, and the stiff test (blocks A to F)
⌘/Ctrl + Enter

Now build the closed form yourself and check it against the series-free expm. Everything graded is real.

🧪 Coding exerciseExercise: e^(At) for a 2x2 matrix in closed form, and the area of a flowed blob
⌘/Ctrl + Enter

Concept checks

✅ Concept check

$J = \begin{pmatrix}-2&1\\0&-2\end{pmatrix}$. What is the upper-right entry of $e^{Jt}$ at $t = 0.5$? (Give a number.)

✅ Concept check

A flow $\mathbf x' = A\mathbf x$ has $\operatorname{tr}A = -3$. By what factor does a blob's area change in time $t = 1$? (Give a number.)

✅ Concept check

Why does summing the series for $e^{At}$ with $A = \mathrm{diag}(-30, -1)$, $t = 1$, return a wildly wrong value for the first entry?

Problems

✏️ Pen & paperA repeated eigenvalue●●○

Compute eAte^{At} for A=(3−111)A = \begin{pmatrix}3&-1\\1&1\end{pmatrix}.

Show worked solution

T=4T = 4, D=3+1=4D = 3 + 1 = 4, so λ=2\lambda = 2 twice. N=A−2I=(1−11−1)≠0N = A - 2I = \begin{pmatrix}1&-1\\1&-1\end{pmatrix} \neq 0 and N2=(1−1−1+11−1−1+1)=0N^2 = \begin{pmatrix}1-1&-1+1\\1-1&-1+1\end{pmatrix} = 0. Because 2I2I commutes with NN, eAt=e2t(I+Nt)=e2t(1+t−tt1−t)e^{At} = e^{2t}(I + Nt) = e^{2t}\begin{pmatrix}1+t&-t\\t&1-t\end{pmatrix}. Check: at t=0t = 0 it is II, and the derivative at 00 is 2I+N=A2I + N = A. (Hand computation; it is the matrix of the repeated-eigenvalue branch of the exercise.)

✏️ Pen & paperArea per turn of a spiral●●○

For A=(−0.5−33−0.5)A = \begin{pmatrix}-0.5&-3\\3&-0.5\end{pmatrix} find eAte^{At}, the period of the rotation, and the factors by which lengths and areas shrink per turn.

Show worked solution

A=−0.5I+3J1A = -0.5I + 3J_1 with J1=(0−110)J_1 = \begin{pmatrix}0&-1\\1&0\end{pmatrix}, and the two commute, so eAt=e−0.5t(cos⁡3t−sin⁡3tsin⁡3tcos⁡3t)e^{At} = e^{-0.5t}\begin{pmatrix}\cos3t&-\sin3t\\\sin3t&\cos3t\end{pmatrix}. One turn takes t=2π/3=2.0944t = 2\pi/3 = 2.0944. The length factor is e−0.5⋅2.0944=0.35092e^{-0.5\cdot2.0944} = 0.35092. Here T=−1T = -1, so the area factor is e−2.0944=0.12316e^{-2.0944} = 0.12316, which is the square of the length factor 0.350920.35092, as it must be for a rotation-scaling. (Hand computation; not produced by the sim.)

✏️ Pen & paperThe oscillator, area and Euler●●●

(a) For A=(01−ω20)A = \begin{pmatrix}0&1\\-\omega^2&0\end{pmatrix} derive eAte^{At} and show det⁡eAt=1\det e^{At} = 1. (b) Show that Euler’s matrix I+hAI + hA has determinant 1+h2ω21 + h^2\omega^2 for every hh, and say what this does to a ring of initial conditions.

Show worked solution

(a) T=0T = 0, D=ω2D = \omega^2, so m=0m = 0, q=−ω2q = -\omega^2 and eAt=cos⁡(ωt)I+sin⁡ωtωA=(cos⁡ωtsin⁡(ωt)/ω−ωsin⁡ωtcos⁡ωt)e^{At} = \cos(\omega t)I + \frac{\sin\omega t}{\omega}A = \begin{pmatrix}\cos\omega t&\sin(\omega t)/\omega\\-\omega\sin\omega t&\cos\omega t\end{pmatrix}. Its determinant is cos⁡2ωt+sin⁡2ωt=1\cos^2\omega t + \sin^2\omega t = 1, as the area law says for T=0T = 0. (b) det⁡(I+hA)=1⋅1−h⋅(−ω2h)=1+h2ω2\det(I + hA) = 1\cdot1 - h\cdot(-\omega^2h) = 1 + h^2\omega^2. Each Euler step multiplies areas by this factor, so a ring of states is inflated step by step: the numerical flow is not area preserving, in contrast with the exact one (for ω=2\omega = 2, h=0.1h = 0.1 this is the 1.041.04 of block D).

Key Takeaways

  • A fundamental matrix Φ(t)\Phi(t) has independent solutions as columns and Φ′=AΦ\Phi' = A\Phi; eAt=Φ(t)Φ(0)−1e^{At} = \Phi(t)\Phi(0)^{-1} is the flow map x0↦x(t)\mathbf x_0 \mapsto \mathbf x(t), with eA(t+s)=eAteAse^{A(t+s)} = e^{At}e^{As}, (eAt)−1=e−At(e^{At})^{-1} = e^{-At} and ddteAt=AeAt\frac{d}{dt}e^{At} = Ae^{At}.
  • e(A+B)t=eAteBte^{(A+B)t} = e^{At}e^{Bt} only if AB=BAAB = BA: for P=(0100)P = \begin{pmatrix}0&1\\0&0\end{pmatrix}, Q=(0010)Q = \begin{pmatrix}0&0\\1&0\end{pmatrix} the largest entry gap is 0.543080.54308.
  • Routes: VeΛtV−1Ve^{\Lambda t}V^{-1}; for N2=0N^2 = 0, eNt=I+Nte^{Nt} = I + Nt; for a Jordan block eJt=eλt(I+Nt)e^{Jt} = e^{\lambda t}(I + Nt), because λI\lambda I commutes with NN, which derives the tt in the corner; and for any 2×22\times2, eAt=emt[cI+sB]e^{At} = e^{mt}[cI + sB] from Cayley–Hamilton.
  • eJωte^{J_\omega t} is rotation by ωt\omega t (Euler’s formula, from Jω2=−ω2IJ_\omega^2 = -\omega^2I); the spiral is that rotation times eαte^{\alpha t}.
  • det⁡eAt=etr⁡(A)t\det e^{At} = e^{\operatorname{tr}(A)t} because (det⁡Φ)′=Tdet⁡Φ(\det\Phi)' = T\det\Phi; T=0T = 0 flows preserve area (1.000001.00000 even when the blob is stretched to a 7663.877663.87-long sliver), and Euler’s det⁡(I+hA)=1.04\det(I + hA) = 1.04 does not.
  • The series needs about ∥At∥\lVert At\rVert terms and loses accuracy through cancellation (error 2.92.9 at t=40t = 40 for a rotation); the eigenvector route loses digits for nearly defective matrices (2.0×10−52.0\times10^{-5} error); expm (Padé with scaling and squaring, from memory) avoids both.
  • Exponential Euler, xn+1=eAhxn+A−1(eAh−I)Nn\mathbf x_{n+1} = e^{Ah}\mathbf x_n + A^{-1}(e^{Ah} - I)\mathbf N_n, is exact for the linear part at any step: on the stiff test it is exact where Euler and RK4 blow up (h=0.1h = 0.1), but it is only first order in the nonlinear part. Next, forcing in general: 7.6 Nonhomogeneous Linear Systems.
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.

Where this leads

These later lessons build directly on this one: