Flow Lab

Syllabus / Module 13: Applications: Control Systems

Lesson 13.7

LQR & State Estimation

Optimal control with LQR, Riccati equations, Luenberger observers and an introduction to the Kalman filter.

Written byClaude Sonnet

🌊
Why this matters for simulation: LQR autopilots and Kalman-filtered sensors are standard in UAVs.

By the end of this lesson you can…

  • Write the quadratic cost, say what $Q$ and $R$ penalise, and obtain the gain $K = R^{-1}B^TP$ from the algebraic Riccati equation.
  • Solve the Riccati equation by hand for a scalar plant and numerically for the cruise plant.
  • Explain how control effort grows with the weight ratio, and choose $Q$ and $R$ with Bryson's rule.
  • Derive the return-difference inequality $|1 + L(j\omega)| \ge 1$ and from it the $60^\circ$ phase margin of full-state LQR.
  • Build a Luenberger observer, test observability, and place its poles; derive the separation principle from the error dynamics.
  • Describe the Kalman filter as the LQR dual and explain how sensor noise trades against estimation speed.
On this page
·
  1. The cost: pricing error against effort
  2. The Riccati equation and the gain
  3. Sweeping the ratio: what the optimiser does to the poles
  4. Guaranteed margins and the rule for choosing weights
  5. Observers: states you cannot read
  6. The separation principle
  7. Noise, the Kalman filter and the dual
  8. A Module 12 connection
  9. Concept checks
  10. Playground and exercise
  11. Problems
  12. What Module 13 gave you, and what comes next
  13. Key Takeaways

13.6 State-Space Control & Pole Placement placed poles by decree and found the bill: the cruise command grew like ωn2\omega_n^2, and the aircraft asked for 12.1512.15 rad of rudder. It also assumed every state could be read off a dial. LQR writes the bill into a cost function, so the compromise between error and effort becomes a number we choose. An observer rebuilds the unmeasured states from the one signal we do measure. Together they close Module 13.

▶ Video A cost that charges for error and for effort, the Riccati equation, the poles sliding as the ratio of the weights changes, the return-difference margins, an observer chasing a state no sensor reads, the separation principle, and the Kalman filter's noise trade-offcaptions only
Transcript
  1. LQR writes the bill into a cost. Score the whole trajectory: J is the integral of x transpose Q x plus u transpose R u. Q weighs the state error and R weighs the control. A designer who cares only about speed uses Q equal diag q, 0, and J becomes the integral of q y squared plus r u squared. Doubling both weights changes nothing about which trajectory is best, so only the ratio q over r matters: one knob, where pole placement had two.
  2. Suppose the best cost from x is x transpose P x. Setting the derivative in u to zero gives u equal minus K x, with K equal R inverse B transpose P: linear state feedback. P solves the algebraic Riccati equation. Kleinman's iteration, from memory, starts at K equal 0, solves a linear Lyapunov equation for P, sets K, and repeats: 9 passes for the default design. A scalar plant by hand: a equal minus 1, q equal 3, r equal 1 gives K equal 1 and a pole at minus 2. For the cruise plant with q equal 1 and r equal 0.01, K is 8.4121 and 0.6378, poles minus 0.8689 plus or minus 0.5000 i, N equal 10.0499, with 0.43 percent overshoot, settling in 4.345 seconds.
  3. Sweep the ratio q over r, with r equal 1. At q over r equal 1 the poles barely leave minus 0.1 and minus 1: effort is spent only when it buys something. At 100 they are minus 0.8689 plus or minus 0.5000 i, at 10 to the 4 minus 2.2919 plus or minus 2.1789 i, and at 10 to the 6 minus 7.0889 plus or minus 7.0532 i. The damping settles toward 0.7: 0.7247, then 0.7089, the Butterworth pattern, from memory. The opening command N equals the square root of 1 plus q over r. LQR does not remove the cost of speed; it makes the exchange rate a single dial.
  4. The ratio law comes from an identity: the magnitude of 1 plus L, squared, equals 1 plus q over r times G squared, so it is at least 1 at every frequency. At crossover it equals 2 sin of PM over 2, so the phase margin is at least 60 degrees; the gain margin is one half to infinity, from memory. The lab gives a minimum of 1.0000, phase margin 87.82 degrees at 0.804 radians per second, and 70.42 degrees at 4.113 for q over r equal 10 to the 4. Full-state only: an observer loses it, Doyle's counterexample, from memory. Bryson's rule, from memory: error 1, throttle 10 gives q equal 1, r equal 0.01.
  5. The car has a speed sensor and no engine dial. An observer runs a copy of the plant, drives it with the same input, and corrects it by the output it gets wrong. The error then obeys e dot equals A minus L C times e. Place those poles with the dual of Ackermann, if the observability matrix has full rank: for the cruise plant its determinant is 0.1. A rule of thumb, from memory: 2 to 5 times faster than the controller's 1.0025: at m equal 3, L is 3.1105 and 58.3443, poles minus 2.1052 plus or minus 2.1478 i. Started wrong, the estimate chases the true engine state.
  6. Close the loop on the estimate: u equals minus K x hat plus N r. In x and e coordinates the matrix is block triangular, so the poles of the combined system are those of A minus B K together with those of A minus L C: minus 2.1052 plus or minus 2.1478 i and minus 0.8689 plus or minus 0.5000 i. That is separation. If e starts at 0 the response is the full-state one: 0.43 percent, 4.345 seconds. But separation fixes poles, not the transient. With the estimate off by 1 and 1, the step has 0.92 percent overshoot, settles in 3.495 seconds, peaks at 12.0693 rather than 10.0499, and the error in a peaks at 8.4183.
  7. A faster observer buys little. Sweeping m, peak command goes 4.8577, 8.7364, 12.0693, 18.2700 and 33.1726 for m equal 1, 2, 3, 5 and 10, while settling barely improves beyond 3: 3.495, 3.430, 3.380 seconds. With sensor noise of 0.05 the rms command jitter is 0.0916, 0.0939, 0.1737, 0.3552 and 0.9188. Without noise the jitter at m equal 5 and 10 is only 0.0054 and 0.0066, so the growth is all noise. A slow observer ignores noise but reacts late; a fast one shakes.
  8. The Kalman filter settles the trade-off, and it is the LQR problem in disguise: the same Riccati equation with A transpose and C transpose, and L equal P C transpose over V. At sigma 0.05, L is 1.1394 and 7.6308, poles minus 1.1197 plus or minus 0.8653 i, only 1.415 radians per second against the controller's 1.0025: below the 2 to 5 rule. Jitter 0.0622 and error 0.0191, against 0.1737 and 0.1082 for m equal 3; the price is a gentler start-up, peak error 1.9660 against 8.4183. At sigma 0.005 the poles move to minus 3.2020 plus or minus 3.1221 i, at 0.2 to minus 0.7124 plus or minus 0.0495 i.
  9. Module 13 went from a block diagram to an autopilot's brain, and here it adds gains chosen by optimisation with an observer to feed them. The answer to 13.6's growing effort is one dial: N equals the square root of 1 plus q over r, 10.0499 at 100. It left out nonlinear control, delay-dominated loops and discrete-time design. The next unbuilt lesson in BACKLOG order is the capstone: a six-degree-of-freedom flight simulator with an autopilot. It is not yet written.

The cost: pricing error against effort

Take the cruise plant of 13.6, x˙=Ax+Bu\dot{\mathbf x} = A\mathbf x + Bu with x=(y,a)\mathbf x = (y, a), A=[−0.10.10−1]A = \begin{bmatrix}-0.1 & 0.1\\0 & -1\end{bmatrix}, B=(0,1)B = (0, 1), C=(1,0)C = (1, 0). Score the whole trajectory:

J=∫0∞(xTQx+uTRu)dt.J = \int_0^\infty \left(\mathbf x^TQ\mathbf x + u^TRu\right)dt .

QQ is a symmetric matrix of weights on the state error and RR a positive weight on the control. JJ is small only if the state returns to zero quickly and the input stays modest. A designer who cares only about the speed error uses Q=diag⁡(q,0)Q = \operatorname{diag}(q, 0): a weight qq on yy and none on the engine state aa. Then J=∫(qy2+ru2) dtJ = \int(qy^2 + ru^2)\,dt.

Doubling both weights changes nothing about which trajectory is best, so only the ratio q/rq/r matters for the gain: one knob, where pole placement had two, and the optimisation decides where the poles go.

The Riccati equation and the gain

Suppose the best cost from a start x\mathbf x is V=xTPxV = \mathbf x^TP\mathbf x for a symmetric PP. In a short time dtdt we pay (xTQx+uTRu) dt(\mathbf x^TQ\mathbf x + u^TRu)\,dt and VV changes by V˙ dt\dot V\,dt; optimality says the best uu makes the total zero:

0=min⁡u[xTQx+uTRu+2xTP(Ax+Bu)].0 = \min_u\left[\mathbf x^TQ\mathbf x + u^TRu + 2\mathbf x^TP(A\mathbf x + Bu)\right].

Setting the derivative in uu to zero gives 2Ru+2BTPx=02Ru + 2B^TP\mathbf x = 0, so

u=−Kx,K=R−1BTP.u = -K\mathbf x, \qquad K = R^{-1}B^TP .

The optimal control is linear state feedback, the form of 13.6, with a gain we did not guess. Substituting back, the bracket must vanish for every x\mathbf x, which is the algebraic Riccati equation (ARE):

ATP+PA−PBR−1BTP+Q=0.A^TP + PA - PBR^{-1}B^TP + Q = 0 .

We want its symmetric positive semi-definite solution, which gives a stable A−BKA - BK when (A,B)(A, B) is controllable and the weighted states are visible in JJ (stated without proof).

Solving it numerically. A simple method for a browser is Kleinman’s iteration (Newton’s method on the ARE, from memory). Start from a stabilising gain, here K=0K = 0 since AA is stable. Repeat: solve the linear Lyapunov equation (A−BK)TP+P(A−BK)=−(Q+KTRK)(A - BK)^TP + P(A - BK) = -(Q + K^TRK) for PP, then set K=R−1BTPK = R^{-1}B^TP. It converges quadratically: 99 passes for the lab’s default design, 1818 for q/r=106q/r = 10^6. The lab and exercise implement exactly this.

Worked example 1: a scalar plant by hand. For x˙=ax+u\dot x = ax + u the ARE is 2aP−P2/r+q=02aP - P^2/r + q = 0, so P=r(a+a2+q/r)P = r\left(a + \sqrt{a^2 + q/r}\right) and

K=a+a2+q/r,closed-loop pole a−K=−a2+q/r.K = a + \sqrt{a^2 + q/r}, \qquad \text{closed-loop pole } a - K = -\sqrt{a^2 + q/r}.

With a=−1a = -1, q=3q = 3 and r=1r = 1 we get P=1P = 1, K=1K = 1 and a pole at −2-2. The more we weigh the error, the further left the pole goes; with q/rq/r small, K≈a+∣a∣K\approx a + |a| is near zero for a stable plant, so the optimiser leaves it alone.

Worked example 2: the cruise plant. Take q=1q = 1 and r=0.01r = 0.01 (explained below). The lab’s Riccati solution is P=[1.46190.08410.08410.0064]P = \begin{bmatrix}1.4619 & 0.0841\\0.0841 & 0.0064\end{bmatrix} and

K=[8.4121, 0.6378],poles of A−BK: −0.8689±0.5000i,N=10.0499,K = [8.4121,\ 0.6378], \qquad \text{poles of } A - BK:\ -0.8689 \pm 0.5000i, \qquad N = 10.0499 ,

with N=−1/(C(A−BK)−1B)N = -1/(C(A - BK)^{-1}B) the feedforward gain of 13.6. Against 13.6’s ωn=1\omega_n = 1, ζ=0.7\zeta = 0.7 (K=[8.7,0.3]K = [8.7, 0.3], N=10N = 10) the speed is the same, but LQR picked ζ=0.867\zeta = 0.867. The full-state step has 0.43%0.43\% overshoot and settles in 4.3454.345 s.

◆ Interactive LQR on the cruise plant: slide the weight on speed error (log Q) and on effort (log R); read K, the Riccati solution P, the poles, the peak command and the return-difference margin

Loading simulation…

Sweeping the ratio: what the optimiser does to the poles

The lab’s first plot sweeps q/rq/r (here r=1r = 1, Q=diag⁡(q,0)Q = \operatorname{diag}(q, 0)):

q/rq/rKKclosed-loop polesNN
11[0.3772, 0.0370][0.3772,\ 0.0370]−0.1421, −0.9949-0.1421,\ -0.99491.41421.4142
100100[8.4121, 0.6378][8.4121,\ 0.6378]−0.8689±0.5000i-0.8689 \pm 0.5000i10.049910.0499
10410^4[95.5212, 3.4838][95.5212,\ 3.4838]−2.2919±2.1789i-2.2919 \pm 2.1789i100.005100.005
10610^6[985.9227, 13.0778][985.9227,\ 13.0778]−7.0889±7.0532i-7.0889 \pm 7.0532i1000.00051000.0005

When error is cheap the poles barely leave −0.1-0.1 and −1-1: the controller spends effort only when it buys something. As the ratio grows the poles sweep left and settle into a pair with ζ→0.7\zeta\to 0.7 (0.72470.7247 at 10410^4, 0.70890.7089 at 10610^6; the Butterworth pattern, from memory). And NN, the command when the step arrives, follows a clean law.

Why N=1+q/rN = \sqrt{1 + q/r}. Break the loop at the plant input: L(s)=K(sI−A)−1BL(s) = K(sI - A)^{-1}B. Write PA+ATP=−P(jωI−A)−(−jωI−AT)PPA + A^TP = -P(j\omega I - A) - (-j\omega I - A^T)P in the ARE, multiply on the left by BT(−jωI−AT)−1B^T(-j\omega I - A^T)^{-1} and on the right by (jωI−A)−1B(j\omega I - A)^{-1}B, and use BTP(jωI−A)−1B=rL(jω)B^TP(j\omega I - A)^{-1}B = rL(j\omega). With Q=qCTCQ = qC^TC,

∣1+L(jω)∣2=1+qr ∣G(jω)∣2,G(s)=C(sI−A)−1B.|1 + L(j\omega)|^2 = 1 + \frac{q}{r}\,|G(j\omega)|^2, \qquad G(s) = C(sI - A)^{-1}B .

At ω=0\omega = 0 the cruise plant has G(0)=1G(0) = 1 and N=1+L(0)N = 1 + L(0), so N2=1+q/rN^2 = 1 + q/r; the lab prints N=10.0499=101N = 10.0499 = \sqrt{101} at q/r=100q/r = 100.

Back to 13.6. There the command went as 10ωn210\omega_n^2 and we could only watch it climb. Here each tenfold increase in q/rq/r multiplies the opening command by about 10\sqrt{10}. LQR does not remove the cost of speed; it makes the exchange rate a single dial.

Guaranteed margins and the rule for choosing weights

The identity says ∣1+L(jω)∣≥1|1 + L(j\omega)| \ge 1 at every frequency, whatever q≥0q \ge 0 and r>0r \gt 0: the Nyquist curve never enters the unit disc centred at −1-1. Two consequences, the second from memory.

  • Phase margin. At gain crossover ∣L∣=1|L| = 1, write L=ej(−180∘+PM)L = e^{j(-180^\circ + \mathrm{PM})}. Then ∣1+L∣=2sin⁡(PM/2)≥1|1 + L| = 2\sin(\mathrm{PM}/2) \ge 1, so PM≥60∘\mathrm{PM} \ge 60^\circ.
  • Gain margin. The curve can cross the negative real axis only left of −2-2, so the gain can rise without limit and fall by half: [12,∞)[\tfrac12, \infty).

The lab confirms it: min⁡∣1+L(jω)∣=1.0000\min|1 + L(j\omega)| = 1.0000 at q/r=100q/r = 100 and 10410^4. The phase margin is 87.82∘87.82^\circ at 0.8040.804 rad/s for the default design and 70.42∘70.42^\circ at 4.1134.113 rad/s for q/r=104q/r = 10^4. At q/r=1q/r = 1 there is no crossover, since ∣L(0)∣=0.414|L(0)| = 0.414.

The fine print. The guarantees hold for full-state feedback with the loop opened at the input. They say nothing about delay or sensor noise, and with an observer they are lost (Doyle’s LQG counterexample, from memory).

Choosing QQ and RR. Bryson’s rule (from memory): Qii=1/yi,max⁡2Q_{ii} = 1/y_{i,\max}^2 and R=1/umax⁡2R = 1/u_{\max}^2, the reciprocals of the largest tolerable deviation of each state and of the input. For an acceptable error of 11 and a throttle limit of 1010: q=1q = 1, r=0.01r = 0.01, worked example 2. The opening command N=10.0499N = 10.0499 sits right at the limit; the rule is a start and the sliders decide the rest. By the identity, keeping N≤10N \le 10 needs q/r≤99q/r \le 99.

Observers: states you cannot read

The car has a speed sensor and no engine-state dial: we have y=Cxy = C\mathbf x and want x\mathbf x. Run a copy of the plant inside the controller, drive it with the same input, and correct it by the output it gets wrong. This is the Luenberger observer:

x^˙=Ax^+Bu+L (y−Cx^).\dot{\hat{\mathbf x}} = A\hat{\mathbf x} + Bu + L\,(y - C\hat{\mathbf x}).

Subtract it from x˙=Ax+Bu\dot{\mathbf x} = A\mathbf x + Bu: the input cancels and the error e=x−x^\mathbf e = \mathbf x - \hat{\mathbf x} obeys

e˙=(A−LC) e.\dot{\mathbf e} = (A - LC)\,\mathbf e .

We place the eigenvalues of A−LCA - LC by choosing LL. This is the controller problem for the dual pair ATA^T, CTC^T, so Ackermann’s formula of 13.6 applies and all poles can be placed when the observability matrix [CCA]\begin{bmatrix}C\\CA\end{bmatrix} has full rank. For the cruise plant it is [10−0.10.1]\begin{bmatrix}1 & 0\\-0.1 & 0.1\end{bmatrix} with determinant 0.10.1. With only the engine state measured, C=(0,1)C = (0, 1), the matrix has rank one: aa does not depend on yy, so the speed could never be recovered.

Worked example 3: placing the observer by hand. For L=(l1,l2)L = (l_1, l_2),

A−LC=[−0.1−l10.1−l2−1],det⁡(sI−A+LC)=s2+(1.1+l1)s+(0.1+l1+0.1 l2).A - LC = \begin{bmatrix}-0.1 - l_1 & 0.1\\-l_2 & -1\end{bmatrix}, \qquad \det(sI - A + LC) = s^2 + (1.1 + l_1)s + (0.1 + l_1 + 0.1\,l_2).

The LQR design has natural frequency det⁡(A−BK)=1.0025\sqrt{\det(A - BK)} = 1.0025. A rule of thumb (from memory) puts the observer two to five times faster, so the estimate has converged before the controller needs it. For m=3m = 3: ωo=3.0075\omega_o = 3.0075, ζ=0.7\zeta = 0.7 gives l1=1.4ωo−1.1=3.1105l_1 = 1.4\omega_o - 1.1 = 3.1105 and l2=(ωo2−0.1−l1)/0.1=58.3443l_2 = (\omega_o^2 - 0.1 - l_1)/0.1 = 58.3443, poles −2.1052±2.1478i-2.1052 \pm 2.1478i (the lab’s readout). The large l2l_2 infers the unseen engine state from the slope of the speed signal.

The separation principle

Now close the loop with the estimate, u=−Kx^+Nru = -K\hat{\mathbf x} + Nr. Write x^=x−e\hat{\mathbf x} = \mathbf x - \mathbf e:

x˙=Ax−BK(x−e)+BNr=(A−BK)x+BK e+BNr,e˙=(A−LC)e.\dot{\mathbf x} = A\mathbf x - BK(\mathbf x - \mathbf e) + BNr = (A - BK)\mathbf x + BK\,\mathbf e + BNr, \qquad \dot{\mathbf e} = (A - LC)\mathbf e .

In (x,e)(\mathbf x, \mathbf e) coordinates the matrix is block triangular, [A−BKBK0A−LC]\begin{bmatrix}A - BK & BK\\0 & A - LC\end{bmatrix}, so its eigenvalues are those of the diagonal blocks. The poles of the combined system are those of A−BKA - BK together with those of A−LCA - LC: design the controller as if the state were measured and the observer as if there were no controller. This is the separation principle.

The lab shows it: at m=3m = 3 the 4×44\times4 matrix of (x,x^)(\mathbf x, \hat{\mathbf x}) has eigenvalues −2.1052±2.1478i-2.1052 \pm 2.1478i and −0.8689±0.5000i-0.8689 \pm 0.5000i. If the estimate starts correct (e(0)=0\mathbf e(0) = 0) the output-feedback response is the full-state one exactly, 0.43%0.43\% and 4.3454.345 s.

What separation does not say. It fixes the poles, not the transient. With the initial estimate off by (1,1)(1, 1) (the lab default) the start-up leaks into the command through BKeBK\mathbf e: at m=3m = 3 the step has 0.92%0.92\% overshoot, settles in 3.4953.495 s, peaks at a command of 12.069312.0693 rather than 10.049910.0499, and ∣ea∣\lvert e_a\rvert peaks at 8.41838.4183. Sweeping mm:

mm112233551010
overshoot (%)3.383.381.151.150.920.920.810.810.740.74
2%2\% settling (s)7.2657.2653.5253.5253.4953.4953.4303.4303.3803.380
peak command4.85774.85778.73648.736412.069312.069318.270018.270033.172633.1726
peak ∣ea∣\lvert e_a\rvert2.60192.60194.66884.66888.41838.418316.934616.934639.404739.4047

Beyond m≈3m \approx 3 the settling barely improves while the command and the transient grow fast.

◆ Interactive Observer with an initial estimation error: pick the observer speed m, watch the estimate (violet) chase the true speed and the unmeasured engine state, then compare the output-feedback loop with full-state LQR

Loading simulation…

Noise, the Kalman filter and the dual

A real sensor adds noise vv to yy, and a fast observer, with its large LL, passes it into the estimate and the command. With the lab’s seeded noise (σ=0.05\sigma = 0.05, held 0.020.02 s at a time, seed 1234512345) the figures after 55 s are:

mm112233551010
rms of u−ufull-stateu - u_{\text{full-state}}0.09160.09160.09390.09390.17370.17370.35520.35520.91880.9188
rms estimation error0.03840.03840.04830.04830.10820.10820.27290.27290.84470.8447

Noise-free, the jitter at m=5m = 5 and 1010 is only 0.00540.0054 and 0.00660.0066, so the growth is all noise. A slow observer ignores noise but reacts late; a fast one shakes. The right LL depends on how noisy the sensor is against how much the plant surprises us.

The Kalman filter settles that trade-off, and it is the LQR problem in disguise. Model unmodelled disturbance as white noise (here unit intensity on the input channel, W=BBTW = BB^T, a choice) and sensor noise as white with variance V=σ2V = \sigma^2. The steady-state gain is L=PCT/VL = PC^T/V with PP from the same Riccati equation with ATA^T and CTC^T in place of AA and BB: the noise levels play the roles of QQ and RR, and the observer is the dual of the controller. A quiet sensor makes the filter trust the measurement and the gain large; a noisy one makes it trust the model.

At σ=0.05\sigma = 0.05 the lab gives L=[1.1394, 7.6308]L = [1.1394,\ 7.6308] and observer poles −1.1197±0.8653i-1.1197 \pm 0.8653i, only 1.4151.415 rad/s against the controller’s 1.00251.0025, below the two-to-five rule (a rule of thumb, not a law). The noisy run has command jitter 0.06220.0622 and estimation error 0.01910.0191, beating the m=3m = 3 placement’s 0.17370.1737 and 0.10820.1082; the price, in the noise-free run, is a gentler start-up (peak ∣ea∣=1.9660\lvert e_a\rvert = 1.9660 against 8.41838.4183) and slower 4.1154.115 s settling. The filter re-tunes with the sensor: σ=0.005\sigma = 0.005 gives poles −3.2020±3.1221i-3.2020 \pm 3.1221i and σ=0.2\sigma = 0.2 gives −0.7124±0.0495i-0.7124 \pm 0.0495i.

Sensor fusion for attitude is the same idea in one dimension. A gyro gives the rate cleanly but its integral drifts; an accelerometer gives the angle without drift but noisily. The observer θ^˙=θ˙gyro+L(θacc−θ^)\dot{\hat\theta} = \dot\theta_{\text{gyro}} + L(\theta_{\text{acc}} - \hat\theta) trusts the gyro at high frequency and the accelerometer at low (a complementary filter), and the Kalman filter picks LL from the noise levels, as in the attitude equations of 12.4 6-DOF Aircraft Equations of Motion.

A Module 12 connection

13.6 moved the Dutch roll of 12.6 Lateral-Directional Modes and found that dragging every pole needed 12.1512.15 rad of rudder per unit of yaw-rate reference: in LQR terms, an rr far too small. Weights set from the rudder travel (Bryson) give poles the rudder can reach. The sensor gap is the other half: a rate gyro reads pp and rr but the sideslip velocity vv is hard to measure, so an observer on the lateral model estimates it.

Concept checks

✅ Concept check

In LQR with cost $\int(\mathbf x^TQ\mathbf x + u^TRu)dt$ and ARE solution $P$, the optimal control is:

✅ Concept check

For $\dot x = u$ (so $a = 0$) with cost $\int(4x^2 + u^2)dt$, what is the LQR gain $K$ in $u = -Kx$?

✅ Concept check

For the cruise plant with $Q = \operatorname{diag}(q, 0)$, the opening command of a unit step is $N = \sqrt{1 + q/r}$. For $q/r = 399$, what is $N$?

✅ Concept checkselect all that apply

Select every statement that is true.

Playground and exercise

The first cell solves the Riccati equation by Kleinman’s iteration; the second builds observers and checks the combined eigenvalues; the exercise has you write the LQR and Kalman gains.

🐍 PythonPlayground: Riccati by Kleinman iteration, a sweep of q/r on the cruise plant
⌘/Ctrl + Enter
🐍 PythonPlayground: observers of three speeds and the combined eigenvalues
⌘/Ctrl + Enter
🧪 Coding exerciseExercise: the LQR gain and the Kalman gain
⌘/Ctrl + Enter

Problems

✏️ Pen & paperScalar LQR by hand●○○

For x˙=x+u\dot x = x + u (an unstable plant, a=1a = 1) with cost ∫(3x2+u2)dt\int(3x^2 + u^2)dt, find PP, KK and the closed-loop pole.

Show worked solution

With a=1a = 1, q=3q = 3, r=1r = 1 the ARE is 2P−P2+3=02P - P^2 + 3 = 0, that is P2−2P−3=0P^2 - 2P - 3 = 0, with roots 33 and −1-1. We want the positive root, P=3P = 3. Then K=P/r=3K = P/r = 3 and the pole is a−K=−2a - K = -2. Check with the formula: K=a+a2+q/r=1+2=3K = a + \sqrt{a^2 + q/r} = 1 + 2 = 3 and the pole is −a2+q/r=−2-\sqrt{a^2 + q/r} = -2.

✏️ Pen & paperObserver poles for the cruise plant●●○

Find L=(l1,l2)L = (l_1, l_2) that puts both poles of A−LCA - LC at −3-3 for the cruise plant.

Show worked solution

The characteristic polynomial is s2+(1.1+l1)s+(0.1+l1+0.1 l2)s^2 + (1.1 + l_1)s + (0.1 + l_1 + 0.1\,l_2), to be matched with (s+3)2=s2+6s+9(s + 3)^2 = s^2 + 6s + 9. Then l1=4.9l_1 = 4.9 and 0.1+4.9+0.1 l2=90.1 + 4.9 + 0.1\,l_2 = 9, so l2=40l_2 = 40. Check: A−LC=[−50.1−40−1]A - LC = \begin{bmatrix}-5 & 0.1\\-40 & -1\end{bmatrix} has trace −6-6 and determinant 5+4=95 + 4 = 9. The gain l2=40l_2 = 40 multiplies the speed error and feeds the engine state, which is only seen through the slope of yy: the observer converts a speed discrepancy into a correction to aa with a large factor.

✏️ Pen & paperCommand limit and the weight ratio●●○

For the cruise plant the opening command of a unit step is N=1+q/rN = \sqrt{1 + q/r} and the throttle accepts at most 1010. Find the largest q/rq/r that fits, then test Bryson’s rule with an acceptable speed error of 0.50.5 and umax⁡=10u_{\max} = 10.

Show worked solution

1+q/r≤10\sqrt{1 + q/r} \le 10 gives q/r≤99q/r \le 99. Bryson gives q=1/0.52=4q = 1/0.5^2 = 4, r=0.01r = 0.01, so q/r=400q/r = 400 and N=401=20.02N = \sqrt{401} = 20.02: twice the throttle available. Accept a larger error, or slow the reference. The rule is a start; the identity says which way to move.

What Module 13 gave you, and what comes next

Module 13 went from a block diagram to an autopilot’s brain: the loop and its time response (13.1, 13.2), PID and its traps (13.3), stability by roots and by frequency (13.4, 13.5), full-state pole placement (13.6) and, here, gains chosen by optimisation and an observer to feed them. It left out nonlinear control, delay-dominated loops and discrete-time design.

The next lesson in BACKLOG order is the capstone on a six-degree-of-freedom flight simulator with an autopilot (16.2 Capstone: 6-DOF Flight Simulator with Autopilot).

Key Takeaways

  • LQR minimises ∫(xTQx+uTRu) dt\int(\mathbf x^TQ\mathbf x + u^TRu)\,dt with u=−Kxu = -K\mathbf x, K=R−1BTPK = R^{-1}B^TP and PP from the Riccati equation; Kleinman’s iteration solves it with a Lyapunov solve per pass.
  • Only q/rq/r matters. On the cruise plant, q/r=100q/r = 100 gives K=[8.4121,0.6378]K = [8.4121, 0.6378] and poles −0.8689±0.5000i-0.8689 \pm 0.5000i; 10410^4 gives −2.2919±2.1789i-2.2919 \pm 2.1789i.
  • The opening step command is 1+q/r\sqrt{1 + q/r} (10.049910.0499 at 100100): the answer to 13.6’s growing effort is a single dial.
  • ∣1+L(jω)∣≥1|1 + L(j\omega)| \ge 1 gives a phase margin of at least 60∘60^\circ for full-state LQR (87.82∘87.82^\circ, 70.42∘70.42^\circ in the lab); not guaranteed once an observer is added.
  • The observer x^˙=Ax^+Bu+L(y−Cx^)\dot{\hat{\mathbf x}} = A\hat{\mathbf x} + Bu + L(y - C\hat{\mathbf x}) has error dynamics A−LCA - LC and needs a full-rank observability matrix.
  • Separation: the combined poles are those of A−BKA - BK and A−LCA - LC, but the start-up leaks into the command (peak 12.069312.0693 against 10.049910.0499).
  • Faster observers amplify noise (0.17370.1737 at m=3m = 3, 0.91880.9188 at m=10m = 10); the Kalman filter, the dual LQR problem, sets the speed from the noise levels (L=[1.1394,7.6308]L = [1.1394, 7.6308] at σ=0.05\sigma = 0.05).
Finished this lesson?

Mark it complete to track your progress (saved in this browser). Reading every section does this for you.

Where this leads

These later lessons build directly on this one: