Left Half-Plane
system identification · experiment design · prediction error

Identify

The cart-pole from Stable is Still, but today's motor answers late, the cart drags on its rail, and the motor's gain is off, none of it in the model. Before the scored run the pendulum hangs, and three pokes are free: a pulse of force on the cart, the cart's velocity recorded. Fit a model to the record, by hand or by least squares, and the balancing controller is built on your numbers. It stands if they were right.

world —runs today 0best today —the true numbers —
Where you stand today

A poke is a pulse from the motor while the pendulum hangs: choose how hard and how long. The record is the cart's velocity through a noisy encoder for a second and a half, drawn on the right; the blue curve is what your three numbers predict for the same pulse, and fit is the root-mean-square gap between them, in millimetres per second, over every poke so far. Least squares searches the same three numbers for the smallest gap. The run is Stable is Still's: thirty seconds of growing gusts, the same cost, the same gains, except that the controller acts on a prediction: the measured state carried forward through the commands in flight, over your delay, through your friction and gain. Get the delay wrong by more than a few hundredths and it falls at the first gust; get it about right and the delay is gone. Hidden today: all three numbers, and which pulse would have shown them best.

The trick

Ask the plant a question it has to answer

Every controller in this series is designed on a model, and every plant hides something the model omits; identification is the step in between, and it has two halves. One is the fit: a family of models, a record, and the model whose prediction is least wrong, which is what your three sliders and the least-squares button both do. The other half is quieter and decides more: the experiment. A pulse that is too short gives the friction nothing to act on, and the drag stays anyone's guess; one that is too long runs the cart to the end of the track. The delay shows in the moment the velocity starts to rise, so a sharp edge tells you more than a gentle one, and noise argues for a harder push. Three pokes, all the same, learn less than three that differ. Then the design: Time Machine's predictor, run on the numbers you found. If it stands, the numbers were right enough; the score is the cost of the run, and the honest reading is how much of it your fit cost you.

Cards
The best model is the one whose prediction is least wrong on the data you have.prediction-error
θ̂ = arg minθ Σt (y(t) − ŷ(t | θ))², with ŷ the model's one-step or simulated predictionLjung: the prediction-error method; least squares when the prediction is linear in θ, a search otherwise, as here

Write the model with unknown numbers, predict the record with it, and pick the numbers that leave the smallest residual. That is nearly the whole of parametric identification; the rest is what counts as a prediction (one step ahead from the measured past, or a simulation from the start, which is what this page compares), how the residual is weighted, and what the noise does to the answer. The residual is also the check: a model that fits the record with nothing but noise left over has learned what the record had to teach.

Here the three numbers do three visibly different things to the curve: the delay shifts it, the gain scales its rise, the friction bends its tail. That separability is luck, and it is what makes a fit by hand possible; with ten numbers acting on the same features the search has to be done by the algorithm, and the experiment has to be designed so that the numbers can be told apart at all.

Where this game's fit and its button · Time Machine's residual, the same idea live · the Kalman filter, the same residual as an innovation
Deeper Ljung, System Identification: Theory for the User, ch. 7 · Söderström & Stoica, System Identification, ch. 7

What you can learn is set before the fit, by the input you chose.experiment-design
cov(θ̂) ≈ (Σ ψ ψᵀ / λ)⁻¹, ψ = ∂ŷ/∂θthe Cramér–Rao bound: a parameter is only as sharp as the sensitivity of the prediction to it, summed over the record

A record is informative about a number only where changing that number would change the record. The delay is visible at the edges of the pulse and nowhere else; the friction only while the cart is moving, and better the longer it moves; the gain in the size of the response, which is where the noise bites. So the sharpness of each estimate is set by the input: its edges, its length, its amplitude, and how many different ones you tried. Persistent excitation is the name for an input rich enough to pin every number; three identical pokes are the opposite.

In practice the experiment is often the expensive part: a plant taken off its job to be poked, a machine that must not be driven to its limits, a patient. The theory of optimal experiment design chooses the input that makes the covariance smallest for the least disturbance; the craft is knowing which number you actually need, and asking for that one.

Where this game's two poke knobs · Recipe, where the range to randomise is the same question in another form
Deeper Goodwin & Payne, Dynamic System Identification: Experiment Design and Data Analysis, 1977 · Ljung, ch. 13 · Gevers, Identification for control, 2005

Identify while you control and you are adaptive; learn too fast and the loop learns you.adaptive
θ̂(t + 1) = θ̂(t) + γ ψ(t) (y(t) − ŷ(t)), u(t) = design(θ̂(t))the certainty-equivalence loop: estimate, design as if the estimate were true, repeat; the gain γ is where it goes wrong

This page does its identification first and its control after, which is the safe order. Adaptive control closes the two into one loop: the estimate updates from the running record while the controller redesigns itself on the current estimate. It works, and it flies aircraft, when the estimate moves slowly against the plant. When it moves fast the estimator and the controller chase each other, the input stops being informative because the controller is doing its job, and the whole thing can burst: the 1967 X-15 crash is the field's founding accident and its founding lesson.

The theory that followed, Lyapunov-based adaptive laws with bounded estimates and slow enough gains, is one of the field's hardest and least visible. In this series it stays a card, but its question is everywhere: Time Machine reads the delay from the residual, Echo learns a disturbance lap by lap, and both are adaptive in the loose sense, with the same danger in the same knob.

Where a card · Echo, whose gain is this γ · Time Machine
Deeper Åström & Wittenmark, Adaptive Control · Ioannou & Sun, Robust Adaptive Control · Dydek, Annaswamy & Lavretsky, Adaptive control and the NASA X-15-3 flight revisited, 2010

References
  1. L. Ljung, System Identification: Theory for the User, 2nd ed. Prentice Hall, 1999.
  2. T. Söderström and P. Stoica, System Identification. Prentice Hall, 1989.
  3. G. C. Goodwin and R. L. Payne, Dynamic System Identification: Experiment Design and Data Analysis. Academic Press, 1977.
  4. K. J. Åström and P. Eykhoff, "System identification — a survey," Automatica, vol. 7, no. 2, pp. 123–162, 1971.
  5. M. Gevers, "Identification for control: From the early achievements to the revival of experiment design," European Journal of Control, vol. 11, nos. 4–5, pp. 335–352, 2005.
  6. K. J. Åström and B. Wittenmark, Adaptive Control, 2nd ed. Addison-Wesley, 1995; Dover reprint, 2008.
  7. P. A. Ioannou and J. Sun, Robust Adaptive Control. Prentice Hall, 1996.
  8. Z. T. Dydek, A. M. Annaswamy, and E. Lavretsky, "Adaptive control and the NASA X-15-3 flight revisited," IEEE Control Systems Magazine, vol. 30, no. 3, pp. 32–48, 2010.