The machine from the front door, kicked at the tip every six seconds, harder each time, until the one kick it cannot take. You choose how stiff it stands and how much it minds where the cart is. Today's track tilts by an angle nobody told the controller, and the motor answers late. Survive the kicks, cheaply, and stay where you were put.
The controller is the lab's, u = −Kx, an LQR whose Q is the lab's scaled by the stiffness knob, with the cart's position and velocity weights also multiplied by the position knob; R stays. Kicks land on the tip as angular-velocity impulses, ten rungs from 0.4 to 1.9 rad/s, one every six seconds. A kick is survived if the machine still stands when the next is due; a kick due while it is down is lost. After a fall it recovers on its own, calm, swing-up, catch, which costs a kick and the swing-up's energy. The learned policy from the front door runs the same minute as a reference row; the knobs do nothing to it. Cost is 10 per kick lost + 5 ∫(u/umax)² dt + ½ ∫x² dt while standing. Hidden today: the actuator delay, and the tilt of the track, a constant force on the cart the controller never sees. Without integral action it stands against that force off-centre, by F/kx, and the wandering term charges for it. Par is the lab as shipped, 1× and 1×, in today's world; it was tuned for a level track and no delay.
The intuition says a bigger kick needs a stiffer controller. Try it. Under a torque limit the stiffest designs saturate first and fall to the smallest kicks; the delay the controller never sees eats their phase margin; and the basin, how big a kick the machine survives at all, is drawn by the 30 N motor far more than by the gains. Softness has its own bill: a soft controller survives by wandering, and today's tilted track makes it stand off-centre by F/kx, which the score charges for. The design is the trade between the two, and it moves with the day's slope and delay. The lab's own tuning was made for a level track with no delay, and in today's world it is beatable by a lot.
A linear analysis says the upright equilibrium is stable and says nothing about how far you may push it. The states that return form the region of attraction, and for a real machine its edge is drawn by the actuator: a 30 N motor accelerates this cart at about 21 m/s², and a tip kick that needs more than that is lost whatever the gains say. Here the wall sits near 1.5 rad/s at zero delay, near 1.0 with the motor halved, and it moves inward with every millisecond of delay.
Estimating it: a Lyapunov sublevel set is a certificate but a conservative one; the lab instead simulates two seconds ahead under the gains, which is a test rather than a proof; and sampling, as the slice above does, is the empirical route. Under saturation and delay, more gain usually shrinks the basin. The intuition that stiffness buys robustness is the one this game exists to break.
Direct collocation turns a trajectory into one large sparse nonlinear program, and a solver (IPOPT through CasADi, offline, in games/tools/) finds the cheapest path the motor can actually deliver: 2.5 s, 18.5 N peak, 1.15 m of cart travel. The path is stored in sim.js as a table; the browser only tracks it. Nothing improvised gets two links from hanging to upright on 30 N.
An open-loop plan dies at the first unmodelled thing, which is why it needs a tracker, and why a kick during the swing-up sends the machine back to calm rather than onward. Planning and feedback are not rivals. One finds a path the other could never invent; the other keeps the first honest.
Linearise the dynamics around the plan at every step, and the same Riccati recursion that gives the balancing gains gives a schedule of gains along the path. The controller is the plan plus a correction; the lab computes the schedule in your browser when it loads, by finite differences of one simulation step.
Its basin is a tube around the plan. Leave the tube and the local model is wrong, which is what a kick mid swing-up does, and why the recovery restarts from calm instead of fighting on. Stitching such tubes together until they cover the whole space is the idea behind LQR-trees.
Switch the controller off on the front door and the machine is a double pendulum, one of the simplest mechanical systems that is chaotic. The equations are deterministic, this browser integrates them the same way every time, and two starts a hair apart still diverge within seconds. Sensitive dependence is not randomness; it is a fact about the flow.
It is why the calm stage never tries to predict where the links will be: prediction is hopeless and unnecessary. Draining energy through the cart needs only the energy, a smooth function of the state that chaos cannot hide. Chaos is what the controller has to bring back, and it is brought back by asking for less.
The lab's Learned controller is a policy: a small network trained offline by reinforcement learning on the same equations this page integrates (games/tools/, PyTorch on four CPU cores, an hour). Reward: both links up and calm, the cart near the middle and not racing, a little effort, a small charge for changing its mind between decisions, and a little for mechanical energy while it is down. It never saw a model, a Riccati equation or the word pole; it saw states and rewards, tens of millions of them across two thousand simulations running side by side, and adjusted six thousand numbers until the reward rose. Then the numbers were written into a file and the browser runs them, deterministically, like any other controller here. It does every stage: from hanging it swings itself up in four or five seconds by a route nobody planned, and it holds. Its input carries a short memory, its own last six commands and the velocities at its last three decisions: from how the machine answered what it was told, a delay or a tilt can be read, which a policy that sees only the present cannot do, and cannot be trained to do.
Where it is strong: at zero delay it takes tip kicks to 1.6 rad/s where the LQR stops at 1.3, because it was trained on them; it was trained across 0 to 50 ms of actuator delay, so at 20 ms it takes 1.4 where the linear rule, designed for no delay, drops to 0.7, at 40 ms it falls to most kicks but is back up within seconds on ten of twelve, and at 50 ms it still swings up and stands every time, where the linear rule is done at 40; and it needs no hand-over, no catch region and no planned path. The memory cost a rung: its memoryless predecessor took 1.72 at zero delay and 1.6 at 20 ms, and could not stand at 50. Where it is weak: it has no proof of anything, its precision at rest is worse than the LQR's, it spends more, and outside the states it was trained on it fails without warning where a model-based controller degrades. It cannot say which stage it is in, because it has none. The linear rule is one function of six numbers, and can be read; the policy is one function of twenty-three, and cannot.
A simulation is a model, and a model lies; a policy that memorises one simulation inherits its lies. Domain randomisation draws the doubtful parameters afresh for every episode, so the only policies that score well are the ones that work across the whole range. It is the reinforcement-learning answer to robust control, bought with data instead of with a norm bound, and it is how most learned controllers reach real machines.
The bill: a policy trained on a wide range is usually less precise and more expensive on any one plant than a policy trained for that plant alone, and a parameter outside the range is as unmodelled as ever. Measured on Kick's ladder: the lab's policy, which saw 0 to 50 ms of delay in training and carries a short memory of its own commands, takes tip kicks to 1.6 rad/s at zero latency, 1.4 at 20 ms and 0.7 at 40, where it falls to the rest and gets back up, and it swings up and stands at 50; its memoryless predecessor, trained to 40 ms, took 1.72, 1.6 and 1.3 at 30, and could not stand at 50; a balance-only twin trained on the model alone took 1.7 at zero latency, 1.85 at 10 ms, and fell to every kick at 20. The LQR, designed for no delay, takes 1.3 and 0.7. That is the whole trade in three numbers each. Recipe makes the choice of range the game.