Core concept

A controller tuned in simulation meets hardware whose timing, actuators and sensors differ from the model. The differences fall into a few classes, and each has a different remedy:

For a feedback loop, the unmodeled dynamics, actuation, sensing and timing rows act through one quantity: the phase the real loop loses relative to the model at the frequency where the loop gain crosses 1. This lesson measures that loss for the Level 8 pendulum controller, predicts its consequence from one number, the delay margin, and turns a failed transfer into a measurement of what the model got wrong.

[!established] The delay margin A loop $L(j\omega)$ crosses unit gain at $\omega_c$ with phase margin $\varphi_m = \pi + \arg L(j\omega_c)$. A pure delay $d$ multiplies the loop by $e^{-j\omega d}$: it removes phase $\omega d$ and leaves the gain unchanged. So the loop tolerates extra delay up to $d^\ast = \varphi_m / \omega_c$, provided the gain crosses 1 once. A controller that holds its output over a period $T$ adds about $T/2$ of delay. MuJoCo 3.14.0 models actuator delay directly: delay (s), a history of nsample past commands, and interp ("linear" here). The history must cover the delay, since nsample caps it without warning (Lesson 18.2), so the script keeps 200 samples (400 ms).

Visual intuition

The lab holds the pendulum at 0.05 rad with the Level 8 controller. Its readouts show the margin predicted from the loop and, on request, the margin found by bisection.

1. With kp 80 and damping ratio 0.7, the predicted margin is about 41 ms. Set the actuator delay to 35 ms: the step rings but settles. Set it to 45 ms: the oscillation grows until the error passes 0.5 rad and the lab stops it.
2. Back at 0 ms, raise kp to 200: the margin falls to about 25 ms. Stiffer gains raise the crossover frequency, and the same delay costs more phase there.
3. Switch sensing to encoder, raw velocity and watch the torque chatter. Filter the velocity at 10 Hz: the chatter drops and so does the margin.
4. Add a 20 ms motor lag, or run the controller at 100 Hz, and watch the prediction and the measurement move together.

```lab sim2real {“dock”: true, “title”: “Delay margin, live”}

## Mathematics

### (1) Predicting the margin

The controller is $u = k_p(q^\ast - q) - k_d\dot q + mgL\sin q$ on a pendulum with $I$ = 0.2501 kg m², holding $q^\ast$ = 0.05 rad. If gravity compensation were exact, the plant would be $I s^2$ and the loop $(k_p + k_d s)e^{-sd}/(Is^2)$. But the compensation passes through the same delay as the rest of the command, so the linearized plant keeps its gravity term and the loop is $(k_p - mgL + k_d s)\,e^{-sd}/(Is^2 + mgL)$. Predicted margins against the extra delay at which the simulated loop stops settling (bisection to 0.1 ms):

| $k_p$ | $\zeta$ | Predicted, plant $Is^2$ | Predicted, gravity in the loop | Measured | Frequency at the margin |
|---|---|---|---|---|---|
| 20 | 1.0 | 71.4 ms at 18.41 rad/s | 71.8 ms at 19.18 rad/s | 71.4 ms | 19.29 rad/s |
| 20 | 0.5 | 78.5 ms at 11.38 rad/s | 87.5 ms at 11.92 rad/s | 86.7 ms | 11.99 rad/s |
| 80 | 0.7 | 40.2 ms at 27.59 rad/s | 40.7 ms at 27.95 rad/s | 40.5 ms | 28.23 rad/s |
| 200 | 0.7 | 25.1 ms at 43.63 rad/s | 25.2 ms at 43.85 rad/s | 25.1 ms | 44.49 rad/s |

The full loop predicts every margin to within 0.8 ms and every oscillation frequency to within 1.5%. The simple model misses the soft, lightly damped controller by 8 ms because there the gravity term ($mgL$ = 4.9 N m/rad) is a large part of the stiffness. The prediction failed by an amount that pointed at the missing term. Stiffer gains raise the crossover, so the same delay costs more phase: kp 200 tolerates a third of what kp 20 does. Past the margin the loop does not always diverge. It can settle into a sustained oscillation, which is why the script's test is whether the loop settles, not whether it grows.

### (2) Actuator and controller mismatch

At kp 80 and $\zeta$ 0.7 (margin 40.5 ms):

| Mismatch | Predicted margin | Measured | Frequency |
|---|---|---|---|
| motor with a 20 ms first-order lag | 24.9 ms | 24.4 ms | 25.84 rad/s |
| motor giving 80% of the modelled torque | 45.9 ms | 45.7 ms | 23.58 rad/s |
| controller at 100 Hz, output held | 36.7 ms | 36.6 ms | 28.13 rad/s |

A 20 ms lag costs 16 ms of margin: well below $1/\tau$ a lag removes phase almost as a delay of $\tau$ does, and here $\omega\tau \approx 0.5$. A weaker motor lowers the crossover and gains margin, but leaves a static error under gravity, because the compensation is now 20% short: 0.75 mrad at 0.05 rad, as predicted. A 100 Hz controller holds each command for 10 ms and loses about 4 ms: half its period, less the half step the 500 Hz loop already had.

### (3) Sensing: chatter against margin

With a 4096-count encoder (1.534 mrad per count) and velocity differenced at 500 Hz, one count of change reads as 0.77 rad/s, and kd turns it into torque:

| Velocity estimate | Torque chatter (std) | Peak to peak | Margin predicted | Measured |
|---|---|---|---|---|
| true angle and velocity | 0.000 N m | 0.000 N m | 40.7 ms | 40.5 ms |
| encoder, unfiltered | 4.345 N m | 9.721 N m | 39.6 ms | 40.4 ms |
| filtered at 100 Hz | 2.546 N m | 6.653 N m | 38.8 ms | 39.5 ms |
| filtered at 30 Hz | 0.909 N m | 2.499 N m | 35.1 ms | 35.9 ms |
| filtered at 10 Hz | 0.349 N m | 0.858 N m | 26.4 ms | 27.2 ms |

The unfiltered loop is as stable as the ideal one, and on hardware it would buzz: a 9.7 N m swing on a 3 N m motor. Every filter that quiets it costs margin, because a low-pass filter is a frequency-dependent delay. With the encoder the measured margins sit 0.7 to 0.8 ms above the linear prediction; quantization is not linear, and the model omits it.

### (4) A transfer experiment that fails informatively

The "real" pendulum has a delay, a motor lag, a torque gain and a mass that the model does not know; it reads a 4096-count encoder filtered at 30 Hz. The kp 200 controller, whose simulated margin is 25.2 ms, does not settle on it. That failure says only that the real loop lost more phase than its margin allowed, not which mismatch took it.

A better experiment uses two controllers with large predicted margins (kp 10 and 40, $\zeta$ 1), adds software delay until each stops settling, and records both the extra delay and the frequency of the oscillation. At the margin the real loop has gain 1 and phase $-\pi$ at that frequency, so each probe measures the unmodeled factor $G(j\omega) = -1/\big(L_\text{model}(j\omega)\,e^{-j\omega d_x}\big)$ at one frequency:

| Probe | Model margin | Real margin | Frequency | $\lvert G\rvert$ | $\arg G$ |
|---|---|---|---|---|---|
| kp 10 | 96.5 ms | 79.7 ms | 12.20 rad/s | 0.825 | −22.4° |
| kp 40 | 44.5 ms | 21.1 ms | 22.51 rad/s | 0.814 | −41.2° |

Fitting $G = k\,e^{-j\omega d}/(1 + j\omega\tau)$ to the two points gives $k$ = 0.830, $\tau$ = 8.8 ms, $d$ = 23.2 ms. The hidden values are a 25 ms delay and an 8 ms lag, with a torque gain of 0.9 on a 1.1 kg pendulum, which a loop sees as about 0.82 (inference: gain over mass ratio). The split between lag and delay is approximate, because at these frequencies $\omega\tau \le 0.2$ and a lag is nearly a delay. The total phase they remove, which is what the next controller depends on, is measured directly. With the fitted factor, the stiffest controller among kp 200, 120, 80 and 40 that keeps a 20 ms margin is kp 40, predicted at 27.9 ms. On the real pendulum its margin is 27.8 ms.

> [!recommendation] Design transfer experiments to measure, not just to pass
> Before deploying, predict a number the experiment can contradict: a delay margin, an oscillation frequency, a static error. Start with controllers that have margin to spare, inject known delay in software until the loop stops settling, and record the frequency. Each such run measures the real loop's gain and phase at one frequency, so two or three runs at different gains show how the mismatch depends on frequency. Then redesign with the measured loss and keep margin for what you did not measure.

## Implementation

```io
INPUT: pendulum.xml under PD with gravity compensation; actuator delay, motor lag (filterexact), gear and mass set per plant; an encoder model
PROCESS: (1) delay margins predicted from the loop and measured by bisection; (2) lag, torque gain, control rate; (3) encoder and velocity filters; (4) probes on a hidden plant, a fit of its gain, lag and delay, and a redesign
OUTPUT: printed tables

```python file=examples/l19_1_sim_to_real.py “"”Lesson 19.1: what latency, actuator mismatch and sensing do to a controller tuned in simulation (Level 19).

INPUT pendulum.xml held at 0.05 rad by the Level 8 controller: PD on the angle with gravity compensation, u = kp (q* - q) - kd qdot + m g L sin q; MuJoCo’s actuator delay (a 200-sample history, linear interpolation), first-order actuator dynamics and gear model the hardware PROCESS (1) the delay margin, the extra actuator delay at which the loop starts to oscillate with growing amplitude, predicted from the loop’s phase margin and crossover, then found by bisection in simulation, for four sets of gains; first with the plant I s^2, then with the gravity term that also passes through the delay; (2) actuator mismatch: a 20 ms motor lag, 80% of the modelled torque, and a 100 Hz controller; (3) sensing: a 4096-count encoder, velocity by differencing, and a low-pass filter on it: torque chatter against delay margin; (4) a transfer experiment on a “real” pendulum whose delay, motor lag, torque gain and mass are hidden: the stiff controller tuned in simulation, then software delay added until the loop oscillates, at two sets of gains, to measure the hidden phase loss and redesign OUTPUT printed tables

Run: python examples/l19_1_sim_to_real.py (about 2 minutes on 24 cores) “””

import cmath import math import os from multiprocessing import Pool

import mujoco import numpy as np

from mjcourse import model_path

FAST = os.environ.get(“MJC_FAST”) == “1” H, I_PIVOT, MGL, TARGET = 0.002, 0.2501, 1.0 * 9.81 * 0.5, 0.05 NOMINAL = {“delay”: 0.0, “lag”: 0.0, “gain”: 1.0, “mass”: 1.0} HIDDEN = {“delay”: 0.025, “lag”: 0.008, “gain”: 0.9, “mass”: 1.1} # the “real” pendulum COUNTS = 4096 # encoder resolution per turn

def pendulum(plant: dict, extra_delay: float = 0.0) -> mujoco.MjModel: “"”The pendulum with the plant’s delay (+ extra), first-order motor lag, torque gain and mass.””” spec = mujoco.MjSpec.from_file(str(model_path(“pendulum”))) spec.body(“pole”).mass = plant[“mass”] act = spec.actuator(“torque”) act.nsample, act.interp, act.delay = 200, 1, plant[“delay”] + extra_delay # 200 samples hold 400 ms act.ctrlrange = [-100.0, 100.0] # stay linear: no saturation act.gear = [plant[“gain”], 0, 0, 0, 0, 0] if plant[“lag”]: act.dyntype = mujoco.mjtDyn.mjDYN_FILTER act.dynprm = [plant[“lag”]] + [0.0] * 9 return spec.compile()

def run(job) -> np.ndarray: “"”Hold TARGET for seconds; returns the angle error per step, inf from the step where the loop left the linear range or MuJoCo reset a diverged state. job = (kp, kd, plant, extra delay, sensing, seconds) with sensing = None (true qvel) or (period in steps, filter cutoff in Hz or 0, quantize: bool).””” kp, kd, plant, extra, sensing, seconds = job model = pendulum(plant, extra) data = mujoco.MjData(model) period, cutoff, quantize = sensing or (1, 0.0, False) alpha = 1.0 if not cutoff else 1.0 - math.exp(-2 * math.pi * cutoff * H) # first-order low-pass n = round(seconds / H) err, ctrl = np.empty(n), np.empty(n) q_prev, v, u = 0.0, 0.0, 0.0 for k in range(n): q = data.qpos[0] if quantize: q = round(q * COUNTS / (2 * math.pi)) * 2 * math.pi / COUNTS v += alpha * ((q - q_prev) / H - v) q_prev = q else: v = data.qvel[0] if k % period == 0: u = kp * (TARGET - q) - kd * v + MGL * math.sin(q) # compensation for the nominal mass data.ctrl[0] = u mujoco.mj_step(model, data) err[k], ctrl[k] = TARGET - data.qpos[0], u if abs(err[k]) > 0.5 or data.time < (k + 1) * H - 1e-9: err[k:] = np.inf break return np.vstack([err, ctrl])

def growing(err: np.ndarray) -> bool: “"”The loop does not settle: it diverged, the spread of its error about the mean over the last quarter exceeds the third quarter’s by 10%, or it sustains an oscillation above 5 mrad RMS (past the margin the pendulum can settle into a limit cycle instead of growing; an encoder alone keeps it moving by about a count; a static offset, such as a mass error leaves, is not an oscillation).””” n = len(err) if np.isinf(err).any(): return True spread = [float(np.std(err[i * n // 4:(i + 1) * n // 4])) for i in range(4)] return spread[3] > 1.1 * spread[2] or (spread[3] > 0.005 and spread[3] > 0.5 * spread[1])

def measure_margin(job) -> tuple[float, float]: “"”Smallest extra delay at which the oscillation grows (bisection, 0.1 ms), and its frequency (rad/s).””” kp, kd, plant, sensing = job seconds = 8.0 if FAST else 20.0 lo, hi = 0.0, 0.38 if growing(run((kp, kd, plant, 0.0, sensing, seconds))[0]): return 0.0, float(“nan”) while hi - lo > 1e-4: mid = 0.5 * (lo + hi) lo, hi = (lo, mid) if growing(run((kp, kd, plant, mid, sensing, seconds))[0]) else (mid, hi) err = run((kp, kd, plant, lo, sensing, seconds))[0] tail = err[len(err) // 2:] crossings = np.flatnonzero(np.diff(np.sign(tail - tail.mean())) != 0) return hi, math.pi / (np.mean(np.diff(crossings)) * H) if len(crossings) > 2 else float(“nan”)

def loop_response(kp, kd, w, plant=NOMINAL, sensing=None, gravity=True) -> complex: “"”The linearized loop at frequency w (rad/s). With gravity, the plant is I s^2 + m g L and the compensation passes through the actuator: g A(s) e^(-s d) (C(s) - m g L) / (I s^2 + m g L); without, C(s) e^(-s d) / (I s^2). A zero-order hold over the control period T contributes e^(-s T/2).””” period, cutoff, quantize = sensing or (1, 0.0, False) inertia = 0.25 * plant[“mass”] + 1e-4 # m L^2 + I_c s = 1j * w if quantize: # differenced, filtered velocity deriv = (1 - cmath.exp(-s * H)) / H if cutoff: a = 1.0 - math.exp(-2 * math.pi * cutoff * H) deriv *= a / (1 - (1 - a) * cmath.exp(-s * H)) else: deriv = s c = kp + kd * deriv act = plant[“gain”] / (1 + s * plant[“lag”]) * cmath.exp(-s * (plant[“delay”] + period * H / 2)) if gravity: return act * (c - MGL) / (inertia * s * s + MGL * plant[“mass”]) return act * c / (inertia * s * s)

def predict_margin(kp, kd, plant=NOMINAL, sensing=None, gravity=True) -> tuple[float, float]: “"”Delay margin = phase margin / crossover frequency, from loop_response.””” lo, hi = 0.5, 2000.0 for _ in range(200): # |loop| falls through 1 once mid = math.sqrt(lo * hi) lo, hi = (mid, hi) if abs(loop_response(kp, kd, mid, plant, sensing, gravity)) > 1 else (lo, mid) w = lo phase_margin = math.pi + cmath.phase(loop_response(kp, kd, w, plant, sensing, gravity)) phase_margin = (phase_margin + math.pi) % (2 * math.pi) - math.pi return phase_margin / w, w

def gains(kp: float, zeta: float, inertia: float = I_PIVOT) -> tuple[float, float]: return kp, 2 * zeta * math.sqrt(kp * inertia)

if name == “main”: print(f”MuJoCo {mujoco.version}; pendulum I = {I_PIVOT} kg m^2, m g L = {MGL:.3f} N m, timestep {1000 * H:.0f} ms, “ f”holding {TARGET} rad”) with Pool(24) as pool: print(“(1) delay margin: predicted from the loop, measured by bisection in simulation”) print(f” {‘kp’:>5} {‘zeta’:>5} {‘kd’:>6} {‘predicted, plant I s^2’:>24} {‘with gravity in the loop’:>26} “ f”{‘measured’:>8} {‘frequency at the margin (rad/s)’:>32}”) cases = [(20, 1.0), (20, 0.5), (80, 0.7), (200, 0.7)] jobs = [(*gains(kp, z), NOMINAL, None) for kp, z in cases] for (kp, z), job, (m, f) in zip(cases, jobs, pool.map(measure_margin, jobs)): p0, w0 = predict_margin(job[0], job[1], gravity=False) p1, w1 = predict_margin(job[0], job[1]) print(f” {kp:>5} {z:>5} {job[1]:>6.3f} {1000 * p0:>13.1f} ms at {w0:>5.2f} {1000 * p1:>15.1f} ms at {w1:>5.2f} “ f”{1000 * m:>5.1f} ms {f:>32.2f}”)

    print("(2) actuator and controller mismatch, kp 80, zeta 0.7 (margin without them in (1))")
    kp, kd = gains(80, 0.7)
    variants = [("motor with a 20 ms first-order lag", {**NOMINAL, "lag": 0.02}, None),
                ("motor giving 80% of the modelled torque", {**NOMINAL, "gain": 0.8}, None),
                ("controller at 100 Hz (every 5 steps), held", NOMINAL, (5, 0.0, False))]
    results = pool.map(measure_margin, [(kp, kd, plant, sensing) for _, plant, sensing in variants])
    for (label, plant, sensing), (m, f) in zip(variants, results):
        p, w = predict_margin(kp, kd, plant, sensing)
        print(f"    {label:<44} predicted {1000 * p:5.1f} ms, measured {1000 * m:5.1f} ms (oscillating at {f:.2f} rad/s)")
    hold = run((kp, kd, {**NOMINAL, "gain": 0.8}, 0.0, None, 6.0))[0]
    print(f"    80% torque also leaves a static error: {1000 * hold[-1]:.2f} mrad at {TARGET} rad, predicted "
          f"{1000 * 0.2 * MGL * math.sin(TARGET) / (0.8 * kp + 0.2 * MGL * math.cos(TARGET)):.2f} mrad")

    print(f"(3) sensing: {COUNTS}-count encoder ({1000 * 2 * math.pi / COUNTS:.3f} mrad), velocity by differencing "
          "at 500 Hz, kp 80, zeta 0.7")
    sens = [("true angle and velocity", None), ("encoder, unfiltered velocity", (1, 0.0, True)),
            ("encoder, velocity filtered at 100 Hz", (1, 100.0, True)), ("encoder, velocity filtered at 30 Hz", (1, 30.0, True)),
            ("encoder, velocity filtered at 10 Hz", (1, 10.0, True))]
    holds = pool.map(run, [(kp, kd, NOMINAL, 0.0, s, 4.0) for _, s in sens])
    margins = pool.map(measure_margin, [(kp, kd, NOMINAL, s) for _, s in sens])
    for (label, s), hold, (m, _) in zip(sens, holds, margins):
        u = hold[1][len(hold[1]) // 2:]
        p, _ = predict_margin(kp, kd, NOMINAL, s)
        print(f"    {label:<38} torque chatter (std) {u.std():6.3f} N m, peak-to-peak {np.ptp(u):6.3f} N m; "
              f"delay margin predicted {1000 * p:5.1f} ms, measured {1000 * m:5.1f} ms")

    print("(4) a transfer experiment; hidden: delay, motor lag, torque gain and mass of the real pendulum")
    stiff = gains(200, 0.7)
    p_sim, _ = predict_margin(*stiff)
    real_stiff = run((*stiff, HIDDEN, 0.0, (1, 30.0, True), 6.0))[0]
    print(f"    deploy kp 200, zeta 0.7 (simulated margin {1000 * p_sim:.1f} ms): "
          + ("the real loop does not settle" if growing(real_stiff) else "the real loop settles"))
    probes = [gains(10, 1.0), gains(40, 1.0)]
    sensing = (1, 30.0, True)
    points = []
    for g, (m, f) in zip(probes, pool.map(measure_margin, [(*g, HIDDEN, sensing) for g in probes])):
        p, w = predict_margin(*g, NOMINAL, sensing)
        # at the margin the real loop has gain 1 and phase -180 deg at f: one point of the hidden factor G(jf)
        nominal = loop_response(*g, f, NOMINAL, sensing) * cmath.exp(-1j * f * m)
        factor = -1 / nominal
        points.append((f, abs(factor), cmath.phase(factor)))
        print(f"    probe kp {g[0]:.0f}, zeta 1: model margin {1000 * p:5.1f} ms at {w:.2f} rad/s; real margin {1000 * m:5.1f} ms, "
              f"oscillating at {f:.2f} rad/s -> hidden factor there: gain {abs(factor):.3f}, phase {math.degrees(cmath.phase(factor)):.1f} deg")
    # G(jw) = k e^(-jwd) / (1 + jw tau): the two gains give tau and k, the phases then give d
    (w1, g1, ph1), (w2, g2, ph2) = points
    ratio = (g1 / g2) ** 2                                                  # = (1 + w2^2 tau^2) / (1 + w1^2 tau^2)
    tau = math.sqrt(max((ratio - 1) / (w2 ** 2 - ratio * w1 ** 2), 0.0))
    k = g1 * math.sqrt(1 + (w1 * tau) ** 2)
    d = float(np.mean([(-ph - math.atan(w * tau)) / w for w, ph in ((w1, ph1), (w2, ph2))]))
    print(f"    fitted: torque gain {k:.3f}, motor lag {1000 * tau:.1f} ms, delay {1000 * d:.1f} ms "
          f"(hidden: gain {HIDDEN['gain']} with mass {HIDDEN['mass']} kg, lag {1000 * HIDDEN['lag']:.0f} ms, delay {1000 * HIDDEN['delay']:.0f} ms)")
    model_fit = {**NOMINAL, "delay": d, "lag": tau, "gain": k}
    for kp_try in (200, 120, 80, 40, 20):
        g = gains(kp_try, 0.7)
        p, _ = predict_margin(*g, model_fit, sensing)
        if p >= 0.02:
            break
    real_margin, _ = measure_margin((*g, HIDDEN, sensing))
    print(f"    stiffest kp keeping a 20 ms margin under the identified delay and lag: {kp_try} "
          f"(predicted margin {1000 * p:.1f} ms); on the real: margin {1000 * real_margin:.1f} ms") ``` Output:

MuJoCo 3.14.0; pendulum I = 0.2501 kg m^2, m g L = 4.905 N m, timestep 2 ms, holding 0.05 rad
(1) delay margin: predicted from the loop, measured by bisection in simulation
       kp  zeta     kd     predicted, plant I s^2     with gravity in the loop   measured    frequency at the margin (rad/s)
       20   1.0  4.473            71.4 ms at 18.41              71.8 ms at 19.18    71.4 ms                              19.29
       20   0.5  2.237            78.5 ms at 11.38              87.5 ms at 11.92    86.7 ms                              11.99
       80   0.7  6.262            40.2 ms at 27.59              40.7 ms at 27.95    40.5 ms                              28.23
      200   0.7  9.901            25.1 ms at 43.63              25.2 ms at 43.85    25.1 ms                              44.49
(2) actuator and controller mismatch, kp 80, zeta 0.7 (margin without them in (1))
    motor with a 20 ms first-order lag           predicted  24.9 ms, measured  24.4 ms (oscillating at 25.84 rad/s)
    motor giving 80% of the modelled torque      predicted  45.9 ms, measured  45.7 ms (oscillating at 23.58 rad/s)
    controller at 100 Hz (every 5 steps), held   predicted  36.7 ms, measured  36.6 ms (oscillating at 28.13 rad/s)
    80% torque also leaves a static error: 0.75 mrad at 0.05 rad, predicted 0.75 mrad
(3) sensing: 4096-count encoder (1.534 mrad), velocity by differencing at 500 Hz, kp 80, zeta 0.7
    true angle and velocity                torque chatter (std)  0.000 N m, peak-to-peak  0.000 N m; delay margin predicted  40.7 ms, measured  40.5 ms
    encoder, unfiltered velocity           torque chatter (std)  4.345 N m, peak-to-peak  9.721 N m; delay margin predicted  39.6 ms, measured  40.4 ms
    encoder, velocity filtered at 100 Hz   torque chatter (std)  2.546 N m, peak-to-peak  6.653 N m; delay margin predicted  38.8 ms, measured  39.5 ms
    encoder, velocity filtered at 30 Hz    torque chatter (std)  0.909 N m, peak-to-peak  2.499 N m; delay margin predicted  35.1 ms, measured  35.9 ms
    encoder, velocity filtered at 10 Hz    torque chatter (std)  0.349 N m, peak-to-peak  0.858 N m; delay margin predicted  26.4 ms, measured  27.2 ms
(4) a transfer experiment; hidden: delay, motor lag, torque gain and mass of the real pendulum
    deploy kp 200, zeta 0.7 (simulated margin 25.2 ms): the real loop does not settle
    probe kp 10, zeta 1: model margin  96.5 ms at 14.18 rad/s; real margin  79.7 ms, oscillating at 12.20 rad/s -> hidden factor there: gain 0.825, phase -22.4 deg
    probe kp 40, zeta 1: model margin  44.5 ms at 27.02 rad/s; real margin  21.1 ms, oscillating at 22.51 rad/s -> hidden factor there: gain 0.814, phase -41.2 deg
    fitted: torque gain 0.830, motor lag 8.8 ms, delay 23.2 ms (hidden: gain 0.9 with mass 1.1 kg, lag 8 ms, delay 25 ms)
    stiffest kp keeping a 20 ms margin under the identified delay and lag: 40 (predicted margin 27.9 ms); on the real: margin 27.8 ms

Debugging

A delay set in the model has less effect than expected. The actuator’s nsample history is too short to hold it; MuJoCo caps the delay at the history length without warning.

The prediction misses one controller and fits the others. A term the simple model left out matters at that controller’s crossover: here the gravity term, for soft gains.

The loop neither diverges nor settles. It is past its margin in a sustained oscillation. Test for settling, not for growth.

A filtered controller oscillates where the unfiltered one did not. The filter’s phase lag used up the margin; raise the cutoff or lower the gains.

A controller works in simulation and buzzes on the robot. Velocity from a quantized position, multiplied by kd. Simulate the encoder and the filter, and budget their margin.

Exercise

Predict, then measure, the delay margin of kp 80, $\zeta$ 0.7 with both a 10 ms motor lag and a controller at 100 Hz. Then find the highest velocity-filter cutoff that keeps the torque chatter below 1 N m and report the margin it leaves.

Challenge

Make the hidden plant’s lag 25 ms and its delay 10 ms, and add a third probe at a stiffer gain. Show how many probes, at which frequencies, separate a lag from a delay to within 2 ms, and redesign the controller for a 30 ms margin.

Research connection

[!research] Measuring a loop by making it oscillate Åström and Hägglund (Automatica 20(5), 1984, pp. 645-651; doi:10.1016/0005-1098(84)90014-1) tuned industrial regulators from an oscillation provoked on purpose, the same idea as part (4): at the edge of stability a loop reveals its gain and phase at one frequency. In learned locomotion, Tan et al. (arXiv:1804.10332) simulated latency and an actuator model before training, and Hwangbo et al. (arXiv:1901.08652) learned an actuator network on the robot. For a policy instead of a PD loop, the margin is harder to compute, but the experiment carries over (inference): add known delay at deployment until behaviour degrades, and compare the delay a policy tolerates on hardware with what it tolerated in simulation.

{"id": "19.1-check", "title": "Knowledge check", "questions": [
  {"kind": "numeric", "q": "A loop crosses unit gain at 25 rad/s with a phase margin of 0.75 rad. How much extra delay (ms) can it tolerate?",
   "answer": 30, "tol": 0.5, "unit": "ms",
   "explain": "<p>$d^\\ast = \\varphi_m/\\omega_c$ = 0.75/25 = 0.030 s.</p>"},
  {"kind": "mcq", "q": "Why does kp 200 tolerate a third of the delay kp 20 tolerates?",
   "options": ["Its gains saturate the motor", "Its crossover frequency is higher, so the same delay removes more phase", "Its damping is lower", "Gravity compensation is worse"],
   "answer": 1,
   "explain": "<p>Phase lost to a delay is ωd; the crossover rises from about 19 to 44 rad/s.</p>"},
  {"kind": "mcq", "q": "Filtering the encoder velocity at 10 Hz cut torque chatter from 4.3 to 0.35 N m. What did it cost?",
   "options": ["Nothing", "About 13 ms of delay margin, because the filter's phase lag acts like a delay", "Steady-state accuracy", "Encoder resolution"],
   "answer": 1,
   "explain": "<p>The margin fell from 40.4 to 27.2 ms.</p>"},
  {"kind": "mcq", "q": "A controller fails on hardware. Which follow-up measures the cause instead of just confirming the failure?",
   "options": ["Retrain with more randomization", "Lower the gains until it works", "Run softer controllers, add known delay until each stops settling, and record the oscillation frequency", "Increase the control rate"],
   "answer": 2,
   "explain": "<p>Each probe gives the real loop's gain and phase at one frequency, which a model of the mismatch can be fitted to.</p>"}
]}

Next

Level 20 opens the engine: the pipeline inside mj_step, the constraint solver, performance, and extending MuJoCo.