Matching Synthetic Speed Profiles to Observed Distributions

The generated speed histogram overlays the observed one almost perfectly. Then somebody plots a single trace and it oscillates between 12 and 48 km/h twice a second, which no vehicle does and no physical model produced — the speeds were sampled, not driven.

Part of Physics-Based Path Generation: the physics gives plausible motion but the wrong speed distribution, and the obvious correction throws the physics away. Getting both is the actual problem.

Root Cause: A Marginal Distribution Says Nothing About Order

Speed observed along a trajectory is a time series with strong autocorrelation: a vehicle at 50 km/h is at roughly 50 km/h a second later, because acceleration is bounded. The marginal distribution of speed discards that entirely.

So a generator that draws each speed independently from the right marginal reproduces the histogram exactly and produces motion that is physically impossible. The mismatch shows up in three statistics, none of which anybody computes unless they have been caught by this before:

  • The acceleration distribution. Independent draws produce accelerations an order of magnitude beyond what any vehicle achieves, and the tail is where it is most obvious.
  • The lag-1 autocorrelation of speed. Observed traces sit above 0.9 at one-second sampling. Independent draws sit at zero, by construction.
  • The distribution of sustained-speed run lengths. Real driving holds a speed band for tens of seconds; sampled speeds leave it every step.
Sampled speeds against driven speeds over the same period The horizontal axis is time in seconds across a little under four minutes of trace. The vertical axis is speed in kilometres per hour. The upper trace is produced by drawing each second's speed independently from the observed speed distribution: it fills the same vertical band, its histogram matches the reference exactly, and it jitters violently from one second to the next, spanning most of the band within a couple of samples. The lower trace comes from a process where speed changes are bounded by an acceleration budget and the target speed changes only occasionally: it moves through the same band smoothly, holding a level for tens of seconds before transitioning. Beside each trace two numbers are printed — the ninety-fifth percentile of absolute acceleration and the lag-one autocorrelation of speed. The sampled trace's acceleration percentile is an order of magnitude beyond anything a road vehicle achieves and its autocorrelation is essentially zero; the driven trace's figures both sit in the observed range. The note underneath makes the point that the histogram is identical in both cases, so no marginal check separates them. Identical histograms; one of them is not driving 0 20 40 60 speed (km/h) Drawn a₉₅ 30.9 m/s² lag-1 r -0.12 drawn from the marginal 0 60 120 180 0 20 40 60 time (s) speed (km/h) Driven a₉₅ 2.1 m/s² lag-1 r 1.00 driven under an accele… Both traces have the same speed histogram to within sampling error. Observed driving sits near 2 m/s² at the ninety-fifth percentile and above 0.9 lag-one correlation at 1 Hz. A histogram overlay is a picture that either generator passes; the acceleration percentile and the autocorrelation are numbers only one of them passes.
Same speed histogram, two generators: the marginal match is exact in both and everything about the dynamics differs.

Prerequisite Check: Establish Both Targets Before Fitting Either

python
def speed_targets(traces, dt: float) -> dict:
    """The two things a generated profile has to match, measured together."""
    speeds, accels, lag1 = [], [], []
    for tr in traces:
        v = [step.speed for step in tr]
        speeds.extend(v)
        accels.extend((v[i + 1] - v[i]) / dt for i in range(len(v) - 1))
        lag1.append(autocorrelation(v, lag=1))
    return {
        "speed_quantiles": quantiles(speeds, [0.05, 0.25, 0.5, 0.75, 0.95]),
        "accel_p95": quantiles([abs(a) for a in accels], [0.95])[0],
        "accel_p999": quantiles([abs(a) for a in accels], [0.999])[0],
        "lag1_median": median(lag1),
        "dt": dt,
    }

Recording dt in the target is not bookkeeping. Autocorrelation and acceleration percentiles are both functions of the sampling interval, so a target measured at 1 Hz and applied to a 5 Hz generator is simply a different target — the same interval dependence that makes cross-release comparison of trajectory statistics awkward everywhere else.

Fix: Generate Dynamics First, Then Map the Quantiles

The construction that works keeps the physics in charge and corrects the distribution afterwards, rather than the reverse.

1 — Generate a physically valid profile with roughly the right shape

python
def drive(route, limits, rng, dt: float) -> list[float]:
    """A first-order speed process bounded by the acceleration budget."""
    v = 0.0
    out = []
    for seg in route:
        target = seg.speed_limit * rng.normal(0.92, 0.08)     # drivers vary around the limit
        for _ in range(int(seg.duration / dt)):
            desired = (target - v) / max(dt, 1e-6)
            a = clamp(desired, -limits.decel_max, limits.accel_max)
            v = max(0.0, v + a * dt)
            out.append(v)
    return out

This produces the right kind of trace — smooth, bounded, autocorrelated — and almost certainly the wrong distribution, because the segment targets were guessed.

2 — Map the quantiles onto the observed distribution

python
def quantile_map(generated: list[float], observed_quantiles, gen_quantiles) -> list[float]:
    """Monotone map from the generated distribution onto the observed one."""
    return [
        interp(v, gen_quantiles, observed_quantiles)
        for v in generated
    ]

Monotonicity is the whole reason this works. A monotone transformation of a trace preserves its ordering, so the autocorrelation survives, the run-length structure survives, and the marginal becomes exactly the target. What it does not preserve is the acceleration budget: stretching the upper tail multiplies the accelerations there.

3 — Re-clip acceleration and iterate

python
def reconcile(generated, targets, limits, dt, rounds: int = 6):
    """Alternate quantile mapping with acceleration clipping until both hold."""
    v = list(generated)
    for _ in range(rounds):
        v = quantile_map(v, targets["speed_quantiles"], quantiles(v, [.05, .25, .5, .75, .95]))
        v = clip_accelerations(v, limits, dt)      # slightly perturbs the marginal again
        if accel_p95(v, dt) <= targets["accel_p95"] * 1.02:
            break
    return v

The alternation converges quickly in practice because the two operations pull on nearly disjoint parts of the trace: quantile mapping moves the values, clipping moves the differences, and after three or four rounds neither is changing the other much. If it does not converge, the target combination is infeasible — an observed distribution with a heavy high-speed tail cannot be reached under an acceleration budget that never gets there, and the right response is to question the budget rather than to raise the round count.

Convergence of the quantile-map and acceleration-clip alternation The horizontal axis is the alternation round, from zero to eight. The vertical axis is relative error, shared by both curves. One curve is the error in the speed marginal, measured as the largest relative deviation across the fitted quantiles; it starts high because the physically generated profile had guessed segment targets, drops sharply at the first quantile map, and is pushed back up slightly by each acceleration clip. The other curve is the excess of the ninety-fifth-percentile acceleration over its budget; it starts high because the first quantile map stretched the upper tail, and falls as each clip removes the excess while the next map re-perturbs it less each time. A shaded tolerance band across the bottom marks where both are acceptable. The curves enter the band together around the fourth round and stay inside it. The note underneath explains why the alternation converges rather than oscillating, and what a failure to converge actually means. Quantile map moves the values; clipping moves the differences 0 2 4 6 8 0.00 0.10 0.20 0.30 0.40 alternation round relative error tolerance band both inside at round 5 marginal error across fitted quantiles excess over the acceleration budget The two operations pull on nearly disjoint parts of the trace, which is why the alternation converges rather than fighting itself. Failure to converge is informative rather than a tuning problem: it means the observed distribution cannot be reached under the stated acceleration budget, and the budget is the assumption worth re-examining first.
The alternation in progress: marginal error and acceleration excess across rounds, converging where they cross.

Verification Step: Assert on Dynamics, Not Only on the Histogram

python
def test_marginal_matches(generated, targets, tol=0.03):
    got = quantiles(generated, [0.05, 0.25, 0.5, 0.75, 0.95])
    for g, want in zip(got, targets["speed_quantiles"]):
        assert abs(g - want) / max(want, 1e-6) < tol, f"{g:.2f} vs {want:.2f}"


def test_accelerations_are_physical(generated, targets, dt, tol=1.05):
    a = [abs((generated[i + 1] - generated[i]) / dt) for i in range(len(generated) - 1)]
    assert quantiles(a, [0.95])[0] < targets["accel_p95"] * tol
    assert quantiles(a, [0.999])[0] < targets["accel_p999"] * tol


def test_autocorrelation_survives(generated, targets, tol=0.08):
    got = autocorrelation(generated, lag=1)
    assert abs(got - targets["lag1_median"]) < tol, f"lag-1 {got:.3f}"


def test_run_lengths_are_realistic(generated, observed_runs, tol=0.2):
    """Time spent inside a 5 km/h band before leaving it."""
    got = median(band_run_lengths(generated, band=5.0))
    assert abs(got - observed_runs) / observed_runs < tol

The last two are the ones that fail on a marginal-only generator, and they are the reason the suite is worth writing: a histogram overlay is a picture that any sampling approach passes, and these are numbers that only a dynamically plausible trace passes.

Four speed generators against four validation statistics A matrix with four generators as rows and four statistics as columns. Drawing each speed independently from the observed marginal passes the quantile check outright and fails acceleration, autocorrelation and run length. A smoothed version of those independent draws — a moving average — passes acceleration and autocorrelation but now fails the quantile check, because smoothing compresses the distribution toward its mean. Physics alone, with guessed segment targets, passes everything about the dynamics and fails the quantile check. Physics reconciled against the observed quantiles passes all four. Reading down the columns, the quantile check alone accepts the first generator, and the acceleration and autocorrelation checks together are what reject it. The note underneath states the practical rule: a validation suite containing only distributional checks will certify a generator that produces physically impossible motion, and adding two cheap time-series statistics closes that gap entirely. Two extra statistics separate a driven profile from a sampled one generator speed quantiles acceleration p95 lag-1 autocorr run lengths independent draws pass fail 10× budget fail r ≈ 0 fail 1 step smoothed draws fail compressed pass pass fail too regular physics, guessed targets fail wrong shape pass pass pass physics + quantile map pass pass pass pass A validation suite containing only distributional checks will certify the first row — a generator producing motion no vehicle can perform. The acceleration percentile and the lag-one autocorrelation each cost one pass over the trace, and together they close the gap that the histogram cannot see.
Which statistic catches which defect: four generators against four checks, and the two checks that separate them.

Edge Cases & Gotchas

Stops are a separate distribution. Observed speed distributions have a spike at zero that is not part of the moving distribution at all. Fitting one distribution across both makes the map wrong everywhere; separate the stopped and moving regimes, match the moving one, and let the stop model supply the zeros.

Speed distributions are conditional. Free-flow motorway speeds and congested urban speeds are different distributions, and a single global target reproduces neither. Conditioning the target on road class costs a lookup and removes most of the residual error.

The observed distribution carries its own measurement error. Speeds derived by differencing noisy positions are inflated at the low end, so a generator matched to them inherits the noise as if it were signal. If the reference comes from position differencing rather than from a speed sensor, the target should be de-noised first or the generator will be asked to reproduce GPS error twice — once here and once in the noise injection stage.

Quantile mapping needs enough generated data. Estimating the generated quantiles from a single short trace is noisy, so the map wobbles from trace to trace. Fit the map once on a large pooled sample and apply it to every trace.

Where the Target Distribution Should Come From

The reconciliation is only as good as the distribution it targets, and the choice of reference is made carelessly more often than the fitting method is.

Prefer a speed sensor to differenced positions. A distribution derived by differencing GPS positions is the true speed distribution convolved with the position noise, widened at both ends and with an artificial floor above zero. Matching to it makes the generator reproduce measurement error as though it were behaviour, and the noise injection stage then adds the same error a second time.

Condition on the thing that actually varies. Speed distributions differ far more between road classes, times of day and vehicle types than they do between weeks. A global distribution fitted across all of them is a mixture that describes no individual case, and a generator matched to it produces motorway speeds in residential streets at exactly the rate the mixture implies.

Check the reference’s own sampling interval. A distribution built from 1 Hz samples and one built from 10-second samples differ systematically: the coarser sample misses short excursions and under-represents both tails. If the generator emits at a different rate than the reference was measured at, the target has to be re-derived at the emission rate rather than reused.

Beware selection in the reference fleet. Speed distributions from commercial telematics come from vehicles that are professionally driven, instrumented, and often speed-limited. Using that as the target for a general population produces a synthetic fleet that is noticeably more disciplined than reality — a bias that survives every check in this article, because every check compares the output to the reference rather than the reference to the world.