Industry Research

Appearance

Continuous-Time Trajectory Splines and Gaussian Process Priors ​

Continuous-Time Trajectory Splines and Gaussian Process Priors curated visual

Visual: continuous trajectory with spline knots, interpolated measurement times, GP prior factors, manifold state, and observability pitfalls.

Most estimators pretend the robot state exists only at discrete keyframes. Real sensors do not. A rolling-shutter camera exposes rows at different times, a spinning LiDAR collects each point at a different azimuth time, an IMU samples at hundreds of hertz, and wheel/GNSS/radar measurements arrive asynchronously. Continuous-time trajectory estimation represents pose as a function T(t) so every measurement can be evaluated at its own acquisition time.

The practical question is not "continuous or discrete?" It is: what assumptions make the function between optimized states well defined, observable, and cheap enough to solve? Splines and Gaussian process (GP) priors are the two dominant answers.

1. Related Docs ​

2. Why Continuous Time Exists ​

For a measurement z_i acquired at time t_i, the residual usually has the form:

text
r_i = z_i - h_i(x(t_i), theta)

where theta may include calibration parameters, landmarks, map variables, or time offsets. A discrete estimator must either:

  1. create a state exactly at t_i,
  2. interpolate between nearby states outside the optimizer, or
  3. assign z_i to a nearby keyframe and absorb timing error into noise.

Continuous-time estimation makes the interpolation part of the model:

text
x(t) = Phi(t; c_0, c_1, ..., c_K)

The variables c_k are control poses, knot states, or GP support states. The optimizer changes them, and every residual updates consistently.

This matters when:

  • LiDAR scans bend because the platform moves during one revolution.
  • Rolling-shutter rows are projected with the wrong camera pose.
  • Camera, IMU, LiDAR, wheel, and GNSS samples have different rates.
  • Calibration jointly estimates extrinsics and time offset.
  • A map or replay tool needs poses at arbitrary timestamps.

3. State on a Manifold ​

For rigid-body motion, pose is not a vector in Euclidean space. It is an element of SE(3):

text
T(t) =
[ R(t)  p(t) ]
[ 0     1    ]

Small updates live in the tangent space:

text
T <- Exp(delta_xi) T
delta_xi in R^6

A continuous-time trajectory may model:

  • position p(t) and orientation R(t) separately,
  • pose T(t) directly on SE(3),
  • velocity and bias states in addition to pose,
  • calibration parameters as time-invariant variables.

The central constraint is that interpolation must respect the manifold. Linear interpolation of quaternion components or transformation matrix entries creates invalid motion unless followed by projection, and that projection changes the intended uncertainty.

4. Spline Trajectories ​

4.1 Intuition ​

A spline trajectory uses a small set of control variables to define a smooth curve. Local support is the key property: changing one control point affects only a bounded time interval. This keeps Jacobians sparse.

For scalar uniform B-splines:

text
x(t) = sum_j B_j,k(u) c_j

where:

  • c_j are control points,
  • B_j,k are degree-k basis functions,
  • u is normalized time between knots.

Higher degree gives smoother derivatives. A cubic B-spline is commonly used because it gives continuous position, velocity, and acceleration in vector spaces while retaining local support.

4.2 Pose Splines ​

For rotations or poses, a common construction composes local increments:

text
R(u) = R_i
       Exp(B_1(u) * log(R_i^T R_(i+1)))
       Exp(B_2(u) * log(R_(i+1)^T R_(i+2)))
       Exp(B_3(u) * log(R_(i+2)^T R_(i+3)))

Analogous cumulative formulations exist for SE(3). The exact convention depends on left/right perturbations and whether translation is interpolated in world, body, or group coordinates. Pick one convention and keep the Jacobians, velocity extraction, and residual definitions consistent.

4.3 Cost Function ​

A spline batch estimator solves:

text
min_c,theta  sum_i || z_i - h_i(Phi(t_i; c), theta) ||^2_Ri
             + smoothness_or_prior_terms

Smoothness may be implicit in the spline degree and knot spacing, or explicit:

text
E_smooth = integral || d^2 x(t) / dt^2 ||^2_Q dt

This says fast changes in acceleration are unlikely unless measurements demand them.

4.4 Knot Spacing ​

Knot spacing is a modeling choice, not just a compute knob.

Knot spacingEffect
Too widecannot represent fast turns, suspension motion, or scan distortion
Too narrowoverfits noise, weakens observability, increases solve cost
Nonuniformuseful for eventful intervals but complicates implementation

A good rule is to make the trajectory bandwidth higher than the motion that must be corrected, then let measurement noise and priors suppress overfitting.

5. Gaussian Process Priors ​

5.1 First Principle ​

A GP trajectory treats state over time as a random function:

text
x(t) ~ GP(mu(t), K(t, t'))

A motion prior defines which functions are likely. In continuous-time robotics, the GP prior is often generated by a stochastic differential equation:

text
x_dot(t) = A(t) x(t) + F(t) w(t)
w(t) ~ white noise with power spectral density Q_c

For a constant-velocity model in vector space:

text
d/dt [ p ] = [ 0 I ] [ p ] + [ 0 ] w(t)
     [ v ]   [ 0 0 ] [ v ]   [ I ]

The process noise says acceleration is unknown but bounded in distribution. Large Q_c allows agile motion; small Q_c enforces smooth motion.

5.2 Exactly Sparse Structure ​

Barfoot, Tong, and Sarkka showed that GP priors derived from linear time-varying SDEs can have exactly sparse inverse covariance when represented at support states. The prior between neighboring support states looks like:

text
r_k = x_(k+1) - Phi_k x_k
Q_k = integral Phi(t_(k+1), tau) F Q_c F^T Phi(t_(k+1), tau)^T d tau

cost_prior = sum_k r_k^T Q_k^-1 r_k

This is the same algebraic shape as a Markov process model or smoother:

text
x_k  --prior--  x_(k+1)  --prior--  x_(k+2)

The GP interpretation adds continuous interpolation. For a time tau between support states i and i+1:

text
x(tau) = Lambda(tau) x_i + Psi(tau) x_(i+1)

where Lambda and Psi are computed from the motion prior. Measurements at arbitrary times connect only to nearby support states, so the factor graph stays sparse.

5.3 GP vs Spline View ​

AspectSpline trajectoryGP prior trajectory
Primary ideadeterministic basis functionsprobabilistic motion prior
Smoothnessdegree, knot spacing, regularizationprocess-noise spectral density
Uncertaintyadded through residual weightsbuilt into interpolation covariance
Localitylocal basis supportMarkov prior and local interpolation
Tuning knobknot spacing and smoothness weightsupport spacing and Q_c

They are not enemies. Both are continuous-time priors. Splines are often easier to implement for calibration and rolling-shutter models. GP priors are often easier to connect to Kalman smoothing, uncertainty, and physical motion models.

6. How It Appears in Perception, SLAM, and Mapping ​

AreaContinuous-time use
LiDAR SLAMdeskew every point by evaluating T(t_point) before registration
visual-inertial calibrationestimate camera-IMU extrinsic and time offset while fitting a smooth trajectory
rolling-shutter visionproject each image row using its exposure time
radar fusioncompensate detections using ego pose at radar acquisition time
dense mappingintegrate depth/range data into TSDF/occupancy maps at point time
incident replayquery aligned vehicle pose at any logged sensor timestamp

Kalibr is a canonical example of the calibration use case: a continuous-time spline trajectory lets camera and IMU residuals share one motion model while extrinsics and temporal offsets are estimated.

7. Observability and Failure Modes ​

Failure modeRoot causeSymptom
Over-smooth trajectoryknots too far apart or prior too strongdeskew leaves curved walls and rolling-shutter residuals
Overfit trajectoryknots too close or prior too weakmotion explains sensor noise; calibration becomes unstable
Bad time offset estimatemotion lacks excitation or timestamps are inconsistentlow residuals with wrong extrinsics or phase lag
Wrong manifold conventionleft/right perturbations mixedoptimizer diverges or velocities have wrong sign
Boundary artifactsweak constraints at first/last knotstrajectory curls at log edges
Degenerate calibration motionno rotations, no acceleration, planar-only sceneextrinsic/time parameters correlate strongly
Hidden latencyusing receipt time instead of acquisition timeresiduals grow with speed and yaw rate
Ignored covarianceall measurements weighted equallyhigh-rate sensor dominates even when biased

Continuous-time models can hide problems by explaining them as motion. A trajectory that looks plausible is not proof that calibration, timing, and map geometry are correct.

8. Implementation Checklist ​

  • Define time semantics: acquisition time, sensor clock, host clock, and offset.
  • Store every measurement timestamp in a common time base before optimization.
  • Choose support spacing from platform dynamics and scan/exposure duration.
  • Use manifold-aware interpolation for SO(3) and SE(3).
  • Derive analytic Jacobians or verify autodiff Jacobians against finite differences on the manifold.
  • Add priors or fixed gauges for global pose, yaw, scale, and clock offset as required by the sensor set.
  • Keep GP/spline support local so each residual touches only nearby states.
  • Include boundary support states outside the measurement interval when using splines with finite support.
  • Validate with synthetic data where the true trajectory, time offset, and calibration are known.
  • Plot residuals versus timestamp, image row, LiDAR azimuth, speed, and yaw rate. Time errors usually appear as structured residuals.
  • Stress test with high angular velocity, braking, bumps, and sensor dropouts.
  • Export both optimized support states and a documented query function T(t_query); downstream code should not reimplement interpolation casually.

9. Minimal Mental Model ​

Continuous-time estimation replaces:

text
measurement -> nearest pose

with:

text
measurement timestamp -> trajectory query -> residual

The trajectory query is not bookkeeping. It is a prior about how the platform moved between optimized states. Spline models express that prior with basis functions; GP models express it with stochastic dynamics and covariance.

10. Sources ​

Public research notes collected from public sources.

Copyright © 2026 kvynlim