SimTrack3D: Monocular 3D Trajectory Reconstruction of Fast-Moving Sports Projectiles via Broadcast-Calibrated Sim-to-Real Transfer

Gino Chang

Project page · 2026

SimTrack3DDemoRelated workMethodObservabilityExperimentsSupplementary

Demo

Abstract

1. Introduction

Professional racket-sports archives comprise decades of broadcast footage, an invaluable historical record for athletic training, tactical analysis and broadcast visualization. Recovering metric 3D kinematics — smash steepness, apex height, net clearance — from a single broadcast camera, however, is fundamentally ill-posed. Multi-camera optical tracking rigs capture high-precision metric kinematics, but their installation and calibration costs restrict them to a handful of elite venues, leaving historical broadcast archives inaccessible to quantitative evaluation. In this work, we propose SimTrack3D, a broadcast-only sim-to-real reconstruction framework that recovers metric 3D projectile trajectories to a median error of 0.42 m using only cues naturally present in broadcast video: calibrated court markings, 2D shuttle detections and player poses.

Deep heatmap regressors such as TrackNet^{[1, 2]} have established reliable 2D projectile localization in high-speed sports broadcasts. Lifting these 2D detections into metric 3D trajectories from a single camera, however, encounters severe geometric degeneracies. Conventional monocular 3D vision resolves metric scale and depth through object-centric size priors, motion parallax or ground-contact dynamics. In broadcast badminton, all these cues collapse simultaneously:

• Absence of optical scale priors. A tournament shuttlecock measures merely ≈ 7 px across a 720p frame, elongating into a textureless streak under motion blur, without resolvable geometric extent or orientation.
• Degeneracy of motion parallax. Launch speeds reach a 99th percentile of 89.5 m/s, compressing a smash into ≈ 15 tracked frames at 30 fps. Because many of these flights travel nearly collinear with the optical axis, the subtle curvature induced by gravity and drag is buried beneath tracking noise (≈ 2 px).
• Absence of ground impacts. Unlike tennis or table tennis^{[3, 4]}, where planar bounces supply explicit metric boundary conditions (z = 0), badminton flights run strictly racket to racket and touch the floor only when the rally ends.

Badminton thus represents the limiting frontier of monocular ballistic estimation: a regime in which visual scale, ground bounces and motion parallax are absent or degenerate at once.

The reprojection fallacy. Because sports broadcasts carry no dense 3D ground truth, prior monocular estimators rely heavily on 2D reprojection residuals or coarse heuristics (e.g., net clearance) as optimization targets and validation proxies. As formalized in Figure 1, reprojection residuals are structurally uninformative for high-speed axial trajectories. Scaling a trajectory along its instantaneous viewing rays generates an ambiguity cone: flights that differ by 6.2 m at launch and 4.0 m at arrival in 3D change the image projection by less than 1.1×10⁻¹³ px, thirteen orders of magnitude below detector jitter. Candidate smashes with a 43-fold difference in 3D error (0.065 m vs. 2.78 m) sit within an indistinguishable 0.20 px residual band. Minimizing 2D reprojection error alone is therefore blind to line-of-sight depth drift.

Core intuition: anchoring depth to calibrated planes. Although a single viewpoint cannot disambiguate depth along an optical ray, broadcast cameras localize planar court lines with sub-pixel precision. Every badminton flight originates and terminates near a player standing on this calibrated floor: the striker at launch (t = 0) and the receiver at arrival (t = T). Detecting the players’ feet with 2D pose estimation and back-projecting them onto the floor plane establishes spatiotemporal floor anchors that constrain depth along the initial and terminal rays. MonoTrack^[5] already uses the players’ court positions, to initialise its fit and to pull the end of the flight toward the receiver; we instead feed them, with a declared uncertainty, to a learned depth estimator and measure what they buy against real 3D. The elevated net cord — the sole non-coplanar static fixture on court — supplies an explicit vertical metric scale that decouples focal length from camera distance.

We realize this intuition in SimTrack3D, centered on Shuttle3D, a ray-depth transformer trained exclusively on aerodynamically simulated strokes. Our contributions are:

• A broadcast-calibrated sim-to-real pipeline. A self-contained framework that reconstructs metric 3D shuttle flights from single-camera video using only court lines, 2D detections and player poses, requires no manual 3D annotation, and applies retrospectively to broadcast archives.
• Sag-gated non-coplanar calibration. A calibration formulation that exploits the elevated net cord to fix the focal length that planar homographies constrain only weakly, verified by an invariant 26 mm catenary sag gate across 32 broadcast cameras.
• Ray-depth lifting with uncertainty-aware anchors. Shuttle3D parameterizes depth strictly along viewing rays, eliminating transverse drift by construction. It conditions lifting on player foot detections with an explicit declared uncertainty σ, so that noisy poses are weighed rather than trusted.
• Null-space feasibility rectification. A second-tier optimizer projects physically invalid trajectories onto the court rules only along the weakest eigenspace of the Gauss–Newton curvature matrix J^⊤J, at a median reprojection cost of 0.00 px.
• Empirical validation and observability theory. On an independent ten-camera real-3D benchmark (2,682 strokes), Shuttle3D attains a median error of 0.42 m, ahead of learned and physics baselines (Tables 1 and 2). An image-computable observability number N predicts the Cramér–Rao lower bound (CRLB ∝ N^−0.98) on synthetic strokes, delineating image-driven from prior-dominated regimes.

2. Related Work

2D projectile tracking in sports. Deep heatmap regressors, pioneered by TrackNet^{[1, 2, 6]}, form the standard front-end for tracking small, fast-moving projectiles across consecutive frames. Subsequent architectures add trajectory rectification and spatio-temporal attention, with extensions adapted to soccer, tennis and table tennis. While motion blur is traditionally treated as corruption, BlurBall^[7] shows that blur streaks encode the ball’s instantaneous velocity. We take off-the-shelf 2D detectors as observational inputs and focus entirely on the ill-posed monocular 3D lifting problem.

Monocular 3D ballistic reconstruction. Reconstructing metric 3D trajectories from single-view footage requires strong physical or geometric priors to resolve depth ambiguities. Existing work falls into three paradigms:

• Physics-fitting optimization. MonoTrack^[5] fits a ballistic launch state and drag coefficient per stroke by non-linear least squares against 2D detections, initialised at the hitter’s court position and softly pulled toward the receiver’s. As an (approximately) unbiased estimator it is bounded by the Cramér–Rao lower bound, and on axial flights such as smashes, where visual curvature is degenerate, the fit breaks down: the official code, given the same player positions, is about four times less accurate than Shuttle3D on synthetic smashes (0.89 m vs. 0.24 m median).
• Impact-conditioned lifting. In table tennis, TT3D^[3], TT4D^[4] and Uplifting Table Tennis^[8] exploit table bounces (z = 0) as explicit metric boundary conditions. Badminton flights run racket to racket and offer no bounce during active play.
• Learned trajectory lifting. SynthNet^[9] and TrajTrans^[10] train networks on synthetic flights to lift 2D detections into 3D coordinates or launch parameters. TrajTrans, the closest prior work to ours, is trained only on simulated shuttle flights and regresses six launch parameters from the pixel track and four court corners; on real video it is evaluated by 2D reprojection (median 23.6 px) and cross-camera consistency, without real 3D. Where Is The Ball^[11] parameterizes tennis and soccer trajectories along viewing rays, a precedent for our ray representation, but assumes flights that start and end on the ground. These methods share two limitations: (i) several calibrate from coplanar court corners, which carry no vertical metric scale; and (ii) they validate on synthetic sets or, on real footage, through 2D reprojection and plausibility proxies, which cannot penalize line-of-sight depth drift (Section 1).

In contrast, SimTrack3D recovers vertical scale from the net cord, feeds the players’ floor positions to a learned depth estimator as uncertainty-aware tokens rather than as a fixed fitting term, and validates directly against an independent ten-camera real 3D reference.

Sim-to-real transfer and system identification. Sim-to-real paradigms span domain randomization^[12], domain adaptation and differentiable simulation. Naive domain randomization samples arbitrary camera distributions, whereas monocular ballistic recovery needs accurate inductive biases. Our framework follows system identification instead: we measure 32 real tournament camera geometries and synthesize flights by solving aerodynamic boundary-value problems anchored to 13,010 real tournament strokes.

Broadcast sports camera calibration. Calibrating a broadcast camera from planar court markings yields an 8-DoF homography H, which constrains focal length f only weakly through Zhang’s planar constraints^[13]. Deep registration networks and vanishing-point estimators (e.g., TVCalib^[14], PnLCalib^[15]) automate field calibration, but at the shallow tilt of broadcast cameras a court without upright fixtures leaves focal length ill-conditioned. We resolve it with the elevated net cord as a non-coplanar vertical metric constraint, and validate every calibration by the cord’s catenary sag, which is withheld from the fit and must match the 26 mm specification.

A longer survey, with the sim-to-real mechanisms and the constrained-estimation literature treated separately, is on the supplementary page.

3. Methodology

Overview. As illustrated in Figure 2, SimTrack3D operates in two regimes: offline simulation and online inference. Offline, we anchor 77,728 aerodynamic boundary-value flights to 13,010 real tournament strokes and project them through 32 broadcast-calibrated cameras to train our core network, Shuttle3D (§3.2, §3.3). At inference, a single broadcast camera is calibrated from the court lines and the elevated net cord (§3.1). 2D shuttle detections are converted into viewing rays, and detected player feet form uncertainty-aware spatiotemporal floor anchors (§3.4). Shuttle3D predicts metric depth strictly along each viewing ray. Finally, a null-space rectifier resolves physical rule violations only along the least-observed eigenspace of the curvature matrix J^⊤J, producing metric, kinematically admissible flights, or flagging the stroke (§3.5).

3.1 Sag-gated non-coplanar calibration

Let X = [X, Y, Z]^⊤ denote a 3D coordinate in the canonical court frame with the floor on Z = 0. Coplanar court markings supply an 8-DoF homography

Under planar constraints alone focal length is ill-conditioned: at broadcast tilt a 10% change in f moves the court image by only a few pixels, and on our broadcast corpus the two planar (Zhang) constraints^[13] disagree on focal length by up to 65%.

To break this projective ambiguity, we incorporate the elevated net cord as a non-coplanar metric reference. By Badminton World Federation (BWF) specifications, the cord is suspended at post height H_p = 1.55 m and deflects under gravity to H_c = 1.524 m at the center across width L = 6.10 m. In court coordinates the cord follows a catenary

Because Δz / L ≪ 1, the shallow parabolic approximation holds:

Joint optimization. We extract sub-pixel line segments and net-cord samples by ridge filtering over temporal median backgrounds, and solve for focal length f, rotation R and translation t:

where d_⊥ is the orthogonal point-to-line residual on the court floor and π(·) is the perspective projection of the 3D net-cord samples Q_j. Crucially, f is fixed by the vertical offset between the net cord and the floor, while the sag Δz is withheld from the optimization.

Physical validation gate. A converged calibration is accepted if and only if its estimated deflection satisfies

The gate rejects degenerate minima whose 2D line residuals look excellent (0.17–0.38 px) but whose geometry is wrong, with sags of 9, −15 and −49 mm. Of the 32 broadcast cameras, 29 are calibrated fully automatically (median runtime 11.7 s per venue) and three from four hand-clicked court corners; the measured sag has a median of 23.1 mm against the 26 mm specification (Figure 3a; Supp. §S3.2).

3.2 Aerodynamic boundary-value synthesis

A tournament shuttlecock experiences strong quadratic aerodynamic drag. With kinematic state s(t) = [x(t)^⊤, v(t)^⊤]^⊤, the equations of motion are

where g = [0, 0, −9.81]^⊤ m/s², γ = ‖g‖ / v_T² ≈ 0.212 m⁻¹, and the terminal velocity is v_T = 6.8 m/s.

Rather than sampling unconstrained launch states, synthetic trajectories are anchored to real match dynamics through two-point boundary-value problems (BVPs) over 13,010 strokes from ShuttleSet. The endpoints x₀ = [X₀, Y₀, Z₀]^⊤ and x_T = [X_T, Y_T, Z_T]^⊤ take their ground coordinates and the flight duration T from the annotation, while the heights Z₀, Z_T are sampled from stroke-specific priors. We solve for the initial velocity v₀ with a fourth-order Runge–Kutta (RK4) shooting method and damped Newton steps:

where Φ(v₀) = x̂(T; v₀) − x_T and J_Φ = ∂Φ / ∂v₀ ∈ ℝ^3×3. The solver converges on 99.9% of problems.

Physical feasibility filtering. Heights drawn from priors alone can yield physically invalid arcs: 10.6% of such solutions pass under the cord. We therefore solve an ensemble of eight height candidates per stroke and keep one that satisfies hard admissibility criteria:

This yields 77,728 physically verified flights, each clearing the net cord by at least 2 cm.

3.3 Shuttle3D: ray-depth transformer

To generalize across cameras, Shuttle3D operates directly in viewing-ray space. Given a 2D detection p_t = [u_t, v_t]^⊤, the unit viewing ray in court coordinates is

where P is the calibrated camera matrix. The floor intersection of r_t on Z = 0 is

where C is the optical center. Each frame is tokenized as a 7D vector [r_t^⊤, X_floor, Y_floor, 1_horiz, t]^⊤, where 1_horiz flags near-horizontal rays whose floor intersections diverge; a global token carries the camera center and the anchors of §3.4. A 6-layer Transformer encoder (d_model = 192, 8 heads) estimates a relative depth d_t along each viewing ray:

where c₀ is the court center. Parameterizing depth along the rays guarantees zero out-of-ray transverse drift (0.000 m) by construction.

Domain-matched noise injection. To bridge the sim-to-real gap, training batches project the synthetic flights through the measured broadcast cameras and corrupt them with noise identified from real match tracking: a per-stroke detection noise σ ∈ [0.8, 12] px drawn from real unconstrained reprojection residuals, 3% outliers (8–40 px), 5% random frame dropouts, contiguous tracking bursts (2–8 frames lost), late acquisition and sudden track ends, and calibration jitter (±1.5 px). Differentiable hinge losses penalize violations of the court rules on the network’s output during training.

3.4 Uncertainty-aware spatiotemporal floor anchors

A single viewpoint cannot resolve axial depth, but the players’ foot contacts with the floor supply natural physical constraints at the trajectory endpoints. We extract spatiotemporal floor anchors from broadcast video with off-the-shelf 2D pose estimation (Figure 3b):

• Striker ankle midpoint (t = 0). Projected by the homography H onto the floor, it locates the launch horizontally.
• Receiver ankle midpoint (t = T). The ground position of the defending player at arrival locates the end of the stroke horizontally.
• Stationary landing point (z = 0). Extracted by temporal clustering on rally-final frames.

Precision-aware encoding. Because players jump and lunge, trusting pose detections blindly corrupts ballistic predictions. We append an 18D global token to the Transformer sequence: the camera center (3), the three anchors with presence bits (9), a launch-height slot with its presence bit (2), and a declared uncertainty σ for each anchor (4). During training σ is sampled over 0.10–1.50 m horizontally and 0.05–0.40 m in height, with synthetic anchor noise scaled to the declared σ. This forces the network to balance visual evidence against anchor confidence, treating anchors as soft contextual tokens rather than rigid constraints. An undetected player enters as an absent anchor (30% of training strokes carry none), so one model serves any subset of anchors. The difference between soft and hard use is measured: on 6,553 broadcast strokes with anchors from a real pose detector, anchors fed as tokens leave the rectifier’s feasibility at 95.9% (96.0% without anchors), whereas imposing them as hard constraints drops it to 79.8%.

3.5 Null-space feasibility rectification

Raw per-frame ray-depth predictions lack explicit temporal dynamics and occasionally violate physical boundaries (e.g., net penetration or floor clipping). We formulate a second-tier rectifier that decouples physical legality from visual fidelity. Given the raw 3D predictions P̂_t, we fit an aerodynamic launch state s₀ = [x₀^⊤, v₀^⊤]^⊤ by non-linear least squares,

with a robust loss ρ, subject to the hard court rules

together with the hall bounds; the last condition applies to every stroke except the last of a rally, since a player cannot strike across the net.

Corrective projection in the visual null space. When the unconstrained fit violates a rule, corrective steps are restricted to the 2D subspace spanned by the eigenvectors [v₅, v₆] of the two smallest eigenvalues of the Gauss–Newton curvature matrix J^⊤J, where J = ∂π / ∂s₀:

As Figure 4 shows on one broadcast stroke, this subspace follows the camera’s viewing rays, where metric shifts barely change the image: the projection lifts the net crossing from 0.97 to 1.55 m while the reprojection error changes only from 7.21 to 7.37 px. Across 6,553 tournament strokes, rectification costs a median of 0.00 px (90th percentile 0.76 px). Strokes with no feasible launch state within a 14 px χ² reprojection budget are explicitly flagged rather than hallucinated. The rectifier buys feasibility, not accuracy (§5.4).

4. Observability and Theoretical Estimation Bounds

To characterize when monocular observations suffice to recover metric ballistic depth, we develop a projective observability formulation that connects image-space trajectory curvature to the Fisher Information Matrix (FIM) and the Cramér–Rao lower bound (CRLB).

4.1 Möbius invariance of unaccelerated projective motion

Consider an unaccelerated 3D trajectory in camera coordinates, X_c(t) = X₀ + Vt. Under pinhole projection with focal length f and principal point [c_x, c_y]^⊤, the image coordinates follow rational linear trajectories:

where {a₁, b₁, a₂, b₂, c} define a 1D Möbius transformation with five degrees of freedom. Any constant-velocity 3D line therefore projects as a Möbius curve in pixel space, and any measurable deviation from it is the sole empirical signature of the acceleration induced by gravity and drag.

4.2 Image-computable observability number N

We quantify the signal-to-noise ratio of trajectory curvature directly in image space, without 3D ground truth. The dimensionless observability number N is the root-mean-square residual of the observed 2D track from its best-fit Möbius projection, normalized by the detector noise σ_2D:

N registers the magnitude of projected dynamic curvature relative to tracking uncertainty. As N → 0, the observed track becomes indistinguishable from an unaccelerated ray and metric ballistic depth becomes unobservable.

4.3 Fisher information and dual geometric degeneracies

Under additive Gaussian tracking noise, p_t ∼ 𝒩(π(P(t; s₀)), σ_2D²I), the FIM over the 6D launch state s₀ = [x₀^⊤, v₀^⊤]^⊤ is

For any unbiased estimator, the covariance of the reconstructed 3D positions is bounded by the CRLB:

Dual degeneracy mechanisms. Singular value decomposition of I(s₀) over 3,993 synthetic strokes shows that the right singular vector of the smallest singular value σ_min aligns with the camera ray to the launch point within a median angle of 1.9° (<10° on 89.0% of strokes). The ray-alignment angle and log N are uncorrelated (r = −0.00), while the angle explains only part of the ill-conditioning (r = +0.35 with log σ_min). Monocular ballistic lifting therefore suffers from two independent degeneracies: (i) optical collinearity with the viewing ray and (ii) short temporal observation arcs. N registers both.

4.4 Bias–variance trade-off: learned priors versus the CRLB

Across 7,994 synthetic strokes, N predicts the CRLB through an empirical power law (Figure 5):

The partial correlation remains strong (ρ = −0.59) after controlling for the number of tracked frames. The bound falls steeply with N, from a median of 4.3 m below N = 1.5 to 0.3 m at N ≈ 15–20, and it is heavy-tailed at the low end: 1.9% of strokes exceed 5 m (up to 61 m). Of these, 95% have N < 2, an image track that is almost a constant-velocity line, and 45% are tracked for ten frames or fewer; they are mostly rushes and drives near the net. For such a stroke the image carries essentially no depth information.

Prior-dominated and image-dominated regimes. The bound delineates two operating regimes:

• Image-dominated regime (high N). On the four stroke types with the highest median N (lift 27.0, clear 23.0, cross-net 15.3, defensive lift 14.9), visual curvature is rich enough for an unbiased physics fit to compete: on 400 synthetic strokes batch MAP is more accurate than Shuttle3D on lifts (0.136 vs. 0.279 m) and clears (0.139 vs. 0.309 m; Supp. §S2.1). Shuttle3D stays above the bound there (ratio 1.51–2.55), paying the variance that the tracking noise demands.
• Prior-dominated regime (low N). On fast axial strokes (tap smash, rear drive, smash, rush and drive; median N ≤ 4.2), visual curvature collapses into tracking jitter and the CRLB exceeds 1 m, reaching 2.54 m on drives. Shuttle3D’s error sits well below the unbiased bound (ratios 0.23–0.65; 0.23× on drives), and over all 15 stroke types it falls below the bound on nine (Supp. Table S1). This does not contradict information theory: the CRLB bounds unbiased estimators only. A network trained on a stroke distribution is a biased estimator; under severe visual ambiguity it trades variance for bias and regresses toward typical tactical trajectories.

Per stroke rather than per type, binning 602 held-out synthetic strokes by N locates the transition between the regimes: the ratio of bin-median error to bin-median CRLB crosses 1 at N ≈ 17 and 1.2 at N ≈ 22.

A design-time diagnostic. N is a design-time diagnostic rather than a deployment filter. Computed directly from 2D pixel coordinates, it lets practitioners evaluate camera placements and identify the shot classes that need auxiliary constraints, such as floor anchors, before a 3D lifter is trained or deployed. It is not a per-stroke quality flag: on 6,928 real broadcast strokes its rank correlation with endpoint error is ρ = +0.13, the opposite sign to the one theory predicts, so we do not use it that way.

5. Experiments and Results

5.1 Experimental setup

Datasets. We evaluate on three benchmarks. (i) The ShuttleSet broadcast corpus: 6,553 tournament strokes from 10 broadcast cameras, with TrackNet detections and annotated player ground positions. (ii) The TrackNetV3 benchmark: 2,592 interior strokes from 22 international venues with frame-level 2D annotations, calibrated from broadcast video by §3.1 (19 venues with no human input). (iii) The multi-view 3D reference: an independent ten-camera indoor capture (48 sessions, 2,083 rallies) whose 3D flights are reconstructed by a learned multi-view method. It is temporally smoothed (a 1–2 cm floor) and was recorded in a training hall, so we treat it as a reference rather than exact ground truth, and to prevent leakage it serves strictly as a test benchmark: 2,682 strokes from 7 sessions are held out from every fit. Model inputs are restricted to what one broadcast camera provides. The reference flights are projected through our 32 measured broadcast cameras (σ = 2 px, 5% dropout), and player anchors are simulated at detector grade: the reference player’s ground position plus a residual drawn from 7,763 residuals of our broadcast pose pipeline on ShuttleSet (median 0.76 m, wrong-player picks included), with 5.2% dropped as undetected.

Evaluation metrics. (i) Physical feasibility rate: the fraction of strokes satisfying every hard court rule — net clearance, the hall and ceiling bounds, floor non-penetration, and crossing the net on every stroke but the last of a rally. (ii) Metric 3D error: median over strokes of the per-stroke median Euclidean error against the multi-view reference. All primary accuracy claims rest on this metric. (iii) Endpoint consistency: horizontal distance between the reconstructed endpoints and the annotated ground positions of the hits on ShuttleSet. The annotation marks a clicked floor position near a player, not the shuttle in the air, so even a perfect reconstruction does not score zero, and one biased toward the players scores better than the truth (Supp. §S2.2). It separates methods whose endpoints are metres apart; it is not an accuracy metric. Naming. SimTrack3D is the full pipeline (calibration, Shuttle3D, foot anchors and the rectifier); Shuttle3D is the network alone, trained only on simulation. Accuracy against real 3D is reported for the network, because the rectifier buys feasibility rather than accuracy (§5.4).

5.2 Benchmarking on real multi-view 3D

We benchmark SimTrack3D against existing monocular paradigms on the 2,682 held-out multi-view strokes (Table 1): physics fitting — the official MonoTrack^[5] code, a non-linear least-squares fit of launch state and drag that takes the hitter’s and receiver’s court positions, given the same foot anchors as ours — and learned sim-to-real lifters — the official Uplifting Table Tennis^[8] code, retrained on our simulated trajectories and measured cameras with badminton court and net-post keypoints in place of its table keypoints and its spin loss switched off; and TrajTrans^[10], which regresses launch parameters from the pixel track and four court corners and which we implement on the same data because no official code is released. Both learned baselines are trained exactly as long as Shuttle3D: 12,120 gradient steps at batch 256 on the same simulated strokes.

On 400 of these strokes, Shuttle3D’s error is 0.450 m along the line of sight and 0.024 m across it (Supp. Table S5).

Physics fitting fails on these flights even with the player positions: MonoTrack’s median error is 1.31 m, three times Shuttle3D’s 0.42 m with the same anchors, and 68.9% of its strokes are more than 1 m wrong. Shuttle3D reaches 0.466 m without anchors, ahead of TrajTrans (0.582 m) and the official Uplifting model (0.840 m, seed 0). Detector-grade foot anchors, a channel neither learned baseline has, improve Shuttle3D to 0.417 m and cut gross failures (>1 m) from 24.4% to 16.2%. Without anchors every learned lifter reverses the depth of 9–12% of strokes — reconstructing a flight as approaching the camera while it recedes — and the anchors remove most of them (2.5% for Shuttle3D). All learned models are reported at seed 0; over three seeds Shuttle3D varies by 0.007 m without anchors (0.459–0.466 m) and by 0.015 m with them (0.417–0.432 m). Figure 6 shows a broadcast stroke and a held-out reference stroke from several viewpoints, with TrajTrans, the official Uplifting and the official MonoTrack run on the same 2D track and camera: on this stroke all three drift along the line of sight.

5.3 Real tournament broadcast evaluation

On the 5,638 ShuttleSet strokes that all four methods attempt and that carry an arrival annotation, we score every method under three protocols to rule out selection bias (Table 2): (i) the certified subset that SimTrack3D’s physical gate accepts, (ii) all strokes with their own output, and (iii) all strokes with every flagged or physically impossible flight charged a harsh 7.35 m. MonoTrack is given the hitter’s and receiver’s detected court positions, as its design requires; the learned baselines take no anchors; SimTrack3D is reported both without anchors and with the same detected feet.

SimTrack3D returns a physically possible flight on 97.3% of these strokes, against 88.2% for TrajTrans, 78.8% for Uplifting and 54.4% for MonoTrack, and charging its flagged strokes the penalty moves its median by at most 0.01 m, so the gate does not win by discarding hard cases. The consistency columns rank the methods differently from real 3D. With the same detected feet SimTrack3D is the most consistent at the arrival end (0.54 m, p90 1.89 m), but without anchors it is slightly less consistent than TrajTrans (0.61 vs. 0.57 m), although TrajTrans is 25% less accurate against real 3D (Table 1: 0.582 vs. 0.466 m). This is the bias §5.1 describes: the annotation marks where the players stand, and a reconstruction pulled toward the players scores better than the true flight (Supp. §S2.2). The same holds for MonoTrack, whose fit pulls the end of the flight toward the receiver’s detected feet: at the arrival end it is within 0.35 m of SimTrack3D, at the launch end, which it is not pulled toward, it is 2.04 m away against 0.81 m. Under coverage sweeps in which each method may abandon strokes by its own physics-fit residual (Supp. Figure S2), MonoTrack’s residual barely selects, and at its own possibility flag it remains behind SimTrack3D at full coverage. The EKF-family estimators are reported in Supp. §S2.1.

5.4 Representation ablation and the sim-to-real gap

Viewing rays versus 3D positions. Table 3 evaluates four alternative parameterizations under identical Transformer backbones on seen and unseen cameras. Pixel-input models that regress 3D positions appear competitive on seen cameras (0.329–0.341 vs. 0.388 m) because they memorize venue-specific depth priors, but they degrade on unseen cameras (0.579–1.408 m). Our ray-depth parameterization keeps the same accuracy on seen (0.388 m) and unseen cameras (0.387 m), with zero out-of-ray drift by construction.

Line-of-sight error decomposition. Against the multi-view reference, Shuttle3D’s error is 0.450 m along the line of sight and only 0.024 m across it, a 19-fold difference (Supp. Table S5). In this protocol, where the 2D track is the reference flight with 2 px noise seen through a calibrated camera, the track and the calibration contribute little of the error; depth along the optical axis is the bottleneck, as §4 predicts.

The synthetic-to-real disparity. Synthetic validation understates real 3D error for every non-trivial method (Supp. Table S5): by 1.4× for Shuttle3D (0.329 → 0.452 m), 2.1× for MonoTrack (0.668 → 1.408 m, with player positions) and 3.0× for the physics fit without priors (0.596 → 1.802 m). The ranking changes with it: the physics-prior and full estimators, the best synthetic cells (0.268 and 0.267 m), lose to Shuttle3D alone on real flights (0.638 and 0.642 vs. 0.452 m), because the fixed-drag flight model itself does not fit real flights closely (median RMS 0.10 m when fitted directly to the reference). With rally-chained inference the rectifier costs 1–3 cm with a tight prior and 9–16 cm with the deployed prior (Supp. §S4.5). Physical feasibility guarantees kinematic legality, not metric precision. Refitting the drag law to the real flights (terminal velocity 6.04 m/s) and retraining does not change the monocular error either (Supp. §S4.2).

5.5 Stress testing: airborne smashes and input degradation

Airborne jump smashes. Floor anchors come from the players’ feet, yet smashes are struck in mid-air. On the 2,682 held-out strokes we measure lift as the striker’s pelvis height at contact minus that player’s standing baseline. Counterintuitively, jumping does not degrade horizontal anchor quality: the striker-to-shuttle distance falls from 1.21 m on grounded strokes to 0.78 m on airborne ones (lift > 0.15 m; correlation −0.48 across lift), because a jumping player gets directly beneath the shuttle. Foot anchors improve airborne strokes from 0.516 to 0.487 m (Table 4); these strokes carry more error with or without anchors because they are short, low-N arcs. On broadcast footage the real pose detector does degrade on jump smashes (1.05 vs. 0.76 m from the annotated player position), which the declared uncertainty absorbs (Supp. §S5).

Tracking loss and anchor uncertainty. We also degrade the inputs at inference (Supp. Table S12). Randomly discarding up to 50% of the tracked frames raises the anchored error only from 0.417 to 0.451 m. A contiguous 8-frame tracking collapse gives 0.442 m with anchors, against 0.523 m without them. Declaring additional anchor error honestly through the precision channel keeps the anchors useful up to about 2.0 m of injected noise before they cross the unanchored baseline, against about 1.0 m when the same error is not declared (Supp. Figure S5).

5.6 Computational runtime

On one NVIDIA RTX 3090, 2D shuttle detection runs at 224 fps (TrackNetV4, 4.46 ms/frame) and player pose estimation at 50 fps (19.81 ms/frame). Shuttle3D lifts 6,100 strokes/s in batch mode (0.16 ms/stroke), and venue calibration takes 11.7 s once per tournament. In batch processing the rectifier handles 32 strokes/s (31.3 ms/stroke), giving an end-to-end throughput of 5.8× real time for broadcast footage (1.3× with player pose). The rectifier is throughput-bound and not an interactive component, and the host carried unrelated load during timing, so these are upper bounds (Supp. Table S14).

6. Discussion and Limitations

Camera rigidity. SimTrack3D assumes that the main broadcast camera stays static during active play. Across 27 broadcast rallies from 9 international venues, masked affine ECC alignment against the first frame gives a median camera drift of 0.034 px (90th percentile 0.064 px, maximum 0.40 px), comparable to the measurement’s own noise floor (0.022–0.040 px). Broadcast cuts and zoomed replays fall outside active rally segments. To support actively panning (PTZ) cameras, per-frame planar court registration can be applied while retaining the sag-calibrated focal length.

Residual depth uncertainty at stroke boundaries. Contact frames are the seam between consecutive trajectory solves, where motion parallax is weakest. At 58 contact frames in 8 venues, a body fitted from the broadcast places the striking wrist 0.52 m from the viewing ray through the detection, consistent with arm and racket length, but the reconstructed depth along that ray scatters with an interquartile range of −1.11 to +1.65 m. Mid-flight depth is tightly bounded, whereas depth at the moment of impact carries an uncertainty of about a metre (Supp. §S5.1).

Aerodynamic model sensitivity. Simulation and estimation share a quadratic drag law with v_T = 6.8 m/s. Perturbing v_T by ±15% inflates the mean (not median) 3D error by up to 63% (0.471 → 0.769 m) while leaving physical feasibility nearly unchanged (98.7% → 96.5%): kinematic feasibility enforces physical boundaries, but metric accuracy depends on aerodynamic calibration.

Scope and generalization. The multi-view reference was recorded in a training hall, whose lighting and playing styles differ from premier tournament venues; because only one of its rallies has video, its player anchors are simulated at a broadcast detector’s measured error level, and its 2D tracks are projections with 2 px noise rather than tracker output. Our geometric decoupling relies on the elevated net cord and its catenary sag. It should extend to other net-divided sports (tennis, volleyball, table tennis); flat-field sports such as soccer or basketball would need other non-coplanar static structures, such as goal posts or backboards. The full discussion is in Supp. §S7.3.

7. Conclusion

Recovering metric 3D projectile trajectories from monocular sports broadcasts is an extreme vision problem in which visual scale, ground impacts and motion parallax degrade at once. We have presented SimTrack3D, a self-contained sim-to-real framework that reconstructs 3D projectile kinematics solely from cues native to standard broadcast video. It resolves monocular depth ambiguity through a series of geometric and physical invariants:

1. a non-coplanar net calibration, verified by an invariant 26 mm catenary sag, fixes the focal length that planar court homographies leave ill-conditioned;
2. Shuttle3D, a ray-depth transformer trained entirely on 77,728 boundary-value simulated flights, parameterizes depth strictly along viewing rays and conditions its estimate on detected player foot positions with declared uncertainty;
3. a null-space feasibility rectifier moves invalid flights into the court rules only along the weakly observed eigenspace of J^⊤J, at a median reprojection cost of 0.00 px.

Evaluated against an independent ten-camera real-3D capture (2,682 held-out strokes), Shuttle3D attains a median 3D error of 0.42 m, ahead of physics-fitting and learned baselines, while SimTrack3D achieves 95.9% physical feasibility on 6,553 tournament strokes. The image-computable observability number N grounds the system by predicting the Cramér–Rao lower bound (CRLB ∝ N^−0.98) from image pixels on synthetic strokes. By removing the need for multi-camera infrastructure or manual 3D annotation, SimTrack3D opens decades of archival sports video to quantitative performance analysis, automated tactical coaching and immersive 3D broadcast visualization.

BibTeX

@misc{chang2026shuttle3d,
  title  = {SimTrack3D: Monocular 3D Trajectory Reconstruction of Fast-Moving
            Sports Projectiles via Broadcast-Calibrated Sim-to-Real Transfer},
  author = {Chang, Gino},
  year   = {2026},
  note   = {Project page}
}

Supplementary Material

Sections, tables and figures carry an S prefix. S1 extends the observability analysis; S2 gathers the protocol-fairness evidence behind Table 2; S3 the net-cord and calibration checks; S4 the validation against the multi-view reference and the drag law; S5 airborne strokes, input degradation and failure modes; S6 the image-level synthesis tools and runtime; S7 further ablations and the extended discussion. Main-text results come from the final model, trained without any 3D measurement and tested with monocular inputs. Some supplementary analyses were run on earlier model generations or deliberately use real 3D to probe the simulator and the anchors; each such table says so in its caption or in the note that opens its section (§S4, §S5).

S1 Extended observability and the Cramér–Rao bound

Per-type values behind §4.4 and Figure 5, and the profile likelihood behind Figure 1(b).

S2 Protocol fairness, baselines and coverage

S2.1 Baseline comparison details

We benchmark against state-of-the-art monocular projectile reconstructors and classical estimators. MonoTrack^[5] optimizes a per-shot launch state and drag coefficient under an L6 reprojection loss plus a soft pull of the flight’s end toward the receiver’s court position; we run its official code (github.com/jhwang7628/monotrack, commit 39e9227) unmodified. Iterated EKF + RTS smoother combines extended Kalman filtering with Rauch–Tung–Striebel backwards smoothing over identical aerodynamic models. Batch MAP performs global maximum a posteriori optimization over the complete temporal frame sequence.

Every baseline is given our calibration and our tracks. All methods consume the identical 2D tracks from the identical calibrated cameras. The EKF family and our physics ablations integrate the same quadratic-drag model through the same RK4 integrator that generates our training data. MonoTrack runs its own official solver — forward Euler with 30 substeps and a per-stroke fitted drag coefficient — with its camera matrix fitted by its own DLT to the four court corners and two net-post tops of our calibration, and its player positions taken from the same pose detections SimTrack3D receives (where a player is undetected, the first or last detection back-projected to MonoTrack’s own 1.7 m launch-height guess). Two properties of the official code are kept as they are: it compares each observed frame with the state one frame after the one it outputs for that frame, and it interpolates missing detections before fitting. Correcting the one-frame offset would change its median error on the multi-view strokes from 1.31 to 1.34 m. Within that frame the rows span four distinct families. Physics-fit: MonoTrack is its official per-stroke seven-parameter fit, drag included; physics-only and physics-with-prior are strict ablations of our own system — the same launch-state parameterization with the null-space search and the anchors removed, which isolates what the geometry contributes from what the prior contributes. Classical recursive estimation: iterated EKF and batch MAP are strong, properly tuned implementations, not straw men — on a noise-free sanity trajectory they recover the flight to 0.0014 m, and on high-N stroke types they beat us, which is the behaviour an efficient unbiased estimator should show and is reported as such below. Trivial lower bounds: constant depth and the ground-plane assumption, which bracket how much of any result is geometry rather than method. Learned alternatives: the representation ablation of §S7.1 retrains our own backbone under four different input spaces, and an end-to-end image-to-3D variant is reported separately. We do not tabulate TT3D or TT4D^{[3, 4]}: each derives its metric scale from a table bounce that badminton does not have, so porting them here would require replacing the component that carries their accuracy, and the resulting number would measure our substitution rather than their method. What we do port is the learned lifting component: the official Uplifting Table Tennis code, retrained on our data with badminton keypoints (main Table 1), and TrajTrans, which has no released code and which we implement ourselves (§S2.2).

What the launch-error column can and cannot be read as. Launch error here is the horizontal distance from the reconstructed launch point to the annotated hit point, and that annotation is a ground coordinate — where the striking player stands, not the airborne shuttle. Contact happens an arm and a racket away from it, so the column has a floor: measuring the racket head directly from per-frame body fits over 68 contacts in 8 venues puts the true contact point a median 1.30 m from the same annotation (IQR 0.67–1.83). Every row of Table S2 carries that floor, so gaps of a metre or more (MonoTrack at the launch end against us) stand; gaps smaller than the floor do not, because a reconstruction biased toward the player can beat the truth on this column (§S2.2). Our 1.00 m is below the floor, which means only that a launch point pulled toward the player's feet scores well here — not that it is accurate to a metre. §S2.5 scores the launch end against references that are actually at the launch point.

The arrival column has the same floor, for the same reason. Because the dataset annotates the end of one flight and the start of the next as a single point, a stroke's arrival reference is the next hit annotation: we find the two identical to the millimetre on 92.8% of consecutive pairs. Only 10 of 6,548 strokes (0.2%) are the last of their rally, where arrival is a genuine floor landing; for the other 99.8% the reference is the receiving player's court position while the shuttle is at their racket. The floor is therefore the same 1.30 m measured above, and our 0.59 m mid-rally arrival error sits below it too. Neither column should be read as a distance to where the shuttle was.

As reported in Table S2, the physics-fitting baselines struggle on real tournament footage: the official MonoTrack, given the detected player positions, yields physically plausible flights on 49.5% of strokes; its arrival endpoints sit a median 0.93 m from the annotated receiver, toward whose detected feet its fit pulls them, while its launch endpoints miss the hitter by 2.05 m.

Crucially, Batch MAP outperforms Shuttle3D on the two highest-N stroke types where sensory evidence is rich (lift: 0.136 m vs. 0.279 m; clear: 0.139 m vs. 0.309 m). This directly corroborates our theoretical prediction: an efficient unbiased estimator attains the CRLB when sensory information is uncompromised. However, on fast smashes where N collapses, Batch MAP deteriorates severely (1.264 m vs. 0.229 m for Shuttle3D), achieving only 22.4% valid flights on real footage. In contrast, SimTrack3D delivers 97.4% physical feasibility on this benchmark subset, achieving an arrival error of 0.57 m.

S2.2 Learned sim-to-real lifters, and what the annotation score rewards

The Uplifting-style rows of this subsection are our own reimplementation, kept as a representation ablation; the comparison in the main text (Table 1) uses the official Uplifting Table Tennis code retrained on our data.

Both learned baselines are re-implementations: no code is released for TrajTrans, and the Uplifting Table Tennis code is built around a table. Every arm trains on the same anchored synthetic set, split, observation-noise model, rule hinge loss and schedule as Shuttle3D (60 epochs, d = 192, 6 layers). At test time the baselines receive keypoints projected by the true calibrated camera, which favours them. Uplifting-style takes per-frame pixel coordinates plus 14 court and net-post keypoints, uses sinusoidal time and sudden-end augmentation, and outputs per-frame 3D. Arm A samples cameras at random from the published range; arm B uses our 32 measured cameras. TrajTrans-style maps pixels plus four court corners to six launch parameters that an RK4 integrator unrolls. Arm C uses the four corners; arm D adds the two net-post tops.

The two representations fail differently. Measured against the true shuttle at the first and last tracked frames of each reference stroke (three seeds each), the Uplifting-style lifter is closer at launch and Shuttle3D is closer at arrival and over the stroke as a whole:

The pulled rows are the point of the table. A flight whose endpoints are wrong by 0.56 / 0.68 m scores 2× better against the players than the true flight does, so a player-annotation score can prefer a biased answer to the correct one. The ShuttleSet annotations are clicked closer to the shuttle than a pelvis is, so their floor is lower than the 1.1–1.4 m measured here, but it is not zero and it cannot be measured without the true shuttle. Between the two lifters, the player score on the reference ranks them the same way their true endpoint errors do (Uplifting-style ahead at launch, Shuttle3D ahead at arrival), so annotation bias alone does not account for ShuttleSet ranking the Uplifting-style lifter ahead at arrival. One mechanism we can confirm is the output head: giving Shuttle3D the same unconstrained 3D output moves its ShuttleSet arrival score from 0.660 to 0.592 m and its raw feasibility from 90.1% to 96.0%, while its reference error grows from 0.482 to 0.525 m (reference sessions outside the 2,682-stroke test set, one seed). An estimate allowed off its viewing ray can clear the net cord by lifting a point a few centimetres, which an estimate on the ray cannot do (§S3.1); the image no longer agrees with it, and the reference says it is less accurate.

S2.3 Every stroke is scored: full analysis

A reader is entitled to ask whether 96.0% feasibility and 0.60 m arrival consistency describe the same strokes — whether, in effect, the 4% we flag are the 4% we could not do — and whether the baselines were ever offered the same right to give up. This subsection answers both (Table S3, Figure S2), and the answer to the first is that the gate is worth one to two centimetres, not the gap to the baseline.

The premise of the objection does not hold in our pipeline. The feasibility flag marks a stroke; it does not delete it. Every flagged stroke retains the reconstruction Tier 1 and the rectifier produced, so a full-sample number exists and can simply be read off. We evaluate it three ways (Table S3) on the intersection of every method's output (6,285 strokes, smaller than 6,553 because MonoTrack does not attempt rally-final strokes; 5,622 of them carry an arrival annotation and 5,713 a launch annotation).

(a) Common subset. Scored only on the strokes we certify, SimTrack3D reaches 0.60 m against 0.88 m for the official MonoTrack (which, unlike this row, is given the receiver’s detected feet) and 0.89 m for our own unrectified Tier 1 output. The ordering is therefore not an artefact of which strokes each method happened to solve.

(b) Penalised full sample. Keeping the flagged strokes' own output moves our median from 0.60 to 0.61 m; charging every flagged stroke a fixed 7.35 m — half the court diagonal, a deliberately harsh stand-in for "no answer" — moves it to 0.62 m. The flag is nonetheless informative rather than arbitrary: the 142 strokes it marks (2.5%) have a median error of 3.45 m against 0.61 m overall, so it is selecting genuine failures. It simply is not carrying the comparison. The same penalty applied to the official MonoTrack's 45.5% physically impossible strokes raises its median from 0.89 to 1.92 m.

(c) Coverage–error curves. The cleanest form of the question is to give every method the same right to abandon and sweep it. Figure S2 rejects strokes by an inference-time score that uses no annotation — the unconstrained physics-fit residual, which each method computes on its own output. Our curve falls from 0.612 m at full coverage to 0.590 at 96%, 0.564 at 90% and 0.542 at 80%; the oracle ordering (by true error, unavailable at inference) reaches 0.473 at 80%, and random rejection is flat, so the residual is a weak but real rejector for us (Spearman 0.26). The official MonoTrack, given the identical mechanism and the receiver’s detected feet, starts at 0.89 m and does not improve as it abandons strokes (0.86, 0.86, 0.88, 0.95 m at 96/90/80/50% coverage): its residual does not rank its own errors (Spearman −0.01). Its own physical-possibility flag keeps 54.5% of strokes at 0.83 m. At no coverage does it reach what SimTrack3D achieves at 100% (0.61 m without anchors, 0.54 m with the same feet).

Two honest limits of this analysis. The Tier-1 row comes from an earlier Tier-1 generation than the final model, labelled as such; Table S10 shows the generations agree to within 2 cm after rectification. And the EKF family was only ever run on 1,200 real strokes, so its row is on its own subset and is not row-comparable with the others.

S2.4 Leave-one-anchor-out real-world validation

Because real footage lacks dense 3D ground truth, we hold out each anchor in turn and measure how consistent the reconstruction stays with the annotated floor positions; these are consistency checks, not accuracy measurements.

Table S4 shows that the floor anchors bring the reconstructed endpoints closer to the annotated positions. Enforcing all anchors as strict hard inequalities during rectification reduces overall feasibility to 79.8%, reflecting noise in visual pose detections, whereas feeding the detected feet to the network only keeps it at 95.9%. Consequently, our final pipeline treats anchors as soft tokens in Tier 1 and reserves hard constraints in Tier 2 strictly for court boundary rules.

What the medians in Table S4 are taken over. The launch and arrival columns are medians over the strokes the rectifier certifies feasible in that configuration — the same convention as Table S2, and the reason the row n varies with the anchor set. This is a reporting convention, not a selection: a flagged stroke keeps its reconstruction in the output file, and nothing is deleted. §S2.3 repeats the entire comparison with every stroke scored, under two different treatments of the flagged minority, and under a rejection curve that grants the baselines the same right to abandon. The conclusion there is that the convention is worth 1–2 cm. The rows here are the final Tier-1 model; Table S10 reports earlier generations through the same rectifier, and they agree to within 2 cm and one percentage point.

S2.5 Scoring the launch end against something that is at the launch end

The launch-error column above is measured against a ground annotation, which bounds it away from zero (§S2). Three alternative references were tried. One of them turns out to be an identity and is reported here so that it is not mistaken for a result.

The viewing ray is an identity for this model. The obvious choice is the perpendicular distance from the reconstructed launch point to the first tracked viewing ray — the two directions the pixels constrain. Over all 6,548 strokes it is 0.000 m, to a maximum of 2.2×10⁻¹⁴. Shuttle3D predicts depth along the detected ray, so its output lies on that ray by construction. The check is vacuous, in the same way that reprojection error is vacuous for any method whose horizontal position is solved from the detected pixel. It does carry one real consequence: the launch error of this model is entirely a depth error, with no transverse component to trade against. (The whole-track pixel residual is not vacuous — a median 11.6 px — because the flight is an integral and only its first point is pinned to a ray.)

The racket head. The one reference physically at the contact point is the striker's racket, which the per-frame SMPL-X fits of §S6.2 place in 3D. Over 68 contacts in 8 venues, the reconstructed launch point sits a median 1.31 m from the fitted racket head in 3D (IQR 0.67–1.83), of which +0.31 m is a signed height error: the reconstruction places contact at a median 1.62 m while the fitted racket head is at 1.28 m. That bias has a direction, which the ground-annotation metric cannot see at all. The reference is itself an estimate — the racket head is taken as the wrist plus 0.435 m along the forearm, which assumes the face follows the forearm, and the fits reproject to a median 3.5 px — so this bounds the launch error rather than resolving it, and it is a pilot: extending it to the full corpus needs a body fit per stroke.

Physical priors on the launch state. The third reference needs no reference point at all, so there is no registration to get wrong: it asks only whether the reconstructed launch state is something a player can produce. Launch height is healthy. Overhead stroke classes come out at a median 2.13 m against 0.96 m for underarm ones (n = 1,873 and 4,186) — the right ordering, separated by 1.17 m — with 1.2% above the 3.3 m a jumping smash can reach and 0.5% below the floor.

Launch speed is where a real bias shows, but only once both sides are conditioned on the same shot. Comparing our per-class median against the per-class median of an independent boundary-value solve is not like-for-like, because the two class labels cover different populations: the solver's “drop” travels a median 7.0 m while the dataset's net drop travels 3.1 m, and that 2.3× difference in distance alone produced an apparent 2.5× disagreement in speed. Restricting both sides to strokes within 1.0 m and 0.12 s of the same target removes it:

Three classes agree. The smash does not: our conditioned median sits below both the independent solve and the lower bound that follows from the drag model alone, and 64.2% of conditioned smashes fall under that bound, against 14.8% for net drops. We report this as the one class where all three references point the same way rather than as a quantified error, because the bound has its own false positive rate — it flags 14.8% of the net drops on which our estimate and the independent solve in fact agree. A mechanism we expected to explain it does not survive its own test: the smash has the shortest observed arc, so we predicted a prior-pulled slow bias that grows as the arc shortens, but the correlation between tracked length and reconstructed speed changes sign across classes (−0.18 smash, +0.30 clear), and within a duration-conditioned set tracked length is largely residual duration, which makes the relationship arithmetic rather than evidence. The bias is real and its cause is open.

S3 The net cord and calibration

S3.1 Diagnostic: net-cord clearance boundary

As illustrated in Figure S3, on real footage 8.9% of unconstrained reconstructions violate net clearance. Our diagnostic confirms that this is not an altitude estimation failure.

• Timing vs. height error. An image ray intersects the known net plane at exactly one height; clearance is determined solely by the estimated frame of net crossing.
• Boundary sensitivity. 89.7% of cord violations are resolved by a uniform depth scaling of merely 1.0% (median shortfall 0.11 m). Net clearance is a high-sensitivity boundary test decided at the third significant figure of depth.
• Real vs. synthetic margin. In tournament play, real players execute net shots clearing the tape by a median of 0.18 m, whereas synthetic BVP generators produce clearances averaging 0.37 m.
• Net-hit trajectories. A ray-plane certificate reveals that 10.3% of apparent violations could not clear the tape at any depth; these represent shots terminating in the net, where the clearance rule is inapplicable.

S3.2 Calibration: automation level and camera stability

Automation level, counted rather than claimed. Of the 32 cameras used throughout this paper, 29 need no clicking at all — 10 ShuttleSet venues, whose court homography ships with the dataset, and 19 TrackNetV3 venues solved from video alone — and 3 TrackNetV3 venues pass only when the ICP stage is initialized from four manually clicked court corners. Every stage is automatic in isolation: line detection is a ridge response registered to the known court specification; net-cord detection is a scan over candidate focal lengths, each of which fully determines the cord's projected curve, scored by ridge support; and the acceptance gate combines sag with a court-coverage test. Only initialization fails on those three venues, and the gate that admits them is bit-identical to the one used on the other 29. Median wall-clock cost is 11.7 s per venue, amortized over every rally that venue contributes.

Is the broadcast camera actually static? The pipeline calibrates once per venue, which presumes the main rally camera does not pan or zoom within a rally — the same assumption made by prior broadcast badminton work^[5], but usually left untested. We test it. For each of 27 rallies (the three longest from each of 9 venues) we take 24 frames spanning the clip, build a per-clip temporal-median background, mask to static structure only, and align every frame to the first with a masked affine ECC warp; drift is the RMS displacement of a grid over the central 80% of the frame. Across 621 frame pairs the median drift is 0.034 px, the p90 is 0.064 px, and the single worst value in the entire sample is 0.395 px. The method's own noise floor, estimated from adjacent frames where no camera motion can have occurred, is 0.022–0.040 px. No rally in the sample shows drift distinguishable from the measurement noise, and every observed value is more than an order of magnitude below the 1.7 px calibration residual it would have to exceed to matter. One venue initially appeared to drift by up to 119 px; inspection showed the first 33–81 frames of those clips are a broadcast replay insert, a different shot entirely, and re-measuring from the start of the continuous rally segment returns them to 0.072–0.119 px. A cut detector, not a PTZ model, is what such clips need. We therefore treat the camera as fixed within a rally, with the caveat that this is a measurement on nine venues of one broadcast style rather than a property of broadcast video in general; a genuinely tracking camera would require per-frame registration, which our formulation accepts but which we have not needed to run.

S4 Validation against the multi-view 3D reference and the drag law

Why this section exists. Every real-domain number above is measured against the floor, because broadcast footage carries no 3D ground truth in mid-flight: our only exact mid-air anchors are the 14 rally-final landings reported in §S7.3. The central claims of this paper — that synthetic test error does not predict real accuracy, and that plausibility is not accuracy — therefore rest on proxies. We close that gap with an independent reference acquired from a ten-camera indoor capture of competitive badminton (48 sessions, 2,083 rallies), reconstructed by a learned multi-view method. We treat it strictly as a multi-view triangulated reference, not as motion capture, and we use it only to audit the monocular system; no multi-view information enters the method.

Which analyses below are tests of the system, and which are not. The main text scores the final model with inputs a single broadcast camera provides, including player anchors at a broadcast pose detector’s error level (§5.1). Several analyses in this section deliberately go further than a deployed system can. §S4.2 fits a drag law to the reference. §S4.3 fine-tunes on it and matches a launch-height distribution to it. §S4.4–S4.5 read player anchors directly from the reference skeletons, choosing the player nearest the true shuttle. These use real 3D by design: they ask what the reference can tell us about the simulator and about the anchor mechanism. None of them enters the final model as data. One design choice was informed by them: the declared-uncertainty anchor channel of §3.4 was adopted after §S4.5 showed an earlier model ignoring anchor precision. The final model is that design retrained on simulation data that uses no reference measurement. The models in Tables S8–S9 are earlier generations, and their anchored numbers are upper bounds on what anchors can do, not results of the system.

Bringing the two world frames together. We solve a 7-parameter similarity from our calibrated court metres to the capture world frame using court white lines only — neither the shuttle nor the players participate. The recovered scale is 1.000, so the reference is already metric. The net cord (1.524 m, never presented to the fit) reprojects at a median of 2.94 px across the rig and 0.0–1.5 px on the eight narrow-angle side cameras, an independent geometric confirmation of the alignment. Independently re-calibrating the two wide-angle end cameras from court lines alone places their centres within 0.55 m and 0.65 m of the capture extrinsics, which were never used in that fit.

Test-set construction. The reference trajectories are resampled from 50 Hz to the 30 Hz of our models and projected through the 32 broadcast cameras measured in §3.1, with the same clean observation model as the synthetic test set (σ = 2 px, 5% dropout; the network itself is trained with heavier, tracker-level noise). The camera distribution is therefore identical to the rest of the paper and the only thing that changes is the trajectory distribution, i.e. the generator itself. Of 12,531 stroke segments, 10,733 survive the visibility filter and are split by capture session into 8,051 for system identification and 2,682 held-out strokes across 7 sessions that enter no fit of any kind. On the held-out set the reference flights clear the net in 97.5% of strokes, stay inside the hall in 100% and never cross the floor; median duration is 32 frames at 30 Hz and median launch speed 17.7 m/s.

Reservations, stated once and carried through. (i) The reference is temporally smoothed: residuals of a fixed-drag fit have lag-1 autocorrelation +0.76 to +0.90 and the curvature is attenuated by α ≈ 0.90, which we estimate as a 1–2 cm noise floor — thirty times smaller than the 0.3–0.9 m errors under test, but not negligible for the flight-law fit, where we correct for it explicitly. (ii) The venue is a training hall, not a tournament arena, so stroke selection and player level differ from broadcast play. (iii) The eight usable cameras view the court from the side, nearly perpendicular to the flight direction; by our own Fisher analysis (§4.3) that is the best-conditioned geometry, not a representative one. These reservations bound the numbers below but do not reverse any of them.

S4.1 The synthetic-to-real gap, measured against real 3D

Table S5 repeats the comparison with the final model; Table S6 below is the earlier-generation run it replaces.

Table S6 is the paper's central negative result measured for the first time against real 3D rather than a proxy. Four consequences follow.

• Synthetic test error understates real error by 1.7× and the understatement is not uniform across methods. Shuttle3D degrades by 1.7×, and the unbiased physics fit by 2.9× (the official MonoTrack, run on the Table S5 strokes, by 2.1×). A ranking read off the synthetic column is therefore not the ranking on real flights: the physics prior that wins on synthetic data (0.255 m, the best cell in the table) loses its advantage over Tier 1 alone on real flights (0.632 m vs. 0.530 m). Reported gains that live entirely inside the generator's own dynamics do not transfer.
• Plausibility is not accuracy, now with 3D truth on both sides. The constrained estimator returns physically valid flights for 100% of strokes in both domains, yet its error grows from 0.265 m to 0.618 m. Conversely, constant depth is better on real flights than synthetic ones (1.507 vs. 2.073 m) while remaining physically absurd. No feasibility rate distinguishes the two columns.
• The entire error is along the line of sight. Decomposing Shuttle3D's error into the viewing direction and its orthogonal complement gives 0.585 m along the ray versus 0.026 m across it; the ratio is 22×. Every method shows the same signature (official MonoTrack 1.53 / 0.13 m on the strokes of Table S5; constant depth 1.78 / 0.026 m; estimator 0.65 / 0.035 m). Calibration and 2D tracking are not the bottleneck — depth is, exactly as §4 predicts.
• The residual sits just above the information bound. Over 3,000 of these strokes the median CRLB is 0.460 m, so Shuttle3D's 0.530 m is 1.15× the bound for an unbiased estimator. The least-observable Fisher direction lies within 10° of the line of sight for 87.5% of strokes (median 2.3°). On real flights, in the best-conditioned camera geometry available, the system is already operating close to what a single view can support.

Downstream tactical quantities inherit the same structure: against the reference, Shuttle3D recovers launch speed to 3.91 m/s (reference median 19.2 m/s), launch angle to 6.4°, apex height to 0.065 m and net clearance to 0.108 m, while the horizontal apex position — a depth-dominated quantity — is off by 0.569 m.

S4.2 The generator's flight law is wrong, and fixing it does not help the learner

A forward-prediction test isolates the dynamics. Fitting a launch state to the first 8 frames of a stroke and then integrating forward with the law held fixed, synthetic trajectories diverge by 0.000 m at every horizon — a direct display of the inverse crime, since they are by construction solutions of that same ODE. Real reference flights diverge by 0.156 / 0.268 / 0.430 / 1.240 m at 0.20 / 0.40 / 0.67 / 1.07 s (n = 588), growing monotonically with launch speed. The reference's smoothing accounts for 1–2 cm of this; the rest is the model.

What the real 3D says about shuttle aerodynamics. Identifying a law of the form deceleration = γ · s^p on the 24 identification sessions, with g free so that the reference's curvature attenuation is absorbed rather than smuggled into the drag term, returns p = 1.972. The quadratic form is therefore correct; the coefficient is not. After dividing out the measured attenuation (α = 0.920), the identified coefficient corresponds to v_T = 6.04–6.10 m/s rather than the 6.8 m/s assumed throughout the literature and in our own generator — the shipped law underestimates drag by about 31%. Allowing γ to vary with speed over six knots (2–50 m/s) yields equivalent terminal velocities of 5.97, 6.07, 5.91, 6.48, 6.59 and 6.53 m/s, i.e. more drag at low speed and less at high speed. Because §S7.3 shows that a 15% error in v_T inflates mean 3D error by 63%, this is a substantive correction for any physics-fitting reconstructor, ours included.

Identification cuts the simulator's disagreement with real 3D from 7.20 cm to 4.53 cm under the convention that flight begins after racket contact, and from 8.09 cm to 5.46 cm when the contact frame is included. The two conventions differ for a measurable reason rather than by choice: dropping the first k frames from both the fit and the score gives 5.51, 4.68, 4.28, 3.51 and 2.55 cm for k = 0, 1, 2, 4, 8, so the residual is concentrated in the first 100–160 ms. An aerodynamic turnover term A e^−t/τ — the natural physical explanation, as a shuttle flips skirt-first after impact — makes the error monotonically worse at every value of A and τ under both conventions. The first frame is contaminated by the racket-and-shuttle blob rather than governed by unmodelled aerodynamics, and is excluded on that evidence.

The correction does not reach the monocular learner. Regenerating the training set with the identified piecewise law and retraining end-to-end gives 0.544 m on the held-out sessions, against 0.508 m (seed 0) and 0.468 m (seed 7) for the shipped law. Because two seeds of one recipe already span 0.040 m on this set, the change is not measurable: correcting the dynamics moved the physics-fitting baselines, which degrade 2.9× in Table S6, but left a prior-dominated learner untouched. A 4 cm shape correction is noise next to a 0.46 m information bound. This is worth stating plainly because the obvious inference from Table S6 — that a more faithful simulator closes the sim-to-real gap — is wrong for this class of model, and we only know it is wrong because we measured it.

S4.3 Where the remaining gap lives: a supervised upper bound

To locate the gap rather than guess at it, we fine-tuned the same architecture on real 3D from the identification sessions and evaluated on the same held-out sessions. This is an ablation and an upper bound, not a proposed system: the main results of this paper use no real 3D labels anywhere.

Three readings, in order of how much they change the paper.

• The supervised model lands below the information bound, in the real domain. 0.320 m against a median CRLB of 0.460 m on these strokes. This is not a violated bound: the CRLB constrains unbiased estimators, and a network trained on one hall's strokes is a biased estimator regressing towards that hall's prior. §4.4 argues this from synthetic data; here it is measured against real 3D, and it is the reason a low number on a matched test set cannot be read as evidence of recovery from the image.
• The gap is in the launch-state distribution, not the dynamics. Matching the conditional launch-height distribution to the reference — median Kolmogorov–Smirnov distance over 24 populated bins falls from 0.221 to 0.170, improving in 19 of 24 bins, with the marginal distance falling from 0.068 to 0.040 and synthetic net clearance moving from 0.60 m to 0.53 m against the 0.41 m measured independently on tournament footage (§S3.1) — gives 0.485 m, again inside the 0.040 m seed band. Neither the dynamics fix (§S4.2) nor the sampler fix is measurable, while supervision on the target hall is worth 0.19 m.
• Therefore the residual is venue-specific tactical prior, which a broadcast generator should not learn. The remaining 0.508 → 0.320 m is the value of knowing how these players, in this hall, choose to hit. Absorbing it into the simulator would buy accuracy on one venue by hard-coding a prior that is wrong elsewhere — precisely the failure mode §S7.1 documents for pixel-space models. The information has to come from the image at inference time, which is what planar anchors do.

S4.4 Planar anchors, validated against real 3D

Table S8 also gives the first measurement of the anchor mechanism of §3.4 against real 3D rather than against held-out floor annotations. Anchors here are the hitter's and receiver's ground positions and the launch height, read from the reference skeletons: cleaner than a pose detector on broadcast video (no identity confusion, no missed far player), which makes these numbers an upper bound on what a deployed detector delivers. Their geometric quality is nonetheless modest — the chosen pelvis sits 1.12 m from the ball at the first tracked frame and 1.36 m at the last, while the other player sits 7.72 m and 7.15 m away, so the selection is unambiguous even though the position is metres from the target.

Feet anchors cut the error from 0.508 m to 0.358 m (−30%) and, more importantly, cut gross failures: strokes worse than 1 m fall from 26.7% to 8.5%, depth-profile reversals from 10.8% to 1.4%. Supplying only the launch-side anchor gives 0.434 m and only the arrival-side anchor 0.379 m, so both ends carry information and the arrival end carries more. Replacing the pelvis with a truth-grade anchor gives 0.315 m, which bounds what better pose estimation can buy. The effect composes with supervision rather than substituting for it: on the fine-tuned model the same anchors give 0.320 → 0.263 m, a comparable relative gain on a much stronger starting point, so prior quality and anchor quality are close to independent contributions. A representative 21-stroke rally shows the mechanism concretely: its worst stroke goes from 5.08 m to 1.38 m with feet anchors, 0.94 m once the launch height is added and 0.55 m with truth-grade anchors, while the depth-scale error on that stroke falls from 14.8% to 3.7% and then to 1.7%.

The one multi-view rally that also comes with its video can be inspected interactively: the rally on the training-hall court overlays the multi-view reference (cyan) and our monocular reconstruction (orange) on the footage of one of the capture cameras, or orbits them over the rectified floor. Two caveats are printed on that page and repeated here: the capture camera stands 8.7 m from the court centre with a 92° lens, outside the broadcast geometry the model was trained on (12–39 m, 19–57°), and the rally is two strokes, the first mostly out of frame; its per-frame median error there is 0.65 m, not the 0.20 m of the broadcast-geometry rally above.

S4.5 Chained inference and precision-aware anchors

Anchors invite two extensions the reference data can now adjudicate. First, strokes chain: the predicted arrival point of stroke k is the launch point of stroke k + 1, and a prediction is a tighter anchor than a pelvis (0.34 m from truth versus 1.36 m). 74.7% of held-out strokes are chainable, 91.5% of non-serve strokes. Second, if anchors differ in precision by an order of magnitude, the model should be told which is which; we therefore sample each anchor's noise as log-uniform σ ∈ [0.10, 1.50] m during training and pass σ itself as an input channel.

Table S9 is deliberately restricted to paired tests and within-model ratios, because the two models are single training runs and the run-to-run spread on this set is 0.040 m — larger than the effects in the first row. Read that way the result is unambiguous. Under a fixed training σ, handing the model an anchor 4.3× more accurate changes nothing (+0.001 m, p = 0.16): it has learned a single anchor-noise level and cannot depart from it. With the precision channel the same substitution is worth −0.013 m at p = 6×10⁻⁹, and the span between a pelvis anchor and a truth-grade anchor widens from 8% to 38% of the error. A label-response sweep confirms the channel is read rather than memorised: holding the anchor values fixed and varying only the declared σ, the error is minimised when the declared value matches the true one (0.340 / 0.317 / 0.319 m at declared σ = 0.8 / 1.2 / 1.5 m against a true 1.25 m), and the series is flat where the anchors are already tight. The channel is not free: at the loose setting the precision-aware model is slightly worse on synthetic held-out cameras (0.272 → 0.290 m), the price of spending capacity on a nuisance input. It pays where it matters — at a declared σ = 0.30 m the same model reaches 0.169 m.

Chaining itself does not accumulate: the cumulative median over strokes up to position k in the rally rises from 0.269 m at the first stroke to 0.300 m by the twelfth, against 0.316 m for pelvis anchors alone and 0.361 m when the chain is used without a pelvis fallback. The chain is a net gain at every rally position and degrades gracefully when it breaks.

Joint estimation over a whole rally. Chaining passes information forward one stroke at a time and commits to each prediction as it goes. Solving the strokes of a rally jointly — one shared unknown per hit, since the arrival point of one stroke and the launch point of the next are the same 3D point — replaces that greedy pass with a single estimate over the rally, and should recover the information the forward pass discards. On the 2,695 held-out strokes (497 rallies) it does not: the one-directional backward pass is already the fixed point. Fusing the backward handover with the pelvis anchor by inverse variance, with the handover's precision declared honestly at σ = 0.50 m, gives 0.296 m, against 0.317 m for pelvis anchors alone, 0.298 m for the plain chain and 0.280 m for a chain fed truth-grade handover points. The backward Gauss–Seidel sweep converges in a single round — rounds 1 through 4 are bit-for-bit identical — and the declared σ is immaterial here (0.296 m at every value in 0.15–0.50 m).

Both bidirectional forms are worse rather than merely equal: Jacobi settles at 0.335–0.349 m and two-sided Gauss–Seidel at 0.346–0.353 m, 2.1–2.4 cm above the chain on a paired Wilcoxon test over the same strokes (p < 10⁻⁶⁰). Forward handover — using the next stroke's predicted launch point as this stroke's arrival anchor — is monotonically harmful in its declared precision: 0.371, 0.353, 0.342, 0.319 and 0.311 m at σ = 0.30, 0.45, 0.55, 0.80 and 1.20 m, reaching the chain's 0.296 m only in the limit where it is switched off. Two reasons, both measured. The forward anchor is the worse of the two (0.66–0.83 m from truth against 0.50 m backward, because the model predicts a stroke's arrival point better than its launch point), and after one sweep it has stopped being an independent observation: the next stroke's launch point was itself produced from this stroke's arrival point, so a second pass feeds the estimate its own output back as evidence.

Handover transports anchor information; it does not create any. Chaining every stroke from its own predecessor with no pelvis anchor anywhere gives 0.462 m — worse than supplying no anchors at all (0.445 m) — and 0.426 m backward-only. Without a player detector the rally structure is worth nothing. The serve-height rule that would otherwise seed the first stroke of each rally is likewise unusable, for a reason we can measure rather than assume: the first segment the reference exposes is frequently not the serve, its true launch height having a median of 1.44 m with only 19.5% at or below the 1.15 m the laws allow a serve.

The second tier costs accuracy on real 3D. Running the rectifier of §3.5 rules-only on the same 2,695 strokes gives 0.326 m at 89.5% feasibility with a tight prior (σ_prior = 0.08 m) and 0.456 m at 86.5% with the deployed one (0.50 m), against 0.296 m for Tier 1 alone; paired, p = 4×10⁻⁶⁵ and 5×10⁻¹⁸². The cause is the flight model itself, and it is visible without any image: fitting the rectifier's aerodynamic model directly to the true 3D — no projection, no detector noise, no constraints — leaves a median per-stroke RMS of 0.104 m (p90 0.451 m), rising from 0.053 m on 20–30-frame strokes to 0.291 m beyond 45 frames. The test set is not at fault: the reference trajectories satisfy the court rules on 96.9% of strokes, the cord being the only rule broken at all (3.13%).

Substituting the drag law identified in §S4.2 helps the rectifier in exactly the way that diagnosis predicts, and not enough to change the ordering. Paired over the same strokes it is worth −0.003 m at σ_prior = 0.08 m (p = 0.025, 51.5% of strokes improved) and −0.031 m at 0.50 m (p = 2×10⁻²³, 62.9% improved): the looser the prior, the more the dynamics are load-bearing. But the law holds only inside the window it was identified on. Against true 3D the best-possible fit improves by 36% and 29% on strokes below 20 frames and of 20–30 frames, and degrades by 8% and 31% on 30–45 and 45+ frames, which are long enough that the segment is probably not a single free flight. Either way Tier 2 remains 1.2 cm behind Tier 1.

A joint form carrying no flight model at all confirms where the cost sits. One depth scale per stroke, solved in closed form across a rally through the shared hit points and aimed directly at the per-stroke depth-level failure mode, reaches its optimum at do not move (median |α − 1| = 0.0007, p = 0.12 against its own Tier-1 input) and still lands at 0.300 m, ahead of the physics rectifier's 0.326 m. Read together: on real 3D, Tier 1 with a backward chain is the best operating point this system has, and Tier 2 buys feasibility (89.5% on these strokes) and smoothness for 1–3 cm of accuracy. That inverts the synthetic ordering of Table S6, where the physics prior is the best cell outright. It is the inverse crime of §S4.2 seen from the other side: on synthetic data the estimator's flight model is the generator's, so imposing it can only help; on real flights it is an approximation with a 0.10 m floor, and imposing it on a learner already inside that floor costs more than it returns.

Deployment with a detected pose. The anchors in Tables S8 and S11 come from reference skeletons, which is an upper bound on anchor quality. The deployed configuration reads the hitter's and receiver's ground positions from a pose detector on broadcast frames, where identity assignment and the far player's scale are genuine failure sources. Table S10 runs the feasibility and leave-one-anchor-out protocol of §S2.4 on 6,553 tournament strokes for three successive generations of the Tier-1 model. After rectification the three are indistinguishable: every paired difference is under 1 cm, and the few that reach significance do so only because n ≈ 5,000.

The generational gain sits in the first tier, and the rectifier hides it. On the raw per-frame network output, with no launch fit and no repair, the fraction of strokes satisfying every court rule rises 88.4 → 88.9 → 90.3% across the three generations on 6,924 strokes, driven almost entirely by cord violations falling 7.2 → 6.4 → 5.6%; with anchors supplied the same series reads 90.4 → 91.2 → 91.6%. The rectifier then repairs good and bad proposals to the same admissible point, which is what it is for, and is why the end-to-end metric cannot see first-tier progress. Reporting only that metric would have made three generations of training-recipe work look like nothing.

The declared-precision channel survives the transfer in form but not in payoff. Holding everything else fixed and telling v9 that the foot anchor is less precise (σ = 1.0 → 1.6 m) moves the reconstruction measurably away from that anchor — distance to the anchor itself rises from 1.208 m to 1.280 m at the launch end and 1.191 m to 1.282 m at the arrival end, paired p < 10⁻⁸⁰ — but buys nothing against the independent human annotation, which moves by at most 0.5 cm either way (0.666 → 0.661 m and 0.488 → 0.507 m), while feasibility stays flat at 80.4 / 80.1 / 80.2%. The limit is anchor quality rather than the channel: the detected feet sit 1.32 m from the annotated hit point and select the wrong player on 8% of strokes, against 1.12 m and no identity confusion for the reference skeletons of §S4.4. The next lever in deployment is the pose detector, not the lifter.

S5 Airborne strokes, input degradation and failure modes

Relation to Table 4 and Table S12. Tables S11 and S13 of this section were computed with an earlier model generation and with anchors read from the reference skeletons plus the true launch height, which a broadcast does not provide. Table 4 of the main text and Table S12 with Figure S5 repeat the airborne and degradation analysis with the final model and detector-grade foot anchors only. The qualitative findings carry over — jumping players stay close to the shuttle horizontally, frame loss is cheap and anchor error is expensive — but the anchored figures here are upper bounds, not results of the system.

Two objections follow directly from how badminton is actually played and how its footage is actually tracked. The floor anchors of §3.4 are taken from the hitter's ankles, but a smash is struck in mid-air with both feet off the ground; and TrackNetV3 does not return every frame, least of all the fastest ones, where the shuttle is a streak. This subsection measures both, on the 2,682 held-out multi-view strokes where real 3D exists and on the 6,553 broadcast strokes where a real pose detector is in the loop.

Jumping is measurable, and the result runs against intuition: the higher the player jumps, the better the anchor. We define lift as the hitter's pelvis height at contact minus that same player's standing baseline in that rally (the 20th percentile of their pelvis height, computed per rally and per half-court, so it is immune to the two skeleton slots swapping). Lift is 0.05 m at the median and 0.30 m at the 95th percentile, and 76.2% of smash-class strokes are classified airborne against 21.3% of the rest, so the detector is finding the right strokes. The anchor error — the distance from the ankle-derived ground point to the shuttle's true horizontal position at contact — then falls monotonically with lift, from 1.37 m in the lowest quartile to 0.78 m in the highest (correlation −0.48). The reason is simple once seen: a player jumping to smash gets underneath the shuttle, which is exactly what makes the ground anchor good. What jumping destroys is the assumption that the pelvis is at standing height, and height is not taken from the anchor at all — it is a separate token.

Table S11 shows that airborne strokes do carry more 3D error than grounded ones (0.374 against 0.300 m with feet anchors), but the no-anchor column moves in step (0.493 against 0.432 m), which says the penalty is the stroke type — short, low-N arcs — and not the anchor. The anchor's relative contribution is present in both groups (−24% airborne, −31% grounded) and only begins to narrow past a lift of 0.40 m, where n = 37.

With a real detector instead of a reference skeleton, the anchor degrades and the answer does not. On 6,553 broadcast strokes with YOLO11m-pose ankles, airborne stroke types (smash, tap smash, net kill; n = 850) have an anchor 1.05 m from the annotated player position against 0.76 m for everything else, and the detected wrist sits at 1.58 m against 1.02 m, which is itself the physically correct signature of an overhead contact. Arrival error is nonetheless 0.56 m against 0.54 m — essentially unchanged — and under the paper's headline rules-only setting the post-repair feasibility is 94.7% against 96.1%. The one place the difference bites is when anchors are imposed as hard constraints, where airborne strokes fall to 66.1% feasible against 82.2%: a 1.05 m ankle inside a tighter admissible radius is simply inconsistent. That is the direct empirical argument for the design we already ship — anchors are soft tokens in Tier 1, and Tier 2 enforces only court rules. Launch error does separate (1.46 against 0.71 m), but roughly half of that is the annotation: a jumping player's annotated hit point is their ground position, and they have left it.

Dropped detections are cheap; a mis-placed anchor is not. Table S13 doses both degradations in the units a practitioner actually faces. Removing an extra 0%, 10%, 20%, 30% and 50% of tracked frames moves the anchored error 0.309 → 0.312 → 0.313 → 0.321 → 0.329 m, and the unanchored error 0.447 → 0.491 m. Losing half the observations costs 6%. Contiguous bursts of 3, 5 and 8 frames — the failure mode a fast blurred smash actually produces — cost 0.314/0.317/0.326 m with anchors, which is indistinguishable from random removal at the same frame count (0.313/0.315/0.321). Without anchors, burst-8 costs 0.497 against 0.466 m for the equivalent random loss, so gap structure only matters when there is nothing pinning the depth.

Anchor noise is an order of magnitude more expensive per metre. Adding σ_extra = 0/0.5/1/2/3/4 m of Gaussian error, with the declared precision widened to match, gives 0.309/0.322/0.346/0.391/0.508/0.635 m; keeping the declared precision fixed at its nominal 1.2 m while the anchor silently degrades gives 0.328/0.367/0.489/0.608/0.688 m. The break-even against no anchor at all is 2.5 m if the pipeline declares its uncertainty honestly and 1.7 m if it does not. A measured YOLO ankle on broadcast is 0.77 m from the annotated player position, leaving roughly a factor of three of headroom, so a jittering far-court skeleton is safe — provided the declared-precision channel is used. This is the first operational value we can attach to that channel. Degradations barely interact: the shape of the anchor-noise curve at 30% frame loss matches its shape at 0%.

This experiment degrades the inputs of an already-trained model at inference. It is the complement of the domain-matched corruption of §3.3, which injects the same families of artefact — heavy-tailed detector noise, burst dropouts, late acquisition, calibration jitter — during training; neither substitutes for the other. The anchor-noise sampling itself carries about ±0.01 m between random seeds, so the break-even points are quoted to 0.1 m only.

S5.1 An independent check on contact-frame depth

Placing a fitted body and a reconstructed shuttle in the same metric frame creates a comparison this problem does not otherwise admit, and it bears directly on the wrist-elevation anchor of §3.4. The reference pixel is the tracker’s raw detection, which the reconstruction consumes but never edits; nothing in the pose fitting sees a shuttle at all, and the body’s depth comes from ankles on the court plane. The two sides of the comparison are therefore independent of each other and of the reconstruction being tested.

Over 58 contact frames in 8 venues, the distance from the striking wrist to the reconstructed 3D shuttle is 1.54 m — beyond an arm plus a racket — while the distance from the same wrist to the line of sight through the raw detection is 0.52 m, which is about one racket. The detection therefore agrees with the fitted body, and the depth along that ray is what disagrees. The reconstructed point itself sits only 0.09 m off the detection’s ray, so the 2D fit is tight; resolved along the ray, its depth is scattered with a slight far bias: a median 0.21 m beyond the wrist’s closest approach, with an interquartile range of −1.11 to +1.65 m.

This is what §4 predicts rather than a contradiction of it. The contact frame is the seam between two solved flights, the instant at which the arc being fitted ends and the next begins, and it carries the least depth information of any frame in the stroke. The measurement turns that qualitative statement into a number, on 58 events rather than the 14 rally-final anchors of §S2.4, and it says that reported depth at the moment of impact should carry an explicit uncertainty of roughly a metre while the same trajectory's mid-flight depth does not.

S6 Image-level synthesis and runtime

A broadcast-registered renderer, built as a tool for future image-level training; the trajectory results of the paper do not use it.

S6.1 Image-level synthesis: a broadcast-registered renderer

The synthetic domain of §3.2 is a domain of trajectories: it transfers because the flight model and the camera distribution are measured, but it cannot exercise a detector, and it cannot tell us whether a reconstructed shuttle is where a human arm could have put it. We therefore extend the synthesis to images. The renderer places, in one metric frame, a measured broadcast camera, the court and net at BWF dimensions, two articulated human bodies, procedural rackets, and a shuttle on a flight from §3.2 — with motion blur at the shutter measured from the footage rather than assumed.

The renderer needs a rigid camera. For venues where §3.1 has run, its output is already one. Where a venue is described instead by four court corners plus a focal length, the projection P = [g₁, g₂, sKr₃, g₃] that keeps the Z = 0 slice exactly equal to the homography is pixel-exact on the court and not a physical camera: its decomposition returns negative focal lengths with det R = −1. Projection is insensitive to that mirror, so no estimation result is affected; a renderer handed it produces a black frame. The mirror is vertical in the image, and flipping the corresponding row of R recovers a proper camera whose reprojection agrees with the original to 3.92 px over nine points spanning the ground, the net-post tops, the players and the shuttle — inside the 6.03 px residual the corner annotations themselves carry.

Players are fitted from the broadcast, not animated. No motion-capture corpus contains badminton, and a stock walk cycle would defeat the purpose. Instead an SMPL-X body is optimised per frame so that its joints project onto the 2D skeleton detected in that frame, through that venue's projection. Three constraints make the fit metric rather than merely plausible. Depth is anchored by back-projecting the ankle pixels onto the court plane — the same sub-pixel ground measurement the rest of this paper rests on — because apparent height is a poor depth cue and fixing the body to a nominal 1.72 m pulls both players about 0.8 m toward the camera; sole contact uses the lowest mesh vertex, not the ankle joint, which sits 7 cm off the floor; and the pose itself is the decoded 32-dimensional latent of a learned pose prior, so that only configurations a human can hold are reachable, with the whole sequence then refined jointly under acceleration penalties on the latent, the root orientation and the translation. The learned prior costs 1.3 px of reprojection against an L2 pull toward the rest pose (2.6 → 3.9 px) and buys poses that stay on the human manifold through lunges and low recoveries, where the L2 prior instead pays for pixels with implausible joint combinations.

Strokes are populated from observed strikes. Fitting 16 clips across eight calibrated venues yields a library of 3,144 poses, each tagged with the racket hand, the wrist heights, and — from the stroke annotations — whether that frame is a contact and which player struck. To render a synthetic stroke, the striker is the observed striking pose whose racket head sits closest to the required launch height, yawed to face the net and translated so that the racket head coincides with the launch point and the feet rest on the floor. Launch point, landing point and duration come from the human annotations exactly as in §3.2; only the height between them is solved.

S6.2 How close the rendered images are to broadcast

A synthetic image domain is only worth building if it is close to the real one, so we measure the distance rather than assert it. We reconstruct one real rally segment in full — same camera, both players fitted per frame from that footage, shuttle on the reconstructed trajectory — and compare the render against the broadcast frame it was built from.

Court alignment lands at the residual the corner annotations themselves carry, and the fitted bodies sit within the detector's own keypoint noise. The remaining shortfall is streak length. The minor axis matches (5.8 against 5.7 px), so the shuttle's size is right and only the exposure is short: the render used the corpus-median shutter of 1/116 s while this clip is slower. The deficit is concentrated at speed — on the fastest fifth of frames the render gives 14.0 px against 16.8, while on the slowest fifth the two agree to 0.2 px — which is the signature of a shutter that is too short rather than a shuttle that is the wrong size. Shutter is measurable per clip from the streaks themselves — median 1/150 s with an eight-fold spread across clips, 1/91 to 1/1708 — and wiring that per-clip value into the renderer is the obvious correction.

These streak figures replace an earlier, contaminated set. We first measured blob shape in a window around the shuttle on the raw frames, which lets court lines inside the window pass the intensity threshold: that gave 11.1 px against 15.2 and an orientation agreement of 0.575, a figure so far below the 0.996 measured on broadcast that it should have been read as a measurement fault rather than a rendering one. Every streak statistic here is measured on the background-difference image — which is also what the detector consumes.

The acceptance test: a detector that has never seen a render. Registration says the geometry is right; it does not say a detector would treat the image as broadcast. So the released TrackNetV5-Lite — trained on real footage only, not fine-tuned here — is run on the 91 rendered frames and on the 91 real frames they reconstruct, and scored against the projected 3D on the render. On the real side this corpus has no human per-frame 2D; the two references available are the raw TrackNetV3 detection (another detector's opinion) and the reconstruction's own reprojection (which carries the reconstruction's error), and both are reported as what they are.

A detector that has never seen a rendered pixel localises the rendered shuttle to 2.7 px at the median with 96% of frames inside 8 px, and reports the same heatmap confidence on the render as on the broadcast (0.66 against 0.69). The only rendered frames it misses by more than 8 px are the contact frames 160–164, where the racket frame occludes the shuttle at 8–9 px; the physics-filled frames, with no player near the shuttle, are its best (2.2 px). What this does not establish is that it fails on the same frames as on real footage: the real side has no ground truth, so a per-frame comparison is a comparison of two detectors, and we do not draw one.

The population step has its own failure mode, and it is a sampling failure rather than a fitting one. A first library built from randomly chosen 100-frame windows contained exactly one overhead strike among 58 striking poses, which makes a synthetic smash impossible to stage. Two explanations were natural and both were tested and rejected: keypoint confidence at overhead contacts is the highest in the corpus (0.97 against 0.94 elsewhere) and the fitted wrist reprojects 5.5 px from the detected one, so the fit follows the data; and the distance from the nearest fitted wrist to the line of sight through the raw detection, swept around each contact, reads 1.08 / 0.74 / 0.52 / 0.66 / 0.95 m at −8 / −4 / 0 / +4 / +8 frames — a clean minimum exactly at contact, so the striker is identified correctly. The cause is that random windows are mostly footwork. Selecting windows by the annotated stroke class, since smash, clear, cut and push are struck above the head, changes the library without changing a line of the fitter.

S6.3 Cost

Every stage was timed on one RTX 3090 with the GPU exclusive, after warm-up, reporting the median of repeated runs. The host was not idle — an unrelated 40-worker job held the 64-core machine at load 22–41 throughout — which inflates the CPU-side feature construction, the kernel-launch-bound rectifier and the calibration wall clock. Every figure below is therefore an upper bound on this hardware.

Table S14 reports the stage costs. The rectifier is throughput-bound, not latency-bound, and we state this rather than round it away. Its cost is a fixed number of sequential RK4, Levenberg–Marquardt and penalty-continuation steps together with their kernel launches, not arithmetic, so wall time per pass barely moves with batch size: 32 strokes take 131.9 s and 13,106 strokes take 410 s — 410× the work for 3.1× the time. The consequence is that repairing a single rally in isolation costs a whole pass, roughly 130 s, and the method must not be described as interactive. Batched over a match or a corpus it is 31 ms per stroke. A low-latency single-rally mode would need fewer sequential steps or CUDA graphs, not a larger GPU.

End to end, a median rally (188 frames, 6.3 s, about 8 strokes) in the deployed rules-only configuration costs 0.84 s of detection plus 0.001 s of lifting plus 0.25 s of amortized repair — 1.1 s for 6.3 s of video, 5.8× real time, with no pose detector required. Adding ground anchors adds the pose stage and brings the total to 4.8 s, still 1.3× real time. The per-frame front end alone is 24.3 ms against a 33.3 ms budget, so even with pose the streaming stages fit inside 30 fps on one GPU. Calibration is excluded from both, being once per venue and amortizing to about 0.02 ms per frame of that venue's footage.

Two caveats we prefer to state. The deployment-batch rectifier rows are amortized throughput read from production logs rather than fresh timings, and come from different checkpoints than the batch-32 row; they agree only through the scaling argument above. And the detector figure is the stride-8 non-overlapping pass; the stride-1 overlap-averaged pass used to produce the paper's tracks costs about 8× that, which is arithmetic rather than measurement.

S7 Ablations and extended discussion

S7.1 Ablation: input space parameterization

To evaluate whether performance gains stem from model capacity or geometric parameterization, we evaluate four input representations under identical Transformer backbones and training schedules on held-out strokes: world rays to depth along the ray (ours), world rays to 3D coordinates, pixels plus the camera matrix to 3D coordinates, and pixels plus a learned camera embedding to 3D coordinates.

The results table is Table 3 of the main text.

As shown in Table 3 of the main text, pixel-space baselines perform well on seen cameras by memorizing venue-specific depth priors, but degrade severely (1.7× to 4.3×) when evaluated on novel camera setups. In contrast, our ray-depth formulation shows no measurable degradation on unseen cameras (0.388 m vs. 0.387 m), confirming robust geometric transfer.

S7.2 Ablation analysis

Table S15 delineates the progression across stages. Incorporating measured tracking noise and rule losses improves raw network validity from 78.9% to 88.4%. The Tier-2 rectifier establishes a 96.0% final return rate of physically valid trajectories on real tournament matches.

Crucially, the synthesis ablation highlights that physical feasibility does not imply metric accuracy: random launch parameter sampling achieves a higher raw feasibility rate (67.9% vs. 64.6%) by generating conservative, high-clearance arcs, but suffers severe metric degradation on real strokes (1.29 m vs. 0.90 m endpoint error).

Removing each design decision in turn. Table S16 takes the four choices that define the method — the sag gate, the ray-depth representation, the rectifier, and the planar anchors — and reports what is lost when each is removed, across every evaluation domain in which a measurement exists. Two entries are blanks we decline to fill with an estimate, and one row reverses sign between domains; both are stated rather than smoothed.

The three columns come from different model generations and different held-out splits, and the two “removals” in the last two rows were measured independently rather than as a crossed design, so the rows should be read for direction and magnitude and not summed.

S7.3 Extended discussion and limitations

Operational scope and tracking continuity. Following standard broadcast sports vision pipelines^[5], our framework operates under the operational assumption of a stationary main broadcast camera. Furthermore, trajectory lifting is performed on stroke intervals with at least 45% visual tracking continuity. Across tournament venues, raw detector recall ranges from 9% to 92%, meaning analyzed rallies naturally reflect trackable game segments.

Aerodynamic model sensitivity. Synthesis and estimation share a quadratic drag model with terminal velocity v_T = 6.8 m/s. To quantify sensitivity to drag misspecification, we evaluated the estimator under perturbed terminal velocities (v_T ± 15%). Perturbing v_T by −15% (5.78 m/s) increases mean 3D error by 63% (0.471 m → 0.769 m) and launch error from 0.309 m to 1.474 m. Remarkably, physical feasibility remains virtually unchanged (98.7% → 96.5%). This reinforces our core thesis: physical plausibility provides a necessary sanity boundary, but does not guarantee metric accuracy.

Ground truth in broadcast sports. Monocular sports vision lacks large-scale 3D sensor ground truth. In our corpus, 14 rally-final strokes provide an exact physical 3D anchor (z = 0 at landing), on which SimTrack3D achieves a median 3D error of 1.00 m and a landing height error of 0.32 m. While these 14 strokes serve as an absolute sanity check rather than a comprehensive benchmark, our leave-one-anchor-out formulation provides a scalable proxy for single-camera sports evaluation. §S5.1 adds a second, geometrically independent probe of the same kind: a human body fitted from the broadcast, which never sees a shuttle, bounds where the striking hand was at 58 contact frames and localises the pipeline’s weakest depth to the stroke seam. The multi-view reference of §S4 supplies dense 3D truth for 2,682 held-out strokes and closes much of this gap, subject to its own limits, stated below.

What the image-level domain does not yet establish. §S6.1 renders images that register to broadcast at 2.2 px on the court and 3.8 px on the players, and §S6.2 shows a detector trained only on broadcast treats them as broadcast — 2.7 px median against the true projection, with the same confidence. What has not been run is the converse: a detector trained on the renders and evaluated on real footage. Until it is, the renderer is validated as a measurement instrument and as an evaluation domain, not yet as training data. Its known gaps are the shutter (per-clip exposure is measured but not wired in), lighting estimated from the real shadows, and the absence of crowd, stands and clothing.

Limits of the multi-view reference. Three properties of the reference bound the conclusions of §S4. It is a learned multi-view reconstruction rather than a motion-capture measurement, and it is temporally smoothed: residuals of a fixed-drag fit show lag-1 autocorrelation of +0.76 to +0.90 and curvature attenuated by α ≈ 0.90, which we quantify as a 1–2 cm floor and correct for explicitly when identifying the drag law, but which leaves the identified v_T dependent on that correction. The capture is a training hall, so its stroke mix and player level differ from tournament play; the launch-height statistics we match in §S4.3 are that hall's, which is exactly why we do not adopt them in the shipped generator. And only eight of ten cameras are usable (the two end cameras carry barrel distortion their pinhole intrinsics cannot represent); those eight view the court from the side, nearly orthogonal to the flight direction, which by our own Fisher analysis is the best-conditioned geometry rather than a representative one. Real broadcast cameras sit 12–39 m away, largely behind the baseline. The reported real-3D errors should therefore be read as favourable-geometry figures. Finally, the anchor results of §S4.4–S4.5 use reference skeletons, which have no pose-detection failures, and the cross-model comparisons in §S4.5 are single training runs on a set whose seed-to-seed spread is 0.040 m; only the paired tests and within-model ratios reported there are robust to that.

We have used badminton because it is the hardest available instance of a general problem, not because the problem is badminton's. A shuttlecock removes appearance scale, removes the bounce that gives table tennis and tennis a metric boundary condition, and flies along the optical axis for about fifteen frames; any monocular ballistic estimator that survives that is being tested at the limit of the regime rather than inside it. Two of the tools we needed there are defined without reference to the sport. The observability number N is computed from a 2D track and a camera alone and predicts the Cramér–Rao bound through CRLB ∝ N^−0.98, locating the crossover at which an estimate stops reading the image and starts reading its prior — below N ≈ 17 in one direction, above N ≈ 21 in the other — while our own real-domain test shows it does not transfer to deployment and should not be read as a quality flag there. The null-space projection is a way of moving an estimate only where the image cannot see, and takes any admissibility set in place of the court rules. We expect both to be useful wherever a single camera has to be asked for metric depth along the direction it measures worst.

References

15 works; 14 resolved against OpenAlex or the arXiv API, and 1 entered by hand from the source document and marked as such.

1. Yu-Chuan Huang, I-No Liao, Ching‐Hsuan Chen, Tsì-Uí İk & Wen-Chih Peng. TrackNet: A Deep Learning Network for Tracking High-speed and Tiny Objects in Sports Applications arXiv (Cornell University) (2019). 10.48550/arxiv.1907.03698
2. Y. C. Chen & Yu-Shuen Wang. TrackNetV3: Enhancing ShuttleCock Tracking with Augmentations and Trajectory Rectification (2023). 10.1145/3595916.3626370
3. Thomas Gossard, Andreas Ziegler & Andreas Zell. TT3D: Table Tennis 3D Reconstruction arXiv (Cornell University) (2025). 10.48550/arxiv.2504.10035
4. Nima Rahmanian, Daniel Kienzle, Thomas Gossard, Dvij Kalaria, Rainer Lienhart & Shankar Sastry. TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videos arXiv (Cornell University) (2026). 10.48550/arxiv.2605.01234
5. Paul Liu & Jui-Hsien Wang. MonoTrack: Shuttle trajectory reconstruction from monocular badminton video arXiv (Cornell University) (2022). 10.48550/arxiv.2204.01899
6. Arjun Raj, Lei Wang & Tom Gedeon. TrackNetV4: Enhancing Fast Sports Object Tracking with Motion Attention Maps (2025). 10.1109/icassp49660.2025.10889364
7. Thomas Gossard, Filip Radović, Andreas Ziegler & Andreas Zell. BlurBall: Joint Ball and Motion Blur Estimation for Table Tennis Ball Tracking arXiv (Cornell University) (2025). 10.48550/arxiv.2509.18387
8. Daniel Kienzle, Katja Ludwig, Julian Lorenz, Shin'ichi Satoh & Rainer Lienhart. Uplifting Table Tennis: A Robust, Real-World Application for 3D Trajectory and Spin Estimation arXiv preprint (2025). 10.48550/arXiv.2511.20250
9. Morten Holck Ertner, Sofus Schou Konglevoll, Magnus Ibh & Stella Graßhof. SynthNet: Leveraging Synthetic Data for 3D Trajectory Estimation from Monocular Video (2024). 10.1145/3689061.3689073
10. Magnus Ibh. 3D Reconstruction of Shuttlecock Trajectories (TrajTrans), Chapter 7 of PhD thesis PhD thesis, IT University of Copenhagen (2024). https://en.itu.dk/-/media/EN/Research/PhD-Programme/PhD-defences/2025/Magnus-Ibh.pdf [manual entry: not indexed by OpenAlex or arXiv]
11. Puntawat Ponglertnapakorn & Supasorn Suwajanakorn. Where Is The Ball: 3D Ball Trajectory Estimation From 2D Monocular Tracking arXiv (Cornell University) (2025). 10.48550/arxiv.2506.05763
12. Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Wojciech Zaremba & Pieter Abbeel. Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World arXiv (Cornell University) (2017). 10.48550/arxiv.1703.06907
13. Z. Zhang. A flexible new technique for camera calibration IEEE Transactions on Pattern Analysis and Machine Intelligence (2000). 10.1109/34.888718
14. Jonas Theiner & Ralph Ewerth. TVCalib: Camera Calibration for Sports Field Registration in Soccer 2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (2023). 10.1109/wacv56688.2023.00122
15. Marc Gutiérrez-Pérez & Antonio Agudo. PnLCalib: Sports Field Registration via Points and Lines Optimization arXiv preprint (2024). 10.48550/arXiv.2404.08401

Court and net geometry follow BWF specifications (6.10 × 13.40 m doubles court; net 1.55 m at the posts, 1.524 m at centre). Shuttle flight model: quadratic drag with 6.8 m/s terminal velocity, RK4. Tracking: TrackNetV3. Rendering: Blender 4.2. Every ordinary number on this page comes from the experiment logs of this project.