MLMI 2026 Workshop · MICCAI 2026 Oral + Poster
PPCNet: Projection-Conditioned Point Cloud Reconstruction of Spinal Vertebrae from Biplanar Radiographs
1Healthcare Technology Innovation Centre (HTIC), Indian Institute of Technology (IIT) Madras, India
2Department of Electrical Engineering, Indian Institute of Technology (IIT) Madras, India
†Corresponding author
We propose PPCNet (Projection-Conditioned Point Cloud Network), a query-refinement framework that reconstructs a dense 8,192-point cloud of spinal vertebrae from two orthogonal digitally reconstructed radiographs (DRRs) and their corresponding calibrated 3×4 projection matrices.
PPCNet employs the calibrated projection geometry as the core 2D-to-3D lifting mechanism, combining occupancy-gated query initialisation with projection-conditioned refinement. Gap-preserving and curvature losses maintain inter-vertebral disc spaces and the spinal lordotic curve.
Evaluated on the lumbar spine (L1–L5) across 105 held-out patients from a 1,037-patient dataset derived from VerSe and CTSpine1K, PPCNet achieves a mean Chamfer distance of 1.981 mm, F-score@5 mm of 0.973, and HD95 of 4.525 mm. Clinical validation across 525 vertebrae confirms a morphometric MAE of 2.80 mm with 97% of vertebrae at Vertebral Compression Level (VCL) Grade A. As a proof-of-concept, phantom-based navigation tracking yields a mean vertebra landmark error of 3.01 mm across L1–L5.
- Point Cloud Reconstruction
- Biplanar Radiographs
- DRR
- Spinal Vertebrae
- Projection-Conditioned Refinement
- Minimally Invasive Spine Surgery
Method
Six stages from two radiographs to 8,192 points
Two DRRs, their calibrated projection matrices, and a patient-specific centre and scale go in. 8,192 points in world space come out.
PPCNet architecture: dual ResNet-34 encoders, 3D feature lifting and fusion, occupancy-gated query initialisation, and three projection-conditioned refinement stages.
Unlike methods that fuse the two views implicitly, PPCNet uses the calibrated matrices to determine where each 3D query projects onto each 2D feature map.
-
01
Dual 2D encoders
Two ResNet-34 backbones with independent weights encode the two DRRs up to layer3, then a 1×1 convolution projects to 192 channels. 512×512 → 32×32.
-
02
Feature lift, 2D to 3D
Each feature map is replicated along a new depth axis (D = 32). A learnable depth embedding gives every slice a distinct signature, so the stack is not a set of identical copies; two Conv3D–GroupNorm–GELU layers refine to 128 channels — one volume per view.
-
03
Biplanar fusion
The AP and LP volumes are permuted onto the coronal and sagittal axes, concatenated (256 channels), and fused by 1×1 then 3×3 Conv3D back to 128.
-
04
Coarse 3D U-Net
A three-level U-Net (128→96→160→224) enriches the fused volume and predicts a 48³ occupancy grid — a coarse map of bone-occupied space for the next stage to aim at.
-
05
Occupancy-gated query initialisation
A 20×20×21 grid subsampled to 8,192 points forms the base queries. A global MLP shifts each one, gated by occupancy — empty-region points barely move, while points near bone are pulled onto the surface. Offsets bounded to ±0.25.
-
06
Projection-conditioned refinement
Three independently weighted stages project every query into both views through the calibrated 3×4 matrices, sample a 520-d vector where it lands — 2D image features, 3D volumetric features, occupancy, its own coordinates and both pixel positions — and predict a displacement via 520→384→384→3. ±0.25 per stage, ±0.75 total.
Training signal 8 losses
- Chamfer
- Bidirectional, truncated at 8 mm for a sharper gradient near the surface.
- Gap penalty ours
- Penalises points landing in empty ground-truth voxels, holding the inter-vertebral disc spaces open.
- Axial density ours
- Matches the ground truth’s cranio-caudal density distribution by KDE, so points are not left bunched along the spine.
- Extent match ours
- Matches per-axis 5th–95th percentile spread. Chamfer alone tolerates predictions that under-reach the extremes; this term penalises them.
- Curvature
- Slices along Z and matches the centroid trajectory, preserving the patient’s lordotic curve.
- Occupancy
- BCE + Dice on the auxiliary 48³ head.
- Sliced Wasserstein
- Distribution match over 50 random directions.
- Projection Chamfer
- 2D Chamfer after reprojecting the prediction back into both views.
Results
Reconstruction on held-out patients
The predicted cloud shown against the ground-truth surface, and on its own.
Across 105 held-out patients the mean Chamfer distance is 1.981 mm, at an F-score of 0.973 for a 5 mm threshold.
Ablation six generations vs ground truth
A baseline plus five increments, one component at a time: gap-occupancy gating, a unified pipeline, a Chamfer-ramp curriculum, phased losses, and finally the curvature and extent losses at 8,192 points. All six are shown against the ground truth.
Comparison with prior work identical splits
All four baselines re-implemented on the same 1,037-patient dataset and the identical 829/103/105 split. The volumetric ones — X2CT-GAN, BX2S-Net, Swin-X2S — output 64³ occupancy grids, so 8,192 points are extracted from each and every method is scored on the same representation.
Clinical validation
Does the geometry hold up as a surgical target?
A 3D-printed lumbar phantom, tracked optically. The same needle placement is navigated twice — once against the CT-derived STL model, once against the PPCNet point cloud.
Fiducial-based registration achieves an FRE of 0.41 mm, against the 0.87 mm reported for clinical CT-navigated instrumentation. Mean vertebra landmark localisation error across L1–L5 is 3.01 mm.
Conclusion
Explicit projection geometry, not implicit view fusion
PPCNet recovers the lumbar spine (L1–L5) as 8,192 points in world space from two DRRs and their 3×4 projection matrices, improving on volumetric and graph-based baselines spanning 2019 to 2025 under identical splits. Treating the calibrated matrices as the lifting mechanism, rather than learning view correspondences implicitly from concatenated features, is what that gain rests on.
Future work open directions
Sim-to-real. Every result reported here is obtained on synthetic DRRs. Closing the gap to clinical radiographs remains the primary open problem.
Calibration robustness. Behaviour under imperfect projection matrices has not yet been evaluated.