GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis

Generating left and right views from front view alone (low overlap) on Metropolis driving scenes. Ground truth (top), SEVA (middle), GroundControl (bottom); the center column is the front input. Conditioned on a satellite image, GroundControl produces higher-quality views that follow the satellite scene layout.

Abstract

---

Multi-view or video diffusion models have recently emerged as state-of-the-art approaches for novel-view synthesis from sparse image collections. While they can produce high-quality images, they tend to hallucinate unrealistic spatial layouts when synthesizing views outside the input views (scene extrapolation). We propose to address this challenge for outdoor scenes by using satellite images as an additional information source to condition the multi-view generative model. We introduce a new satellite-conditioned multi-view diffusion model that combines conditioning from a satellite image and street-level input images when generating novel views. Our cross-attention based conditioning allows us to retain pretrained multi-view diffusion priors to generate seamless, realistic novel views from a handful of source views, while also adhering to the scene layout as seen from satellite images. Our model does not use 3D scene reconstruction, and only relies on posed street and satellite views. Our experiments on outdoor datasets show that our model is better than both satellite-conditioned or street-view conditioned baselines on novel-view synthesis, especially on highly challenging cases where generated views have little to no overlap with source views.

Method

---

GroundControl is a satellite-conditioned multi-view diffusion model for outdoor novel-view synthesis. Given sparse posed street-level photos and a single satellite image, it generates photorealistic target views that follow the satellite layout without reconstructing a 3D scene, estimating a height map, or requiring pixel-accurate 6DoF ground–satellite alignment. The model needs only metric relative poses between cameras and a coarse 3DoF (X, Y, heading) satellite alignment.

Ray-guided cross-attention

Satellite images differ sharply from street-level photos in both appearance and viewpoint, so letting noisy diffusion tokens attend to the full top-down image is hard to learn. GroundControl instead uses ray-guided cross-attention: each street-view token attends only to satellite tokens along its 3D ray, with Plücker ray and timestep embeddings added to the query.

These layers are added as residual blocks on a frozen SEVA backbone and zero-initialized, so the pretrained multi-view generative prior is preserved at the start of training.

Dual classifier-free guidance

At inference we use dual classifier-free guidance, with separate scales for input-view and satellite conditioning. A single CFG coefficient would couple the two: turning up satellite adherence would also over-steer street-level appearance. Dual CFG lets us strengthen layout fidelity independently, and trade photorealism for satellite adherence depending on the setting.

Comparisons

---

Satellite conditioning improves layout fidelity when synthesizing views with little overlap from the inputs.

On driving data, GroundControl recovers buildings, roads, and street furniture that a street-view-only diffusion model invents or omits. Each row below shows a satellite crop (input cameras in red, the shown target in green), an example input view, the ground-truth target, then predictions from GroundControl, SEVA, and Sat2Density.

In-distribution results on Metropolis and nuScenes. Sat2Density panoramas are cropped to the target pinhole FOV.

The same satellite prior also helps on DL3DV, which is outside our driving-centric training distribution. These results use the geolocalized outdoor split from Cross-View Splatter. Sparse handheld captures still leave background structure underconstrained; GroundControl uses the top-down image to place buildings and open space more faithfully.

Out-of-distribution results on the geolocalized DL3DV split from Cross-View Splatter.

Results

---

In-distribution. We evaluate on held-out nuScenes (150 scenes) and Metropolis (36 scenes) splits that match our training domain. SEVA produces plausible street views from its multi-view prior but weaker layout adherence. Sat2Density, which uses satellite imagery alone, struggles to recover street-level appearance. GroundControl improves both image quality and semantic layout, with the largest gains on Metropolis, where target cameras have very little overlap with the inputs.

Sparse-view novel-view synthesis on in-distribution driving datasets. Semantic IoU uses SAM3 road and building masks.

Out-of-distribution. On 39 geolocalized outdoor DL3DV sequences from Cross-View Splatter (3 input views, 37 target views), GroundControl generalizes despite being trained mainly on driving data. It improves PSNR, SSIM, and multi-view consistency over SEVA, while remaining comparable in perceptual metrics. While DL3DV has more input-target view overlap than Metropolis, GroundControl still improves multi-view consistency (lower TSED).

DL3DV benchmark. *ViewCrafter uses a continuous trajectory and a different input–target split.

Ablating design choices.

Ablations on the Metropolis dataset show that simply treating the satellite image as another input camera view is not enough: a specialized satellite encoder is required. The use of ray-guided cross-attention, ray and timestep query embeddings, and encoder-side conditioning all matter for good satellite-conditioned generation.

Dual CFG adds a separate coefficient for satellite conditioning vs. input-image conditioning, and lets us trade perceptual quality for satellite adherence.

Ablation study on the Metropolis dataset.

Related work

---

• Stable Virtual Camera: Generative View Synthesis with Diffusion Models. Jensen Zhou, Hang Gao, Vikram Voleti, Aaryaman Vasishta, Chun-Han Yao, Mark Boss, Philip Torr, Christian Rupprecht, and Varun Jampani. ICCV, 2025.
• Cross-View Splatter: Feed-Forward View Synthesis with Georeferenced Images. Matias Turkulainen, Akshay Krishnan, Filippo Aleotti, Mohamed Sayed, Guillermo Garcia-Hernando, Juho Kannala, Arno Solin, Gabriel Brostow, and Daniyar Turmukhambetov. CVPR, 2026.

Acknowledgements

---

We thank Filippo Aleotti, Gabriel Brostow, and Alan Paul for their support and insightful feedback, and Jinghao Zhou for clarifications on SEVA.

@inproceedings{krishnan2027groundcontrol,
    title     = {{GroundControl: Satellite-Conditioned Multi-View Diffusion for Reconstruction-Free Novel View Synthesis}},
    author    = {Krishnan, Akshay and Turkulainen, Matias and Wynn, Jamie and Qureshi, Zawar and Sayed, Mohamed and Garcia-Hernando, Guillermo and Vicente, Sara and Hays, James and Firman, Michael and Turmukhambetov, Daniyar},
    booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
    year      = {2027}
}