Rapid Improvements on NeRFs!
In two recent articles, Local Radiance Fields (LocalRF) and Google's Research team present their unique approaches to reconstructing realistic, large-scale scenes. LocalRF focuses on casual video captures and Google focused on using high quality inputs from DSLRs. Both approaches led to some really clean Neural Radiance Fields (NeRFs)!
LocalRF reconstructs large-scale scenes by jointly estimating camera poses with radiance fields progressively, allocating new local radiance fields in a dynamic manner, and scaling to handle large unbounded scenes. LocalRF addresses the challenges of accurate camera pose estimation and the limitations of a single global radiance field for longer trajectories. It demonstrates higher robustness against pose drifts, and the progressive optimization significantly improves the reconstruction's quality.
They show some really nice results in their paper and on their website. The team did mention they did not perform any global bundle adjustment or loop closures, I would assume both would lead to an even better result.
On the other hand, Google's Research team uses NeRF reconstruction to create photorealistic, interactive indoor experiences in a service titled Immersive View that focuses on indoor venues, such as cafes, restaurants, and bars.
Google's approach captures scenes with DSLR cameras, processes images for Structure-from-Motion, and trains a new NeRF model on each captured location. They utilize 360° videos to provide an interactive user experience, enabling users to virtually explore spaces while maintaining a controllable environment and delivering high-fidelity visual content via NeRF.
Overall, both approaches showcase the significant advancements and potential of NeRF reconstruction in wide-ranging applications, be it large-scale outdoor scenes or immersive indoor experiences. Their goals are to expand the NeRF's capabilities, such as scene segmentation, adapting to outdoor collections, and enabling real-time 3D exploration on-device, to create universally accessible AI-powered immersive experiences.
The pace of progress in this space is quite incredible.