Exploiting SfM Data from Community Collections

Introduction

On public photo sharing sites such as flickr, millions of photographs of monuments are easily available. These are popularly known as community photo collections. Progress in the field of 3D Computer Vision has led to the development of robust SfM (Structure from Motion) algorithms. These SfM algorithms, allow us to build 3D models of the monuments using community photo collections. In this domain of SfM algorithms, we work on improving their speed and accuracy. We have also developed tools which exploit the rich geometry information present in the SfM datasets. This geometry allows us to perform fast image localization, feature triangulation and visualization of user photographs.

Projects involving GPU's and CUDA

1. Visibility Probability Structure from SfM Datasets

VisProbStruct

Large scale reconstructions of camera matrices and point clouds have been created using structure from motion from community photo collections. Such a dataset is rich in information; it represents a sampling of the geometry and appearance of the underlying space. In this work, we encode the visibility information between and among points and cameras as visibility probabilities. We combine this visibility probability structure with a distance measure to prioritize points for fast guided search for the image localization problem. We also define the dual problem of feature triangulation as finding the 3D coordinates of a given image feature and solve it efficiently, again using the visibility probability structure.

GdbBrowser

2. Geometry Directed Photo-Browsing for Personal Photos

Browsers of personal digital photographs all essentially follow the slide show paradigm, sequencing through the photos in the order they are taken. A more engaging way to browse personal photographs, especially of a large space like a popular monument, should involve the geometric context of the space. In this work, we develop a geometry directed photo browser that enables users to browse their personal pictures with the underlying geometry of the space to guide the process. We believe personal photo browsers can provide an enhanced sense of one's own experience with a monument using the underlying 3D-geometric context of the monument.

3. Bundle Adjustment on GPU

Large-scale 3D reconstruction has received a lot of attention recently. Bundle adjustment is a key component of the reconstruction pipeline and often its slowest and most computational resource intensive. Its parallel implementations are rare. In this work, we developed a hybrid implementation of sparse bundle adjustment on the GPU using CUDA, with the CPU working in parallel. The algorithm is decomposed into smaller steps, each of which is scheduled on the GPU or the CPU. We developed efficient kernels for the steps and made use of existing libraries for additional performance. Our implementation outperforms the CPU implementation significantly, achieving a speedup of 30-40 times over the standard CPU implementation for datasets with upto 500 images on an Nvidia Tesla C2050 GPU.

Related Publications

  • Aditya Deshpande, Siddharth Choudhary, P J Narayanan , Krishna Kumar Singh, Kaustav Kundu, Aditya Singh, Apurva Kumar - Geometry Directed Browser for Personal Photographs Proceedings of the 8th Indian Conference on Vision, Graphics and Image Processing, 16-19 Dec. 2012, Bombay, India. [PDF]

  • Siddharth Choudhary and P J Narayanan - Visibility Probability Structure from SfM Datasets and Applications Proceedings of 12th European Conference on Computer Vision, 7-13 Oct. 2012, Vol. ECCV 2012, Part-VI, LNCS 7577, Firenze, Italy. [PDF]

  • Siddharth Choudhary, Shubham Gupta and P. J. Narayanan - Practical Time Bundle Adjustment for 3D Reconstruction on GPU Proceedings of ECCV Workshop on Computer Vision on GPU (CVGPU'10),5-11 Sep. 2010, Crete, Greece. [PDF]

 

Associated People

  • Aditya Singh
  • Kaustav Kundu
  • Shubham Gupta
  • Apurva Kumar
  • Rajvi Shah
  • Siddharth Choudhary
  • Krishna Kumar Singh
  • Aditya Deshpande
  • Prof. P J Narayanan