High Quality Image Reconstruction by Optimal Capture and Accurate Registration

Himanshu Arora (homepage)

Generating high-resolution and high quality images from degraded images has a variety of applications in space imaging, medical imaging, commercial videography and recognition algorithms. Recently, camera-equipped cell-phones are widely being used for day-to-day photography. Limitations of the image capturing hardware and environmental attenuations introduce degradations into the captured image. The quality of reconstructed image obtained from most of the algorithms are highly sensitive to accurate computation of different underlying parameters such as blur, noise, geometric deformation, etc. Variations in blur, illumination and noisy conditions together with occlusions further make the computation of these underlying parameters difficult.

One of the ways of generating a high-quality image is by fusing multiple images, which are displaced at sub-pixel levels. This method is also popularly known as multi-frame Super-resolution (SR). Most multi-image SR algorithms assume that the exact registration and blur parameters between the constituent frames are known. Traditional approaches for image registration are either sensitive to image degradations such as variations in blur, illumination and noise, or are limited in the class of image transformations that can be estimated. These conditions are frequently violated in real-world imaging, where specular surfaces, close light sources, small sensors and lenses create large variations in illumination, noise, and blur within the scene. Interestingly, these are the exact situations, where one would like to employ SR algorithms.

We explore an alternate solution to the problem of robustness in the registration step of a SR algorithm. We present an accurate registration algorithm that uses the local phase information, which is robust to the above degradations. The registration step is formulated as optimization of the local phase alignment at various spatial frequencies. We derive the theoretical error rate of the estimates in presence of non-ideal band-pass behavior of the filter and show that the error converges to zero over iterations. We also show the invariance of local phase to a class of blur kernels. Experimental results on images taken under varying conditions are demonstrated.

Recently, Lin and Shum has shown an upper limit on multi-frame SR techniques. For practical purposes this limit is very small. Another class of SR algorithms formulate the high-quality image generation as an inference problem. High-resolution image is inferred from a set of learned training patches. This class of algorithm works well for natural structures but for many man-made structures this technique does not produce accurate results. We propose to capture the images at optimal zoom from the perspective of image super-resolution. The images captured at this zoom has sufficient amount of information so that it can be magnified further by using any SR algorithm which promotes step edges and certain features. This can have a variety of applications in consumer cameras, large-scale automated image mosaicing, robotics and improving the recognition accuracy of many computer vision algorithms. Existing efforts are limited to image a pre-determined object at the right zoom. In the proposed approach, we learn the patch structures at various down-sampling factors. To further enhance the output we impose the local context around the patch in a MRF framework. Several constraints are introduced to minimize the extent of zoom-in.

Projector-Camera systems are used for various applications in computer vision, immersive environments, visual servoing, etc. Due to gaps between neighboring pixels on the projector's image plane and variations in scene depth, the image projected onto a scene shows pixelation and blurring artifacts. In certain vision and graphics applications, it is required that high quality composition of the scene and the projected image is captured, excluding the artifacts, while retaining the scene characteristics. The localization and restoration of projected pixels is difficult due to various factors such as spatially varying blur, background texture, noise, shapes of scene objects, and color transformations of projector and camera. We extend the usage of local phase, computed using the Gabor filter, to isolate each of the projected pixels distinctively. The local phase is also invariant to a class of blur kernels. For restoration of the captured images, we reproject the projected pixels such that these artifacts are absent. To improve the quality further we propose a mechanism to virtualize a high-resolution projector.

Related Publications

• Himanshu Arora and Anoop M. Namboodiri - How much zoom is the righ zoom from the perspective of Super-Resolution? IEEE Sixth Indian Conference on Computer Vision, Graphics & Image Processing (ICVGIP 2008), pp. 142-149, 16-19 Dec,2008, Bhubaneswar, India. [PDF]

• Himanush Arora and Anoop M. Namboodiri - Projected Pixel Localization and Artifact Removal in Captured Images IEEE TENCON, November 18-21,2008, Hyderabad, India. [PDF]

• Himanshu Arora, Anoop M. Namboodiri and C. V. Jawahar - Robust Image Registration with Illumination Blur and Noice Variations for Super_Resolution IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP 2008),Mar 30-April 4, Las Vegas, USA. [PDF]
  

• Himanshu Arora, Anoop M. Namboodiri and C.V. Jawahar - Accurate Image Registration from Local Phase Information Proc. of The Thirteen National Conference on Communications(NCC 2007), Kanpur, 2007. [PDF]
  

Downloads