FISH: Fast Interactive Image Search in Huge Databases


Introduction

Automated management of the content rich and voluminous multimedia data has attracted tremendous attention over the last decades. Any promising solution should ideally have easy web based access with intuitive querying methods and should be designed to respond with majority of relevant results within a few fractions of a seconds, from amongst millions of real images. This requires the building blocks of the solution like the user interface, representation and indexing, comparison and retrieval, learning and memorization, result presentation etc. all to perform together. Literature abounds with work focusing on small subsets of these blocks but a work unifying the blocks is missing. A practical solution requisites optimal integration all the blocks.


Solution Summary

In FISH we present a complete end-to-end practical system for fast image search in huge databases FISH uses a query-by-example based simplistic interface which maximizes user input. Our proposal uses a set of standard MPEG-7 visual descriptors for representing images. We achieve this with a highly adaptive B+ - tree based indexing scheme which supports the inherent organization of data into similarity clusters. We also adopt a set of popular relevance feedback approaches for incremental short term learning and propose a highly suited scheme for inexpensive and adaptive inter-query or long term learning. We also use the long term learning based long term memory for adaptive improving ROI extraction from images. In our experiments FISH demonstrated commendable retrieval performance from a million real images within fractions of a second.


Some Results

We present here the results from some of our major experiments while redirect the reader to the paper for a detailed discussion.

screen results1 screen results3
Learning based improvement across feedback iterations over a randomly picked train query

 

 

chotaTimeline2  chotaTimeline 
  Long term memory based extraction of ROI shown for a few sample images from the database.

 


 

Related Publications

  • Pradhee Tandon, Piyush Nigam, Vikram Pudi, C. V. Jawahar - FISH: A Practical System for Fast Interactive Image Search in Huge Databases, Proceedings of the 7th ACM International Conference on Image and Video Retrieval (CIVR '08), July 7-9, 2008, Niagara Falls, Canada.

Associated People


Additionals

 

paper

snap 
Paper [pdf] Demo

 

 

Terrain Data Set


Introduction

This is a complex scene with around twelve objects. It has a base terrain object, many trees and grass objects. These objects are ac3d objects which have been imported into the tool and placed to a complex scene. This scene is a dynamic scene unlike the other two data sets which are static. This is a data set which is ideal for testing motion tracking algorithms.


Terrain Data

TerrainDataThis data set includes the following ::

  • Images of the scene in Images directory.
  • Depth-maps of the scene in DepthMaps directory.
  • Alpha-maps of the scene in AlphaMaps directory.
  • Object-maps of the scene in ObjMaps directory.
  • Scene file used for creating this data BoxTrees.scene.
  • Models used in the scene models directory.
  • Each directory in this data set consists of various frames in the scene.

Box & Trees Data Set


This is a complex scene with around eight objects. It has around five trees and some grass objects. These objects are ac3d objects which have been imported into the tool and placed to a complex scene. This is a data set which is ideal for testing alpha matte algorithms. The thin corners of the leaves of plants are difficult to seperate out. This data set provides a set of images rendered at various resolutions.


Box Trees Data

BoxTreesThis data set includes the following ::

  • Images of the scene in Images directory.
  • Depth-maps of the scene in DepthMaps directory.
  • Alpha-maps of the scene in AlphaMaps directory.
  • Object-maps of the scene in ObjMaps directory.
  • Scene file used for creating this data BoxTrees.scene.
  • POV-Ray scene description of the scene BoxTrees.pov.
  • Models used in the scene models directory.
  • Each directory includes different resolutions of the representations.

Comic Scene Data Set


Introduction

This is a complex scene with around five objects. It has two ape objects, a tux object, a globe object and finally a box object. These objects are ac3d objects which have been imported into the tool and placed to a complex scene. This is a data set which is ideal for testing segmentation algorithms. This scene has may parts of the objects occuluded and could be a good test for accuracy of such algorithms.


Comic Scene Data

ComicSceneThis data set includes the following ::

  • Images of the scene in Images directory.
  • Depth-maps of the scene in DepthMaps directory.
  • Alpha-maps of the scene in AlphaMaps directory.
  • Object-maps of the scene in ObjMaps directory.
  • Scene file used for creating this data BoxTrees.scene.
  • POV-Ray scene description of the scene BoxTrees.pov.
  • Models used in the scene models directory.

DGTk Project Page


Introduction

DGTk is a unique tool which provides the UI of a standard 3D authoring tool and at the same time enables the users to generate various representations like depth-maps, alpha-maps, object-maps, etc which are very precious for the CV and IBR researchers. Two years in the making, this tool has evolved from a simple command line based tool to a fully 3D visualization and rendering software. The user can with ease create dynamic scenes with complex animations. The major goal in development of this tool was to reduce the time spent by researchers in creating or finding data sets for testing their algorithms. Another major goal was to make sharing of data as easy as possible. We have developed a new high level scene representation format which enables users to exchange the data generation medium rather than the data itself.


Downloads

The idea behind creating this tool was to provide the CV and IBR research community with a tool which would provide a standard method for creating test data. This tool is hence provided for the community under GPL along with some of the data sets generated.

  • Download source code of DGTk [Click]. Contains::
    • Source files (.cpp and .h)
    • ReadMe.txt (a brief description about our tool).
    • INSTALL (process of getting started with the tool).
  • User manual for DGTk [Click].
  • Download the manual for the scene file description [Click].

Example Data Sets

 BoxTrees ComicScene   TerrainData

 


Associated People

  • V. Vamsi Krishna
  • Prof. P. J. Narayanan