Image Retrieval using Textual Cues

Anand Mishra, Karteek Alahari and C V Jawahar

ICCV 2013

Abstract

We present an approach for the text-to-image retrieval problem based on textual content present in images. Given the recent developments in understanding text in images, an appealing approach to address this problem is to localize and recognize the text, and then query the database, as in a text retrieval problem. We show that such an approach, despite being based on state-of-the-art methods, is insufficient, and propose a method, where we do not rely on an exact localization and recognition pipeline. We take a query-driven search approach, where we find approximate locations of characters in the text query, and then impose spatial constraints to generate a ranked list of images in the database. The retrieval performance is evaluated on public scene text datasets as well as three large datasets, namely IIIT scene text retrieval, Sports-10K and TV series-1M, we introduce.

Contributions

  • Query driven approach for scene text based image retrieval
  • Scene text indexing without explicit text localization and recognition
  • Category as well as instance retrieval
  • The largest datasets for the problem

Paper

ICCV 2013 Paper / Poster

Datasets

IIIT Scene Text Retrieval (IIIT STR)

Video Scene Text Retrieval Datasets (TV series-1M and Sports-10K)

Extended results

TBA 

BibTeX

@InProceedings{Mishra13,
  author    = "Mishra, A. and Alahari, K. and Jawahar, C.~V.",
  title     = "Image Retrieval using Textual Cues",
  booktitle = "Proceedings of IEEE International Conference on Computer Vision",
  year      = "2013"
}

Acknowledgements

Anand Mishra is partly supported by MSR India PhD Fellowship 2012.

Copyright Notice

The documents contained in these directories are included by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their works here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright.