Biometric Authentication


Biometrics deals with recognizing people based on their physiological or behavioral characteristics. Our work primarily concentrates on three different aspects in biometrics:

  • Enhancing Weak Biometrics for Authentication: Weak biometrics (hand-geometry, face, voice, keystrokes) are the traits that possess low discriminating content and they change over time for each individual. However, there are several traits of weak biometrics such as social acceptability, ease of sensing, and lack of privacy concerns that make weak biometrics ideally suited for civilian applications. Methods that we developed can effectively handle the problems of low discriminative power and low feature stability of weak biometrics, as well as time-varying population in civilian applications.
  • Writer Identification from Handwritten Documents: Handwriting is a behavioural biometric that contains distinctive traits aquired by a person over time. Traditional approaches to writer identification tries to compute feature vectors that capture traits of handwriting that are known to experts as discriminative. In contrast we concentrate on automatic extraction of features that are suitable to specific applications such as writer identification in civilian domain and in problems such as forgery and repudiation in forensics.
  • Use of Camera as a Biometric Sensor: Camera has been used for capturing face images for authentication in the past. However, with biometrics traits such as fingerprints and iris, a specialized sensor is often preferred due to the high quality of data that they provide. Recent advances in image sensors have made digital cameras both inexpensive and technically capable for achieving high quality images. However, many problems such as variations in pose, illumination and scale restrict the use of cameras as sensors for many biometric traits. We are working on the use of models of imaging process to overcome these problems, to capture high quality data for authentication.

Enhancing Weak Biometric based Authentication


Weak biometrics (hand-geometry, face, voice, keystrokes) are the traits which possess low discriminating content and they change over time for each individual. Thus they show low accuracy of the system as compared to the strong biometrics (eg. fingerprints, iris, retina, etc.) However, due to exponentially decreasing costs of the hardware and computations, biometrics has found immense use in civilian applications (Time and Attendance Monitoring, Physical Access to Building, Human-Computer Interface, etc.) other than forensics (e.g. criminal and terrorist identification). Various factors need to be considered while selecting a biometric trait for civilian application; most important of which are related to user psychology and acceptability, affordability, etc. Due to these reasons, weak biometric traits are often better suited for civilian applications than the strong biometric traits. In this project, we address issues such as low and unstable discriminating information, which are present in weak biometrics and variations in user population in civilian applications.

schdaDue to the low discriminating content of the weak biometric traits, they show poor performance during verification. We have developed a novel feature selection technique called Single Class Hierarchical Discriminant Analysis (SCHDA), specifically for authentication purpose in biometric systems. SCHDA builds an optimal user-specific discriminant space for each individual where the samples of the claimed identity are well-separated from the samples of all the other users.

The second problem which leads to low accuracy of authentication is the poor stability or permanence of weak biometric traits due to various reasons (eg. ageing, the person gaining or losing weight, etc.) Civilian applications usually operate in cooperative or monitored mode wherein the users can give feedback to the system on occurrence of any errors. An intelligent adaptive framework is used, which uses feedback to incrementally update the parameters of the feature selection and verification framework for each individual.

The third factor that has been explored to improve the performance of an authentication system for civilian applications is the pattern of participation of each enrolled user. As the new users are enrolled into the system, a degradation is observed in performance due to increasing number of users. An interesting observation is that although the number of users enrolled into the system is very high, the number of users who regularly participate in the authentication process is comparatively low. We model the variation in participating population using Markov models. The prior probability of participation of each individual is computed and incorporated into the feature selection framework, providing more relevance to the parameters of regularly participating users. Both the structured and unstructured modes of variation of participation are explored.

Text Independent Writer Identification from Online Handwriting

Handwriting Individuality is a quantitative measure of writer specific information that can be used to identify authorship of the documents and study of comparison of writing habits, evaluation of the significance of their similarities and differences. It is an discrimitive process like fingerprint identification, firearms identification and DNA analysis. Individuality in handwriting lies in the habits that are developed and become consistant to some degree in the process of writing.

Discriminating elements of handwriting lies in various factors such as i) Arrangement, Connections, Constructions, Design, Dimensions, Slant or Slope, Spacings, CLass and choice of allographs, 2) Language styles such as Abbreviation, Commencements and terminations, diacritics and punctuation, line continuity, line quality or fluency, 3) Physical traits such as pen control, pen hold, pen position, pen pressure and writing movement, 4) Consistancy or natural variations and persistance, and 4) Lateral expansion and word proportions.

The framework that we utilize tries to capture the consistent information at various levels and automatically extract discriminative features from them.

Features of our Approach:clusters

  • Text-independent algorithm: Writer can be identified from any text given in underlined script. Comparison of features are not done for the similar charcters.
  • Script dependent framework: Applicablity is verified on different scripts like Devanagiri, Arabic,Roman, Chinese and Hebrew.
  • Use of Online Information: Online data is used for verification purpose. Offline information is also applicable with similar framework with appropriate change in feature extraction.
  • Authentication with small amount of data: Around 12 words in Devanagiri we get accuracy of 87%.



Underlying process of identification:

Represent   velocity 
  • Primitive Definition:

    Primitives are the discrimitive features of handwriting documents. First step is to identify primitive. Primitives can be individuality features like size, shape, distribution of curves in handwritten document. We choose subcharcter level curves as basic primitives

  • Extraction and Representation of primitive:

    Extraction of primitive is done using velocity profile of the stroke shown in the figure. Minimum velocity points are critical points of primitive. Primitives are extracted using size and shape features as shown in diagram.

  • Identification of Consistant Primitives:

    Repeating curves are consitent primitives. To extract consistent curves, unsupervised clustering algorithm is used to cluster them into different groups.

  • Classification:

    Variation in distribution, size and shape of curves in each cluster is used to discriminate writer from other writers.

Related Publications

  • Vandana Roy and C. V. Jawahar - Modeling Time-Varying Population for Biometric Authentication In International Conference on computing: Theory and Applications(ICCTA), Kolkatta, 2007. [PDF]

  • Anoop M. Namboodiri and Sachin Gupta - Text Independent Writer Identification from Online Handwriting, International Workshop on Frontiers in Handwriting Recognition(IWFHR'06), October 23-26, 2006, La Baule, Centre de Congreee Atlantia, France. [PDF]

  • Vandana Roy and C. V. Jawahar, - Hand-Geometry Based Person Authentication Using Incremental Biased Discriminant Analysis, Proceedings of the National Conference on Communication(NCC 2006), Jan 2006 Delhi, January 2006, pp 261-265. [PDF]

  • Vandana Roy and C. V. Jawahar, - Feature Selection for Hand-Geometry based Person Authentication, Proceedings of the Thirteenth International Conference on Advanced Computing and Communications, Coimbatore, December 2005. [PDF]


Associated People

Contours, Textures, Homography and Fourier Domain


The aim of this study is to come up with a Fourier representation of contours and then utilise it to estimate two view relationships like homography and also come up with novel invariants. Ordering in Contours is a very important geometrical information which had been given very less attention till now. We have proposed novel representation for contour sequences in transform domain which helps us exploit the ordering information. This representation was also extended to build affine invariants which could be used in computer vision problems.

A similar transform domain relationship was developed for textures in images. This was used in estimation of homography.


Some of the major contributions of this study are ::

  • Fourier representation of contours.
  • Development of invariants which were demonstrated to be useful in planar shape recognition.
  • Algorithms for homography estimation from textures and contours.
  • Use of invariants to build a polygonal approximation of contours which was used for homography estimation.
  • Successful estimation of geometric relationships like homography and measures like invariants with higher order primitives like contours and conics.
  • Alegraic constratints on a moving point configuration were developed.





Related Publications

  • Paresh Kumar Jain and C.V. Jawahar - Homography Estimation from Planar Contours, Third International Symposium on 3D Data Processing, Visualization and Transmission North Carolina, Chappel Hill, June 14-16, 2006. [PDF]

  • M. Pawan Kumar, Saurabh Goyal, Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Discrete Contours in Multiple Views: Approximation and Recognition Journal of Image and Vision Computin, Vol. 22, No. 14, December 2004, pp. 1229--1239. [PDF]

  • M. Pawan Kumar, Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Planar Homography from Fourier Domain Representation, Proceedings of the International Conference on Signal Processing and Communications(SPCOM), Dec. 2004, Bangalore, India. [PDF]

  • M. Pawan Kumar, C. V. Jawahar and P. J. Narayanan, Geometric Structure Computation from Conics, Proceedings of the Indian Conference on Vision, Graphics and Image Processing(ICVGIP), Dec. 2004, Calcutta, India, pp. 9-14. [PDF]

  • M. Pawan Kumar, C. V. Jawahar and P. J. Narayanan, Building Blocks for Autonomous Navigation using Contour Correspondences, Proceedings of the International Conference on Image Processing(ICIP), Oct. 2004, Singapore, pp. 1381-1384. [PDF]

  • Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Fourier Domain Representation of Planar Curves for Recognition in Multiple Views, Pattern Recognition, Vol. 37, No. 4, April 2004, pp. 739--754. [PDF]

  • Sujit Kuthirummal, C.V. Jawahar and P.J. Narayanan - Algebraic Constraints on Moving Points in Multiple Views, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 311--316. [PDF]

  • M. Pawan Kumar, Saurabh Goyal, C.V. Jawahar, and P.J. Narayanan - Polygonal Approximation of Closed Curves Across Multiple Views, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 317--322. [PDF]

  • Sujit Kuthirummal, C.V. Jawahar and P.J. Narayanan - Multiview Constraints for Recognition of Planar Curves in Fourier Domain, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 323--328. [PDF]

  • Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan, Planar Shape Recognition across Multiple Views, Proceedings of the International Conference on Pattern Recognition(ICPR), Aug. 2002, Quebec City, Canada, pp. 482--488. [PDF]

Associated People

Robotic Vision


Our research activity is primarily concerned with the geometric analysis of scenes captured by vision sensors and the control of a robot so as to perform set tasks by utilzing the scene intepretation. The former problem is popular in literature as 'Structure from Motion', while the later is often refered as the 'Visual Servoing' problem.

Visual servoing consists in using the information provided by a vision sensor to control the movements of a dynamic system. This research topic is at the intersection of the fields of Computer Vision and Robotics. These fields are the subject of profitable research since many years and are particularly interesting by their very broad scientific and application spectrum. More specifically, we are concerned with enhancing the visual servoing algorithms, both in performance and in applicability so as to widen their use.

Performance Enhancement of Visual Servoing Techniques

Visual servoing is an interesting robotic vision area increasingly being applied to real-world problems. Such an application, however calls for an in-depth analysis of robustness and performance issues in visual servoing tasks. Typically, robustness issues involve handling errors in feature correspondence / pose and depth estimation. On the other hand, performance issues involve generating consistent input in-spite of noisy / varying parameters. We have developed algorithms that incorporate multiple cues in order to achieve consistent performance in presence of noisy features.

Visual Servoing in Uncoventional Environments

vsueMost robotic vision algorithms are proposed by envisaging robots operating in structured environments where the world is assumed rigid and planar. These algorithms fail to provide optimum behavior when the robot has to be controlled with respect to active non-rigid non-planar targets. We have developed a new framework for visual servoing that accomplishes the robot-positioning task even in such unconventional environments. We introduced a novel space-time representation scheme for modeling the deformations of a non-rigid object and proposed a new vision-based approach that exploited the two-view geometry induced by the space-time features to perform the servoing task.

Visual Tracking by Integration of Multiple Cuestracking

Object tracking is an important task in robotic vision, particularly for visual servoing. The tracking problem has been modeled in the robotic literature as a motion estimation problem. Thus 3D model based tracking is considered as a pose estimation problem and 2D planar object tracking as a homography estimation problem. There are two major sources of visual features that are used in marker-less visual tracking, edges and texture. Both visual features have advantages and disadvantages that make them suitable/unsuitable in many scenarios. we are designing a robust integration framework using both edge and texture features. This frame work probabilistically integrates the visual information collected from contour and texture. The integration is based on probabilistic goodness weights for each type of feature.    

Probabilistic Robotic Vision We are also currently investigating the utility of applying the rich literature available in the field of Probabilistic Robotics to Computer Vision Problems. Computer Vision problems often involve processing of noisy data. Probabilistic approaches are then appropriate as they allow for uncertainty to be modeled and propagated through the solution process.

Related Publication

  • D. Santohs and C.V. Jawahar - Visual Servoing in Non-Regid Environment: A Space-Time Approach Proc. of IEEE International Conference on Robotics and Automation(ICRA'07), Roma, Italy, 2007. [PDF]

  • A.H. Abdul Hafez and C. V. Jawahar - Probabilistic Integration of 2D and 3D Cues for Visual Servoing, 9th International Conference on Control,Automation,Robotics and Vision(ICARCV'06), Singapore, 5-8 December, 2006. [PDF]

  • A.H. Abdul Hafez and C. V. Jawahar - Integration Framework for Improved Visual Servoing in Image and Cartesian Spaces, International Conference on Intelligent Robots and Systems(IROS'06), Beijing, China,October 9-15, 2006. [PDF]

  • D. Santosh Kumar and C.V. Jawahar - Visual Servoing in Presence of Non-Rigid Motion, Proc. 18th IEEE International Conference on Pattern Recognition(ICPR'06), Hong Kong, Aug 2006. [PDF]

  • A.H. Abdul Hafex and C.V. Jawahar - Target Model Estimation Using Particle Filters for Visual Servoing, Proc. 18th IEEE International Conference on Pattern Recognition(ICPR'06), Hong Kong, Aug 2006. [PDF]

  • Abdul Hafez, Piyush Janawadkar and C.V. Jawahar - Novel view prediction for improved visual servoing, National Conference on Communcations (NCC) 2006, New Delhi
  • Abdul Hafez, and C.V. Jawahar - Minimizing a Class of Hybrid Error Functions for Optimal Pose Alignment, International Conference on Control, Robotics, Automation and Vision (ICARCV) 2006, Singapore.
  • D. Santosh Kumar and C. V. Jawahar - Robust Homography-based Control for Camera Positioning in Piecewise Planar Environments, Indain Conference on Computer Vision, Graphics and Image Processing (ICVGIP) 2006, Madurai
  • Abdul Hafez, Visesh Chari and C.V. Jawahar - Combine Texture and Edges based on Goodness Weights for Planar Object Tracking International Conference on Robotics and Automation (ICRA) 2007, Rome
  • Abdul Hafez and C. V. Jawahar - A Stable Hybrid Visual Servoing Agorithm, International Conference on Robotics and Automation (ICRA) 2007, Rome

Associated People


Content Based Image Retrieval - CBIR

FISH: A Practical System for Fast Interactive Image Search in Huge Database


The problem of search and retrieval of images using relevance feedback has attracted tremendous attention in recent years from the research community. A real-world-deployable interactive image retrieval system must (1) be accurate, (2) require minimal user-interaction, (3) be efficient, (4) be scalable to large collections (millions) of images, and (5) support multi-user sessions. For good accuracy, we need effective methods for learning the relevance of image features based on user feedback, both within a user-session and across sessions. Efficiency and scalability require a good index structure for retrieving results. The index structure must allow for the relevance of image features to continually change with fresh queries and user-feedback. The state-of-the-art methods available today each address only a subset of these issues. In this paper, we build a complete system FISH -- Fast Image Search in Huge databases. In FISH, we integrate selected techniques available in the literature, while adding a few of our own. We perform extensive experiments on real datasets to demonstrate the accuracy, efficiency and scalability of FISH. Our results show that the system can easily scale to millions of images while maintaining interactive response time.

[Project Homepage]

Private Content Based Image Retrieval


For content level access, very often database needs the query as a sample image. However, the image may contain private information and hence the user does not wish to reveal the image to the database. Private Content Based Image Retrieval (PCBIR) deals with retrieving similar images from an image database without revealing the content of the query image. not even to the database server. We propose algorithms for PCBIR, when the database is indexed using hierarchical index structure or hash based indexing scheme. Experiments are conducted on real datasets with popular features and state of the art data structures. It is observed that specialty and subjectivity of image retrieval (unlike SQL queries to a relational database) enables in computationally efficient yet private solutions.

[Project Homepage]

Virtual Textual Representation for Efficient Image Retrieval

vie2The state of the art in contemporary visual object categorization and classification is dominated by “Bag Of Words” approaches. These use either discriminative or generative learning models to learn the object or scene model. In this paper, we propose a novel “Bag of words” approach for content based image retrieval. Images are converted to virtual text documents and a new relevance feedback algorithm is applied on these documents. We explain how our approach is fundamentally different to existing ones and why it is ideally suited for CBIR. We also propose a new hybrid relevance feedback learning model. This merges the best of generative and discriminative approaches to achieve a robust and discriminative visual words based description of a visual concept. Our learning model and “Bag Of Words” approach achieve a balance between good classification and efficient image retrieval.

[Project Homepage]

Effecient Region Based Indexing and Retrieval for Images with Elastic Bucket Tries


Retrieval and indexing in multimedia databases has been an active topic both in the Information Retrieval and com- puter vision communities for a long time. In this paper we propose a novel region based indexing and retrieval scheme for images. First we present our virtual textual description using which, images are converted to text documents con- taining keywords. Then we look at how these documents can be indexed and retrieved using modified elastic bucket tries and show that our approach is one order better than stan- dard spatial indexing approaches. We also show various operations required for dealing with complex features like relevance feedback. Finally we analyze the method compar- atively and and validate our approach.

[Project Homepage]

A Rule-based Approach to Image Retrievalrule1

Imagine the world if computers could comprehend and decipher our verbal descriptions of scenes from the real world and present us with possible pictures of our thoughts. This proved motivation enough for a team from CVIT to exploring the possibility of an image retrieval system which took natural language descriptions of what they were looking for and processed it and closely matched it with the images in the database and presented the users with a select set of retrieved results. A sample query could be like - reddish orage upper egde and bright yellowish centre. The system is a rule-based system where rules describe the image content.

[Project Homepage]

Related Publications

  • Dhaval Mehta, E.S.V.N.L.S.Diwakar, and C. V. Jawahar, A Rule-based Approach to Image Retrieval, Proceedings of the IEEE Region 10 Conference on Convergent Technologies(TENCON), Oct. 2003, Bangalore, India, pp. 586--590. [PDF]


  • Suman Karthik, C.V. Jawahar - Analysis of Relevance Feedback in Content Based Image Retrieval, Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision (ICARCV), 2006, Singapore. [PDF]
  • Suman Karthik, C.V. Jawahar - Virtual Textual Representation for Efficient Image Retrieval, Proceedings of the 3rd International Conference on Visual Information Engineering(VIE), 26-28 September 2006 in Bangalore, India. [PDF]
  • Suman Karthik, C.V. Jawahar - Effecient Region Based Indexing and Retrieval for Images with Elastic Bucket Tries, Proceedings of the International Conference on Pattern Recognition(ICPR), 2006. [PDF]

Associated People

  • Dr. C. V. Jawahar
  • Pradhee Tandon
  • Pramod Sankar
  • Praveen Dasigi
  • Piyush Nigam
  • P. Suman Karthik
  • Natraj J.
  • Saurabh K. Pandey
  • Dhaval Mehta
  • E. S. V. N. L. S. Diwakar

Retrieval from Video Databases


Broadcast Television is one of the primary and popular sources of information. With the advent of video recorders, TV programs could be recorded and stored locally. Digital Libraries of broadcast videos could be easily built with existing technology. The storage, archival, search and retrieval of broadcast videos provide a large number of challenges for the research community. The following projects address these challenges.

Building a Digital Library of Broadcast Videostvserverss

Digital libraries of broadcast videos allows one to archive television content for later viewing and reference. The importance and significance of such a library is similar to building a digital library for all books. The collection in the library is built by recording TV. Since it is difficult to record and store all channels simultaneously, a schedule is chosen to record programmes across various channels. An UI allows users to choose the schedule to be recorded for later viewing.

The videos are stored over multiple nodes that act as a storage cluster. An explicit file system structure is maintained for storing the videos. The file system incorporates the meta level information regarding the videos, such as date, time and channel recorded from. This allows users to easily browse and search the library for programs using these details.

Indexing Broadcast Newsnewsss2

Broadcast news is a class of multimedia that is of importance to both the scientific community and the general public. In broadcast news, the requirement is to provide content-level search and retrieval, which is very challenging. Though many broadcast news datasets are available, there is no collection pertaining to news in the Indian context. We built a system to automatically record and build a respository of Indian news broadcasts.

Our present collection consists of more than a month's news telecasts, recorded from 5 different news channels, covering 3 languages. The size of the collection is an ever increasing number, and is currently limited by the storage space available.

The videos are first divided into stories, by detecting the anchor-person. A keyframe is extracted from each of the shots in the story. An effective and intuitive way of visualising the videos is designed such that the user can get a feel of the content without actually needing to stream the videos and see them. Each video is presented as a slide show of the thumbnails of the constituent shots. User can zoom in on any thumbnail by hovering the mouse pointer over it. The adjacent figure shows a screenshot of this UI.

Searching News Using Overlaid Text (Without Characrter Recognition)


There have been several appraoches to video indexing and retrieval, based on spatio-temporal visual features. However, they are not always reliable and do not allow for easy querying. News video have many clues regarding the content of the video, in the form of overlaid text. This text is reliable for video indexing and retrieval. However, the recognition of overlaid text is difficult, due to the limited accuracy of Optical Character Recognisers (OCR). The inaccuracies are more pronounced in the context of Indian languages, which have a complex script and writing style.

To avoid explicit recognition, we use a novel recognition-free approach that matches words in the image space. Each word extracted from the videos is represented as a feature vector, the features carefully chosen to provide invariance to font type, style and size variations. Given a textual query, the word is rendered into an image and features are extracted from it. The query features are compared to those in the database, using a Dynamic Time Warping (DTW) based distance measure. The words in the databse that have high similarity with the query, are obtained, and the source videos are retrieved for the user.

Automatic Annotation of Cricket Videos


Sports videos are another popular class of multimedia. Sports videos are generally long where only a small part of the video is of real interest. The video is a sequence of action scenes, which occur semi-regularly. It was further observed that, for sports such as Cricket, detailed descriptions of these scenes is provided in the form of textual commentary available from online websites (such as This description is excellent for annotation and retrieval.

However, there is no explicit synchronisation between the textual descriptions, and the video segments that they correspond to. The synchronisation is achieved by using the text to drive the segmentation of the video. The scene categories are modeled in a suitable (visual) feature space, and the text is used to obtain the category of each scene in the video. A hypothetical video is generated from the text, which is aligned with the real video using Dynamic Programming. The scenes in the real video are segmented based on the known scene boundaries of the hypothetical video.

Once segmented, the scenes are automatically annotated with the detailed textual descriptions which allows us to build a Table-of-Contents representation for the entire video. This interface is very intutitve and allows for easy browsing of the video using the corresponding text. The text can now be used to index the videos, which allows for video retrieval using texttual queries (of semantic concepts).

Ongoing Work

In ongoing work, we are addressing various novel directions for enabling retrieval from video collections. In one of the projects for annotating news videos, we are using the people in the news to identify the news content. Faces in the news are annotated with the name of the person, and the news stories can be queried based on the people involved in it.

In another ongoing project, we are exploring the use of various compressed domain techniques for video data retrieval and mining. Videos are generally stored in the MPEG format which is a compressed domain representation. The use of a large number of exisiting techniques for compressed domain, avoids explicit decoding of video, and can convey further information without visual recognition and understanding.

Related Publications

  • Pramod Sankar K., Saurabh Pandey and C. V. Jawahar - Text Driven Temporal Segmentation of Cricket Videos , 5th Indian Conference on Computer Vision, Graphics and Image Processing, Madurai, India, LNCS 4338 pp.433-444, 2006. [PDF]

  • C. V. Jawahar, Balakrishna Chennupati, Balamanohar Paluri and Nataraj Jammalamadaka, Video Retrieval Based on Textual Queries , Proceedings of the Thirteenth International Conference on Advanced Computing and Communications, Coimbatore, December 2005. [PDF]


Associated People