Handwriting Analysis

Introduction

The work in handwriting analysis at CVIT concentrates on Recognition, Synthesis, Annotation, Search, and Classification of handwritten data. We primarily concentrate on online handwriting, where the temporal information of the writing process is available in the handwritten data, although many of the approaches we use are extensible to offline handwriting as well. Specifically, recognition of online handwriting in Indian languages has special significance, as it can form an effective mechanism of data input, as opposed to keyboards that needs multiple keystrokes and control sequences to input many characters.

Handwriting Synthesis

Handwriting synthesis is the problem of generating data close to how a human would write the text. The characterstics of the generated data could be that of specific writer or that from a generic model. Systhesis of handwriting pose a challange as writer spcecific features need to captured and preserved, yet at the same time, variability between handwriting should also be taken into account. Even with the given model, synthesis should not be deterministic since the variation that are found in human handwriting are stochastic.

Application of handwriting synthesis, includes, automatically creation of personalized handwritten documents, large amount of annotated handwritten data for training of recognition engines and writer independent matching and retrieval of handwritten documents.

For synthesis of Indic scripts, we model the handwriting at two levels. A stroke level model is used to capture the writing style and hand movements of the individual strokes. A space-time layout model is then used to arrange the synthesized strokes to form the words. Both the stroke model and the layout model can be learned from examples, and the method can learn from a single example, as well as a large collection to capture the variations. The model also allows us to synthesize the words in multiple Indic scripts through transliteration.

Annotation and Search of Handwritten data

Annotation of handwriting is the process of labeling input data for training for the purpose of a variety of handwriting analysis problems like Handwriting recognition and writer identification systems. However manual annotation of large datasets is tedious, expensive, and error prone process, especially at character and stroke level. Lack of proper linguistic resources in form of annotated data sets is a major hurdle in building recognizers for them.

In many practical situations, plain transcripts of handwritten data is available, which can be used to make the process of annotation, easier. Data collection process can be carried out in different setting like, unrestricted data, designed text, dictation, and data generation (using handwriting synthesis). A parallel text is available in all the above case, except that of unrestricted data. For annotation, we use the model based handwriting synthesis unit described above to map the text corpora to handwriting space and annotation is propagated to word and character levels using elastic matching of handwriting. Stroke level annotation for online handwriting recognition is currently being done using semi-automatic tools.

Online Handwriting Recognition

We aim at building robust and accurate recognition engine for Indian languages, specifically for Hindi, Telugu and Malayalam. There are some features for Indian Languages and their writing, which necessitates a different approach for recognition as compared to English:

• The primary unit of words is an akshara, which is a combination of multiple consonants, and ending in a vowel.
• Each language has a very large set of aksharas - usually multiple thousands
• Each akshara is composed of a single or multiple strokes, and no partial strokes.

A robust and accurate recognition system poses a variety of research challenges, especially for Indian languages. We concentrate on a variety of problems such as building large-class hierarchical classifiers, specifically for handwriting recognition and OCR, Discriminating classifiers for differenting similar looking time-series data (strokes), Compact representation of class models, Efficient spell checkers for languages with large number of work form variations, etc.

Writer Identification

Writer Identification is a process of identifying the authorship of handwritten documents. Relevance of document in civil and criminal litigations, primarily dependent on our ability to assign authorship to the particular document. For more information on writer identification click here.

Related Publications

• Anoop M. Namboodiri and Sachin Gupta - Text Independent Writer Identification from Online Handwriting, International Workshop on Frontiers in Handwriting Recognition(IWFHR'06), October 23-26, 2006, La Baule, Centre de Congreee Atlantia, France. [PDF]
  

• Anand Kumar, A. Balasubramanian, Anoop M. Namboodiri and C.V. Jawahar - Model-Based Annotation of Online Handwritten Datasets, International Workshop on Frontiers in Handwriting Recognition(IWFHR'06), October 23-26, 2006, La Baule, Centre de Congreee Atlantia, France. [PDF]

• Karteek Alahari, Satya Lahari Putrevu and C.V. Jawahar - Learning Mixtures of Offline and Online Features for Handwritten Stroke Recognition, Proc. 18th IEEE International Conference on Pattern Recognition(ICPR'06), Hong Kong, Aug 2006, Vol. III, pp.379-382. [PDF]

• C. V. Jawahar and A. Balasubramanian - Synthesis of Online Handwriting in Indian Languages, International Workshop on Frontiers in Handwriting Recognition(IWFHR'06), October 23-26, 2006, La Baule, Centre de Congree Atlantia, France. [PDF]
  

• Karteek Alahari, Satya Lahari P and C. V. Jawahar - Discriminant Substrokes for Online Handwriting Recognition, Proceedings of Eighth International Conference on Document Analysis and Recognition(ICDAR), Seoul, Korea 2005, Vol 1, pp 499-503. [PDF]
  

• A. Bhaskarbhatla, S. Madhavanath, M. Pavan Kumar, A. Balasubramanian, and C. V. Jawahar - Representation and Annotation of Online Handwritten Data, Proceedings of the International Workshop on Frontiers in Handwriting Recognition(IWFHR), Oct. 2004, Tokyo, Japan, pp. 136--141. [PDF]
  

• Pranav Reddy and C. V. Jawahar, The Role of Online and Offline Features in the Development of a Handwritten Signature Verification System, Proceedings of the National Conference on Document Analysis and Recognition(NCDAR), Jul. 2001, Mandya, India, pp. 85--94. [PDF]
  

 Associated People

Biometric Authentication

Introduction

Biometrics deals with recognizing people based on their physiological or behavioral characteristics. Our work primarily concentrates on three different aspects in biometrics:

• Enhancing Weak Biometrics for Authentication: Weak biometrics (hand-geometry, face, voice, keystrokes) are the traits that possess low discriminating content and they change over time for each individual. However, there are several traits of weak biometrics such as social acceptability, ease of sensing, and lack of privacy concerns that make weak biometrics ideally suited for civilian applications. Methods that we developed can effectively handle the problems of low discriminative power and low feature stability of weak biometrics, as well as time-varying population in civilian applications.
• Writer Identification from Handwritten Documents: Handwriting is a behavioural biometric that contains distinctive traits aquired by a person over time. Traditional approaches to writer identification tries to compute feature vectors that capture traits of handwriting that are known to experts as discriminative. In contrast we concentrate on automatic extraction of features that are suitable to specific applications such as writer identification in civilian domain and in problems such as forgery and repudiation in forensics.
• Use of Camera as a Biometric Sensor: Camera has been used for capturing face images for authentication in the past. However, with biometrics traits such as fingerprints and iris, a specialized sensor is often preferred due to the high quality of data that they provide. Recent advances in image sensors have made digital cameras both inexpensive and technically capable for achieving high quality images. However, many problems such as variations in pose, illumination and scale restrict the use of cameras as sensors for many biometric traits. We are working on the use of models of imaging process to overcome these problems, to capture high quality data for authentication.

Enhancing Weak Biometric based Authentication

Weak biometrics (hand-geometry, face, voice, keystrokes) are the traits which possess low discriminating content and they change over time for each individual. Thus they show low accuracy of the system as compared to the strong biometrics (eg. fingerprints, iris, retina, etc.) However, due to exponentially decreasing costs of the hardware and computations, biometrics has found immense use in civilian applications (Time and Attendance Monitoring, Physical Access to Building, Human-Computer Interface, etc.) other than forensics (e.g. criminal and terrorist identification). Various factors need to be considered while selecting a biometric trait for civilian application; most important of which are related to user psychology and acceptability, affordability, etc. Due to these reasons, weak biometric traits are often better suited for civilian applications than the strong biometric traits. In this project, we address issues such as low and unstable discriminating information, which are present in weak biometrics and variations in user population in civilian applications.

Due to the low discriminating content of the weak biometric traits, they show poor performance during verification. We have developed a novel feature selection technique called Single Class Hierarchical Discriminant Analysis (SCHDA), specifically for authentication purpose in biometric systems. SCHDA builds an optimal user-specific discriminant space for each individual where the samples of the claimed identity are well-separated from the samples of all the other users.

The second problem which leads to low accuracy of authentication is the poor stability or permanence of weak biometric traits due to various reasons (eg. ageing, the person gaining or losing weight, etc.) Civilian applications usually operate in cooperative or monitored mode wherein the users can give feedback to the system on occurrence of any errors. An intelligent adaptive framework is used, which uses feedback to incrementally update the parameters of the feature selection and verification framework for each individual.

The third factor that has been explored to improve the performance of an authentication system for civilian applications is the pattern of participation of each enrolled user. As the new users are enrolled into the system, a degradation is observed in performance due to increasing number of users. An interesting observation is that although the number of users enrolled into the system is very high, the number of users who regularly participate in the authentication process is comparatively low. We model the variation in participating population using Markov models. The prior probability of participation of each individual is computed and incorporated into the feature selection framework, providing more relevance to the parameters of regularly participating users. Both the structured and unstructured modes of variation of participation are explored.

Text Independent Writer Identification from Online Handwriting

Handwriting Individuality is a quantitative measure of writer specific information that can be used to identify authorship of the documents and study of comparison of writing habits, evaluation of the significance of their similarities and differences. It is an discrimitive process like fingerprint identification, firearms identification and DNA analysis. Individuality in handwriting lies in the habits that are developed and become consistant to some degree in the process of writing.

Discriminating elements of handwriting lies in various factors such as i) Arrangement, Connections, Constructions, Design, Dimensions, Slant or Slope, Spacings, CLass and choice of allographs, 2) Language styles such as Abbreviation, Commencements and terminations, diacritics and punctuation, line continuity, line quality or fluency, 3) Physical traits such as pen control, pen hold, pen position, pen pressure and writing movement, 4) Consistancy or natural variations and persistance, and 4) Lateral expansion and word proportions.

The framework that we utilize tries to capture the consistent information at various levels and automatically extract discriminative features from them.

Features of our Approach:

• Text-independent algorithm: Writer can be identified from any text given in underlined script. Comparison of features are not done for the similar charcters.
• Script dependent framework: Applicablity is verified on different scripts like Devanagiri, Arabic,Roman, Chinese and Hebrew.
• Use of Online Information: Online data is used for verification purpose. Offline information is also applicable with similar framework with appropriate change in feature extraction.
• Authentication with small amount of data: Around 12 words in Devanagiri we get accuracy of 87%.

Underlying process of identification:

• Primitive Definition:

  Primitives are the discrimitive features of handwriting documents. First step is to identify primitive. Primitives can be individuality features like size, shape, distribution of curves in handwritten document. We choose subcharcter level curves as basic primitives

• Extraction and Representation of primitive:

  Extraction of primitive is done using velocity profile of the stroke shown in the figure. Minimum velocity points are critical points of primitive. Primitives are extracted using size and shape features as shown in diagram.

• Identification of Consistant Primitives:

  Repeating curves are consitent primitives. To extract consistent curves, unsupervised clustering algorithm is used to cluster them into different groups.

• Classification:

  Variation in distribution, size and shape of curves in each cluster is used to discriminate writer from other writers.

Related Publications

• Vandana Roy and C. V. Jawahar - Modeling Time-Varying Population for Biometric Authentication In International Conference on computing: Theory and Applications(ICCTA), Kolkatta, 2007. [PDF]
  

• Anoop M. Namboodiri and Sachin Gupta - Text Independent Writer Identification from Online Handwriting, International Workshop on Frontiers in Handwriting Recognition(IWFHR'06), October 23-26, 2006, La Baule, Centre de Congreee Atlantia, France. [PDF]
  

• Vandana Roy and C. V. Jawahar, - Hand-Geometry Based Person Authentication Using Incremental Biased Discriminant Analysis, Proceedings of the National Conference on Communication(NCC 2006), Jan 2006 Delhi, January 2006, pp 261-265. [PDF]
  

• Vandana Roy and C. V. Jawahar, - Feature Selection for Hand-Geometry based Person Authentication, Proceedings of the Thirteenth International Conference on Advanced Computing and Communications, Coimbatore, December 2005. [PDF]
  

Associated People

• Vandana Roy
• Sachin Gupta
• Prof. C. V. Jawahar
• Anoop M. Namboodiri

Contours, Textures, Homography and Fourier Domain

Objective

The aim of this study is to come up with a Fourier representation of contours and then utilise it to estimate two view relationships like homography and also come up with novel invariants. Ordering in Contours is a very important geometrical information which had been given very less attention till now. We have proposed novel representation for contour sequences in transform domain which helps us exploit the ordering information. This representation was also extended to build affine invariants which could be used in computer vision problems.

A similar transform domain relationship was developed for textures in images. This was used in estimation of homography.

Contributions

Some of the major contributions of this study are ::

• Fourier representation of contours.
• Development of invariants which were demonstrated to be useful in planar shape recognition.
• Algorithms for homography estimation from textures and contours.
• Use of invariants to build a polygonal approximation of contours which was used for homography estimation.
• Successful estimation of geometric relationships like homography and measures like invariants with higher order primitives like contours and conics.
• Alegraic constratints on a moving point configuration were developed.

Related Publications

• Paresh Kumar Jain and C.V. Jawahar - Homography Estimation from Planar Contours, Third International Symposium on 3D Data Processing, Visualization and Transmission North Carolina, Chappel Hill, June 14-16, 2006. [PDF]
  

• M. Pawan Kumar, Saurabh Goyal, Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Discrete Contours in Multiple Views: Approximation and Recognition Journal of Image and Vision Computin, Vol. 22, No. 14, December 2004, pp. 1229--1239. [PDF]
  

• M. Pawan Kumar, Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Planar Homography from Fourier Domain Representation, Proceedings of the International Conference on Signal Processing and Communications(SPCOM), Dec. 2004, Bangalore, India. [PDF]
  

• M. Pawan Kumar, C. V. Jawahar and P. J. Narayanan, Geometric Structure Computation from Conics, Proceedings of the Indian Conference on Vision, Graphics and Image Processing(ICVGIP), Dec. 2004, Calcutta, India, pp. 9-14. [PDF]
  

• M. Pawan Kumar, C. V. Jawahar and P. J. Narayanan, Building Blocks for Autonomous Navigation using Contour Correspondences, Proceedings of the International Conference on Image Processing(ICIP), Oct. 2004, Singapore, pp. 1381-1384. [PDF]
  

• Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan - Fourier Domain Representation of Planar Curves for Recognition in Multiple Views, Pattern Recognition, Vol. 37, No. 4, April 2004, pp. 739--754. [PDF]
  

• Sujit Kuthirummal, C.V. Jawahar and P.J. Narayanan - Algebraic Constraints on Moving Points in Multiple Views, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 311--316. [PDF]
  

• M. Pawan Kumar, Saurabh Goyal, C.V. Jawahar, and P.J. Narayanan - Polygonal Approximation of Closed Curves Across Multiple Views, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 317--322. [PDF]

• Sujit Kuthirummal, C.V. Jawahar and P.J. Narayanan - Multiview Constraints for Recognition of Planar Curves in Fourier Domain, Proceedings of the Indian Conference on Computer Vision, Graphics and Image Processing(ICVGIP), Dec. 2002, Ahmedabad, India, pp. 323--328. [PDF]
  

• Sujit Kuthirummal, C. V. Jawahar and P. J. Narayanan, Planar Shape Recognition across Multiple Views, Proceedings of the International Conference on Pattern Recognition(ICPR), Aug. 2002, Quebec City, Canada, pp. 482--488. [PDF]
  

Associated People

• Sujit Kuthirummal
• Paresh Kumar Jain
• M. Pawan Kumar
• Saurabh Goyal
• Dr. C. V. Jawahar
• Prof. P. J. Narayanan

Content Based Image Retrieval - CBIR

FISH: A Practical System for Fast Interactive Image Search in Huge Database

The problem of search and retrieval of images using relevance feedback has attracted tremendous attention in recent years from the research community. A real-world-deployable interactive image retrieval system must (1) be accurate, (2) require minimal user-interaction, (3) be efficient, (4) be scalable to large collections (millions) of images, and (5) support multi-user sessions. For good accuracy, we need effective methods for learning the relevance of image features based on user feedback, both within a user-session and across sessions. Efficiency and scalability require a good index structure for retrieving results. The index structure must allow for the relevance of image features to continually change with fresh queries and user-feedback. The state-of-the-art methods available today each address only a subset of these issues. In this paper, we build a complete system FISH -- Fast Image Search in Huge databases. In FISH, we integrate selected techniques available in the literature, while adding a few of our own. We perform extensive experiments on real datasets to demonstrate the accuracy, efficiency and scalability of FISH. Our results show that the system can easily scale to millions of images while maintaining interactive response time.

[Project Homepage]

Private Content Based Image Retrieval

For content level access, very often database needs the query as a sample image. However, the image may contain private information and hence the user does not wish to reveal the image to the database. Private Content Based Image Retrieval (PCBIR) deals with retrieving similar images from an image database without revealing the content of the query image. not even to the database server. We propose algorithms for PCBIR, when the database is indexed using hierarchical index structure or hash based indexing scheme. Experiments are conducted on real datasets with popular features and state of the art data structures. It is observed that specialty and subjectivity of image retrieval (unlike SQL queries to a relational database) enables in computationally efficient yet private solutions.

[Project Homepage]

Virtual Textual Representation for Efficient Image Retrieval

The state of the art in contemporary visual object categorization and classification is dominated by “Bag Of Words” approaches. These use either discriminative or generative learning models to learn the object or scene model. In this paper, we propose a novel “Bag of words” approach for content based image retrieval. Images are converted to virtual text documents and a new relevance feedback algorithm is applied on these documents. We explain how our approach is fundamentally different to existing ones and why it is ideally suited for CBIR. We also propose a new hybrid relevance feedback learning model. This merges the best of generative and discriminative approaches to achieve a robust and discriminative visual words based description of a visual concept. Our learning model and “Bag Of Words” approach achieve a balance between good classification and efficient image retrieval.

[Project Homepage]

Effecient Region Based Indexing and Retrieval for Images with Elastic Bucket Tries

Retrieval and indexing in multimedia databases has been an active topic both in the Information Retrieval and com- puter vision communities for a long time. In this paper we propose a novel region based indexing and retrieval scheme for images. First we present our virtual textual description using which, images are converted to text documents con- taining keywords. Then we look at how these documents can be indexed and retrieved using modified elastic bucket tries and show that our approach is one order better than stan- dard spatial indexing approaches. We also show various operations required for dealing with complex features like relevance feedback. Finally we analyze the method compar- atively and and validate our approach.

[Project Homepage]

A Rule-based Approach to Image Retrieval

Imagine the world if computers could comprehend and decipher our verbal descriptions of scenes from the real world and present us with possible pictures of our thoughts. This proved motivation enough for a team from CVIT to exploring the possibility of an image retrieval system which took natural language descriptions of what they were looking for and processed it and closely matched it with the images in the database and presented the users with a select set of retrieved results. A sample query could be like - reddish orage upper egde and bright yellowish centre. The system is a rule-based system where rules describe the image content.

[Project Homepage]

Related Publications

• Dhaval Mehta, E.S.V.N.L.S.Diwakar, and C. V. Jawahar, A Rule-based Approach to Image Retrieval, Proceedings of the IEEE Region 10 Conference on Convergent Technologies(TENCON), Oct. 2003, Bangalore, India, pp. 586--590. [PDF]
  

• Suman Karthik, C.V. Jawahar - Analysis of Relevance Feedback in Content Based Image Retrieval, Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision (ICARCV), 2006, Singapore. [PDF]
• Suman Karthik, C.V. Jawahar - Virtual Textual Representation for Efficient Image Retrieval, Proceedings of the 3rd International Conference on Visual Information Engineering(VIE), 26-28 September 2006 in Bangalore, India. [PDF]
• Suman Karthik, C.V. Jawahar - Effecient Region Based Indexing and Retrieval for Images with Elastic Bucket Tries, Proceedings of the International Conference on Pattern Recognition(ICPR), 2006. [PDF]

Associated People

• Dr. C. V. Jawahar
• Pradhee Tandon
• Pramod Sankar
• Praveen Dasigi
• Piyush Nigam
• P. Suman Karthik
• Natraj J.
• Saurabh K. Pandey
• Dhaval Mehta
• E. S. V. N. L. S. Diwakar