Fine-Grain Annotation of Cricket Videos

Abstract: 

 It's a sample video taken of IPL channel from Youtube

The recognition of human activities is one of the key problems in video understanding. Action recognition is challenging even for specific categories of videos, such as sports, that contain only a small set of actions. Interestingly, sports videos are accompanied by detailed commentaries available online, which could be used to perform action annotation in a weakly-supervised setting.

For the specific case of Cricket videos, we address the challenge of temporal segmentation and annotation of actions with semantic descriptions. Our solution consists of two stages. In the first stage, the video is segmented into ``scenes'', by utilizing the scene category information extracted from text-commentary. The second stage consists of classifying video-shots as well as the phrases in the textual description into various categories. The relevant phrases are then suitably mapped to the video-shots. The novel aspect of this work is the fine temporal scale at which semantic information is assigned to the video. As a result of our approach, we enable retrieval of specific actions that last only a few seconds, from several hours of video. This solution yields a large number of labelled exemplars, with no manual effort, that could be used by machine learning algorithms to learn complex actions.

Contribution:

In this paper, we present a solution that enables rich semantic annotation of Cricket videos at a fine temporal scale. Our approach circumvents technical challenges in visual recognition by utilizing information from online text-commentaries. We obtain a high annotation accuracy, as evaluated over a large video collection. The annotated videos shall be made available for the community for benchmarking, such a rich dataset is not yet available publicly. In future work, the obtained labelled datasets could be used to learn classifiers for fine-grain activity recognition and understanding.

Solution:

In the first stage, the goal is to align the two modalities at a “scene” level. This stage consists of a joint synchronisation and segmentation of the video with the text commentary.

Scene Segmentation:

A typical scene in a Cricket match follows the sequence of events depicted in figure

Model Learning

It was observed that the visual-temporal patterns of the scenes are conditioned on the outcome of the event. In other words, a 1-Run outcome is visually different from a 4-Run outcome

Four Run Model	One Run Model

While the visual model described above could be useful in recognizing the scene category for a given video segment, it cannot be immediately used to spot the scene in a full-length video. Conversely, the temporal segmentation of the video is not possible without using an appropriate model for the individual scenes themselves. This chicken-and-egg problem can be solved by utilizing the scene-category information from the parallel text-commentary.

Dataset:

Our dataset is collected from the YouTube channel for the Indian Premier League(IPL) tournament.
The Commentary data is collected from CricInfo by web Crawling.

Results:

Related Publications:

Rahul Anand Sharma, Pramod Sankar, C. V. Jawahar - Fine-Grain Annotation of Cricket Videos Proceedings of the Third Asian Conference on Pattern Recognition 3-6 Nov 2015, Kuala Lumpur, Malaysia. [PDF]

Code and Dataset:

• Code can be Downloaded here Code
• Crinfo Commentary Dataset can be Downloaded here Dataset
• IPL Dataset is available on IPL's official youtube channel IPL
• Divide Each Match into 10 over chunks and run the above code. Refer README inside code directory.
• In case of any queries/doubts please contact      This email address is being protected from spambots. You need JavaScript enabled to view it.

Associated People:

• Rahul Anand Sharma
• Pramod Sankar K
• C. V. Jawahar