Detection is one of the classical computer vision problems, where axis parallel bounding boxs needs to be identified over each of the objects in an image, from a given set of labels. There could be multiple instances of the object, in which case separate bounding boxes needs to be identified for each instance. This is a well studied problem for common objects with many datasets available (cite Imagenet, MSCOCO cite{MSCOCO}).

Detection for autonomous naviation is also an active area of reasearch. Previous challenges like KITTI, Cityscapses are based on western conditions. Autorikshaws is not part of their label set.

Data Set

The full dataset consists of 1000 images from indian roads, with arbitrary perspectives. The participants will be given 800 images with bounding box annotations of autorickshaws for training/validation. 200 images will be test images.

Please register for getting a sample dataset of 350 training images here : register

The full training set will be released according to the timeline.

Evaluation

For each of the test image $I$, and each ground truth bounding box $g_{k,I}$, IOUs will be computed with each of the predicted bounding boxes $p_{k,I}$. The score of the ground truth $g_{k,I}$ is $\max_{k} IOU(g_{k,I}, p_{k,I})$ and the score for an image $I$ is average of the scores for each of the ground truth bounding boxes in the image. Finally the score for the test dataset is the averages of the score for each of the images in the dataset.

The score of the test dataset will be considered for ranking the participants of the challenge.

Scheme for evaluating results

The 200 test images given without any annotations will be used for calculating the scores on which the participants will be ranked. The participants will be be asked to upload the outputs of their algorithms in a standard format and the scores will be calculated using the ground truths available with the organization team. The winning teams will also be required to run their binaries against the test data at the time of the workshop, in presence of the organizers

Timeline

October 6th : Final relase of the 800 training images with annotations

November 1st : Relase of the 200 testing images

Contact

Dr. Girish Varma,
Machine Learning Lab,
KCIS,