SemanticEdge Labeling usingDepth cues

Nishit Soni

Abstract

Contours are critical inhumanperception of a scene. Theyprovide information about object boundaries, surface planes and surface intersections. This information helps to isolate objects from a scene. In computer vision, contours have similar importance. It has been shown that labelled edges can contribute to segmentation, reconstruction and recognition problems. This thesis has addressed edge labeling of images in indoor and outdoor scenes using depth and RGB data. We classify the contours as occluding, planar (depth discontinuity), and convex, concave (surface normal discontinuity). This task is not straightforward and it is one of the fundamental problems in computer vision.We propose a novel algorithm using random forest for classifying edge pixels into occluding, planar, convex and concave entities.We approach the problem by first focusing on indoor images where we use depth information fromKinect. We release an indoor data set withmore than 500 RGBD images with pixel-wise ground labels. Our method produces promising results and achieves an F-score of 0.84. We also test the approach onmore complex images from from NYU kinect data set and we obtain F-Score of 0.74. While addressing this problem in outdoor images where we use depth from stereo, we realise the need for additional features. Stereo depth of outdoor scenes has artifacts and errors which cannot confidently represent an edge type locally.We show that a simple feature based on semantic classes helps improving the labeling. On Kitti outdoor driving stereo data set, we obtain occluding and planar average F-Score of 0.77 while the approach works poorly to classify curvature edges i.e convex and concave edges. We find this to be because of stereo depth errors and low resolution depth at far distance, which gives poor feature extraction. However,we acknowledge the potential of using semantic classes to improve edge labeling and with large amount of ground truth edgelabels andbetter semantic segmentation, there is ahope of improving the classification.

Related Publications

Downloads