Human Shape Capture and Tracking at Home


Human body tracking typically requires specialized capture set-ups. Although pose tracking is available in consumer devices like Microsoft Kinect, it is restricted to stick figures visualizing body part detection. In this paper, we propose a method for full 3D human body shape and motion capture of arbitrary movements from the depth channel of a single Kinect, when the subject wears casual clothes. We do not use the RGB channel or an initialization procedure that requires the subject to move around in front of the camera. This makes our method applicable for arbitrary clothing textures and lighting environments, with minimal subject intervention. Our method consists of 3D surface feature detection and articulated motion tracking, which is regularized by a statistical human body model. We also propose the idea of a Consensus Mesh (CMesh) which is the 3D template of a person created from a single view point. We demonstrate tracking results on challenging poses and argue that using CMesh along with statistical body models can improve tracking accuracies. Quantitative evaluation of our dense body tracking shows that our method has very little drift which is improved by the usage of CMesh.



Major Contributions

  • Temporal tracking on monocular depth input for subjects wearing casual clothes, using a statistical human body model.
  • Creation of 3D template mesh (Consensus mesh) of a person using a single Kinect.
  • Simple capture setup of one Kinect, captured dataset and code to be released soon.

Related Publications

  • Gaurav Mishra, Saurabh Saini, Kiran Varanasi and P.J. Narayanan, Human Shape Capture and Tracking at Home, IEEE Winter Conference on Applications of Computer Vision (WACV 2018), Lake Tahoe, CA, USA, 2018.. [PDF] [Supp] [ WACV Presentation ]

Code and Dataset:

We will release code and captured dataset for this project soon.


If you use this work or dataset, please cite :

    title={Human Shape Capture and Tracking at Home},
    author={Mishra, Gaurav and Saini, Saurabh and Varanasi, Kiran and Narayanan, P.~J.},
    booktitle={IEEE Winter Conference on Applications of Computer Vision (WACV 2018), Lake Tahoe, CA, USA},