Human Pose Estimation with Iterative Error Feedback

August, 2015 (last update May 2016)

Abstract

Hierarchical feature extractors such as Convolutional Networks (ConvNets) have achieved impressive performance on a variety of classification tasks using purely feed-forward processing. Feedforward architectures can learn rich representations of the input space but do not explicitly model dependencies in the output spaces, that are quite structured for tasks such as articulated human pose estimation or object segmentation. Here we propose a framework that expands the expressive power of hierarchical feature extractors to encompass both input and output spaces, by introducing top-down feedback. Instead of directly predicting the outputs in one go, we use a self-correcting model that progressively changes an initial solution by feeding back error predictions, in a process we call Iterative Error Feedback (IEF). IEF shows excellent performance on the task of articulated pose estimation in the challenging MPII and LSP benchmarks, matching the state-of-the-art without requiring ground truth scale annotation.


Code

Here you can find MATLAB code for testing models learned on the MPII Human Pose Dataset. These models are described in our November revision of the paper, but trained longer. They get 81.8 on the MPII validation set.
Given an image of a person and a point anywhere on the torso - e.g. obtained from a person detector - our system predicts articulated 2D pose. We provide both matconvnet and CAFFE versions. We will make code available for training new models soon.

For the CAFFE version see here

Download the matconvnet version here:
Training code: IEF-Train
Testing code: IEF-test-v1.tgz
Pretrained models: IEF-models-v1.tgz


Publications

Human Pose Estimation with Iterative Error Feedback. J. Carreira, P. Agrawal, K. Fragkiadaki and J. Malik. arXiv, July 2015. pdf

When using our system, please cite this work. The bibtex entry is provided below for your convenience.

@inproceedings{IEF2015human,
 author = {Carreira, Joao and Agrawal, Pulkit and Fragkiadaki, Katerina and Malik, Jitendra},
 title = {Human Pose Estimation with Iterative Error Feedback},
 journal={arXiv preprint arXiv:1507.06550},
 Year = {2015}}