Methods and Theory : Hyper-Features

Overview

During his Ph.D. work at Berkeley, Andras Ferencz addressed the important topic of object identification, the task of determining, from two images, whether two objects are the same or different. He approached the problem by modeling the difference between two images, rather than my modeling the images themselves. The innovation in his work was to develop conditional models of the difference in appearance between two objects in which the conditioning was a function of the appearance of one of the objects.


The rationale behind this approach was as follows. Modeling the appearance of images directly is very difficult. While the appearance of images is extremely complex, certain simple appearance features of images, such as color, can provide critical conditioning features for modeling image differences. That is, by conditioning on certain simple features of the appearance of an image, one can reduce the complexity of the image difference modeling problem, and make it tractable. Andras referred to these conditioning features as hyper-features. This novel approach to modeling image differences conditioned on image appearance was very successful in both vehicle identification and in face identification. You can learn more about hyper-features and their applications from the papers below, or from the project page at Berkeley, or from the UMass face recognition page.
Recently, Vidit Jain has trained the original hyper-feature model in a discriminative fashion (BMVC 06) to improve its accuracy. He has also combined the basic hyper-feature model with a natural language model (ICCV 07) to produce a joint model of text and image differences for clustering face images and text about people. He calls this model People LDA. See the papers below for more information.

Faculty


Students


Collaborators


Publications

  • Andras Ferencz, Erik Learned-Miller, and Jitendra Malik.
    Learning to locate informative features for visual identification.
    To appear International Journal of Computer Vision: Special Issue on Learning and Vision, 2007.
    [pdf]
  • Vidit Jain, Erik Learned-Miller, and Andrew McCallum.
    People-LDA: Anchoring topics to people using face recognition.
    International Conference on Computer Vision (ICCV), 2007.
    [pdf]
  • Gary B. Huang, Vidit Jain, and Erik Learned-Miller.
    Unsupervised joint alignment of complex images.
    International Conference on Computer Vision (ICCV), 2007.
    [pdf]
  • Vidit Jain, Andras Ferencz and Erik Learned-Miller.
    Discriminative training of hyper-feature models for object identification.
    Proceedings of the British Machine Vision Conference (BMVC), Volume 1, pp. 357-366, 2006.
    [pdf]
  • Andras Ferencz, Erik Learned-Miller and Jitendra Malik.
    Learning hyper-features for visual identification.
    In Neural Information Processing Systems (NIPS) 17, pp. 425-432, 2005.
    [pdf]
  • Andras Ferencz, Erik Learned-Miller, and Jitendra Malik.
    Building a classification cascade for visual identification from one example.
    In International Conference on Computer Vision (ICCV), pp. 286-293, 2005.
    [pdf]