Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Group Meeting Jan 28, 2009: Learning Hierarchies of Invariant Visual Features..
- Speaker: Yann LeCun (NYU)
- Abstract:
There is considerable evidence that object recognition in the brain involves multiple stages of processing from the retina to the inferotemporal cortex. There is theoretical and empirical evidence that complex tasks, such as invariant object recognition, require such “deep” architectures, composed of multiple layers of trainable non-linear modules. Devising efficient learning algorithms for such deep architectures has been a difficult challenge for machine learning, particularly when the number of labelled training samples is limited.
Supervised learning methods applied to neural networks whose architectures is inspired by the visual cortex have been very successful, but have required large numbers of labelled training samples.
Several methods have recently been proposed to train (or pre-train) deep architectures in an unsupervised fashion. Each layer of the deep architecture is composed of a feed-forward encoder which computes a feature vector from the input, and a feed-back decoder which reconstructs the input from the features. A number of such layers can be stacked and trained sequentially, thereby learning a deep hierarchy of features with increasing levels of abstraction. The training of each layer can be seen as shaping an energy landscape with low valleys around the training samples and high plateaus everywhere else.
A particular class of methods for deep energy-based unsupervised learning will be described that imposes sparsity constraints on the features. The method can learn multiple levels of sparse and overcomplete representations of data. When applied to natural image patches, the method produces hierarchies of filters similar to those found in the mammalian visual cortex. Using sparsity constraints on overlapping groups of features produces complex cell-like features with local invariance organized in topographic maps.
An application to category-level object recognition with invariance to pose and illumination will be described. It achieves state of the art performance on a standard dataset with 100 object categories and only 30 labelled samples per categories. Another application to vision-based navigation for off-road mobile robots will be described (with videos). The system autonomously learns to discriminate obstacles from traversable areas at long range.
This is joint work with Koray Kavakcuoglu, and Marc’Aurelio Ranzato, Karol Gregor, Y-Lan Boureau, Raia Hadsell, and Fu-Jie Huang,
- Notes: Wed 4pm