Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Group Meeting Feb 2, 2009: Unsupervised Learning of Feature Hierarchies
- Speaker: Marc'Aurelio Ranzato (NYU)
- Abstract:
The applicability of machine learning methods is often limited by the amount of available labeled data, and by the ability (or inability) of the designer to produce good internal representations of the data and good similarity measures to compare them. The main focus of this work has been to alleviate these two limitations by devising unsupervised algorithms that learn good internal representations, and invariant feature hierarchies from unlabeled data.
The first part of the talk will introduce the Energy-Based Model framework for Unsupervised Learning. This framework describes any algorithm in terms of (1) a model architecture, (2) an inference procedure and (3) a loss functional. The model assign a scalar energy to each input data vector. The goal of learning is to make the energy lower around areas of high data density. I will describe the principles of Unsupervised Learning and propose efficient ways to learn the parameters of the model, as well as to infer the representation. I will use this framework to devise a novel sparse coding algorithm, and I will show how a simple extension of this algorithm can learn features that are not only sparse, but also invariant to learned transformations.
These unsupervised algorithms are used as building block to train deep networks, models that are composed by a sequence of non-linear transformations and that produce feature hierarchies. I will demonstrate these algorithms on a variety of tasks, such as visual object recognition and text document classification.