Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Group Meeting Feb 9, 2009: To recognize shapes, first learn to generate reconstruct images.
- Speaker: Hugo Larochelle
- Abstract:
The work on Deep Belief Networks (DBNs) has exposed the usefulness of greedy layer-wise unsupervised learning for successfully training deep neural networks. In DBNs, greedily training a stack of Restricted Boltzmann Machines (RBMs) was proposed and justified as an approach to approximately optimize the likelihood of input vectors according to a DBN. It was also found that the same approach is useful to initialize a standard feed-forward neural networks with many layers of hidden units.
In this talk, I’ll discuss how autoencoders (or autoassociators) can also be useful for training deep neural networks. I’ll present a series of experiments confirming that initializing a deep neural network using autoencoders instead of RBMs can also improve its generalization by successfully leveraging the addition of hidden layers and computing a more complex representation of the input. Moreover, these experiments shed some light on the similarity between RBMs and autoencoders, as well as on the role of greedy unsupervised learning as a regularizer. Finally, I’ll present more recent work on the denoising autoencoder and on a variant of the standard autencoder that allows lateral interactions between units in a hidden layer. These variations are able to improve even more the quality of the representation learned by autoencoders.
- Notes: in PT266