Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Group Meeting Sep 15, 2008: Learning Multilayer Boltzmann Machines
- Speaker: Ruslan Salakhutdinov
- Abstract:
We present a new learning algorithm for Boltzmann machines that contain many layers of hidden variables. Data-dependent expectations are estimated using a variational approximation that tends to focus on a single mode. Expectations with respect to the model distribution are approximated by applying a stochastic approximation procedure that uses Markov chain Monte Carlo (MCMC). The use of two quite different techniques for estimating the two types of expectation that enter into the gradient of the log-likelihood makes it practical to learn Boltzmann machines with multiple hidden layers and millions of parameters. The learning can be made more efficient by using a layer-by-layer “pre-training” phase that allows variational inference to be initialized by a single bottom-up pass. After learning, annealed importance sampling can be used to produce an accurate estimate of a variational lower bound on the log-probability of test data and this allows us to demonstrate that multilayer Boltzmann machines learn good generative models.