Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Tea Talk Mar 25, 2009: "Topmoumoute online natural gradient algorithm" by Le Roux, Manzagol, Bengio.
- Speaker: Vlad Mnih
- Link: http://www.google.com/url?sa=U&start=3&q=http://research.microsoft.com/en-us/people/nicolasl/tonga.pdf&ei=EfH1Sc2cMJPIMqXf2bwP&usg=AFQjCNE6zdX3G1cezVh5Ca7h93nvul6zeg
- Abstract: Guided by the goal of obtaining an optimization algorithm that is both fast and yielding good generalization, we study the descent direction maximizing the decrease in generalization error or the probability of not increasing generalization error. The surprising result is that from both the Bayesian and frequentist perspectives this can yield the natural gradient direction. Although that direction can be very expensive to compute we develop an efficient, general, online approximation to the natural gradient descent which is suited to large scale problems. We report experimental results showing much faster convergence in computation time and in number of iterations with TONGA (Topmoumoute Online natural Gradient Algorithm) than with stochastic gradient descent, even on very large datasets.