Machine Learning Research Projects

(ICASSP) International Conference on Acoustics, Speech and Signal Processing: (Papers)

George E. Dahl and Jack W. Stokes and Li Deng and Dong Yu . Large-Scale Malware Classification Using Random Projections and Neural Networks. 2013. in ICASSP.
George E. Dahl and Tara N. Sainath and Geoffrey E. Hinton . Improving Deep Neural Networks for LVCSR Using Rectified Linear Units and Dropout. 2013. in ICASSP.
George E. Dahl and Dong Yu and Li Deng and Alex Acero. Large vocabulary continuous speech recognition with context-dependent DBN-HMMS. 2011. in ICASSP, pages 4688-4691.
Abdel-rahman Mohamed and Tara N. Sainath and George E. Dahl and Bhuvana Ramabhadran and Geoffrey E. Hinton and Michael A. Picheny. Deep Belief Networks using discriminative features for phone recognition. 2011. in ICASSP, pages 5060-5063.
Navdeep Jaitly and Geoffrey E. Hinton. Learning a better Representation of Speech Sound Waves using Restricted Boltzmann Machines. 2010. in International Conference on Acoustics, Speech and Signal Processing.

(ICML) International Conference on Machine Learning: (Papers)

George E. Dahl and Ryan P. Adams and Hugo Larochelle. Training Restricted Boltzmann Machines on Word Observations. 2012. in Proceedings of the 29th International Conference on Machine Learning (ICML-12), pages 679--686. Omnipress, New York, NY, USA.
Daniel Tarlow and Dhruv Batra and Pushmeet Kohli and and Vladimir Kolmogorov. Dynamic Tree Block Coordinate Ascent. 2011.
Tyler Lu and Craig Boutilier. Learning Mallows Models with Pairwise Preferences. 2011. in International Conference on Machine Learning.
Ilya Sutskever and James Martens and Geoff Hinton. Generating Text with Recurrent Neural Networks. 2011. in International Conference on Machine Learning.
James Martens and Ilya Sutskever. Learning Recurrent Neural Networks with Hessian-Free Optimization. 2011. in International Conference on Machine Learning.

(CVPR) IEEE Conference on Computer Vision and Pattern Recognition: (Papers)

Daniel Tarlow and Ryan Adams. Revisiting Uncertainty in Graph Cut Solutions. 2012. in Computer Vision and Pattern Recognition.
Patrick Li and Inmar Givoni and Brendan J. Frey. Learning Better Image Representations Using 'Flobject Analysis'. 2011. in Computer Vision and Pattern Recognition.
Marc'Aurelio Ranzato and Josh Susskind and Volodymyr Mnih and Geoffrey E. Hinton. On Deep Generative Models with Applications to Recognition. 2011. in Computer Vision and Pattern Recognition.

(AISTATS) International Conference on Artificial Intelligence and Statistics: (Papers)

Jasper Snoek, Ryan Prescott Adams and Hugo Larochelle. On Nonparametric Guidance for Learning Autoencoder Representations. 2012. in International Conference on Artificial Intelligence and Statistics.
Daniel Tarlow and Richard Zemel. Structured Output Learning with High Order Loss Functions. 2012. in International Conference on Artificial Intelligence and Statistics.
Daniel Tarlow and Ryan Adams and Richard Zemel. Randomized Optimum Models for Structured Prediction. 2012. in International Conference on Artificial Intelligence and Statistics.
Hugo Larochelle and Iain Murray. The Neural Autoregressive Distribution Estimator. 2011. in International Conference on Artificial Intelligence and Statistics. (Oral and Notable Paper award.)
Ryan Adams and Hanna Wallach and Zoubin Ghahramani. Learning the Structure of Deep Sparse Graphical Models. 2010. in International Conference on Artificial Intelligence and Statistics.
Shai Ben-David and Tyler Lu and Teresa Luu and David Pal. Impossibility Theorems for Domain Adaptation. 2010. in International Conference on Artificial Intelligence and Statistics.
Nevena Lazic and Brendan J. Frey and Parham Aarabi. Solving the Uncapacitated Facility Location Problem Using Message Passing Algorithms. 2010. in International Conference on Artificial Intelligence and Statistics.
Tyler Lu and David Pal and Martin Pal. Showing Relevant Ads via Context Multi-Armed Bandits. 2010. in International Conference on Artificial Intelligence and Statistics.
Marc'Aurelio Ranzato and Alex Krizhevsky and Geoff Hinton. Factored 3-Way Restricted Boltzmann Machines For Modeling Natural Images. 2010. in International Conference on Artificial Intelligence and Statistics.
James Martens and Ilya Sutskever. Parallelizable Sampling for MRFs. 2010. in International Conference on Artificial Intelligence and Statistics.
Iain Murray and Ryan P. Adams and David J.C. MacKay. Elliptical Slice Sampling. 2010. in International Conference on Artificial Intelligence and Statistics.
Ruslan Salakhutdinov and Hugo Larochelle. Efficient Learning of Deep Boltzmann Machines. 2010. in International Conference on Artificial Intelligence and Statistics.
Ilya Sutskever and Tijmen Tieleman. On the Convergence Properties of Contrastive Divergence. 2010. in International Conference on Artificial Intelligence and Statistics.
Daniel Tarlow and Inmar Givoni and Richard Zemel. HOP-MAP: Efficient Message Passing with High Order Potentials. 2010. in International Conference on Artificial Intelligence and Statistics.

(KDD) International Conference on Knowledge Discovery and Data Mining: (Papers)

Anitha Kannan, Inmar Givoni, Rakesh Agrawal and Ariel Fuxman. Matching Unstructured Product Offers to Structured Product Specifications. 2011. in International Conference on Knowledge Discovery and Data Mining.

(NIPS) Conference on Neural Information Processing Systems: (Papers)

Ruslan Salakhutdinov and Josh Tenenbaum and Antonio Torralba. Learning to Learn with Compound Hierarchical-Deep Models. 2011. in Neural Information Processing Systems.
Joseph Lim and Ruslan Salakhutdinov and Antonio Torralba. Transfer Learning by Borrowing Examples. 2011. in Neural Information Processing Systems.
Rina Foygel and Ruslan Salakhutdinov and Ohad Shamir and Nathan Srebro. Learning with the Weighted Trace-norm under Arbitrary Sampling Distributions. 2011. in Neural Information Processing Systems.
Marc'Aurelio Ranzato and Volodymyr Mnih and Geoffrey E. Hinton. How to Generate Realistic Images Using Gated MRF's. 2010. in Neural Information Processing Systems.
George E. Dahl and Marc'Aurelio Ranzato and Abdel-rahman Mohamed and Geoffrey E. Hinton. Phone Recognition with the Mean-Covariance Restricted Boltzmann Machine. 2010. in Neural Information Processing Systems.
Hugo Larochelle and Geoffrey E. Hinton. Learning to Combine Foveal Glimpses with a Third-order Boltzmann Machine. 2010. in Neural Information Processing Systems.
Paolo Viappiani and Craig Boutilier. Optimal Bayesian Recommendation Sets and Myopically Optimal Choice Query Sets. 2010. in Neural Information Processing Systems.
Iain Murray and Ryan Prescott Adams. Slice Sampling Hyperparameters of Latent Gaussian Models. 2010. in Neural Information Processing Systems.
Ryan Prescott Adams and Zoubin Ghahramani and Michael I. Jordan. Tree-Structured Stick Breaking for Hierarchical Data. 2010. in Neural Information Processing Systems.

(UAI) Conference on Uncertainty in Artificial Intelligence: (Papers)

Daniel Tarlow and Inmar Givoni and Richard Zemel and Brendan Frey. Graph Cuts is a Max-Product Algorithm. 2011. in Conference on Uncertainty in Artificial Intelligence. (Runner up for best student paper award.)
Laurent Charlin and Richard Zemel and Craig Boutilier. A Framework for Optimizing Paper Matching. 2011. in Conference on Uncertainty in Artificial Intelligence.
Inmar Givoni and Clement Chung and Brendan Frey. Hierarchical Affinity Propagation. 2011. in Conference on Uncertainty in Artificial Intelligence.

Alternative Splicing: The coding capacity of the vertebrate genome is greatly expanded by alternative splicing, which enables a single gene to produce more than one distinct protein.The Frey and Blencowe labs at the University of Toronto have combined forces to develop a 'splicing code' that accurately predicts how hundreds of RNA features work together to regulate tissue-dependent alternative splicing for thousands of exons. It has been used to predict how alternative splicing may play important roles in development and neurological processes, and has provided insights into mechanisms of splicing regulation. The code has also been incorporated into a web tool.

Yoseph Barash and John A. Calarco and Weijun Gao and Qun Pan and Xinchen Wang and Ofer Shai and Benjamin J. Blencowe and Brendan J. Frey. Deciphering the splicing code.. 2010. Nature, 465:53-59.

Audio, Speech and Language Processing: (papers)

Geoffrey E. Hinton and Li Deng and Dong Yu and George E. Dahl and Abdel-rahman Mohamed and Navdeep Jaitly and Andrew Senior and Vincent Vanhoucke and Patrick Nguyen and Tara N. Sainath and Brian Kingsbury. Deep Neural Networks for Acoustic Modeling in Speech Recognition . 2012. IEEE Signal Processing Magazine, vol 29.
Dahl, G.E. and Dong Yu and Li Deng and Acero, A. Context-Dependent Pre-Trained Deep Neural Networks for Large-Vocabulary Speech Recognition. 2012. Audio, Speech, and Language Processing, IEEE Transactions on, 20(1):30 -42.
Mohamed, A. and Dahl, G.E. and Hinton, G. Acoustic Modeling Using Deep Belief Networks. 2012. Audio, Speech, and Language Processing, IEEE Transactions on, 20(1):14 -22.

Affinity Propagation: An algorithm that identifies exemplars among data points and forms clusters of data points around these exemplars. It operates by simultaneously considering all data point as potential exemplars and exchanging messages between data points until a good set of exemplars and clusters emerges.

Inmar Givoni and Brendan J. Frey. Semi-Supervised Affinity Propagation with Instance-Level Constraints. 2009. in AISTATS.
Inmar Givoni and Brendan J. Frey. A Binary Variable Model for Affinity Propagation. 2009. Neural Computation.
Daniel Tarlow, Richard Zemel and Brendan J. Frey. Flexible Priors for Exemplar-based Clustering. 2008. in Uncertainty in Artificial Intelligence.

Neural Processing:

Rama Natarajan and Iain Murray and Ladan Shams and Richard Zemel. Characterizing response behavior in multisensory perception with conflicting cues. 2009. in Advances in Neural Information Processing Systems 21, pages 1153--1160.

Protein Sequence Classification and Motif Discovery:

R. Min and A. Bonner J. Li and Z. Zhang. Learned Random-Walk Kernels and Empirical-Map Kernels for Protein Sequence Classification. 2009. Journal of Computational Biology, 16(3):457--474.
R. Min and R. Kuang and A. Bonner and Z. Zhang. Learning Random-Walk Kernels for Protein Remote Homology Identification and Motif Discovery. 2009. (null).

Symbolic Reasoning and Learning:

Sutskever, I. and Hinton G.E. Using matrices to model symbolic relationships. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.

Cumulative Distribution Networks:

Jim C. Huang and Brendan J. Frey. Structured ranking learning using cumulative distribution networks. 2009. in Advances in Neural Information Processing Systems 21.
Jim C. Huang and Brendan J. Frey. Cumulative distribution networks and the derivative-sum-product algorithm. 2008.

Inverting Generative Black Boxes:

Vinod Nair and Josh Susskind and Geoffrey E. Hinton. Analysis-by-Synthesis by Learning to Invert Generative Black Boxes. 2008. in Artificial Neural Networks - ICANN 2008 , 18th International Conference, Prague, Czech Republic, September 3-6, 2008, Proceedings, Part I, pages 971-981.

Mimicking Go Experts with Convolutional Neural Networks:

Sutskever, I. and Nair, V. Mimicking Go Experts with Convolutional Neural Networks. 2008.

Temporal Kernel Recurrent Neural Networks:

Sutskever, I. and Hinton, G. Temporal Kernel Recurrent Neural Networks. 2008. (null).

Language Modeling:

Zhang Yuecheng and Andriy Mnih and Geoffrey E. Hinton. Improving a statistical language model by modulating the effects of context words. 2008. in ESANN 2008, 16th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 23-25, 2008, Proceedings, pages 493-498.
Andriy Mnih and Geoffrey E. Hinton. Three new graphical models for statistical language modelling. 2007. in Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvalis, Oregon, USA, June 20-24, 2007, pages 641-648.

Combining Discriminative Features To Infer Complex Trajectories: A conditional model for time-series regression.

Identification of microRNA Targets:

Jim C. Huang and Tomas Babak and Timothy W. Corson and Gordon Chua and Sophia Khan and Brenda L. Gallie and Timothy R. Hughes and Benjamin J. Blencowe and Brendan J. Frey and Quaid D. Morris. Using expression profiling data to identify human microRNA targets. 2007. Nature Methods, 4:1045--1049.
Jim C. Huang and Quaid D. Morris and Brendan J. Frey. Detecting microRNA targets by linking sequence, microRNA and gene expression data. 2006.

Modeling Human Motion:

Graham W. Taylor and Geoffrey E. Hinton and Sam T. Roweis. Modeling Human Motion Using Binary Latent Variables. 2006. in Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pages 1345-1352.

Multiple-Cause Vector Quantization: Learning parts-based models of data.

Inferring Motor Programs from Images of Handwritten Digit:

Geoffrey E. Hinton and Vinod Nair. Inferring Motor Programs from Images of Handwritten Digits. 2005. in Advances in Neural Information Processing Systems 18 [Neural Information Processing Systems, NIPS 2005, December 5-8, 2005, Vancouver, British Columbia, Canada].

Dimensionality Reduction, Embedding, and Data Visualization:

L.J.P. van der Maaten and G.E. Hinton. Visualizing High-Dimensional Data Using t-SNE. 2008. Journal of Machine Learning Research, 9:2579-2605.
Cook, J. and Sutskever, I. and Mnih, A. and Hinton, G. Visualizing similarity data with a mixture of maps. 2007.
R. R. Salakhutdinov and G. E. Hinton. Learning a non-linear embedding by preserving class neighbourhood structure. 2007. in AI and Statistics.
G. E. Hinton and R. R. Salakhutdinov . Reducing the Dimensionality of Data with Neural Networks . 2006. Science, 313(5786):504-507.
A. Globerson and S. Roweis. Metric Learning by Collapsing Classes. 2006. in Advances in Neural Information Processing Systems 19 (NIPS'05), pages 451--458.
Roland Memisevic and Geoffrey E. Hinton. Improving dimensionality reduction with spectral gradient descent. 2005. Neural Networks, 18(5-6):702-710.
Roland Memisevic and Geoffrey E. Hinton. Multiple Relational Embedding. 2004. in Advances in Neural Information Processing Systems 17 [Neural Information Processing Systems, NIPS 2004, December 13-18, 2004, Vancouver, British Columbia, Canada].
Jacob Goldberger and Sam T. Roweis and Geoffrey E. Hinton and Ruslan Salakhutdinov. Neighbourhood Components Analysis. 2004. in Advances in Neural Information Processing Systems 17 [Neural Information Processing Systems, NIPS 2004, December 13-18, 2004, Vancouver, British Columbia, Canada].
G. Hinton and S. T. Roweis. Stochastic Neighbor Embedding. 2003. in Advances in Neural Information Processing Systems 15 (NIPS'02), pages 857--864.
Sam T. Roweis and Lawrence K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. 2000. Science, 290(5500):2323-2326. [PDF]

Restricted Boltzmann Machines and Deep Belief Networks: The group does a significant amount of work on the unsupervised discovery of data representations using Restricted Boltzmann Machines and Deep Belief Networks.

Iain Murray and Ruslan Salakhutdinov. Evaluating probabilities under high-dimensional latent variable models. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.
Tanya Schmah and Geoffrey E Hinton and Richard Zemel and Steven L Small and Stephen Strother. Generative versus discriminative training of RBMs for classification of fMRI images. 2009. in Advances in Neural Information Processing Systems 21, pages 1409--1416.
Ruslan Salakhutdinov and Iain Murray. On the quantitative analysis of Deep Belief Networks. 2008. in Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pages 872--879. Omnipress.
Tijmen Tieleman. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. 2008. in Proceedings of the International Conference on Machine Learning.
Sutskever, I. and Hinton, G.E. Learning multilevel distributed representations for high-dimensional sequences. 2007.
Sutskever, I. and Hinton, G.E. Deep, narrow sigmoid belief networks are universal approximators. 2008. Neural Computation, 20(11):2629--2636.
Sutskever, I., Hinton G.E. and Taylor, G.W. The Recurrent Temporal Restricted Boltzmann Machine. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.
Vinod Nair and Geoffrey E. Hinton. Implicit Mixtures of Restricted Boltzmann Machines. 2008. in Advances in Neural Information Processing Systems 21, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 8-11, 2008.
Roland Memisevic and Geoffrey E. Hinton. Unsupervised Learning of Image Transformations. 2007. in 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), 18-23 June 2007, Minneapolis, Minnesota, USA.
Ruslan Salakhutdinov and Andriy Mnih and Geoffrey E. Hinton. Restricted Boltzmann machines for collaborative filtering. 2007. in Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvalis, Oregon, USA, June 20-24, 2007, pages 791-798.
Ruslan Salakhutdinov and Geoffrey E. Hinton. Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes. 2007. in Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007.
Geoffrey E. Hinton and Simon Osindero and Yee Whye Teh. A Fast Learning Algorithm for Deep Belief Nets. 2006. Neural Computation, 18(7):1527-1554.
Geoffrey E. Hinton. What kind of graphical model is the brain?. 2005. in IJCAI-05, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, UK, July 30-August 5, 2005, pages 1765-.
Geoffrey E. Hinton and Max Welling and Andriy Mnih. Wormholes Improve Contrastive Divergence. 2003. in Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003, December 8-13, 2003, Vancouver and Whistler, British Columbia, Canada].

Glove-Talk: A neural network that converts gestures into real-time speech.

Helmholtz Machines: Unsupervised learning using bottom-up recognition models.

Hinton, G. E. and Zemel, R. S. Autoencoders, Minimum Description Length, and Helmholtz Free Energy. 1994. in Advances in Neural Information Processing Systems 6. [PDF]
R. S. Zemel and G. E Hinton. Learning Population Codes by Minimizing Description Length. 1995. Neural Computation, 7:549-564. [PDF]

Elastic Models: Using deformable models to recognize hand-written digits.