Machine Learning Research Projects

Restricted Boltzmann Machines and Deep Belief Networks: The group does a significant amount of work on the unsupervised discovery of data representations using Restricted Boltzmann Machines and Deep Belief Networks.

Iain Murray and Ruslan Salakhutdinov. Evaluating probabilities under high-dimensional latent variable models. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.
Tanya Schmah and Geoffrey E Hinton and Richard Zemel and Steven L Small and Stephen Strother. Generative versus discriminative training of RBMs for classification of fMRI images. 2009. in Advances in Neural Information Processing Systems 21, pages 1409--1416.
Ruslan Salakhutdinov and Iain Murray. On the quantitative analysis of Deep Belief Networks. 2008. in Proceedings of the 25th Annual International Conference on Machine Learning (ICML 2008), pages 872--879. Omnipress.
Tijmen Tieleman. Training Restricted Boltzmann Machines using Approximations to the Likelihood Gradient. 2008. in Proceedings of the International Conference on Machine Learning.
Sutskever, I. and Hinton, G.E. Learning multilevel distributed representations for high-dimensional sequences. 2007.
Sutskever, I. and Hinton, G.E. Deep, narrow sigmoid belief networks are universal approximators. 2008. Neural Computation, 20(11):2629--2636.
Sutskever, I., Hinton G.E. and Taylor, G.W. The Recurrent Temporal Restricted Boltzmann Machine. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.
Vinod Nair and Geoffrey E. Hinton. Implicit Mixtures of Restricted Boltzmann Machines. 2008. in Advances in Neural Information Processing Systems 21, Proceedings of the Twenty-Second Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 8-11, 2008.
Roland Memisevic and Geoffrey E. Hinton. Unsupervised Learning of Image Transformations. 2007. in 2007 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2007), 18-23 June 2007, Minneapolis, Minnesota, USA.
Ruslan Salakhutdinov and Andriy Mnih and Geoffrey E. Hinton. Restricted Boltzmann machines for collaborative filtering. 2007. in Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvalis, Oregon, USA, June 20-24, 2007, pages 791-798.
Ruslan Salakhutdinov and Geoffrey E. Hinton. Using Deep Belief Nets to Learn Covariance Kernels for Gaussian Processes. 2007. in Advances in Neural Information Processing Systems 20, Proceedings of the Twenty-First Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 3-6, 2007.
Geoffrey E. Hinton and Simon Osindero and Yee Whye Teh. A Fast Learning Algorithm for Deep Belief Nets. 2006. Neural Computation, 18(7):1527-1554.
Geoffrey E. Hinton. What kind of graphical model is the brain?. 2005. in IJCAI-05, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, UK, July 30-August 5, 2005, pages 1765-.
Geoffrey E. Hinton and Max Welling and Andriy Mnih. Wormholes Improve Contrastive Divergence. 2003. in Advances in Neural Information Processing Systems 16 [Neural Information Processing Systems, NIPS 2003, December 8-13, 2003, Vancouver and Whistler, British Columbia, Canada].

Affinity Propagation: An algorithm that identifies exemplars among data points and forms clusters of data points around these exemplars. It operates by simultaneously considering all data point as potential exemplars and exchanging messages between data points until a good set of exemplars and clusters emerges.

Inmar Givoni and Brendan J. Frey. Semi-Supervised Affinity Propagation with Instance-Level Constraints. 2009. in AISTATS.
Inmar Givoni and Brendan J. Frey. A Binary Variable Model for Affinity Propagation. 2009. Neural Computation.
Daniel Tarlow, Richard Zemel and Brendan J. Frey. Flexible Priors for Exemplar-based Clustering. 2008. in Uncertainty in Artificial Intelligence.

Neural Processing:

Rama Natarajan and Iain Murray and Ladan Shams and Richard Zemel. Characterizing response behavior in multisensory perception with conflicting cues. 2009. in Advances in Neural Information Processing Systems 21, pages 1153--1160.

Protein Sequence Classification and Motif Discovery:

R. Min and A. Bonner J. Li and Z. Zhang. Learned Random-Walk Kernels and Empirical-Map Kernels for Protein Sequence Classification. 2009. Journal of Computational Biology, 16(3):457--474.
R. Min and R. Kuang and A. Bonner and Z. Zhang. Learning Random-Walk Kernels for Protein Remote Homology Identification and Motif Discovery. 2009. (null).

Symbolic Reasoning and Learning:

Sutskever, I. and Hinton G.E. Using matrices to model symbolic relationships. 2009. in Advances in Neural Information Processing Systems 21, pages 1137--1144.

Cumulative Distribution Networks:

Jim C. Huang and Brendan J. Frey. Structured ranking learning using cumulative distribution networks. 2009. in Advances in Neural Information Processing Systems 21.
Jim C. Huang and Brendan J. Frey. Cumulative distribution networks and the derivative-sum-product algorithm. 2008.

Inverting Generative Black Boxes:

Vinod Nair and Josh Susskind and Geoffrey E. Hinton. Analysis-by-Synthesis by Learning to Invert Generative Black Boxes. 2008. in Artificial Neural Networks - ICANN 2008 , 18th International Conference, Prague, Czech Republic, September 3-6, 2008, Proceedings, Part I, pages 971-981.

Mimicking Go Experts with Convolutional Neural Networks:

Sutskever, I. and Nair, V. Mimicking Go Experts with Convolutional Neural Networks. 2008.

Temporal Kernel Recurrent Neural Networks:

Sutskever, I. and Hinton, G. Temporal Kernel Recurrent Neural Networks. 2008. (null).

Language Modeling:

Zhang Yuecheng and Andriy Mnih and Geoffrey E. Hinton. Improving a statistical language model by modulating the effects of context words. 2008. in ESANN 2008, 16th European Symposium on Artificial Neural Networks, Bruges, Belgium, April 23-25, 2008, Proceedings, pages 493-498.
Andriy Mnih and Geoffrey E. Hinton. Three new graphical models for statistical language modelling. 2007. in Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvalis, Oregon, USA, June 20-24, 2007, pages 641-648.

Combining Discriminative Features To Infer Complex Trajectories: A conditional model for time-series regression.

Identification of microRNA Targets:

Jim C. Huang and Tomas Babak and Timothy W. Corson and Gordon Chua and Sophia Khan and Brenda L. Gallie and Timothy R. Hughes and Benjamin J. Blencowe and Brendan J. Frey and Quaid D. Morris. Using expression profiling data to identify human microRNA targets. 2007. Nature Methods, 4:1045--1049.
Jim C. Huang and Quaid D. Morris and Brendan J. Frey. Detecting microRNA targets by linking sequence, microRNA and gene expression data. 2006.

Modeling Human Motion:

Graham W. Taylor and Geoffrey E. Hinton and Sam T. Roweis. Modeling Human Motion Using Binary Latent Variables. 2006. in Advances in Neural Information Processing Systems 19, Proceedings of the Twentieth Annual Conference on Neural Information Processing Systems, Vancouver, British Columbia, Canada, December 4-7, 2006, pages 1345-1352.

Multiple-Cause Vector Quantization: Learning parts-based models of data.

Inferring Motor Programs from Images of Handwritten Digit:

Geoffrey E. Hinton and Vinod Nair. Inferring Motor Programs from Images of Handwritten Digits. 2005. in Advances in Neural Information Processing Systems 18 [Neural Information Processing Systems, NIPS 2005, December 5-8, 2005, Vancouver, British Columbia, Canada].

Dimensionality Reduction, Embedding, and Data Visualization:

L.J.P. van der Maaten and G.E. Hinton. Visualizing High-Dimensional Data Using t-SNE. 2008. Journal of Machine Learning Research, 9:2579-2605.
Cook, J. and Sutskever, I. and Mnih, A. and Hinton, G. Visualizing similarity data with a mixture of maps. 2007.
R. R. Salakhutdinov and G. E. Hinton. Learning a non-linear embedding by preserving class neighbourhood structure. 2007. in AI and Statistics.
G. E. Hinton and R. R. Salakhutdinov . Reducing the Dimensionality of Data with Neural Networks . 2006. Science, 313(5786):504-507.
A. Globerson and S. Roweis. Metric Learning by Collapsing Classes. 2006. in Advances in Neural Information Processing Systems 19 (NIPS'05), pages 451--458.
Roland Memisevic and Geoffrey E. Hinton. Improving dimensionality reduction with spectral gradient descent. 2005. Neural Networks, 18(5-6):702-710.
Roland Memisevic and Geoffrey E. Hinton. Multiple Relational Embedding. 2004. in Advances in Neural Information Processing Systems 17 [Neural Information Processing Systems, NIPS 2004, December 13-18, 2004, Vancouver, British Columbia, Canada].
Jacob Goldberger and Sam T. Roweis and Geoffrey E. Hinton and Ruslan Salakhutdinov. Neighbourhood Components Analysis. 2004. in Advances in Neural Information Processing Systems 17 [Neural Information Processing Systems, NIPS 2004, December 13-18, 2004, Vancouver, British Columbia, Canada].
G. Hinton and S. T. Roweis. Stochastic Neighbor Embedding. 2003. in Advances in Neural Information Processing Systems 15 (NIPS'02), pages 857--864.
Sam T. Roweis and Lawrence K. Saul. Nonlinear Dimensionality Reduction by Locally Linear Embedding. 2000. Science, 290(5500):2323-2326. [PDF]

Glove-Talk: A neural network that converts gestures into real-time speech.

Helmholtz Machines: Unsupervised learning using bottom-up recognition models.

Hinton, G. E. and Zemel, R. S. Autoencoders, Minimum Description Length, and Helmholtz Free Energy. 1994. in Advances in Neural Information Processing Systems 6. [PDF]
R. S. Zemel and G. E Hinton. Learning Population Codes by Minimizing Description Length. 1995. Neural Computation, 7:549-564. [PDF]

Elastic Models: Using deformable models to recognize hand-written digits.