Roland Memisevic I am a research scientist at PNYLab LLC in Princeton, New Jersey, USA. Until summer 2008 I have been a postdoctoral fellow in the machine learning group at the University of Toronto, where I also obtained my PhD in 2008, working with Prof. Geoffrey Hinton. My research interests are in machine learning and pattern recognition. I design algorithms for solving data-intensive problems in areas like Computer Vision, time series prediction, and statistical modelling. To contact me write to roland[at]cs[dot]toronto[dot]edu.

New things...:

• Code for computing Kernel Information Embeddings in Python is now available. Unlike LLE, kernel PCA, Laplacian Eigenmaps, etc., these embeddings can generalize to new, unseen data, since they come with both forward- and backward-mappings.
  
  
• Download 'Monte python': Gradient based learning in pure Python.
  Monte is a framework for building gradient based learning machines, such as neural networks, logistic regression, energy based models, etc. Over time it accumulated some more general learning methods like conditional random fields, nonlinear conditional random fields, sparse regression (LASSO), KNN, and other things.
  
  
• Unsupervised learning is normally about finding good representations for data. This report describes a model that learns to represent data transformations instead.
  
  
• Here is a tutorial-paper (ps, pdf) on modeling structured responses. The paper covers conditional random fields, max-margin-Markov networks and other recent developments, all extending the applicability of supervised learning to problems that are exponentially more interesting than those handled by standard methods.
  
  

Recent talks

• April 2008 Memisevic, R., Hinton, G. E.
  Learning with factorized three-way interactions
  Cifar 2008 workshop on Interneurons.
• December 2007 Memisevic, R., Hinton, G. E.
  Learning the allowable transformations of simple shapes using 3-way interactions
  Cifar Meeting 2007 "Ways to Achieve Perceptual Invariance."
• September 2007 Memisevic, R., Thiesson, B.
  Building a compact multi-resolution index structure
  Microsoft Research, Redmond.
• March 2007 Memisevic, R., Hinton, G. E.
  Unsupervised learning of image transformations
  University of Toronto Machine Learning Group
• July 2006 Memisevic, R., Hinton, G. E.
  Unsupervised learning of image transformations
  CIAR summer school on Neural Computation and Adaptive Perception
• June 2006 Memisevic, R.
  Kernel Information Embeddings
  ICML 2006
• Dec 2005 Memisevic, R.
  CRFs in the dual.
  NIPS Workshop on Kernel Methods and Structured Domains.
• Dec 2005 Memisevic, R., Srebro, N., Roweis, S.
  Exploring the regularization path for adaptive Gaussian kernel SVMs.
  NIPS Workshop on the Accuracy-Regularization Frontier.

Papers

• 2009 Sigal, L., Memisevic, R., Fleet, D.
  Shared kernel information embedding for discriminative inference.
  Computer Vision and Pattern Recognition (CVPR 2009).
  
  
• 2008 Memisevic, R.
  Non-linear latent factor models for revealing structure in high-dimensional data.
  PhD thesis, University of Toronto. [pdf]
  
  
• 2007 Memisevic, R. and Hinton, G. E.
  Unsupervised learning of image transformations.
  Computer Vision and Pattern Recognition (CVPR 2007). [pdf], [bibtex]
  There is also a longer technical report available on the same topic. [pdf]
  
  
• 2007 Samulowitz, H. and Memisevic, R.
  Learning to solve QBF.
  Twenty-Second Conference on Artificial Intelligence (AAAI 2007). Vancouver, British Columbia. [pdf][bibtex]
  
  
• 2006 Memisevic, R.
  An introduction to structured discriminative learning. Technical report.
  Toronto, 2006. [ps],[pdf]
  
  
• 2006 Memisevic, R.
  Dual optimization of conditional probability models. Technical report.
  Toronto, 2006. [ps],[pdf]
  
  
• 2006 Memisevic, R.
  Kernel Information Embeddings.
  23rd International Conference on Machine Learning (ICML 2006). Pittsburgh, Pennsylvania. [pdf],[bibtex]
  
  
• 2006 Memisevic, R.
  Propagating Errors and Beliefs for Large Scale Nonlinear Structure Prediction.
  North East Student Colloquium on Artificial Intelligence. Ithaca, NY.
  
  
• 2005 Meinicke, P., Klanke, S., Memisevic, R., Ritter, H.
  Principal Surfaces from Unsupervised Kernel Regression.
  IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), vol. 27, no. 9, pp. 1379-1391. [bibtex]
  
  
• 2005 Memisevic, R. and Hinton, G. E.
  Improving dimensionality reduction with spectral gradient descent.
  Neural Networks, 18, pp 702-710. [online version],[bibtex]
  
  
• 2005 Memisevic, R. and Hinton, G. E.
  Embedding via clustering: Using spectral information to guide dimensionality reduction.
  International Joint Conference on Neural Networks 2005.
  
  
• 2004 Memisevic, R. and Hinton, G. E.
  Multiple Relational Embedding.
  Advances in Neural Information Processing Systems (NIPS), 17, MIT Press, Cambridge, MA [ps.gz],[pdf],[bibtex]
  
  
• 2003 Memisevic, R.
  Unsupervised Kernel Regression for Nonlinear Dimensionality Reduction. [pdf]
  Master's Thesis, Bielefeld University, 2003.
  The thesis describes a framework, based on kernel density estimation, that reconciles two common, but previously
  separate, views of dimensionality reduction:
  Spectral embedding (such as Laplacian Eigenmaps, ISOMAP, LLE) which allows for efficent, global optimization,
  vs. generative models (aka. latent variable models: Factor Analysis, GTM, Principal Curves, etc.), that provide
  proper density estimates and out-of-sample generalizations but are inefficient and based on non-global optimization.
  Check out this talk (pdf) for the basic ideas.
  

Miscellaneous

• Spam-features:
  Spam filtering is the "prototype" application for simple supervised machine learning. Here is a script, written in Python, that takes two mailboxes (one spam, the other no-spam), extracts word-features (as well as some other features) from the emails, and dumps these into a file for further processing by some classifier. The word-features are extracted using an estimate of the mutual information between words and class-labels. They are created after dropping all the words listed in the file stopwords.txt (in the same directory). For more info, check out the script itself.
• CSC 321: Here are the slides for the tutorials on Matlab for novices, Matlab for machine learning and Probabilities for machine learning)
• minimize.py is a Python adaptation of Carl Rasmussen's matlab function minimize.m
• Slides and a few code-snippets for a ten-minute mini-tutorial on switching from Matlab to Python.

Contact information