Machine Learning Meetings and Events
Group Meetings: Group meetings are held Mondays from 11am to Noon (talk starts 11:10am) in D.L. Pratt 290C unless otherwise noted. Meetings are coordinated by Hugo Larochelle.
Tea Talks: Tea talks are held every Wednesday at 4:00pm in D.L. Pratt 290C. Talks should be simple, accessible, and not exceed 15 minutes. Speakers bring snacks, make tea, and provide a copy of the presented paper.
Group Meeting Sep 22, 2008: Visualizing high-dimensional data using t-SNE
- Speaker: Geoffrey Hinton
- Abstract:
Over the last decade, many new methods have been developed for visualizing high-dimensional data by giving each data-point a location in a two-dimensional map. The goal is to represent the separations of pairs of data-points by the separations of their corresponding map-points, with an emphasis on representing the small separations accurately. I will describe a new method, called t-SNE, that is based on two ideas. The first idea is to convert the set of pairwise distances between data-points into a set of probabilities of selecting pairs of data-points. The selection probability of a pair of points is proportional to a Gaussian function of their separation. If the distances between map-points are converted into pairwise probabilities in the same way, any given arrangement of map-points can be evaluated by measuring the divergence between the probability distributions obtained from the data-points and the map-points. A good arrangement of map-points is then found by performing gradient descent in this divergence.
Unfortunately, if the probabilities of pairs of map-points are computed using a Gaussian function of their separation, the difference between the distributions of pairwise distances in high-dimensional and low-dimensional spaces causes the map-points to be crowded together in the center of the map. This problem can be largely overcome by using a heavy-tailed t-distribution when computing the selection probabilities of pairs of map-points. This leads to maps that look much better than those produced by other recent methods. In particular, t-SNE is very good at preserving clusters in the data at many different scales simultaneously.
The talk describes joint work with Laurens van der Maaten that will appear in the Journal of Machine Learning Research.