tdistributed stochastic neighbor embedding
Machine learning and data mining 

Machine learning venues

tdistributed stochastic neighbor embedding (tSNE) is a machine learning algorithm for dimensionality reduction developed by Laurens van der Maaten and Geoffrey Hinton.^{[1]} It is a nonlinear dimensionality reduction technique that is particularly well suited for embedding highdimensional data into a space of two or three dimensions, which can then be visualized in a scatter plot. Specifically, it models each highdimensional object by a two or threedimensional point in such a way that similar objects are modeled by nearby points and dissimilar objects are modeled by distant points.
The tSNE algorithm comprises two main stages. First, tSNE constructs a probability distribution over pairs of highdimensional objects in such a way that similar objects have a high probability of being picked, whilst dissimilar points have an infinitesimal probability of being picked. Second, tSNE defines a similar probability distribution over the points in the lowdimensional map, and it minimizes the Kullback–Leibler divergence between the two distributions with respect to the locations of the points in the map. Note that whilst the original algorithm uses the Euclidean distance between objects as the base of its similarity metric, this should be changed as appropriate.
tSNE has been used in a wide range of applications, including computer security research,^{[2]} music analysis,^{[3]} cancer research,^{[4]} and bioinformatics.^{[5]}
Details
Given a set of highdimensional objects , tSNE first computes probabilities that are proportional to the similarity of objects and , as follows:
The bandwidth of the Gaussian kernels , is set in such a way that the perplexity of the conditional distribution equals a predefined perplexity using a binary search. As a result, the bandwidth is adapted to the density of the data: smaller values of are used in denser parts of the data space.
tSNE aims to learn a dimensional map (with ) that reflects the similarities as well as possible. To this end, it measures similarities between two points in the map and , using a very similar approach. Specifically, is defined as:
Herein a heavytailed Studentt distribution (with onedegree of freedom, which is the same as a Cauchy distribution) is used to measure similarities between lowdimensional points in order to allow dissimilar objects to be modeled far apart in the map.
The locations of the points in the map are determined by minimizing the (nonsymmetric) Kullback–Leibler divergence of the distribution from the distribution , that is:
The minimization of the Kullback–Leibler divergence with respect to the points is performed using gradient descent. The result of this optimization is a map that reflects the similarities between the highdimensional inputs well.
References
 ↑ van der Maaten, L.J.P.; Hinton, G.E. (Nov 2008). "Visualizing HighDimensional Data Using tSNE" (PDF). Journal of Machine Learning Research. 9: 2579–2605.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
 ↑ Gashi, I.; Stankovic, V.; Leita, C.; Thonnard, O. (2009). "An Experimental Study of Diversity with Offtheshelf AntiVirus Engines". Proceedings of the IEEE International Symposium on Network Computing and Applications: 4–11.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
 ↑ Hamel, P.; Eck, D. (2010). "Learning Features from Music Audio with Deep Belief Networks". Proceedings of the International Society for Music Information Retrieval Conference: 339–344.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
 ↑ Jamieson, A.R.; Giger, M.L.; Drukker, K.; Lui, H.; Yuan, Y.; Bhooshan, N. (2010). "Exploring Nonlinear Feature Space Dimension Reduction and Data Representation in Breast CADx with Laplacian Eigenmaps and tSNE". Medical Physics. 37 (1): 339–351. doi:10.1118/1.3267037.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
 ↑ Wallach, I.; Liliean, R. (2009). "The ProteinSmallMolecule Database, A NonRedundant Structural Resource for the Analysis of ProteinLigand Binding". Bioinformatics. 25 (5): 615–620. doi:10.1093/bioinformatics/btp035.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
Software
 tDistributed Stochastic Neighbor Embedding http://homepage.tudelft.nl/19j49/tSNE.html