Relevance vector machine

From Infogalactic: the planetary knowledge core
Jump to: navigation, search

In mathematics, a relevance vector machine (RVM) is a machine learning technique that uses Bayesian inference to obtain parsimonious solutions for regression and probabilistic classification.[1] The RVM has an identical functional form to the support vector machine, but provides probabilistic classification.

It is actually equivalent to a Gaussian process model with covariance function:

k(\mathbf{x},\mathbf{x'}) = \sum_{j=1}^N \frac{1}{\alpha_j} \varphi(\mathbf{x},\mathbf{x}_j)\varphi(\mathbf{x}',\mathbf{x}_j)

where \varphi is the kernel function (usually Gaussian),\alpha_j's as the variances of the prior on the weight vector w \sim N(0,\alpha^{-1}I) ,and \mathbf{x}_1,\ldots,\mathbf{x}_N are the input vectors of the training set.[citation needed]

Compared to that of support vector machines (SVM), the Bayesian formulation of the RVM avoids the set of free parameters of the SVM (that usually require cross-validation-based post-optimizations). However RVMs use an expectation maximization (EM)-like learning method and are therefore at risk of local minima. This is unlike the standard sequential minimal optimization (SMO)-based algorithms employed by SVMs, which are guaranteed to find a global optimum (of the convex problem).

The relevance vector machine is patented in the United States by Microsoft.[2]

See also


  1. Tipping, Michael E. (2001). "Sparse Bayesian Learning and the Relevance Vector Machine". Journal of Machine Learning Research. 1: 211–244.<templatestyles src="Module:Citation/CS1/styles.css"></templatestyles>
  2. US 6633857, Michael E. Tipping, "Relevance vector machine" 


External links