Von Mises–Fisher distribution

In directional statistics, the von Mises–Fisher distribution (named after Richard von Mises and Ronald Fisher), is a probability distribution on the sphere in If
the distribution reduces to the von Mises distribution on the circle.

The expected value of the Von Mises–Fisher distribution is not on the unit hypersphere, but instead has a length of less than one. This length is given by as defined above. For a Von Mises–Fisher distribution with mean direction and concentration , the expected value is:
For , the expected value is at the origin. For finite , the length of the expected value is strictly between zero and one and is a monotonic rising function of
The empirical mean (arithmetic average) of a collection of points on the unit hypersphere behaves in a similar manner, being close to the origin for widely spread data and close to the sphere for concentrated data. Indeed, for the Von Mises–Fisher distribution, the expected value of the maximum likelihood estimate based on a collection of points is equal to the empirical mean of those points.

Von Mises Fisher (VMF) distributions are closed under orthogonal linear transforms. Let be a by orthogonal matrix. Let and apply the invertible linear transform: The inverse transform is , because the inverse of an orthogonal matrix is its transpose: The Jacobian of the transform is , for which the absolute value of its determinant is 1, also because of the orthogonality. Using these facts and the form of the VMF density, it follows that:

One may verify that since and are unit vectors, then by the orthogonality, so are and .

When , the Von Mises–Fisher distribution, on simplifies to the uniform distribution on The density is constant with value Pseudo random samples can be generated by generating samples in from the standard multivariate normal distribution, followed by normalization to unit norm.

For , let be any component of The marginal distribution for has the density:

where is the beta function. This distribution may be better understood by highlighting its relation to the beta distribution:

where the Legendre duplication formula is useful to understand the relationships between the normalization constants of the various densities above. Note that the components of are not independent, so that the uniform density is not the product of the marginal densities; and cannot be assembled by independent sampling of the components.

In machine learning, especially in image classification, to be classified inputs (e. g. images) are often compared using cosine similarity, which is the dot product between intermediate representations in the form of unitvectors (termed embeddings). The dimensionality is typically high, with at least several hundreds. The deep neural networks that extract embeddings for classification should learn to spread the classes as far apart as possible and ideally this should give classes that are uniformly distributed on For a better statistical understanding of across class cosine similarity, the distribution of dot products between unitvectors independently sampled from the uniform distribution may be helpful. Let be unitvectors in , independently sampled from the uniform distribution. Define:

where is the dot product and are transformed versions of it. Then the distribution for is the same as the marginal component distribution given above;) has the density

supported on the Stiefel manifold of orthonormal p frames , where is an arbitrary real matrix.

Ulrich, in designing an algorithm for sampling from the VMF distribution, makes use of a family of distributions named after and explored by John G. Saw. A Saw distribution is a distribution on the sphere, , with modal vector and concentration , and of which the density function has the form:

where is a non negative, increasing function; and where is the normalization constant. The above mentioned radial tangential decomposition generalizes to the Saw family and the radial compoment, has the density:

where is the beta function. Also notice that the left hand factor of the radial density is the surface area of
By setting , one recovers the VMF distribution.