Статистикада еркіндік дәрежесінің анықтамасы

Number of values in the final calculation of a statistic that are free to vary

In statistics, the number of degrees of freedom is the number of values in the final calculation of a statistic that are free to vary. Estimates of statistical parameters can be based upon different amounts of information or data. The number of independent pieces of information that go into the estimate of a parameter is called the degrees of freedom. In general, the degrees of freedom of an estimate of a parameter are equal to the number of independent scores that go into the estimate minus the number of parameters used as intermediate steps in the estimation of the parameter itself. For example, if the variance is to be estimated from a random sample of independent scores, then the degrees of freedom is equal to the number of independent scores (N) minus the number of parameters estimated as intermediate steps (one, namely, the sample mean) and is therefore equal to
Mathematically, degrees of freedom is the number of dimensions of the domain of a random vector, or essentially the number of "free" components (how many components need to be known before the vector is fully determined). The term is most often used in the context of linear models (linear regression, analysis of variance), where certain random vectors are constrained to lie in linear subspaces, and the number of degrees of freedom is the dimension of the subspace. The degrees of freedom are also commonly associated with the squared lengths (or "sum of squares" of the coordinates) of such vectors, and the parameters of chi squared and other distributions that arise in associated statistical testing problems. While introductory textbooks may introduce degrees of freedom as distribution parameters or through hypothesis testing, it is the underlying geometry that defines degrees of freedom, and is critical to a proper understanding of the concept.

Although the basic concept of degrees of freedom was recognized as early as 1821 in the work of German astronomer and mathematician Carl Friedrich Gauss, its modern definition and usage was first elaborated by English statistician William Sealy Gosset in his 1908 Biometrika article "The Probable Error of a Mean", published under the pen name "Student". While Gosset did not actually use the term 'degrees of freedom', he explained the concept in the course of developing what became known as Student's t distribution. The term itself was popularized by English statistician and biologist Ronald Fisher, beginning with his 1922 work on chi squares.

In equations, the typical symbol for degrees of freedom is ν (lowercase Greek letter nu). In text and tables, the abbreviation "d. f." is commonly used. R. A. Fisher used n to symbolize degrees of freedom but modern usage typically reserves n for sample size.

When the results of structural equation models (SEM) are presented, they generally include one or more indices of overall model fit, the most common of which is a χ2 statistic. This forms the basis for other indices that are commonly reported. Although it is these other statistics that are most commonly interpreted, the degrees of freedom of the χ2 are essential to understanding model fit as well as the nature of the model itself. Degrees of freedom in SEM are computed as a difference between the number of unique pieces of information that are used as input into the analysis, sometimes called knowns, and the number of parameters that are uniquely estimated, sometimes called unknowns. For example, in a one factor confirmatory factor analysis with 4 items, there are 10 knowns (the six unique covariances among the four items and the four item variances) and 8 unknowns (4 factor loadings and 4 error variances) for 2 degrees of freedom. Degrees of freedom are important to the understanding of model fit if for no other reason than that, all else being equal, the fewer degrees of freedom, the better indices such as χ2 will be. It has been shown that degrees of freedom can be used by readers of papers that contain SEMs to determine if the authors of those papers are in fact reporting the correct model fit statistics. In the organizational sciences, for example, nearly half of papers published in top journals report degrees of freedom that are inconsistent with the models described in those papers, leaving the reader to wonder which models were actually tested.

A common way to think of degrees of freedom is as the number of independent pieces of information available to estimate another piece of information. More concretely, the number of degrees of freedom is the number of independent observations in a sample of data that are available to estimate a parameter of the population from which that sample is drawn. For example, if we have two observations, when calculating the mean we have two independent observations; however, when calculating the variance, we have only one independent observation, since the two observations are equally distant from the sample mean. In fitting statistical models to data, the vectors of residuals are constrained to lie in a space of smaller dimension than the number of components in the vector. That smaller dimension is the number of degrees of freedom for error, also called residual degrees of freedom.

In statistical testing problems, one usually is not interested in the component vectors themselves, but rather in their squared lengths, or Sum of Squares. The degrees of freedom associated with a sum of squares is the degrees of freedom of the corresponding component vectors. The three population example above is an example of one way Analysis of Variance. The model, or treatment, sum of squares is the squared length of the second vector,

with 2 degrees of freedom. The residual, or error, sum of squares is

with 3(n−1) degrees of freedom. Of course, introductory books on ANOVA usually state formulae without showing the vectors, but it is this underlying geometry that gives rise to SS formulae, and shows how to unambiguously determine the degrees of freedom in any given situation. Under the null hypothesis of no difference between population means (and assuming that standard ANOVA regularity assumptions are satisfied) the sums of squares have scaled chi squared distributions, with the corresponding degrees of freedom. The F test statistic is the ratio, after scaling by the degrees of freedom. If there is no difference between population means this ratio follows an F distribution with 2 and 3n − 3 degrees of freedom. In some complicated settings, such as unbalanced split plot designs, the sums of squares no longer have scaled chi squared distributions. Comparison of sum of squares with degrees of freedom is no longer meaningful, and software may report certain fractional 'degrees of freedom' in these cases. Such numbers have no genuine degrees of freedom interpretation, but are simply providing an approximate chi squared distribution for the corresponding sum of squares. The details of such approximations are beyond the scope of this page.

Several commonly encountered statistical distributions (Student's t, chi squared, F) have parameters that are commonly referred to as degrees of freedom. This terminology simply reflects that in many applications where these distributions occur, the parameter corresponds to the degrees of freedom of an underlying random vector, as in the preceding ANOVA example. Another simple example is: if are independent normal random variables, the statistic

follows a chi squared distribution with n − 1 degrees of freedom. Here, the degrees of freedom arises from the residual sum of squares in the numerator, and in turn the n − 1 degrees of freedom of the underlying residual vector
In the application of these distributions to linear models, the degrees of freedom parameters can take only integer values. The underlying families of distributions allow fractional values for the degrees of freedom parameters, which can arise in more sophisticated uses. One set of examples is problems where chi squared approximations based on effective degrees of freedom are used. In other applications, such as modelling heavy tailed data, a t or F distribution may be used as an empirical model. In these cases, there is no particular degrees of freedom interpretation to the distribution parameters, even though the terminology may continue to be used.

Many non standard regression methods, including regularized least squares (e. g., ridge regression), linear smoothers, smoothing splines, and semiparametric regression, are not based on ordinary least squares projections, but rather on regularized (generalized and/or penalized) least squares, and so degrees of freedom defined in terms of dimensionality is generally not useful for these procedures. However, these procedures are still linear in the observations, and the fitted values of the regression can be expressed in the form

where is the vector of fitted values at each of the original covariate values from the fitted model, y is the original vector of responses, and H is the hat matrix or, more generally, smoother matrix. For statistical inference, sums of squares can still be formed: the model sum of squares is ; the residual sum of squares is However, because H does not correspond to an ordinary least squares fit (i. e. is not an orthogonal projection), these sums of squares no longer have (scaled, non central) chi squared distributions, and dimensionally defined degrees of freedom are not useful. The effective degrees of freedom of the fit can be defined in various ways to implement goodness of fit tests, cross validation, and other statistical inference procedures. Here one can distinguish between regression effective degrees of freedom and residual effective degrees of freedom.