Consistency (statistics)

Consistency (statistics)
This article is licensed under the GNU Free Documentation License. It uses material from the Wikipedia article "Consistency_(statistics)".

In statistics, a consistent sequence of estimators is one which converges in probability to the true value of the parameter. Often, the sequence of estimators is indexed by sample size, and so the consistency is as sample size (n) tends to infinity. Often, the term consistent estimator is used, which refers to the whole sequence of estimators, resp. to a formula that is used to obtain a term of the sequence.

Consistency with convergence in probability can be referred to as weak consistency. The notion of consistency can be extended to other modes of Convergence of random variables; for example, a sequence is said to be strongly consistent if it converges almost surely to the true parameter.

content

1 Definition
2 Example: Sample Mean for Normal Random Variables
3 Establishing Consistency
4 Properties
- 4.1 Proof
5 Scale factors
6 References
7 Further reading

Definition

Let $Θ$ be a class of a parameters, and let $\{T_n; n \ge 1\}$ be a sequence of estimates that can be used to estimate any parmeter $\theta\in\Theta.$ Usually, $T n$ is based on the first n observations of a sample $X_1,X_2,\ldots$ , from a given probability distribution $\ f(x,\theta).$ Then the sequence $T n$ is (weakly) consistent on $Θ$ (¹, p332) if and only if for all $\theta\in\Theta$ and for all $\varepsilon > 0$ we have

$\lim_{n\rightarrow\infty}P(|T_n-\theta|\geq\varepsilon)=0$

Example: Sample Mean for Normal Random Variables

Suppose one has a sequence of observations $X_1,X_2,\ldots$ from a $\ N(\mu,\sigma^2)$ distribution. To estimate $μ$ based on the first n observations, one usually uses the sample mean, $T_n=(X_1+\ldots+X_n)/n$ . Note that this defines a sequence of estimates, indexed by the sample size n.

Now, from the properties of the normal distribution, it is known that $T n$ is itself normally distributed, with mean $μ$ and variance $\ \sigma^2/n$ . Equivalently, $\sqrt{n}(T_n-\mu)/\sigma$ has a standard normal distribution. Then

$P(T_n-\mu \ge\varepsilon) = P\left ( \frac{\sqrt{n}}{\sigma}(T_n-\mu) \ge \frac{\sqrt{n}}{\sigma} \varepsilon \right ) \to 0$

as n tends to infinity, for any fixed $\varepsilon>0$ . Similarly, $P(T_n-\mu \le -\varepsilon) \to 0$ . Therefore, the sequence $T n$ of sample means is consistent for the population mean $μ$ .

Establishing Consistency

For general problems, finding the exact distribution of $T n$ , and hence establishing consistency directly, can be intractable. However, the closely related Markov and Chebychev inequalities are powerful tools in proving consistency. In context, these inequalities can be expressed as

$P(|T_n - \theta| \ge \varepsilon) \le \frac{ E|T_n-\theta|^2 }{\epsilon^2}.$

As a consequence, it suffices that the Mean Squared Error (equivalently, both the bias and variance) of $T n$ converges to 0. See Lehman, Page 332, Lemma 1.1 and Theorem 1.1.

Another useful result is the continuous mapping theorem: If $T n$ is consistent for $θ$ , and g is a continuous real-valued function, then $g (T n)$ is consistent for $g (θ).$

Properties

Suppose $U$ is an estimator of $θ$ such that the sequence ${U n}$ is consistent. If $\alpha_n\to\alpha\in\mathbb{R}$ and $\beta_n\to\beta\in\mathbb{R}^m$ are two convergent sequences of constants with $0<|\alpha|<\infty$ and $|\beta|<\infty$ , then the sequence ${V n}$ , defined by $V_n \triangleq \alpha_n U_n+\beta_n$ is a consistent estimate of $αθ + β$ .

Proof

First, observe that

$\begin{align} |V_n-(\alpha\theta+\beta)| &= |\alpha_n U_n+\beta_n-\alpha\theta-\beta|\\ &\le |\alpha_nU_n-\alpha\theta|+|\beta_n-\beta|\\ &= |\alpha_nU_n-\alpha_n\theta+\alpha_n\theta-\alpha\theta|+|\beta_n-\beta|\\ &\le |\alpha_nU_n-\alpha_n\theta|+|\alpha_n\theta-\alpha\theta|+|\beta_n-\beta|\\ &= |\alpha_n||U_n-\theta|+|\alpha_n-\alpha||\theta|+|\beta_n-\beta|. \end{align}$

This implies

$\begin{align} & P(|V_n-(\alpha\theta+\beta)|\ge\varepsilon) \\ &\le P(|\alpha_n||U_n-\theta|+|\alpha_n-\alpha| |\theta|+|\beta_n-\beta|\ge\varepsilon) \\ &= P\left(|U_n-\theta|\ge\frac{\varepsilon-|\beta_n-\beta|-|\alpha_n-\alpha||\theta|}{|\alpha_n|}\right). \end{align}$

As $n\to\infty$ , we have $|\beta_n-\beta|\to 0$ , $|\alpha_n-\alpha||\theta|\to 0$ , and $|\alpha_n|\to|\alpha|\ne 0$ . So the last expression goes to $0$ as $n\to\infty$ . Therefore,

$\lim_{n\to\infty}P(|V_n-(\alpha\theta+\beta)|\ge\varepsilon)=0,$

and thus ${V n}$ is a consistent sequence of estimators of $αθ + β$ . Q.E.D.

Scale factors

Main article: Scale parameter#Estimation

If an estimator converges to a multiple of a parameter, such as how various unscaled measures of statistical dispersion converge to a multiple of a scale parameter, one can make it into a consistent estimator by multiplying the estimator by a scale factor, namely the true value divided by the asymptotic value of the estimator.

References

^ Lehman, Theory of Point Estimation, Wiley, 1983

Contents