Illustration of the central limit theorem

We start with a probability density function. This function, although discontinuous, is far from the most pathological example that could be created. It is a piecewise polynomial, with pieces of degrees 0 and 1. The mean of this distribution is 0 and its standard deviation is 1.

Density of a sum of two variables.

Next we compute the density of the sum of two independent variables, each having the above density. The density of the sum is the convolution of the above density with itself.

The sum of two variables has mean 0. The density shown in the figure at right has been rescaled by ${\displaystyle {\sqrt {2}}}$, so that its standard deviation is 1.

This density is already smoother than the original. There are obvious lumps, which correspond to the intervals on which the original density was defined.

Density of a sum of three variables.

We then compute the density of the sum of three independent variables, each having the above density. The density of the sum is the convolution of the first density with the second.

The sum of three variables has mean 0. The density shown in the figure at right has been rescaled by √3, so that its standard deviation is 1.

This density is even smoother than the preceding one. The lumps can hardly be detected in this figure.

Density of a sum of four variables

Finally, we compute the density of the sum of four independent variables, each having the above density. The density of the sum is the convolution of the first density with the third (or the second density with itself).

The sum of four variables has mean 0. The density shown in the figure at right has been rescaled by √4, so that its standard deviation is 1.

This density appears qualitatively very similar to a normal density. No lumps can be distinguished by the eye.

Suppose the probability distribution of a discrete random variable X puts equal weights on 1, 2, and 3:

${\displaystyle X=\left\{{\begin{matrix}1&{\mbox{with}}\ {\mbox{probability}}\ 1/3,\\2&{\mbox{with}}\ {\mbox{probability}}\ 1/3,\\3&{\mbox{with}}\ {\mbox{probability}}\ 1/3.\end{matrix}}\right.}$

The probability mass function of the random variable X may be depicted by the following bar graph:

Clearly this looks nothing like the bell-shaped curve of the normal distribution. Contrast the above with the depictions below.

Sum of all permutations of length 2 selected from the set of integers 1, 2, 3

Now consider the sum of two independent copies of X:

${\displaystyle \left\{{\begin{matrix}1+1&=&2\\1+2&=&3\\1+3&=&4\\2+1&=&3\\2+2&=&4\\2+3&=&5\\3+1&=&4\\3+2&=&5\\3+3&=&6\end{matrix}}\right\}=\left\{{\begin{matrix}2&{\mbox{with}}\ {\mbox{probability}}\ 1/9\\3&{\mbox{with}}\ {\mbox{probability}}\ 2/9\\4&{\mbox{with}}\ {\mbox{probability}}\ 3/9\\5&{\mbox{with}}\ {\mbox{probability}}\ 2/9\\6&{\mbox{with}}\ {\mbox{probability}}\ 1/9\end{matrix}}\right\}}$

The probability mass function of this sum may be depicted thus:

This still does not look very much like the bell-shaped curve, but, like the bell-shaped curve and unlike the probability mass function of X itself, it is higher in the middle than in the two tails.

Sum of all permutations of length 3 selected from the set of integers 1, 2, 3

Now consider the sum of three independent copies of this random variable:

${\displaystyle \left\{{\begin{matrix}1+1+1&=&3\\1+1+2&=&4\\1+1+3&=&5\\1+2+1&=&4\\1+2+2&=&5\\1+2+3&=&6\\1+3+1&=&5\\1+3+2&=&6\\1+3+3&=&7\\2+1+1&=&4\\2+1+2&=&5\\2+1+3&=&6\\2+2+1&=&5\\2+2+2&=&6\\2+2+3&=&7\\2+3+1&=&6\\2+3+2&=&7\\2+3+3&=&8\\3+1+1&=&5\\3+1+2&=&6\\3+1+3&=&7\\3+2+1&=&6\\3+2+2&=&7\\3+2+3&=&8\\3+3+1&=&7\\3+3+2&=&8\\3+3+3&=&9\end{matrix}}\right\}=\left\{{\begin{matrix}3&{\mbox{with}}\ {\mbox{probability}}\ 1/27\\4&{\mbox{with}}\ {\mbox{probability}}\ 3/27\\5&{\mbox{with}}\ {\mbox{probability}}\ 6/27\\6&{\mbox{with}}\ {\mbox{probability}}\ 7/27\\7&{\mbox{with}}\ {\mbox{probability}}\ 6/27\\8&{\mbox{with}}\ {\mbox{probability}}\ 3/27\\9&{\mbox{with}}\ {\mbox{probability}}\ 1/27\end{matrix}}\right\}}$

The probability mass function of this sum may be depicted thus:

Not only is this bigger at the center than it is at the tails, but as one moves toward the center from either tail, the slope first increases and then decreases, just as with the bell-shaped curve.

The degree of its resemblance to the bell-shaped curve can be quantified as follows. Consider

Pr(X₁ + X₂ + X₃ ≤ 7) = 1/27 + 3/27 + 6/27 + 7/27 + 6/27 = 23/27 = 0.85185... .

How close is this to what a normal approximation would give? It can readily be seen that the expected value of Y = X₁ + X₂ + X₃ is 6 and the standard deviation of Y is the square root of 2. Since Y ≤ 7 (weak inequality) if and only if Y < 8 (strict inequality), we use a continuity correction and seek

${\displaystyle {\mbox{Pr}}(Y\leq 7.5)={\mbox{P}}\left({Y-6 \over {\sqrt {2}}}\leq {7.5-6 \over {\sqrt {2}}}\right)={\mbox{Pr}}(Z\leq 1.0606602\dots )=0.85558\dots }$

where Z has a standard normal distribution. The difference between 0.85185... and 0.85558... seems remarkably small when it is considered that the number of independent random variables that were added was only three.

The following image shows the result of a simulation based on the example presented in this page. The extraction from the uniform distribution is repeated 1,000 times, and the results are summed.

Since the simulation is based on the Monte Carlo method, the process is repeated 10,000 times. The results shows that the distribution of the sum of 1,000 uniform extractions resembles the bell-shaped curve very well.