McNemar's test

If either b or c is small (b + c < 25) then ${\displaystyle \chi ^{2}}$ is not well-approximated by the chi-squared distribution. An exact binomial test can then be used, where b is compared to a binomial distribution with size parameter n = b + c and p = 0.5. Effectively, the exact binomial test evaluates the imbalance in the discordants b and c. To achieve a two-sided P-value, the P-value of the extreme tail should be multiplied by 2. For b ≥ c:

${\displaystyle {\text{exact-P-value}}=2\sum _{i=b}^{n}{n \choose i}0.5^{i}(1-0.5)^{n-i},}$

which is simply twice the binomial distribution cumulative distribution function with p = 0.5 and n = b + c.

Edwards[4] proposed the following continuity corrected version of the McNemar test to approximate the binomial exact-P-value:

${\displaystyle \chi ^{2}={(|b-c|-1)^{2} \over b+c}.}$

The mid-P McNemar test (mid-p binomial test) is calculated by subtracting half the probability of the observed b from the exact one-sided P-value, then double it to obtain the two-sided mid-P-value:[5][6]

${\displaystyle {\text{mid-p-value}}=2\left(\sum _{i=b}^{n}{n \choose i}0.5^{i}(1-0.5)^{n-i}-0.5{n \choose b}0.5^{b}(1-0.5)^{n-b}\right)}$

This is equivalent to:

${\displaystyle {\text{mid-p-value}}={\text{exact-p-value}}-{n \choose b}0.5^{b}(1-0.5)^{n-b}}$

where the second term is the binomial distribution probability mass function and n = b + c. Binomial distribution functions are readily available in common software packages and the McNemar mid-P test can easily be calculated.[6]

The traditional advice has been to use the exact binomial test when b + c < 25. However, simulations have shown both the exact binomial test and the McNemar test with continuity correction to be overly conservative.[6] When b + c < 6, the exact-P-value always exceeds the common significance level 0.05. The original McNemar test was most powerful, but often slightly liberal. The mid-P version was almost as powerful as the asymptotic McNemar test and was not found to exceed the nominal significance level.

In the 1970s, it was conjectured that retaining one's tonsils might protect against Hodgkin's lymphoma. John Rice wrote:[9]

85 Hodgkin's patients [...] had a sibling of the same sex who was free of the disease and whose age was within 5 years of the patient's. These investigators presented the following table:

${\displaystyle {\begin{array}{c|c|c}\hline &{\text{Tonsillectomy}}&{\text{No tonsillectomy}}\\\hline {\text{Hodgkins}}&41&44\\\hline {\text{Control}}&33&52\end{array}}}$

They calculated a chi-squared statistic [...] [they] had made an error in their analysis by ignoring the pairings.[...] [their] samples were not independent, because the siblings were paired [...] we set up a table that exhibits the pairings:

${\displaystyle {\begin{array}{cc}&{\text{Sibling}}\\{\text{Patient}}&{\begin{array}{c|c|c}\hline &{\text{No tonsillectomy}}&{\text{Tonsillectomy}}\\\hline {\text{No tonsillectomy}}&37&7\\\hline {\text{Tonsillectomy}}&15&26\end{array}}\end{array}}}$

It is to the second table that McNemar's test can be applied. Notice that the sum of the numbers in the second table is 85—the number of pairs of siblings—whereas the sum of the numbers in the first table is twice as big, 170—the number of individuals. The second table gives more information than the first. The numbers in the first table can be found by using the numbers in the second table, but not vice versa. The numbers in the first table give only the marginal totals of the numbers in the second table. McNemar's test allows the 15 and 7 pairs where the siblings had previously had differing treatment to their tonsils to be compared, as being relevant to the hypothesis, while ignoring the less informative 37 and 26 where the siblings had previously both had the treatment or to their tonsils or neither had.