z test for the difference between two proportions - overview

This page offers structured overviews of one or more selected methods. Add additional methods for comparisons (max. of 3) by clicking on the dropdown button in the right-hand column. To practice with a specific method click the button at the bottom row of the table

$z$ test for the difference between two proportions
McNemar's test
Sign test
You cannot compare more than 3 methods
Independent/grouping variableIndependent variableIndependent variable
One categorical with 2 independent groups2 paired groups2 paired groups
Dependent variableDependent variableDependent variable
One categorical with 2 independent groupsOne categorical with 2 independent groupsOne of ordinal level
Null hypothesisNull hypothesisNull hypothesis
H0: $\pi_1 = \pi_2$

Here $\pi_1$ is the population proportion of 'successes' for group 1, and $\pi_2$ is the population proportion of 'successes' for group 2.

Let's say that the scores on the dependent variable are scored 0 and 1. Then for each pair of scores, the data allow four options:

  1. First score of pair is 0, second score of pair is 0
  2. First score of pair is 0, second score of pair is 1 (switched)
  3. First score of pair is 1, second score of pair is 0 (switched)
  4. First score of pair is 1, second score of pair is 1
The null hypothesis H0 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) = P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the null hypothesis are:

  • H0: $\pi_1 = \pi_2$, where $\pi_1$ is the population proportion of ones for the first paired group and $\pi_2$ is the population proportion of ones for the second paired group
  • H0: for each pair of scores, P(first score of pair is 1) = P(second score of pair is 1)

  • H0: P(first score of a pair exceeds second score of a pair) = P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
  • H0: the population median of the difference scores is equal to zero
A difference score is the difference between the first score of a pair and the second score of a pair.
Alternative hypothesisAlternative hypothesisAlternative hypothesis
H1 two sided: $\pi_1 \neq \pi_2$
H1 right sided: $\pi_1 > \pi_2$
H1 left sided: $\pi_1 < \pi_2$

The alternative hypothesis H1 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) $\neq$ P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is not the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the alternative hypothesis are:

  • H1: $\pi_1 \neq \pi_2$
  • H1: for each pair of scores, P(first score of pair is 1) $\neq$ P(second score of pair is 1)

  • H1 two sided: P(first score of a pair exceeds second score of a pair) $\neq$ P(second score of a pair exceeds first score of a pair)
  • H1 right sided: P(first score of a pair exceeds second score of a pair) > P(second score of a pair exceeds first score of a pair)
  • H1 left sided: P(first score of a pair exceeds second score of a pair) < P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
  • H1 two sided: the population median of the difference scores is different from zero
  • H1 right sided: the population median of the difference scores is larger than zero
  • H1 left sided: the population median of the difference scores is smaller than zero
AssumptionsAssumptionsAssumptions
  • Sample size is large enough for $z$ to be approximately normally distributed. Rule of thumb:
    • Significance test: number of successes and number of failures are each 5 or more in both sample groups
    • Regular (large sample) 90%, 95%, or 99% confidence interval: number of successes and number of failures are each 10 or more in both sample groups
    • Plus four 90%, 95%, or 99% confidence interval: sample sizes of both groups are 5 or more
  • Group 1 sample is a simple random sample (SRS) from population 1, group 2 sample is an independent SRS from population 2. That is, within and between groups, observations are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
Test statisticTest statisticTest statistic
$z = \dfrac{p_1 - p_2}{\sqrt{p(1 - p)\Bigg(\dfrac{1}{n_1} + \dfrac{1}{n_2}\Bigg)}}$
Here $p_1$ is the sample proportion of successes in group 1: $\dfrac{X_1}{n_1}$, $p_2$ is the sample proportion of successes in group 2: $\dfrac{X_2}{n_2}$, $p$ is the total proportion of successes in the sample: $\dfrac{X_1 + X_2}{n_1 + n_2}$, $n_1$ is the sample size of group 1, and $n_2$ is the sample size of group 2.
Note: we could just as well compute $p_2 - p_1$ in the numerator, but then the left sided alternative becomes $\pi_2 < \pi_1$, and the right sided alternative becomes $\pi_2 > \pi_1.$
$X^2 = \dfrac{(b - c)^2}{b + c}$
Here $b$ is the number of pairs in the sample for which the first score is 0 while the second score is 1, and $c$ is the number of pairs in the sample for which the first score is 1 while the second score is 0.
$W = $ number of difference scores that is larger than 0
Sampling distribution of $z$ if H0 were trueSampling distribution of $X^2$ if H0 were trueSampling distribution of $W$ if H0 were true
Approximately the standard normal distribution

If $b + c$ is large enough (say, > 20), approximately the chi-squared distribution with 1 degree of freedom.

If $b + c$ is small, the Binomial($n$, $P$) distribution should be used, with $n = b + c$ and $P = 0.5$. In that case the test statistic becomes equal to $b$.

The exact distribution of $W$ under the null hypothesis is the Binomial($n$, $P$) distribution, with $n =$ number of positive differences $+$ number of negative differences, and $P = 0.5$.

If $n$ is large, $W$ is approximately normally distributed under the null hypothesis, with mean $nP = n \times 0.5$ and standard deviation $\sqrt{nP(1-P)} = \sqrt{n \times 0.5(1 - 0.5)}$. Hence, if $n$ is large, the standardized test statistic $$z = \frac{W - n \times 0.5}{\sqrt{n \times 0.5(1 - 0.5)}}$$ follows approximately the standard normal distribution if the null hypothesis were true.
Significant?Significant?Significant?
Two sided: Right sided: Left sided: For test statistic $X^2$:
  • Check if $X^2$ observed in sample is equal to or larger than critical value $X^{2*}$ or
  • Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
If $b + c$ is small, the table for the binomial distribution should be used, with as test statistic $b$:
  • Check if $b$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $b$ and check if it is equal to or smaller than $\alpha$
If $n$ is small, the table for the binomial distribution should be used:
Two sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Right sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find right sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Left sided:
  • Check if $W$ observed in sample is in the rejection region or
  • Find left sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$

If $n$ is large, the table for standard normal probabilities can be used:
Two sided: Right sided: Left sided:
Approximate $C\%$ confidence interval for $\pi_1 - \pi_2$n.a.n.a.
Regular (large sample):
  • $(p_1 - p_2) \pm z^* \times \sqrt{\dfrac{p_1(1 - p_1)}{n_1} + \dfrac{p_2(1 - p_2)}{n_2}}$
    where the critical value $z^*$ is the value under the normal curve with the area $C / 100$ between $-z^*$ and $z^*$ (e.g. $z^*$ = 1.96 for a 95% confidence interval)
With plus four method:
  • $(p_{1.plus} - p_{2.plus}) \pm z^* \times \sqrt{\dfrac{p_{1.plus}(1 - p_{1.plus})}{n_1 + 2} + \dfrac{p_{2.plus}(1 - p_{2.plus})}{n_2 + 2}}$
    where $p_{1.plus} = \dfrac{X_1 + 1}{n_1 + 2}$, $p_{2.plus} = \dfrac{X_2 + 1}{n_2 + 2}$, and the critical value $z^*$ is the value under the normal curve with the area $C / 100$ between $-z^*$ and $z^*$ (e.g. $z^*$ = 1.96 for a 95% confidence interval)
--
Equivalent toEquivalent toEquivalent to
When testing two sided: chi-squared test for the relationship between two categorical variables, where both categorical variables have 2 levels. Two sided sign test is equivalent to
Example contextExample contextExample context
Is the proportion of smokers different between men and women? Use the normal approximation for the sampling distribution of the test statistic.Does a tv documentary about spiders change whether people are afraid (yes/no) of spiders?Do people tend to score higher on mental health after a mindfulness course?
SPSSSPSSSPSS
SPSS does not have a specific option for the $z$ test for the difference between two proportions. However, you can do the chi-squared test instead. The $p$ value resulting from this chi-squared test is equivalent to the two sided $p$ value that would have resulted from the $z$ test. Go to:

Analyze > Descriptive Statistics > Crosstabs...
  • Put your independent (grouping) variable in the box below Row(s), and your dependent variable in the box below Column(s)
  • Click the Statistics... button, and click on the square in front of Chi-square
  • Continue and click OK
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the McNemar test
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the Sign test
JamoviJamoviJamovi
Jamovi does not have a specific option for the $z$ test for the difference between two proportions. However, you can do the chi-squared test instead. The $p$ value resulting from this chi-squared test is equivalent to the two sided $p$ value that would have resulted from the $z$ test. Go to:

Frequencies > Independent Samples - $\chi^2$ test of association
  • Put your independent (grouping) variable in the box below Rows, and your dependent variable in the box below Columns
Frequencies > Paired Samples - McNemar test
  • Put one of the two paired variables in the box below Rows and the other paired variable in the box below Columns
Jamovi does not have a specific option for the sign test. However, you can do the Friedman test instead. The $p$ value resulting from this Friedman test is equivalent to the two sided $p$ value that would have resulted from the sign test. Go to:

ANOVA > Repeated Measures ANOVA - Friedman
  • Put the two paired variables in the box below Measures
Practice questionsPractice questionsPractice questions