This page offers structured overviews of one or more selected methods. Add additional methods for comparisons (max. of 3) by clicking on the dropdown button in the right-hand column. To practice with a specific method click the button at the bottom row of the table
Goodness of fit test
$z$ test for a single proportion
Sign test
You cannot compare more than 3 methods
Independent variable
Independent variable
Independent variable
None
None
2 paired groups
Dependent variable
Dependent variable
Dependent variable
One categorical with $J$ independent groups ($J \geqslant 2$)
One categorical with 2 independent groups
One of ordinal level
Null hypothesis
Null hypothesis
Null hypothesis
H0: the population proportions in each of the $J$ conditions are $\pi_1$, $\pi_2$, $\ldots$, $\pi_J$
or equivalently
H0: the probability of drawing an observation from condition 1 is $\pi_1$, the probability of drawing an observation from condition 2 is $\pi_2$, $\ldots$,
the probability of drawing an observation from condition $J$ is $\pi_J$
H0: $\pi = \pi_0$
Here $\pi$ is the population proportion of 'successes', and $\pi_0$ is the population proportion of successes according to the null hypothesis.
H0: P(first score of a pair exceeds second score of a pair) = P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
H0: the population median of the difference scores is equal to zero
A difference score is the difference between the first score of a pair and the second score of a pair.
Alternative hypothesis
Alternative hypothesis
Alternative hypothesis
H1: the population proportions are not all as specified under the null hypothesis
or equivalently
H1: the probabilities of drawing an observation from each of the conditions are not all as specified under the null hypothesis
H1 two sided: $\pi \neq \pi_0$
H1 right sided: $\pi > \pi_0$
H1 left sided: $\pi < \pi_0$
H1 two sided: P(first score of a pair exceeds second score of a pair) $\neq$ P(second score of a pair exceeds first score of a pair)
H1 right sided: P(first score of a pair exceeds second score of a pair) > P(second score of a pair exceeds first score of a pair)
H1 left sided: P(first score of a pair exceeds second score of a pair) < P(second score of a pair exceeds first score of a pair)
If the dependent variable is measured on a continuous scale, this can also be formulated as:
H1 two sided: the population median of the difference scores is different from zero
H1 right sided: the population median of the difference scores is larger than zero
H1 left sided: the population median of the difference scores is smaller than zero
Assumptions
Assumptions
Assumptions
Sample size is large enough for $X^2$ to be approximately chi-squared distributed. Rule of thumb: all $J$ expected cell counts are 5 or more
Sample is a simple random sample from the population. That is, observations are independent of one another
Sample size is large enough for $z$ to be approximately normally distributed. Rule of thumb:
Significance test: $N \times \pi_0$ and $N \times (1 - \pi_0)$ are each larger than 10
Regular (large sample) 90%, 95%, or 99% confidence interval: number of successes and number of failures in sample are each 15 or more
Plus four 90%, 95%, or 99% confidence interval: total sample size is 10 or more
Sample is a simple random sample from the population. That is, observations are independent of one another
Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
Test statistic
Test statistic
Test statistic
$X^2 = \sum{\frac{(\mbox{observed cell count} - \mbox{expected cell count})^2}{\mbox{expected cell count}}}$
Here the expected cell count for one cell = $N \times \pi_j$, the observed cell count is the observed sample count in that same cell, and the sum is over all $J$ cells.
$z = \dfrac{p - \pi_0}{\sqrt{\dfrac{\pi_0(1 - \pi_0)}{N}}}$
Here $p$ is the sample proportion of successes: $\dfrac{X}{N}$, $N$ is the sample size, and $\pi_0$ is the population proportion of successes according to the null hypothesis.
$W = $ number of difference scores that is larger than 0
Approximately the chi-squared distribution with $J - 1$ degrees of freedom
Approximately the standard normal distribution
The exact distribution of $W$ under the null hypothesis is the Binomial($n$, $P$) distribution, with $n =$ number of positive differences $+$ number of negative differences, and $P = 0.5$.
If $n$ is large, $W$ is approximately normally distributed under the null hypothesis, with mean $nP = n \times 0.5$ and standard deviation $\sqrt{nP(1-P)} = \sqrt{n \times 0.5(1 - 0.5)}$. Hence, if $n$ is large, the standardized test statistic
$$z = \frac{W - n \times 0.5}{\sqrt{n \times 0.5(1 - 0.5)}}$$
follows approximately the standard normal distribution if the null hypothesis were true.
Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
Two sided:
Check if $z$ observed in sample is at least as extreme as critical value $z^*$ or
Find two sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
Right sided:
Check if $z$ observed in sample is equal to or larger than critical value $z^*$ or
Find right sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
Left sided:
Check if $z$ observed in sample is equal to or smaller than critical value $z^*$ or
Find left sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
If $n$ is small, the table for the binomial distribution should be used:
Two sided:
Check if $W$ observed in sample is in the rejection region or
Find two sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Right sided:
Check if $W$ observed in sample is in the rejection region or
Find right sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
Left sided:
Check if $W$ observed in sample is in the rejection region or
Find left sided $p$ value corresponding to observed $W$ and check if it is equal to or smaller than $\alpha$
If $n$ is large, the table for standard normal probabilities can be used:
Two sided:
Check if $z$ observed in sample is at least as extreme as critical value $z^*$ or
Find two sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
Right sided:
Check if $z$ observed in sample is equal to or larger than critical value $z^*$ or
Find right sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
Left sided:
Check if $z$ observed in sample is equal to or smaller than critical value $z^*$ or
Find left sided $p$ value corresponding to observed $z$ and check if it is equal to or smaller than $\alpha$
n.a.
Approximate $C\%$ confidence interval for $\pi$
n.a.
-
Regular (large sample):
$p \pm z^* \times \sqrt{\dfrac{p(1 - p)}{N}}$
where the critical value $z^*$ is the value under the normal curve with the area $C / 100$ between $-z^*$ and $z^*$ (e.g. $z^*$ = 1.96 for a 95% confidence interval)
With plus four method:
$p_{plus} \pm z^* \times \sqrt{\dfrac{p_{plus}(1 - p_{plus})}{N + 4}}$
where $p_{plus} = \dfrac{X + 2}{N + 4}$ and the critical value $z^*$ is the value under the normal curve with the area $C / 100$ between $-z^*$ and $z^*$ (e.g. $z^*$ = 1.96 for a 95% confidence interval)
-
n.a.
Equivalent to
Equivalent to
-
When testing two sided: goodness of fit test, with a categorical variable with 2 levels.
When $N$ is large, the $p$ value from the $z$ test for a single proportion approaches the $p$ value from the binomial test for a single proportion. The $z$ test for a single proportion is just a large sample approximation of the binomial test for a single proportion.
Is the proportion of people with a low, moderate, and high social economic status in the population different from $\pi_{low} = 0.2,$ $\pi_{moderate} = 0.6,$ and $\pi_{high} = 0.2$?
Is the proportion of smokers amongst office workers different from $\pi_0 = 0.2$? Use the normal approximation for the sampling distribution of the test statistic.
Do people tend to score higher on mental health after a mindfulness course?
Put your categorical variable in the box below Test Variable List
Fill in the population proportions / probabilities according to $H_0$ in the box below Expected Values. If $H_0$ states that they are all equal, just pick 'All categories equal' (default)
Put your dichotomous variable in the box below Test Variable List
Fill in the value for $\pi_0$ in the box next to Test Proportion
If computation time allows, SPSS will give you the exact $p$ value based on the binomial distribution, rather than the approximate $p$ value based on the normal distribution
Put the two paired variables in the boxes below Variable 1 and Variable 2
Under Test Type, select the Sign test
Jamovi
Jamovi
Jamovi
Frequencies > N Outcomes - $\chi^2$ Goodness of fit
Put your categorical variable in the box below Variable
Click on Expected Proportions and fill in the population proportions / probabilities according to $H_0$ in the boxes below Ratio. If $H_0$ states that they are all equal, you can leave the ratios equal to the default values (1)
Frequencies > 2 Outcomes - Binomial test
Put your dichotomous variable in the white box at the right
Fill in the value for $\pi_0$ in the box next to Test value
Under Hypothesis, select your alternative hypothesis
Jamovi will give you the exact $p$ value based on the binomial distribution, rather than the approximate $p$ value based on the normal distribution
Jamovi does not have a specific option for the sign test. However, you can do the Friedman test instead. The $p$ value resulting from this Friedman test is equivalent to the two sided $p$ value that would have resulted from the sign test. Go to:
ANOVA > Repeated Measures ANOVA - Friedman
Put the two paired variables in the box below Measures