Two sample t test - equal variances assumed - overview

This page offers structured overviews of one or more selected methods. Add additional methods for comparisons (max. of 3) by clicking on the dropdown button in the right-hand column. To practice with a specific method click the button at the bottom row of the table

Two sample $t$ test - equal variances assumed
Marginal Homogeneity test / Stuart-Maxwell test
McNemar's test
You cannot compare more than 3 methods
Independent/grouping variableIndependent variableIndependent variable
One categorical with 2 independent groups2 paired groups2 paired groups
Dependent variableDependent variableDependent variable
One quantitative of interval or ratio levelOne categorical with $J$ independent groups ($J \geqslant 2$)One categorical with 2 independent groups
Null hypothesisNull hypothesisNull hypothesis
H0: $\mu_1 = \mu_2$

Here $\mu_1$ is the population mean for group 1, and $\mu_2$ is the population mean for group 2.
H0: for each category $j$ of the dependent variable, $\pi_j$ for the first paired group = $\pi_j$ for the second paired group.

Here $\pi_j$ is the population proportion in category $j.$

Let's say that the scores on the dependent variable are scored 0 and 1. Then for each pair of scores, the data allow four options:

  1. First score of pair is 0, second score of pair is 0
  2. First score of pair is 0, second score of pair is 1 (switched)
  3. First score of pair is 1, second score of pair is 0 (switched)
  4. First score of pair is 1, second score of pair is 1
The null hypothesis H0 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) = P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the null hypothesis are:

  • H0: $\pi_1 = \pi_2$, where $\pi_1$ is the population proportion of ones for the first paired group and $\pi_2$ is the population proportion of ones for the second paired group
  • H0: for each pair of scores, P(first score of pair is 1) = P(second score of pair is 1)

Alternative hypothesisAlternative hypothesisAlternative hypothesis
H1 two sided: $\mu_1 \neq \mu_2$
H1 right sided: $\mu_1 > \mu_2$
H1 left sided: $\mu_1 < \mu_2$
H1: for some categories of the dependent variable, $\pi_j$ for the first paired group $\neq$ $\pi_j$ for the second paired group.

The alternative hypothesis H1 is that for each pair of scores, P(first score of pair is 0 while second score of pair is 1) $\neq$ P(first score of pair is 1 while second score of pair is 0). That is, the probability that a pair of scores switches from 0 to 1 is not the same as the probability that a pair of scores switches from 1 to 0.

Other formulations of the alternative hypothesis are:

  • H1: $\pi_1 \neq \pi_2$
  • H1: for each pair of scores, P(first score of pair is 1) $\neq$ P(second score of pair is 1)

AssumptionsAssumptionsAssumptions
  • Within each population, the scores on the dependent variable are normally distributed
  • The standard deviation of the scores on the dependent variable is the same in both populations: $\sigma_1 = \sigma_2$
  • Group 1 sample is a simple random sample (SRS) from population 1, group 2 sample is an independent SRS from population 2. That is, within and between groups, observations are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
  • Sample of pairs is a simple random sample from the population of pairs. That is, pairs are independent of one another
Test statisticTest statisticTest statistic
$t = \dfrac{(\bar{y}_1 - \bar{y}_2) - 0}{s_p\sqrt{\dfrac{1}{n_1} + \dfrac{1}{n_2}}} = \dfrac{\bar{y}_1 - \bar{y}_2}{s_p\sqrt{\dfrac{1}{n_1} + \dfrac{1}{n_2}}}$
Here $\bar{y}_1$ is the sample mean in group 1, $\bar{y}_2$ is the sample mean in group 2, $s_p$ is the pooled standard deviation, $n_1$ is the sample size of group 1, and $n_2$ is the sample size of group 2. The 0 represents the difference in population means according to the null hypothesis.

The denominator $s_p\sqrt{\dfrac{1}{n_1} + \dfrac{1}{n_2}}$ is the standard error of the sampling distribution of $\bar{y}_1 - \bar{y}_2$. The $t$ value indicates how many standard errors $\bar{y}_1 - \bar{y}_2$ is removed from 0.

Note: we could just as well compute $\bar{y}_2 - \bar{y}_1$ in the numerator, but then the left sided alternative becomes $\mu_2 < \mu_1$, and the right sided alternative becomes $\mu_2 > \mu_1$.
Computing the test statistic is a bit complicated and involves matrix algebra. Unless you are following a technical course, you probably won't need to calculate it by hand.$X^2 = \dfrac{(b - c)^2}{b + c}$
Here $b$ is the number of pairs in the sample for which the first score is 0 while the second score is 1, and $c$ is the number of pairs in the sample for which the first score is 1 while the second score is 0.
Pooled standard deviationn.a.n.a.
$s_p = \sqrt{\dfrac{(n_1 - 1) \times s^2_1 + (n_2 - 1) \times s^2_2}{n_1 + n_2 - 2}}$--
Sampling distribution of $t$ if H0 were trueSampling distribution of the test statistic if H0 were trueSampling distribution of $X^2$ if H0 were true
$t$ distribution with $n_1 + n_2 - 2$ degrees of freedomApproximately the chi-squared distribution with $J - 1$ degrees of freedom

If $b + c$ is large enough (say, > 20), approximately the chi-squared distribution with 1 degree of freedom.

If $b + c$ is small, the Binomial($n$, $P$) distribution should be used, with $n = b + c$ and $P = 0.5$. In that case the test statistic becomes equal to $b$.

Significant?Significant?Significant?
Two sided: Right sided: Left sided: If we denote the test statistic as $X^2$:
  • Check if $X^2$ observed in sample is equal to or larger than critical value $X^{2*}$ or
  • Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
For test statistic $X^2$:
  • Check if $X^2$ observed in sample is equal to or larger than critical value $X^{2*}$ or
  • Find $p$ value corresponding to observed $X^2$ and check if it is equal to or smaller than $\alpha$
If $b + c$ is small, the table for the binomial distribution should be used, with as test statistic $b$:
  • Check if $b$ observed in sample is in the rejection region or
  • Find two sided $p$ value corresponding to observed $b$ and check if it is equal to or smaller than $\alpha$
$C\%$ confidence interval for $\mu_1 - \mu_2$n.a.n.a.
$(\bar{y}_1 - \bar{y}_2) \pm t^* \times s_p\sqrt{\dfrac{1}{n_1} + \dfrac{1}{n_2}}$
where the critical value $t^*$ is the value under the $t_{n_1 + n_2 - 2}$ distribution with the area $C / 100$ between $-t^*$ and $t^*$ (e.g. $t^*$ = 2.086 for a 95% confidence interval when df = 20).

The confidence interval for $\mu_1 - \mu_2$ can also be used as significance test.
--
Effect sizen.a.n.a.
Cohen's $d$:
Standardized difference between the mean in group $1$ and in group $2$: $$d = \frac{\bar{y}_1 - \bar{y}_2}{s_p}$$ Cohen's $d$ indicates how many standard deviations $s_p$ the two sample means are removed from each other.
--
Visual representationn.a.n.a.
Two sample t test - equal variances assumed
--
Equivalent ton.a.Equivalent to
One way ANOVA with an independent variable with 2 levels ($I$ = 2):
  • two sided two sample $t$ test is equivalent to ANOVA $F$ test when $I$ = 2
  • two sample $t$ test is equivalent to $t$ test for contrast when $I$ = 2
  • two sample $t$ test is equivalent to $t$ test multiple comparisons when $I$ = 2
OLS regression with one categorical independent variable with 2 levels:
  • two sided two sample $t$ test is equivalent to $F$ test regression model
  • two sample $t$ test is equivalent to $t$ test for regression coefficient $\beta_1$
-
Example contextExample contextExample context
Is the average mental health score different between men and women? Assume that in the population, the standard deviation of mental health scores is equal amongst men and women.Subjects are asked to taste three different types of mayonnaise, and to indicate which of the three types of mayonnaise they like best. They then have to drink a glass of beer, and taste and rate the three types of mayonnaise again. Does drinking a beer change which type of mayonnaise people like best?Does a tv documentary about spiders change whether people are afraid (yes/no) of spiders?
SPSSSPSSSPSS
Analyze > Compare Means > Independent-Samples T Test...
  • Put your dependent (quantitative) variable in the box below Test Variable(s) and your independent (grouping) variable in the box below Grouping Variable
  • Click on the Define Groups... button. If you can't click on it, first click on the grouping variable so its background turns yellow
  • Fill in the value you have used to indicate your first group in the box next to Group 1, and the value you have used to indicate your second group in the box next to Group 2
  • Continue and click OK
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the Marginal Homogeneity test
Analyze > Nonparametric Tests > Legacy Dialogs > 2 Related Samples...
  • Put the two paired variables in the boxes below Variable 1 and Variable 2
  • Under Test Type, select the McNemar test
Jamovin.a.Jamovi
T-Tests > Independent Samples T-Test
  • Put your dependent (quantitative) variable in the box below Dependent Variables and your independent (grouping) variable in the box below Grouping Variable
  • Under Tests, select Student's (selected by default)
  • Under Hypothesis, select your alternative hypothesis
-Frequencies > Paired Samples - McNemar test
  • Put one of the two paired variables in the box below Rows and the other paired variable in the box below Columns
Practice questionsPractice questionsPractice questions