Using the $C$% confidence interval for $\mu$ to perform two sided paired sample $ t$ test: full explanation

If $\mu_0$ is not in the $C$% confidence interval, the sample mean of the difference scores is significantly different from $\mu_0$ at significance level $\alpha = 1 - C/100$


confidence interval as significance test

If $\mu_0$ is not in the $C$% confidence interval, the sample mean of the difference scores is significantly different from $\mu_0$ at significance level $\alpha = 1 - C/100$. In order to understand why, we have to be aware of the following three points, which are illustrated in the figure above:
  1. The lower bound of the $C$% confidence interval for $\mu$ is $\bar{y} - t^* \times SE$, the upper bound is $\bar{y} + t^* \times SE$. That is, we subtract $ t^* \times SE$ from the sample mean of the difference scores, and we add $ t^* \times SE$ to the sample mean of the difference scores. Here:
    • $ SE$ is the standard error of $\bar{y}$, which is $ s / \sqrt{N}$
    • the critical value $ t^*$ is the value under the $ t$ distribution with area $C / 100$ between $ -t^*$ and $ t^*$, and therefore area $\frac{1\,-\,C/100}{2}$ to the right of $ t^*$.
    In the figure above, the red dots represent sample means of the difference scores, the blue arrows represent the distance $ t^* \times SE$.
  2. If we set the significance level $\alpha$ at $1 - C/100$, the critical value $ t^*$ used for the significance test is the value under the $ t$ distribution with area $\frac{\alpha}{2} = \frac{1\,-\,C/100}{2}$ to the right of $ t^*$. This is the same critical value $ t^*$ as the $ t^*$ used for the $C$% confidence interval. We reject the null hypothesis at $\alpha = 1 - C/100$ if the $ t$ value $ t = \frac{\bar{y} - \mu_0}{SE}$ is at least as extreme as $ t^*$. This means that we reject the null hypothesis at $\alpha = 1 - C/100$ if the sample mean of the difference scores $\bar{y}$ is at least $ t^* \times SE$ removed from $\mu_0$. In the figure above, the rejection region for $\bar{y}$ is represented by the green area.
  3. If $\mu_0$ is not in the $C$% confidence interval [$\bar{y} - t^* \times SE \,;\, \bar{y} + t^* \times SE$], then $\bar{y}$ must be at least $ t^* \times SE$ removed from $\mu_0$. In the figure above, the first confidence interval does not contain $\mu_0$ and thus the sample mean of the difference scores must be at least $ t^* \times SE$ removed from $\mu_0$. The second confidence interval does contain $\mu_0$, and thus the sample mean of the difference scores must be less than $ t^* \times SE$ removed from $\mu_0$.
In sum, if $\mu_0$ is not in the $C$% confidence interval [$\bar{y} - t^* \times SE \,;\, \bar{y} + t^* \times SE$], we know that the sample mean of the difference scores is significantly different from $\mu_0$ at significance level $\alpha = 1 - C/100$.