Using the $C$% confidence interval for $\mu_1 - \mu_2$ to perform two sided two sample $t$ test (equal variances assumed): full explanation

If 0 is not in the $C$% confidence interval, the difference between the two sample means is significantly different from 0 at significance level $\alpha = 1 - C/100$

 If 0 is not in the $C$% confidence interval, the difference between the two sample means is significantly different from 0 at significance level $\alpha = 1 - C/100$. In order to understand why, we have to be aware of the following three points, which are illustrated in the figure above: The lower bound of the $C$% confidence interval for $\mu_1 - \mu_2$ is $\bar{y} - t^* \times SE$, the upper bound is $\bar{y} + t^* \times SE$. That is, we subtract $t^* \times SE$ from the difference between the two sample means, and we add $t^* \times SE$ to the difference between the two sample means. Here: $SE$ is the standard error of $\bar{y}_1 - \bar{y}_2$, which is $s_p \sqrt{\frac{1}{n_1} + \frac{1}{n_2}}$ the critical value $t^*$ is the value under the $t$ distribution with area $C / 100$ between $-t^*$ and $t^*$, and therefore area $\frac{1\,-\,C/100}{2}$ to the right of $t^*$. In the figure above, the red dots represent differences between two sample means, the blue arrows represent the distance $t^* \times SE$. If we set the significance level $\alpha$ at $1 - C/100$, the critical value $t^*$ used for the significance test is the value under the $t$ distribution with area $\frac{\alpha}{2} = \frac{1\,-\,C/100}{2}$ to the right of $t^*$. This is the same critical value $t^*$ as the $t^*$ used for the $C$% confidence interval. We reject the null hypothesis at $\alpha = 1 - C/100$ if the $t$ value $t = \frac{(\bar{y}_1 - \bar{y}_2) - 0}{SE}$ is at least as extreme as $t^*$. This means that we reject the null hypothesis at $\alpha = 1 - C/100$ if the difference between the two sample means $\bar{y}_1 - \bar{y}_2$ is at least $t^* \times SE$ removed from 0. In the figure above, the rejection region for $\bar{y}_1 - \bar{y}_2$ is represented by the green area. If 0 is not in the $C$% confidence interval [$\bar{y} - t^* \times SE \,;\, \bar{y} + t^* \times SE$], then $\bar{y}_1 - \bar{y}_2$ must be at least $t^* \times SE$ removed from 0. In the figure above, the first confidence interval does not contain 0 and thus the difference between the two sample means must be at least $t^* \times SE$ removed from 0. The second confidence interval does contain 0, and thus the difference between the two sample means must be less than $t^* \times SE$ removed from 0. In sum, if 0 is not in the $C$% confidence interval [$\bar{y} - t^* \times SE \,;\, \bar{y} + t^* \times SE$], we know that the difference between the two sample means is significantly different from 0 at significance level $\alpha = 1 - C/100$.