s
Confidence interval as significance test - explanation - two sample z test - right sided
Using the $C$% confidence interval for $\mu_1 - \mu_2$ to perform right sided two sample $ z$ test: full explanation
If 0 is not in the $C$% confidence interval and the difference between the two sample means is larger than 0 (as expected), the difference between the two sample means is significantly larger than 0 at significance level $\alpha = \frac{1 \, - \, C/100}{2}$
If 0 is not in the $C$% confidence interval and the difference between the two sample means is larger than 0, the difference between the two sample means is significantly larger than 0 at significance level $\alpha = \frac{1 \, - \, C/100}{2}$. In order to understand why, we have to be aware of the following three points, which are illustrated in the figure above:
The lower bound of the $C$% confidence interval for $\mu_1 - \mu_2$ is $(\bar{y}_1 - \bar{y}_2) - z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$, the upper bound is $(\bar{y}_1 - \bar{y}_2) + z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$. That is, we subtract $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ from the difference between the two sample means, and we add $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ to the difference between the two sample means. Here:
$\sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ is the standard deviation of $\bar{y}_1 - \bar{y}_2$
the critical value $ z^*$ is the value under the $ z$ distribution with area $C / 100$ between $ -z^*$ and $ z^*$, and therefore area $\frac{1\,-\,C/100}{2}$ to the right of $ z^*$.
In the figure above, the red dots represent differences between two sample means, the blue arrows represent the distance $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$.
If we set the significance level $\alpha$ at $\frac{1 \, - \, C/100}{2}$, the critical value $ z^*$ used for the significance test is the value under the $ z$ distribution with area $\alpha = \frac{1\,-\,C/100}{2}$ to the right of $ z^*$. This is the same critical value $ z^*$ as the $ z^*$ used for the $C$% confidence interval. We reject the null hypothesis at $\alpha = \frac{1\,-\,C/100}{2}$ if the $ z$ value $ z = \frac{(\bar{y}_1 - \bar{y}_2) - 0}{\sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}}$ is at least as large as $ z^*$. This means that we reject the null hypothesis at $\alpha = \frac{1\,-\,C/100}{2}$ if difference between the two sample means $\bar{y}_1 - \bar{y}_2$ is at least $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ above 0. In the figure above, the rejection region for $\bar{y}_1 - \bar{y}_2$ is represented by the green area.
If 0 is not in the $C$% confidence interval $\Bigg[(\bar{y}_1 - \bar{y}_2) - z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}} \,;\, (\bar{y}_1 - \bar{y}_2) + z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}\Bigg]$ and the difference between the two sample means $\bar{y}_1 - \bar{y}_2$ is larger than 0, then $\bar{y}_1 - \bar{y}_2$ must be at least $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ above 0. In the figure above, the first confidence interval does not contain 0 and the difference between the two sample means is larger than 0, so the difference between the two sample means must be at least $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ above 0. The second confidence interval does contain 0, so the difference between the two sample means must be less than $ z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}$ above 0.
In sum, if 0 is not in the $C$% confidence interval $\Bigg[(\bar{y}_1 - \bar{y}_2) - z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}} \,;\, (\bar{y}_1 - \bar{y}_2) + z^* \times \sqrt{\frac{\sigma^2_1}{n_1} + \frac{\sigma^2_2}{n_2}}\Bigg]$ and the the difference between the two sample means is larger than 0, we know that the the difference between the two sample means is significantly larger than 0 at significance level $\alpha = \frac{1 \, - \, C/100}{2}$.