An alpha level represents the number of times out of 100 you are willing to be incorrect if you reject the null hypothesis. If you choose an alpha level of 0.05, 5 times out of 100 you will be incorrect if you reject the null hypothesis. Those five times, both means would come from the same population (Case III).