Introduction to Statistics (Part-III)
By: Antony Christopher
Tags: Confusion Matrix, Statistics, Analysis, Hypothesis Testing

Why You Should Perform Statistical Hypothesis Testing?

Hypothesis testing is a form of inferential statistics that allows us to draw conclusions about an entire population based on a representative sample. You gain tremendous benefits by working with a sample. In most cases, it is simply impossible to observe the entire population to understand its properties. The only alternative is to collect a random sample and then use statistics to analyze it.

What is a Hypothesis Statement?

If you are going to propose a hypothesis, it’s customary to write a statement. Your statement will look like this:

“If I…(do this to an independent variable)….then (this will happen to the dependent variable).”

For example:

If I (decrease the amount of water given to herbs) then (the herbs will increase in size).

If I (give patients counseling in addition to medication) then (their overall depression scale will decrease).

If I (give exams at noon instead of 7) then (student test scores will improve).

If I (look in this certain location) then (I am more likely to find new species).

Hypothesis Testing

Hypothesis testing in statistics is a way for you to test the results of a survey or experiment to see if you have meaningful results. You’re basically testing whether your results are valid by figuring out the odds that your results have happened by chance. If your results may have happened by chance, the experiment won’t be repeatable and so has little use.

Null Hypothesis

The Null Hypothesis, denoted by Ho, assumes that there is a “NO” difference between two sets of values. The null hypothesis states there is no relationship between the measured phenomenon (the dependent variable) and the independent variable.

Alternative Hypothesis

The inverse of the null hypothesis.

States that there is a statistical significance between two variables

Holds true if the null hypothesis is rejected

Usually what the researcher thinks is true and is testing

Denoted by H1 or Ha

Null hypothesis:

If one plant is fed lemonade for one month and another is fed plain water, there will be no difference in growth between the two plants

Alternative Hypothesis:

If one plant is fed lemonade for one month and another is fed plain water, the plant that is fed lemonade will grow more than the plant that is fed plain water

Definition of One-tailed Test

One-tailed test alludes to the significance test in which the region of rejection appears on one end of the sampling distribution. It represents that the estimated test parameter is greater or less than the critical value. When the sample tested falls in the region of rejection, i.e. either left or right side, as the case may be, it leads to the acceptance of the alternative hypothesis rather than the null hypothesis. It is primarily applied in chi-square distribution; that ascertains the goodness of fit.

In this statistical hypothesis test, all the critical region, related to α, is placed in any one of the two tails. The one-tailed test can be:

Left-tailed test: When the population parameter is believed to be lower than the assumed one, the hypothesis test carried out is the left-tailed test.

Right-tailed test: When the population parameter is supposed to be greater than the assumed one, the statistical test conducted is a right-tailed test.

Definition of Two-tailed Test

The two-tailed test is described as a hypothesis test, in which the region of rejection or say the critical area is on both the ends of the normal distribution. It determines whether the sample tested falls within or outside a certain range of values. Therefore, an alternative hypothesis is accepted in place of the null hypothesis, if the calculated value falls in either of the two tails of the probability distribution.

In this test, α is bifurcated into two equal parts, placing half on each side, i.e. it considers the possibility of both positive and negative effects. It is performed to see, whether the estimated parameter is either above or below the assumed parameter, so the extreme values, work as evidence against the null hypothesis.

Confusion Matrix

In predictive analytics, a table of confusion (sometimes also called a confusion matrix) is a table with two rows and two columns that reports the number of false positives, false negatives, true positives, and true negatives. This allows more detailed analysis than mere proportion of correct classifications (accuracy). Accuracy will yield misleading results if the data set is unbalanced; that is, when the numbers of observations in different classes vary greatly.

Given a sample of 13 pictures, 8 of cats and 5 of dogs, where cats belong to class 1 and dogs belong to class 0,

actual = [1,1,1,1,1,1,1,1,0,0,0,0,0],

assume that a classifier that distinguishes between cats and dogs is trained, and we take the 13 pictures and run them through the classifier, and the classifier makes 8 accurate predictions and misses 5: 3 cats wrongly predicted as dogs (first 3 predictions) and 2 dogs wrongly predicted as cats (last 2 predictions).

prediction = [0,0,0,1,1,1,1,1,0,0,0,1,1]

With these two labelled sets (actual and predictions) we can create a confusion matrix that will summarize the results of testing the classifier:

In this confusion matrix, of the 8 cat pictures, the system judged that 3 were dogs, and of the 5 dog pictures, it predicted that 2 were cats. All correct predictions are located in the diagonal of the table (highlighted in bold), so it is easy to visually inspect the table for prediction errors, as they will be represented by values outside the diagonal.

In abstract terms, the confusion matrix is as follows:

P = Positive; N = Negative; TP = True Positive; FP = False Positive; TN = True Negative; FN = False Negative.

Happy learning :)