Significant versus Importance
- With a large enough sample size, even a small difference can be found statistically
significant – that is, the difference is hard to explain by chance alone.
This does not necessarily make the difference important.
- On the other hand, an important difference may not be statistically significant
if the sample size is too small.
Example: The population consists of 100 students.
The hypotheses to be tested are as follows:
H0 : The class consists of all males (100% males).
H1 : The class does not consist of all males. (<100% males).
The decision rule is, Reject H0 if you observe at least one female
in your sample.
Suppose the class actually consists of 2 females and 98 males; that is, H1
is actually true.
If you take a sample of size n=2,
how likely would it be to obtain a female in your sample and thus reject the
null hypothesis?
If you take a sample of size n=80,
how likely would it be to obtain a female in your sample and thus reject the
null hypothesis?