BitWithBite
Statistics · Quick Reference
Hypothesis Testing Cheat Sheet
In one line: hypothesis testing is a formal procedure for deciding whether there's enough evidence in sample data to support a claim about a population.
Key Ideas
1Null Hypothesis (H0). The default assumption of 'no effect' or 'no difference' that we test against, such as 'this new drug has no effect'.
2Alternative Hypothesis (H1). The claim we're trying to find evidence for, such as 'this new drug does have an effect'.
3P-value. The probability of observing results at least as extreme as what was found, assuming the null hypothesis is true. A small p-value suggests the null hypothesis is unlikely.
4Significance Level (alpha). A threshold (commonly 0.05) set in advance; if the p-value is less than alpha, the result is considered 'statistically significant' and we reject the null hypothesis.
5Type I and Type II Errors. Type I error: rejecting a true null hypothesis (a false alarm). Type II error: failing to reject a false null hypothesis (a missed effect).
Worked Examples
A test gives a p-value of 0.02, with significance level alpha = 0.05. What is the conclusion?
Reject the null hypothesis — there is significant evidence for the alternative hypothesis
A test gives a p-value of 0.20, with significance level alpha = 0.05. What is the conclusion?
Fail to reject the null hypothesis — not enough evidence for the alternative
A jury wrongly convicts an innocent person. Relate this to Type I and Type II errors, where H0 = 'innocent'.
This is a Type I error (false positive)
Formulas
Reject H0 if p-value < alpha
Practice Yourself
If p-value = 0.01 and alpha = 0.05, do we reject H0?
Yes, since 0.01 < 0.05
If p-value = 0.08 and alpha = 0.05, do we reject H0?
No, since 0.08 > 0.05
What does the null hypothesis typically represent?
The default assumption of no effect or no difference