Hypothesis Testing

The five main steps

1. Set hypotheses: null and alternative
2. Set level of significance
3. Identify test statistic, distribution, and rejection criteria
4. Compute observed test statistic value
5. Conclude

Step 1: Set hypothesis

In general, the position of null hypothesis is adopted - it is typically the default assumption.

We usually let hypothesis that we want to prove be the alternative hypothesis.

Possible outcomes

Reject or fail to reject

Type of test:

• one-sided
• two-sided

Step 2: Level of significance

Outcomes:

• Reject , conclude
• Do not reject , conclude

> [!definition] Type I error > > Rejecting the null hypothesis $H_{0}$ when it is true > > > [!definition] Level of significance > > $$ > > \alpha = P(\text{type 1 error}) = P(\text{reject } H_{0} | H_{0} \text{ is true}) > > $$

Type II error

Not rejecting null hypothesis when it is false

Power 1 - \beta, where

Type I error is considered a serious error.

Step 3: Test statistic, distribution, rejection

Select a suitable test statistic - it quantifies the likeliness of observing the sample, assuming null hypothesis holds.

The allows to find a decision rule such that it divides the set of all possible values of the test statistic into two regions, one being the rejection region (or critical region), and the other the acceptance region.

Step 4 & 5: Calculation and Conclusion

Once sample is taken, we check whether the obtained value of the test statistic is within rejection region:

• if it is, sample too improbable assuming holds, hence rejected.
• if it is not, we failed to reject

We cannot prove H_{0} is true: use the term "fail to reject H_{0}"

Value approach to testing

p-value

The p-value is the probability of obtaining a test statistic at least as extreme ( or ) than the observed sample value, given is true. Also known as the observed level of significance.

For hypothesis tests, the value can be described as the probability that the test statistic falls into the the rejection region. If the probability is smaller than the significance level , we then can reject .

Hypothesis Test for the Mean

Known Variance

Conditions

• Population variance is known
• either
  • underlying distribution is normal
  • is sufficiently large ()

For the null hypothesis :

• • Rejection regions for test statistic: or
  • value:
• • Rejection regions for test statistic:
  • value:
• • Rejection regions for test statistic:
  • value:

Unknown Variance

Conditions

• Population variance is known
• underlying distribution is normal

For the null hypothesis :

• • Rejection regions for test statistic: or
  • value:
• • Rejection regions for test statistic:
  • value:
• • Rejection regions for test statistic:
  • value:

Confidence intervals for two-sided tests

The two-sided hypothesis test procedure is equivalent to finding a 100(1-\alpha) confidence interval for \mu.

Consider the hypothesis , with the test statistic .

The confidence interval for in this case is given by

If it contains ,

Based on the above inequality, falls out of the rejection region defined by the test statistic.

This means that when the confidence interval contains \mu_0, H_{0} will not be rejected at level \alpha.

Similarly, this also means that if it does not contain , will be rejected at level .

Hypothesis Tests for Comparing Means

Independent Samples with Known Different Variances

Condition

• population variances are known
• either
  • distributions are normal
  • are sufficiently large

For the null hypothesis :

• • Rejection regions for test statistic: or
  • value:
• • Rejection regions for test statistic:
  • value:
• • Rejection regions for test statistic:
  • value:

Independent Samples with Unknown Different Variances

Condition

• population variances are unknown
• are sufficiently large

For the null hypothesis :

• • Rejection regions for test statistic: or
  • value:
• • Rejection regions for test statistic:
  • value:
• • Rejection regions for test statistic:
  • value:

Independent Samples with Equal Variances

Condition

• population variances are unknown but equal
• distributions are normal
• are small

For the null hypothesis :

• • Rejection regions for test statistic: or
  • value:
• • Rejection regions for test statistic:
  • value:
• • Rejection regions for test statistic:
  • value:

Dependent Samples

For paired data, define

For the null hypothesis ,

Depending on the sample size, the test statistic follows:

• small:
• large: