Lesson 12: Significance Testing
Review:
- Small Sample Estimation
- Exam 2 on Thu, Oct 17
Presentation:
- Significance Testing
- aka Hypothesis Testing
- Purpose: to evaluate data for evidence of significant agreement or disagreement
- Significance testing is like paternity testing.
- When you check father-child DNA for a match you can prove one person is or is not the father.
- The same test does not prove another person is the father.
- You’re evaluating only one possibility at a time.
- Step 1. Setup the null hypothesis (Ho) and alternate hypothesis (Ha)
- Step 2. Calculate the appropriate test statistic
- Step 3. Find the P-value (probability of obtaining result by chance)
- Step 4. Compare P-value to α = 0.05 ; if P-value < 0.05, “Reject Ho” else “Fail to Reject Ho”
- Significance Testing video (unit 25)
- Example A: Do Math SAT scores improve significantly with coaching?
- National Math SAT scores are normally distributed with mean = 505 and stdev = 62.
- Sample of 1,000 students who received coaching.
- Sample mean score was 509.
- Are these results significantly better than the national average?
- Step 1: Setup the hypothesis test
- μ = 505, σ = 62, x̅ = 509, n = 1,000
- Ho: μ = 505
Ha: μ > 505
- Ho: μ = 505
- μ = 505, σ = 62, x̅ = 509, n = 1,000
- Step 2: Calculate the appropriate test statistic
- Z-test = (x̅ – μ)/(σ/√n)
- (509 – 505)/(62/√1000) = 4/1.96 = 2.04
- Step 3: Find P-value
- P = 1 – P(Z<2.04) = 1 – 0.9793 = 0.0207
- Step 4: Interpret results
- 0.0207 ≅ 2.07% probability of getting this result by chance
- 0.0207 < 0.05
- Reject Ho
- Coaching seems to improve scores significantly
Activity:
- Portfolio Check
- Problem 1.1. More than 200,000 people worldwide take the GMAT examination each year as they apply for MBA programs. Their scores vary Normally with mean about μ = 525 and standard deviation about σ = 100. One hundred students, n = 100, go through a rigorous training program designed to raise their GMAT scores. The students who go through the program have an average score of x̅ = 541.4. Is there evidence to suggest the training program significantly improves GMAT scores?
- Problem 1.2. A newly installed rooftop solar system has been producing energy for n = 100 days. Average energy production is 41.8 kWh per day with a standard deviation of 13.9 kWh. The solar panel manufacturer claims the panels typically produce 40 kWh per day. Is the newly installed system producing significantly more energy than estimated by the manufacturer?
* Most example and activity problems presented above are derived from Moore, D.S., McCabe, G.P., and Craig, B.A., 2009. Introduction to the Practice of Statistics, 6th Edition. New York: W.H. Freeman and Company.
