STOR 435 Midterm 2 Study Guide

Bayes and independence · Random variables · The famous distributions · Joint distributions and covariance · Normal, limit theorems and inequalities

Download the guide as PDF · Follow-along worksheet PDF (write-in boxes + answer key) · Midterm 1 guide

0. Before you start

What is in scope

The syllabus listed slide decks 5–8, possibly some of 9. Two things settle it: the instructor has confirmed that deck 9 is not on the midterm, and her own STOR 435 Study Guide: Lectures 5–8 lists exactly what is. That scope is decks 5–8, which runs from conditional probability through the Normal distribution and ends, at the back of deck 8, with the central limit theorem, the law of large numbers and the Markov and Chebyshev inequalities (the syllabus had filed those under deck 9, but the slides and the official guide put them in deck 8). Deck 9 proper is jointly continuous random variables, which is left out. The homework is the best map of what will be asked, because the quizzes and exams follow the homework.

Sources: the filled-in and in-class-annotated slides for decks 5–8, Homeworks 6–10, the posted Midterm 2 practice problems, and the instructor's Lectures 5–8 Study Guide. Every result that guide marks with its warning sign is in a blue box here, with the explanation and the worked slide examples the guide itself leaves out.

How to use this guide

• It assumes you missed class. Every section starts from the definition and builds up. Blue boxes are the formulas and facts you must know cold. Orange boxes are the mistakes that cost points. Green boxes say how a question is likely to be phrased.
• Every homework problem is worked here, inside the section that teaches the idea it needs. The in-class quizzes were variants of the homework, and the exam will be too.
• Do each "Check yourself" with pen and calculator before opening the answer. On the exam you need the actual number to four decimals, not just the setup.
• Section 16 is the posted practice set, fully worked, followed by a second practice exam in the same style. Section 17 is a one-page distribution table and formula sheet for the night before.
• The worksheet PDF is the same text with every answer replaced by a ruled box, and all answers collected in a key at the end. Print it if you learn by writing.

If you have one day

1. Section 1 (30 minutes): make sure the series, integrals and Midterm 1 facts are at your fingertips.
2. Sections 2–3 (45 minutes): total probability, Bayes, independence. Do every Check-yourself.
3. Sections 4–7 (90 minutes): random variables, pmf/cdf/pdf, expectation and variance, MGFs. This is the core vocabulary; everything after it is applications.
4. Sections 8–10 (90 minutes): Binomial, Poisson, Uniform, Exponential. Learn the distribution table in section 17 as you go.
5. Sections 11–14 (90 minutes): geometric, joint distributions, indicators, covariance, Normal, CLT and the inequalities.
6. Section 16 Part A (75 minutes, timed): the posted practice problems. Then read every solution, including the ones you got right.
7. Section 17 before bed and again before the exam.

With two days, add Part B of section 16 on day two and redo every Check-yourself you got wrong.

1. The toolkit: what you need from before

Midterm 2 uses the Midterm 1 material constantly, and from section 5 onward it uses calculus. This section is the minimum. If any line here looks unfamiliar, fix that first; everything later assumes it.

1.1 Probability facts from Midterm 1

1.2 Series you must recognize on sight

Why these four: the geometric series is how you sum "first success on trial k" probabilities (section 3 and section 11). The exponential series is why the Poisson probabilities add to 1 and where the Poisson MGF comes from (section 9). The binomial theorem is why the Binomial probabilities add to 1 and gives the Binomial MGF (section 8). The limit is how the Binomial turns into the Poisson.

1.3 Calculus you must do fast

Worked integral. ∫₀^∞ x λe^−λx dx with u = x, dv = λe^−λx dx, so du = dx, v = −e^−λx: uv from 0 to ∞ is 0 (x e^−λx → 0), and −∫ v du = ∫₀^∞ e^−λx dx = 1/λ. This is the mean of the Exponential (section 10).

1.4 Calculator habits

• Keep four decimals until the end; round only the final number. Interval answers are usually generous, but (0.25, 0.4] versus (0.4, 0.5] hinges on the fourth decimal sometimes.
• Know where nCr, x!, e^x and ln are on your calculator. 0! = 1 and e⁰ = 1.
• Powers like 0.95¹⁰ are fine on the calculator; 365! is not. Multiply fractions one at a time.
• Have ready: e⁻¹ = 0.3679, e⁻² = 0.1353, e⁻³ = 0.0498, e⁻⁴ = 0.0183, e⁻⁵ = 0.0067, ln 2 = 0.6931, √(2π) = 2.5066, √(2/π) = 0.7979.

1.5 Check yourself

Deck 52. Total probability and Bayes' formula

The idea. Many probabilities are easiest to compute by splitting the world into cases ("does she have the disease or not?", "which bus did I take?"), computing the probability inside each case, and averaging with the case probabilities. That is the law of total probability. Bayes' formula is the same computation read backwards: it turns "probability of the evidence given the cause" into "probability of the cause given the evidence", which is what a doctor, a juror or a spam filter actually wants.

2.1 The law of total probability

The recipe the slides use every time: name the events first, write the formula in symbols, and only then plug in numbers. Most errors in this topic come from plugging numbers into the wrong slots.

Example: mammograms (slides). In the population described, 0.8% of women have breast cancer. A woman with cancer has a positive mammogram with probability 0.90; a woman without cancer, with probability 0.07. Let C = has cancer, T = positive test. The cases are C and C^c:

P(T) = P(T | C) P(C) + P(T | C^c) P(C^c) = 0.90(0.008) + 0.07(0.992) = 0.0072 + 0.06944 = 0.07664.

About 7.7% of these women test positive, and most of those positives (0.069 of the 0.077) come from women without cancer.

Example: triage (slides). 15% of ER patients are critical, 45% serious, 40% stable. Death rates are 50%, 20% and 2%. With D = dies,

P(D) = 0.50(0.15) + 0.20(0.45) + 0.02(0.40) = 0.075 + 0.09 + 0.008 = 0.173, so P(survives) = 0.827.

Example: grad school (slides). Eva gets into school A, B or C with probabilities 0.20, 0.40, 0.40, and the chances of a great job J afterwards are 0.95, 0.75, 0.05. Then P(J) = 0.95(0.20) + 0.75(0.40) + 0.05(0.40) = 0.19 + 0.30 + 0.02 = 0.51.

Example: the class-selection problem (slides). A class has 24 students; 6 have already been called on. Today a first group of 3 is picked at random, then a second group of 3 is picked at random (from all 24, so it may overlap anything). Find P(N), where N = the second group contains nobody from the original 6 and nobody from the first group.

The difficulty: how many students are "already chosen" when the second group is drawn depends on how much the first group overlapped the original 6. So condition on that. Let A_j = the first group contains exactly j of the original 6, j = 0, 1, 2, 3. These four events partition the sample space.

1. P(A_j) = (6j)(183−j)(243) (choose j from the 6 and 3 − j from the other 18; (243) = 2024).
2. Given A_j, the number of distinct already-chosen students is 6 + 3 − j = 9 − j, so the second group must come from the other 15 + j: P(N | A_j) = (15+j3)(243).
3. P(N) = Σ_j=0³ P(N | A_j) P(A_j) ≈ 0.2649. So P(at least one overlap) = 1 − 0.2649 = 0.7351.

Shortcut check from the slides: choose the second group first. It must avoid the original 6, (183) ways; then the first group can be any 3 of the remaining 21 students, (213) ways; so P(N) = (183)(213)/(243)² = 816 · 1330/2024² ≈ 0.2649. Same answer, no conditioning needed. On an exam, use whichever you see first, but know that "condition on the first stage" always works.

2.2 Bayes' formula: flipping the condition

There is nothing new to memorize: Bayes is the definition of conditional probability, P(F | E) = P(F ∩ E)/P(E), with the numerator written as P(E | F) P(F) by the multiplication rule and the denominator expanded by total probability.

Example: mammograms, continued. A woman from this population tests positive. The chance she has cancer is

P(C | T) = P(T | C) P(C)P(T) = 0.90(0.008)0.07664 = 0.00720.07664 ≈ 0.094.

About 9%, even though the test catches 90% of cancers. When Gigerenzer asked 24 doctors this question, their answers ranged from 1% to 90%. The reason the correct answer is small: cancer is rare here, so the 7% false-positive rate applied to the large healthy group produces far more positives than the 90% true-positive rate applied to the small cancer group.

Example: HIV test in a low-risk group (slides, from Calculated Risks). P(HIV) = 0.0001; P(positive | HIV) = 0.999; P(negative | no HIV) = 0.9999, so P(positive | no HIV) = 0.0001. Then

P(H | T) = 0.999(0.0001)0.999(0.0001) + 0.0001(0.9999) = 0.00009990.0000999 + 0.00009999 ≈ 0.50.

A positive result in this population is a coin flip, which is why a confirmatory test is always run. The slides' story of Betty (told she was HIV positive, two years of medication, then a negative retest and a $600,000 jury award) is the human cost of forgetting this.

Example: grad school, continued. Given that Eva has a great job, the chance she went to A is P(A | J) = 0.95(0.20)0.51 = 0.190.51 ≈ 0.373.

Example: a negative test (practice set, problem 5). 40% of adults are addicted to the Internet. A scan is positive for 70% of addicted adults and for 20% of non-addicted adults. Someone tests negative; chance they are addicted? The evidence is now "negative", so use P(negative | addicted) = 0.3 and P(negative | not) = 0.8:

P(addicted | negative) = 0.4(0.3)0.4(0.3) + 0.6(0.8) = 0.120.60 = 0.20.

2.3 Natural frequencies: Bayes without the formula

The slides recommend the way doctors are now taught: imagine a large population and count. For the HIV example, take 100,000 low-risk men.

Among the roughly 20 positives, 10 are real: P(HIV | positive) ≈ 10/20 = 50%. For the mammogram: 100,000 women, 800 with cancer of whom 720 test positive; 99,200 without cancer of whom 6,944 test positive; 720/(720 + 6,944) = 0.094. Use this to sanity-check any Bayes computation: if the counts and the formula disagree, one of them has a slot filled wrong.

2.4 Two courtroom lessons from the slides

Sally Clark. After two of her sons died of SIDS, a pediatrician testified that the chance of two SIDS deaths in an affluent family was (1/8500)² = 1 in 72 million. Two things are wrong. Multiplying treats the deaths as independent, ignoring shared genes, environment and medical factors (section 3). Worse, the court was told P(two deaths | innocent) when the question was P(innocent | two deaths); a tiny probability of the evidence under one explanation says nothing by itself about the probability of that explanation. She was convicted in 1999 and released in 2003.

O.J. Simpson. The defense argued that fewer than 1 in 2500 battered women are killed by their partner, so battery is a poor predictor of murder. But the woman in question had been murdered. Let B = battered, M = murdered, M_p = murdered by the partner, M^* = murdered by someone else. The relevant quantity is

P(M_p | B ∩ M) = P(M_p ∩ B)P(M_p ∩ B) + P(M^* ∩ B) = P(M_p | B)P(M_p | B) + P(M^* | B) = 1/25001/2500 + 5/100,000 = 89 ≈ 88%,

using the FBI figure that about 5 in 100,000 women are murdered by someone other than a partner. The slides are careful: this is a statement about the calculation, not about the verdict. The exam lesson is the same as in the Clark case: conditioning on all the evidence changes the answer completely.

2.5 Check yourself

Deck 53. Independence and independent trials

The idea. Two events are independent when learning that one happened tells you nothing about the other. Independence is what lets you multiply probabilities, and almost every distribution in this course (Binomial, Poisson, Geometric) is built from independent repetitions of a simple experiment. It is a property you check or assume, not a feeling that two things are "unrelated".

3.1 Definition

Why the two versions agree: if P(E ∩ F) = P(E)P(F) then P(E | F) = P(E ∩ F)/P(F) = P(E)P(F)/P(F) = P(E). Conversely, multiply P(E | F) = P(E) by P(F). The product form is the official definition because it also works when a probability is 0.

• Two coin tosses. E = first is H, F = second is T. S = {HH, HT, TH, TT}; E ∩ F = {HT}; P(E ∩ F) = 1/4 = (1/2)(1/2). Independent.
• One card. E = club, F = a 6. P(E ∩ F) = 1/52 = (1/4)(1/13). Independent: the suit tells you nothing about the rank.
• Two dice, F = first die is 4 (P = 1/6). E₁ = sum is 6: P(E₁) = 5/36 and E₁ ∩ F = {(4, 2)} has probability 1/36, but (5/36)(1/6) = 5/216 ≠ 1/36. Not independent. E₂ = sum is 7: P(E₂) = 6/36 = 1/6, E₂ ∩ F = {(4, 3)}, and (1/6)(1/6) = 1/36. Independent. Changing the target from 6 to 7 changes the answer; you must check.

3.2 Complements, and the disjoint trap

3.3 Three or more events

Why E is independent of F ∪ G (slides): P(E ∩ (F ∪ G)) = P(E ∩ F) + P(E ∩ G) − P(E ∩ F ∩ G) = P(E)[P(F) + P(G) − P(F)P(G)] = P(E)P(F ∪ G).

HW6 problem 3. E₁, E₂, E₃ independent. P(E₁ | E₂ ∩ E₃) = P(E₁), because E₁ is independent of E₂ ∩ E₃ (built from the other two).

HW6 problem 4. A, B, C independent with P(A) = 2/3, P(B) = 3/4, P(C) = 1/5.

• P(A ∩ B^c ∩ C^c) = (2/3)(1/4)(4/5) = 8/60 = 2/15 ≈ 0.1333.
• P(exactly one occurs) = P(A B^c C^c) + P(A^c B C^c) + P(A^c B^c C) = 8/60 + (1/3)(3/4)(4/5) + (1/3)(1/4)(1/5) = 8/60 + 12/60 + 1/60 = 21/60 = 0.35.

HW6 problem 5. E, F, G independent with probabilities 0.2, 0.3, 0.1. P(E ∪ F^c ∪ G) = 1 − P(E^c ∩ F ∩ G^c) = 1 − (0.8)(0.3)(0.9) = 1 − 0.216 = 0.784. Unions of independent events: go through the complement (De Morgan), then multiply.

Practice set, problem 4. E, F, G, H, I independent with probabilities 0.2, 0.3, 0.1, 0.4, 0.5. P((E ∪ F^c ∪ G) ∩ H | I): the event is built from E, F, G, H only, so it is independent of I and the conditioning does nothing. Then (E ∪ F^c ∪ G) and H are independent (disjoint groups), so the answer is 0.784 × 0.4 = 0.3136, in (0.25, 0.5].

3.4 Independent trials

HW6 problem 7. A coin with P(H) = 1/3 is flipped 10 times; E_i = toss i is H. The event "first 6 heads, next 4 tails" is F = E₁ ∩ ··· ∩ E₆ ∩ E₇^c ∩ ··· ∩ E₁₀^c, and P(F) = (1/3)⁶(2/3)⁴ ≈ 0.00027.

HW6 problem 8(1). Eva botches each order with probability 1/2, independently. P(first 10 correct, 11th wrong) = (1/2)¹⁰(1/2) = (1/2)¹¹. (This is the Geometric distribution of section 11 in disguise.)

HW6 problem 8(2): the race. Tim errs with probability 1/6, Eva with 1/2, per customer, independently. Eva takes the trip if she errs strictly before Tim does (a tie means the game stops and Eva stays). For Eva to lose on customer n, both must be perfect on customers 1 to n − 1, probability [(1/2)(5/6)]ⁿ⁻¹ = (5/12)ⁿ⁻¹, then Eva errs and Tim does not, probability (1/2)(5/6) = 5/12:

P(Eva takes the trip) = Σ_n=1^∞ (5/12)ⁿ⁻¹ (5/12) = 5/121 − 5/12 = 57 ≈ 0.7143.

By the same method P(Tim first) = (1/12)/(7/12) = 1/7 and P(tie) = (1/12)/(7/12) = 1/7, and 5/7 + 1/7 + 1/7 = 1. Any "who fails first" question is a geometric series: (probability nothing decisive happens)ⁿ⁻¹ × (probability of the decisive outcome).

HW6 problem 9. Roll two dice repeatedly; E_n = no 3 or 5 on the first n − 1 rolls and a 3 on roll n. P(sum 3) = 2/36 = 1/18, P(sum 5) = 4/36, P(neither) = 1 − 6/36 = 5/6. So P(E_n) = (5/6)ⁿ⁻¹(1/18). E₄ and E₅ are disjoint (E₅ requires no 3 on roll 4, E₄ requires a 3 there), so P(E₄ ∪ E₅) = (5/6)³(1/18) + (5/6)⁴(1/18) ≈ 0.0589: option (E). Options with a subtracted product are wrong because there is no overlap to remove, and options with factors 3 and 4 in front count the trials incorrectly.

Example: the flat tire (slides). Four friends independently name which of four tires went flat. Donatello and Michelangelo pick uniformly (1/4 each). Leonardo picks each left tire with twice the probability of each right tire: 2q + 2q + q + q = 1 gives q = 1/6, so left tires 1/3 each, right tires 1/6 each. Raphael is the mirror image. For any fixed tire the four answers agree with probability (1/4)(1/4)(1/3)(1/6) = 1/288, because one biased friend contributes 1/3 and the other 1/6 whichever tire it is. Four tires: P(all match) = 4/288 = 1/72. The structure the slides stress: first a sum over disjoint cases, then independence inside each case.

3.5 First-step analysis: "which happens first?"

Example: 5 before 7 (slides). Roll two dice repeatedly. P(a sum of 5 appears before a sum of 7)? Let p be this probability and condition on the first roll: sum 5 (probability 4/36), sum 7 (6/36), or neither (26/36). If neither, the game restarts from scratch, so the conditional probability is p again:

p = 1 · 436 + 0 · 636 + p · 2636, so p(1 − 2636) = 436, p = 410 = 25.

Alternating turns (practice set, problem 3). A rolls a 4-sided and a 6-sided die aiming for sum 3 (outcomes (1,2), (2,1): p_A = 2/24 = 1/12); then B rolls aiming for sum 4 ((1,3), (2,2), (3,1): p_B = 3/24 = 1/8); they alternate until someone succeeds. Condition on the first round: A wins immediately with probability p_A; otherwise, if B also fails, the game restarts:

p = p_A + (1 − p_A)(1 − p_B) p, so p = p_A1 − (1 − p_A)(1 − p_B) = 1/121 − (11/12)(7/8) = 1/1219/96 = 819 ≈ 0.4211,

in (0.25, 0.45]. Going first is worth something but A's target is harder, so A is still the underdog.

3.6 Check yourself

Deck 64. Random variables: pmf and cdf

The idea. A random variable is a number produced by a random experiment: the number of heads in three tosses, the largest ball drawn, the price of a stock next Monday at 9 a.m. The sample space might be hopelessly complicated (the state of the universe next Monday), but we only care about one measurement of it. From now on the course describes randomness through random variables and their distributions rather than through events and sample spaces, although every statement still is one.

4.1 Definition and the two types

• Example 1a. Toss 3 fair coins; Y = number of heads. Y takes values {0, 1, 2, 3}.
• Example 1b. Draw 3 balls without replacement from balls numbered 1 to 20; X = the largest number drawn. X takes values {3, 4, …, 20} (the largest of three distinct numbers is at least 3).
• Example 1c. Pick a point "at random" from [0, 1]; X = the point. X takes uncountably many values, and we will need a new way to describe it.

4.2 The probability mass function (discrete)

Example 1a worked. The 8 equally likely outcomes are HHH, HHT, HTH, THH, HTT, THT, TTH, TTT with Y = 3, 2, 2, 2, 1, 1, 1, 0. So p(0) = 1/8, p(1) = 3/8, p(2) = 3/8, p(3) = 1/8. Check: they add to 1.

Example 1b worked. All (203) = 1140 sets of three balls are equally likely. For the largest to equal k, ball k must be in the set and the other two must come from {1, …, k − 1}: (k−12) sets. So

P(X = k) = (k−12)(203), k = 3, 4, …, 20.

For instance P(X = 20) = (192)/1140 = 171/1140 = 0.15, and P(X = 3) = 1/1140.

HW7 problem 2: the second largest. Same urn, X = the second largest of the three numbers, values 2 to 19. For X = i you need one ball below i (i − 1 choices) and one above (20 − i choices):

p(i) = (i − 1)(20 − i)(203), i = 2, …, 19: option (D).

The wrong options have (i + 1) or (20 − i + 1), which would allow a ball to equal i twice. Sanity-check a formula at an edge: p(19) = 18 · 1/1140 (one ball must be ball 20, the other anything below 19); p(2) = 1 · 18/1140.

Biased coin (slides). Three tosses, P(H) = 1/4 on each, independent. Y = number of heads. Each sequence with k heads has probability (1/4)^k(3/4)^3−k and there are (3k) such sequences:

This is the Binomial(3, 1/4) distribution of section 8. Notice that "0 heads" and "1 head" are equally likely here, and that the values 0, 1, 2, 3 are certainly not equally likely.

HW7 problem 3 (a table). X takes values −1.3, 1.2, 2.7, 3.3, 9.8 with probabilities 0.1, 0.25, 0.35, 0.1, 0.2. A pmf given as a table is the most common exam format; the questions then ask for probabilities of ranges, the cdf at a point, or expectations (section 6).

4.3 The cumulative distribution function

For the biased coin: F(x) = 0 for x < 0, 27/64 on [0, 1), 54/64 on [1, 2), 63/64 on [2, 3), and 1 for x ≥ 3. The value at a jump is the upper level (F(1) = 54/64 includes P(X = 1)).

HW7 problem 1. F(x) = 0 for x < −2.5; 2/12 on [−2.5, 0); 5/12 on [0, 1); 8/12 on [1, 6.5); 1 for x ≥ 6.5. Jumps: at −2.5 of size 2/12, at 0 of size 3/12, at 1 of size 3/12, at 6.5 of size 4/12. So the pmf is

HW7 problem 3(a). For the table above, F(−1.5) = P(X ≤ −1.5) = 0, because the smallest value, −1.3, is larger than −1.5 (nothing sits at or below −1.5). And F(3.5) = 0.1 + 0.25 + 0.35 + 0.1 = 0.8: everything at or below 3.5, which excludes only the value 9.8. The cdf at a point that is not a possible value is just the running total of the pmf up to that point.

Practice set, problem 8 (reading a cdf graph). The graph of F is 0 left of −2, 0.1 on [−2, −0.75), 0.3 on [−0.75, 0), 0.6 on [0, 1), 0.8 on [1, 2), and 1 from 2 on. Jumps: 0.1 at −2, 0.2 at −0.75, 0.3 at 0, 0.2 at 1, 0.2 at 2. That is the pmf; the expected value (section 6) is then (−2)(0.1) + (−0.75)(0.2) + 0(0.3) + 1(0.2) + 2(0.2) = −0.2 − 0.15 + 0.2 + 0.4 = 0.25, in (0.2, 0.3].

4.4 Indicator random variables

4.5 Check yourself

Deck 65. Continuous random variables: pdf and cdf

The idea. A point chosen at random from [0, 1] has uncountably many possible values, so we cannot give each value a positive probability (they would add up to infinity). Instead we describe how probability is spread along the line with a density function, and probabilities of intervals are areas under it. Every discrete formula with a sum has a continuous twin with an integral.

5.1 The probability density function

Uniform on [0, 1] (slides). We want every point in [0, 1] equally likely, so the density is a constant C on [0, 1] and 0 elsewhere. Since 1 = ∫₀¹ C dx = C, the density is f(x) = 1 on [0, 1]. Then

• P(X > 0.25) = ∫_0.25¹ 1 dx = 0.75.
• P(0.5 ≤ X ≤ 1.7) = ∫_0.5¹ 1 dx = 0.5 (the density is 0 above 1).
• P(X = 0.787) = 0.

For a uniform variable, probability = length of the piece of the interval you are asking about, divided by the total length.

5.2 The cdf of a continuous variable

Uniform on [0, 1]. F(x) = 0 for x < 0, F(x) = ∫₀^x 1 dt = x for 0 ≤ x ≤ 1, and F(x) = 1 for x > 1. Differentiating the middle piece gives back f = 1. A continuous, non-step cdf is the signature of a continuous random variable.

5.3 Finding the constant, then the probabilities

A density is often given as "c · (something)". The first step is always the same: set the total integral equal to 1 and solve for c. Three homework examples.

HW10 problem 3 (a power-law density). f(x) = c/x⁴ for x ≥ 10, else 0. These densities model wealth, city sizes and social networks.

1. 1 = ∫₁₀^∞ c x⁻⁴ dx = c · 10⁻³3 = c3000, so c = 3000.
2. P(X > 20) = ∫₂₀^∞ 3000 x⁻⁴ dx = 3000 · 20⁻³3 = 10008000 = 0.125.

(The mean and variance of this X are in section 6.)

HW10 problem 6 (a symmetric density). f(x) = c x² on [−4, 4].

1. 1 = c ∫₋₄⁴ x² dx = c · 2 · 643 = 128c3, so c = 3/128.
2. P(|X| = 1/2) = 0 (a single value for a continuous variable, or rather two values, ±1/2, each of probability 0).
3. P(|X| > 2) = P(X > 2) + P(X < −2) = 2 · 3128 ∫₂⁴ x² dx = 2 · 3128 · 64 − 83 = 112128 = 0.875.

HW10 problem 5 (the Laplace density). f(x) = ½ e^−|x| on the whole line: two Exponential(1) halves glued back to back, each carrying probability 1/2.

1. P(|X| ≤ 2) = 2 ∫₀² ½ e^−x dx = 1 − e⁻² ≈ 0.8647.
2. {|X| ≤ 2 or X ≤ −1} = {X ≤ 2} (the second set swallows the left part), so the probability is P(X ≤ 2) = 1 − P(X > 2) = 1 − ½ e⁻² ≈ 0.9323.
3. P(X = 1 or X = 2 or …) = 0: a countable union of probability-zero events.

5.4 Conditional probability with a continuous variable

Nothing changes: P(A | B) = P(A ∩ B)/P(B), with both probabilities computed as integrals.

HW10 problem 4. A bus arrives at a time X uniform on (0, 30) minutes after 10:00. At 10:20 it has not come; P(you wait at least 5 more minutes)? That is P(X ≥ 25 | X > 20) = P(X ≥ 25)P(X > 20) = 5/3010/30 = 12. Given the bus is late, its arrival is uniform on the remaining 10 minutes.

5.5 The distribution of Y = g(X): the cdf method

HW10 problem 6(4). X has density (3/128) x² on [−4, 4]; Y = X². Range: [0, 16]. For 0 ≤ y ≤ 16,

F_Y(y) = P(X² ≤ y) = P(−√y ≤ X ≤ √y) = ∫_−√y^√y 3128 x² dx = 3128 · 2 y^3/23 = y^3/264.

f_Y(y) = 3128 √y for 0 ≤ y ≤ 16, and 0 otherwise.

Check: ∫₀¹⁶ (3/128) y^1/2 dy = (3/128)(2/3)(16^3/2) = (1/64)(64) = 1.

Practice set, problem 21. X uniform on [−0.5, 0.5] (density 1 there), Y = X³. Range: [−1/8, 1/8]. The cube is increasing, so for y in the range

F_Y(y) = P(X ≤ y^1/3) = y^1/3 + 0.5, f_Y(y) = 13 y^−2/3.

At y = 0.05: (1/3)(0.05)^−2/3 ≈ (1/3)(7.368) ≈ 2.456, in (1.5, 3]. A density above 1 is normal; Y is squeezed into an interval of length 1/4.

A linear change. If Y = aX + b with a > 0, then F_Y(y) = F_X((y − b)/a) and f_Y(y) = 1a f_X(y − ba). So if X is uniform on (0, 1), then 30X is uniform on (0, 30), and if Z is standard Normal then μ + σZ is Normal with mean μ and standard deviation σ (section 13).

5.6 Check yourself

Deck 66. Expectation, variance and functions of a random variable

The idea. The expected value is the long-run average of a random variable: what you would get per trial if you repeated the experiment forever. The variance measures how spread out the values are around that average. Both are computed from the pmf (sum) or the pdf (integral), and both obey a few algebra rules that you will use on every exam question.

6.1 Expected value of a discrete random variable

Why this is the long-run average (slides). Repeat the experiment n times. The average of the results is Σ_k x_k · (number of times x_k occurred)/n, and as n grows each frequency (count)/n settles down to p(x_k). So the average tends to Σ x_k p(x_k). HW7 problem 5 phrases it this way: among a huge number of candidates like Brad, the average number of interviews would be close to E(X).

• A fair die. E(X) = (1 + 2 + 3 + 4 + 5 + 6)/6 = 3.5. The mean need not be a value X can take; do not be confused by the word "expected".
• An indicator. I = 1 if A occurs, 0 otherwise. E(I) = 1 · P(A) + 0 · P(A^c) = P(A). Expectation of an indicator is a probability; this will matter a lot in section 11.
• Brad's interviews (HW7 problem 5). X = 0, 1, 2, 3 with probabilities 0.1, 0.2, 0.6, 0.1: E(X) = 0 + 0.2 + 1.2 + 0.3 = 1.7.
• HW7 problem 3(b). Values −1.3, 1.2, 2.7, 3.3, 9.8 with probabilities 0.1, 0.25, 0.35, 0.1, 0.2: E(X) = −0.13 + 0.30 + 0.945 + 0.33 + 1.96 = 3.405.
• From a cdf (practice set, problem 8). Read the pmf off the jumps first (section 4.3), then E(X) = 0.25.

6.2 Expectation of a function of X

Example (slides). X = −1, 0, 1 with probabilities 0.2, 0.5, 0.3 and Y = X². The long way: Y = 0 with probability 0.5 and Y = 1 with probability 0.5, so E(Y) = 0.5. The plug-in way: 0.2(−1)² + 0.5(0)² + 0.3(1)² = 0.5. Same number, and the second way needs no new pmf.

HW7 problem 3(b), continued. E(e^X) = 0.1e^−1.3 + 0.25e^1.2 + 0.35e^2.7 + 0.1e^3.3 + 0.2e^9.8 ≈ 0.027 + 0.830 + 5.206 + 2.711 + 3606.75 ≈ 3615.5. One large value dominates; that is typical of e^X, and it is the reason MGFs (section 7) are so informative.

HW7 problem 4 (stocking a product). Each unit sold earns $5; each unsold unit loses $15. Demand X = 2, 3, 4 with probabilities 0.1, 0.2, 0.7. If the store stocks s units, profit = 5 · min(X, s) − 15 · max(s − X, 0). Compute the expected profit for each s:

• s = 2: every demand buys both units, profit 10 always, E P(2) = 10.
• s = 3: demand 2 gives 10 − 15 = −5 (probability 0.1); demand 3 or 4 gives 15 (probability 0.9). E P(3) = −0.5 + 13.5 = 13.
• s = 4: demand 2 gives 10 − 30 = −20 (0.1); demand 3 gives 15 − 15 = 0 (0.2); demand 4 gives 20 (0.7). E P(4) = −2 + 0 + 14 = 12.

So E P(2) < E P(4) < E P(3): option (B). Stock 3. The method is the whole point: list the values of the function for each outcome, then weight by the probabilities.

6.3 Linearity

HW7 problem 5(8). Each interview costs 10 hours of preparation and the applications took 20 hours, so Y = 10X + 20 and E(Y) = 10(1.7) + 20 = 37 hours.

6.4 Variance and standard deviation

The mean says nothing about spread: W = 0, Y = ±1 (each with probability 1/2) and Z = ±100 all have mean 0. The variance distinguishes them.

Why the two forms agree (slides): E[(X − μ)²] = E[X² − 2μX + μ²] = E(X²) − 2μ E(X) + μ² = E(X²) − 2μ² + μ² = E(X²) − μ², using linearity and the fact that μ is a constant.

• A fair die. E(X²) = (1 + 4 + 9 + 16 + 25 + 36)/6 = 91/6. Var(X) = 91/6 − (7/2)² = 91/6 − 49/4 = 35/12 ≈ 2.917; SD ≈ 1.708.
• Brad (HW7 problem 5). E(X²) = 0 + 0.2 + 2.4 + 0.9 = 3.5; Var(X) = 3.5 − 1.7² = 3.5 − 2.89 = 0.61.
• W, Y, Z above. Var(W) = 0, Var(Y) = 1, Var(Z) = 10,000.

HW7 problem 5(9). Var(Y) = Var(10X + 20) = 100 · 0.61 = 61 hours²; SD(Y) = √61 ≈ 7.81 hours.

6.5 The continuous versions

Uniform on (0, 1). E(X) = ∫₀¹ x · 1 dx = 1/2. E(X²) = ∫₀¹ x² dx = 1/3. Var(X) = 1/3 − 1/4 = 1/12.

The chocolate bar (slides). A 1-foot bar is broken at a uniform point U. Amy gets the piece containing a fixed point p. If U < p her piece is [U, 1], length 1 − U; if U > p it is [0, U], length U. So the length L = g(U) and

E(L) = ∫₀^p (1 − u) du + ∫_p¹ u du = (p − p²2) + (12 − p²2) = 12 + p − p².

At p = 1/2 this is 3/4: the piece containing the middle is on average three quarters of the bar, because whichever side the break lands on, Amy gets the bigger side plus a bit. The plug-in rule with a piecewise g is exactly this: split the integral where the formula for g changes.

HW10 problem 3, continued. f(x) = 3000/x⁴ for x ≥ 10.

• E(X) = ∫₁₀^∞ 3000 x⁻³ dx = 3000 · 10⁻²2 = 15.
• E(X²) = ∫₁₀^∞ 3000 x⁻² dx = 3000 · 10⁻¹ = 300, so Var(X) = 300 − 225 = 75.
• E(2X² + 3X) = 2(300) + 3(15) = 645 by linearity.

(With c/x³ instead of c/x⁴ the variance integral would diverge: heavy-tailed variables can have a mean but infinite variance. The homework chose the exponent 4 so that everything exists.)

6.6 Check yourself

Deck 67. Moment generating functions

The idea. The moment generating function packs every moment E(X), E(X²), E(X³), … of a random variable into one function of t. Differentiate it at 0 and the moments fall out. More importantly, the MGF is the "DNA" of a distribution: two random variables with the same MGF have the same distribution, so you can recognize a Binomial or a Poisson from its MGF, and you can find the distribution of a sum of independent variables by multiplying MGFs.

7.1 Definition

HW7 problem 5(3). X = 0, 1, 2, 3 with probabilities 0.1, 0.2, 0.6, 0.1. M_X(t) = 0.1 + 0.2e^t + 0.6e^2t + 0.1e^3t. For a finite table the MGF is just "probability times e^{t · value}, added up".

7.2 Derivatives at 0 give the moments

Why (slides): differentiating e^tX with respect to t gives X e^tX, so M′(t) = E(X e^tX) and M′(0) = E(X e⁰) = E(X). Differentiate again: M″(t) = E(X² e^tX), M″(0) = E(X²). (Moving the derivative inside the sum or integral is allowed whenever the MGF exists near 0.)

HW7 problem 5(4)–(7). M′(t) = 0.2e^t + 1.2e^2t + 0.3e^3t, so M′(0) = 0.2 + 1.2 + 0.3 = 1.7 = E(X). M″(t) = 0.2e^t + 2.4e^2t + 0.9e^3t, M″(0) = 3.5 = E(X²), Var(X) = 3.5 − 2.89 = 0.61. Exactly the numbers from section 6, which is the point: the MGF is a machine that produces them.

7.3 The MGF determines the distribution

HW7 problem 10 (Sherlock's scrap of paper). M_X(t) = 34 e^2t + 18 e^t + 18. Match the pattern: X = 2 with probability 3/4, X = 1 with probability 1/8, X = 0 with probability 1/8 (the constant is e^{0 · t}). Then E(X) = 2(3/4) + 1(1/8) = 1.625, E(X²) = 4(3/4) + 1(1/8) = 3.125, Var(X) = 3.125 − 1.625² = 3.125 − 2.6406 = 0.4844 ≈ 0.48. Differentiating twice gives the same thing; reading off the pmf is faster.

HW9 problem 8 (Bert and Ernie). M_X(t) = (1/3 + (2/3)e^t)⁵ is the Binomial MGF (q + pe^t)ⁿ with n = 5, p = 2/3 (section 8), and M_Y(t) = e^{2(et − 1)} is the Poisson MGF with λ = 2 (section 9). Recognizing them gives E(X) = 10/3, Var(X) = 10/9, E(Y) = 2, Var(Y) = 2 instantly; the rest of that problem (Z = XY) is finished in section 12.

7.4 MGFs of linear transforms and of sums

Practice set, problem 15. X, Y independent; the MGF of 3X − 2Y is M_X(3t) M_Y(−2t): option (C). Options with powers like (M_X(t))³ confuse "3X" with "X₁ + X₂ + X₃" (three independent copies would give M_X(t)³); options with sums of MGFs are never right.

HW10 problem 8. (1) M_X−2Y(t) = M_X(t) M_Y(−2t). (2) Without independence, no: the MGF of X − 2Y depends on the joint distribution. If Y = X (perfectly dependent) then X − 2Y = −X has MGF M_X(−t), while an independent copy gives M_X(t)M_X(−2t); same marginals, different answers.

HW9 problem 5. X ~ Binomial(4, 0.3) and Y ~ Poisson(2) independent; E(e^X+Y) = M_X+Y(1) = M_X(1) M_Y(1) = (0.7 + 0.3e)⁴ · e^2(e−1) ≈ 5.2748 × 31.080 ≈ 163.94. An MGF evaluated at t = 1 is just E(e^X).

Sums of famous variables. Multiply the MGFs and recognize the result: Binomial(n, p) + Binomial(m, p) = Binomial(n + m, p); Poisson(λ) + Poisson(μ) = Poisson(λ + μ); Normal + Normal = Normal with the means and variances added. These are used in sections 9, 12 and 13.

7.5 MGFs you should recognize

7.6 Check yourself

Deck 68. Bernoulli and Binomial random variables

The idea. The slides call the famous distributions the "X-men" and study each one with the same workflow: discrete or continuous? origin story? properties (mean, variance, MGF)? applications? The Binomial is the first: it counts successes in a fixed number of independent, identical yes/no trials. If you can say "n independent trials, each a success with probability p, and I am counting successes", you have a Binomial.

8.1 Bernoulli trials

8.2 The Binomial distribution

Where the formula comes from (slides, 3 trials, p = 1/4). P(X = 0) = P(FFF) = (3/4)³. P(X = 1) = P(SFF) + P(FSF) + P(FFS) = 3 · (1/4)(3/4)²: each sequence with exactly one success has the same probability, and there are (31) = 3 positions for the success. In general each sequence with i successes has probability pⁱ(1 − p)ⁿ⁻ⁱ (independence), and there are (ni) such sequences. The probabilities add to 1 by the binomial theorem: Σ (ni) pⁱ(1 − p)ⁿ⁻ⁱ = (p + 1 − p)ⁿ = 1.

8.3 Mean, variance and MGF

The MGF (slides). M_X(t) = Σ_k e^tk (nk) p^k q^n−k = Σ_k (nk) (pe^t)^k q^n−k = (q + pe^t)ⁿ, by the binomial theorem with x = q and y = pe^t. Then M′(t) = n(q + pe^t)ⁿ⁻¹ pe^t, so E(X) = M′(0) = np. One more derivative (product rule) gives M″(0) = np + n(n − 1)p² = E(X²), hence Var(X) = np + n²p² − np² − n²p² = np(1 − p). (Section 12 gives the quick proof: X is a sum of n independent Bernoullis, so means and variances add.)

The variance is largest at p = 1/2 and shrinks to 0 at p = 0 or 1: a nearly certain outcome has little spread.

8.4 Worked examples

HW7 problem 6: headlights. Each of 50 cars forgets its lights with probability 0.04, independently. X ~ B(50, 0.04). P(X = 2) = (502)(0.04)²(0.96)⁴⁸ = 1225 · 0.0016 · 0.1409 ≈ 0.2762.

HW7 problem 9: Brad's exam. 80 questions, 5 choices each. Brad knows the answer with probability 0.75; otherwise he guesses. Per question, P(correct) = 0.75 + 0.25(1/5) = 0.80 (total probability). Questions are independent, so X ~ B(80, 0.8): E(X) = 64, Var(X) = 80(0.8)(0.2) = 12.8.

HW7 problem 8: flu vaccines. 300 employees all get the GS vaccine, which is 80% effective, so each still gets the flu with probability 0.2, independently: X ~ B(300, 0.2) employees fall ill. Cost C = 300 · $40 + $300 · X = 12,000 + 300X. By linearity and the scaling rule, E(C) = 12,000 + 300(60) = $30,000 and Var(C) = 300² · Var(X) = 90,000 · 300(0.2)(0.8) = 90,000 · 48 = 4,320,000.

Practice set, problems 9–10 (same story, 40 employees, 70% effective, $70 per vaccine, $1000 per sick employee). X ~ B(40, 0.3): E(X) = 12, SD(X) = √(40 · 0.3 · 0.7) = √8.4 ≈ 2.90, option (D). C = 2800 + 1000X, so Var(C) = 1000² · 8.4 = 8,400,000, in [10,000, ∞), option (E). The fixed vaccine cost contributes nothing to the variance.

Motco's boxes (slides): a Binomial inside a Binomial. A box holds 10 packets; each packet has raisins with probability 0.05, independently. A box is returned if more than one packet has raisins. Eva buys 5 boxes; P(she returns exactly one)?

1. Inside one box: Y = number of raisin packets ~ B(10, 0.05). The box is returned with probability p = P(Y ≥ 2) = 1 − P(Y = 0) − P(Y = 1) = 1 − 0.95¹⁰ − 10(0.05)(0.95)⁹ = 1 − 0.5987 − 0.3151 ≈ 0.0861.
2. Across boxes: each box is returned with probability p independently, so X = number returned ~ B(5, p) and P(X = 1) = (51) p (1 − p)⁴ = 5(0.0861)(0.9139)⁴ ≈ 0.3004.

HW7 problem 7: pooled testing. 1000 employees, each infected with probability 0.05; samples are pooled 10 at a time, so there are 100 pools. A pool tests positive if at least one of its 10 members is infected: probability 1 − 0.95¹⁰ ≈ 0.4013. A negative pool costs 1 test; a positive pool costs 1 + 10 tests (everyone is retested). Expected tests per pool = 1 + 10(0.4013) = 5.013; expected total = 100 × 5.013 ≈ 501.3, about half of testing everyone individually.

Airplane engines (slides, from the textbook). Engines fail independently with probability 1 − p; a plane needs a majority of its engines. A 3-engine plane succeeds with probability 3p²(1 − p) + p³; a 5-engine plane with (53)p³(1 − p)² + (54)p⁴(1 − p) + p⁵. At p = 0.6 these are 0.648 and 0.683; at p = 0.4 they are 0.352 and 0.317; at p = 1/2 both are 1/2. The 5-engine plane is preferable exactly when p > 1/2: redundancy only helps when the components are reliable to begin with. (The slides quote Downer's article on redundancy in complex systems for the engineering context.)

8.5 Recognizing a Binomial (and when it is not one)

8.6 Check yourself

Deck 69. Poisson random variables and the Poisson process

The idea. The Poisson distribution counts rare events: misprints on a page, bombs falling on a quarter-square-kilometer of London, goals by Gretzky in a game, customers at a post office. It is what a Binomial becomes when the number of trials is huge and the success probability tiny, and it is the distribution of the number of events in a time interval when events occur "at random" at a constant rate. One parameter, λ, is both its mean and its variance.

9.1 Definition and basic properties

Why e^−λ (slides). The probabilities must add to 1: Σ_i e^−λ λⁱ/i! = e^−λ Σ_i λⁱ/i! = e^−λ e^λ = 1, by the Taylor series of e^λ. The e^−λ is exactly the normalizing constant.

The MGF (slides). M(t) = Σ e^ti e^−λ λⁱ/i! = e^−λ Σ (λe^t)ⁱ/i! = e^−λ e^λet = e^{λ(et − 1)}. Then M′(t) = λe^t e^{λ(et − 1)}, so E(X) = M′(0) = λ; M″(t) = (λe^t)² e^{λ(et − 1)} + λe^t e^{λ(et − 1)}, so E(X²) = λ² + λ and Var(X) = λ.

9.2 The Poisson as a limit of the Binomial

Why (slides). Put p = λ/n. Then P(Y = k) = (nk) (λ/n)^k (1 − λ/n)^n−k = n(n−1)···(n−k+1)n^k · λ^kk! · (1 − λ/n)ⁿ · (1 − λ/n)^−k. As n → ∞ the first factor → 1, (1 − λ/n)ⁿ → e^−λ, and the last factor → 1, leaving e^−λλ^k/k!. The mean np = λ and the variance np(1 − p) ≈ λ match too.

Evidence from the slides. 537 bombs fell on 576 quarter-square-kilometer squares of South London, λ = 537/576 = 0.93 per square. Predicted number of squares with 0, 1, 2, 3, 4, 5+ bombs: 226.7, 211.4, 98.5, 30.6, 7.1, 1.6. Observed: 229, 211, 93, 35, 7, 1. Gretzky's points per game (λ = 1669/696 ≈ 2.4) fit a Poisson just as well. Many tiny chances, added up, give a Poisson.

Brad's dorm (slides). 180 other students; X = number sharing Brad's birthday. Exactly, X ~ B(180, 1/365); approximately, Poisson with λ = 180/365 ≈ 0.4932.

• (a) P(X ≥ 1) ≈ 1 − e^−0.4932 ≈ 0.3893. Exact: 1 − (364/365)¹⁸⁰ ≈ 0.3897.
• (b) P(X = 1) ≈ 0.4932 e^−0.4932 ≈ 0.3012. Exact: 180(1/365)(364/365)¹⁷⁹ ≈ 0.3018.
• (c) P(X ≥ 2) ≈ 1 − e^−0.4932(1 + 0.4932) ≈ 0.0881.

HW8 problem 1: yelling runners. Brad passes n runners; each yells with probability 0.03.

1. n = 100: X ≈ Poisson(3). P(X > 2) = 1 − e⁻³(1 + 3 + 4.5) = 1 − 8.5e⁻³ ≈ 0.5768. (Exact Binomial: 0.5802.)
2. P(X ≥ 1) = 1 − e^−0.03n ≥ 0.95 ⇔ e^−0.03n ≤ 0.05 ⇔ n ≥ ln(20)/0.03 ≈ 99.86, so n = 100 runners.

Practice set, problem 12. 200 runners, probability 0.01: X ≈ Poisson(2), P(X > 2) = 1 − e⁻²(1 + 2 + 2) = 1 − 5e⁻² ≈ 0.3233, in (0.25, 0.40].

9.3 More worked examples

HW8 problems 2–3. P(X = 1 | X ≤ 1) = λe^−λe^−λ + λe^−λ = λ1 + λ = 0.5, so λ = 1. Conditional probability with a Poisson: both numerator and denominator are sums of pmf terms, and e^−λ cancels.

HW8 problem 4: does the drug work? (Bayes with a Poisson likelihood). Colds per year are Poisson(5); the drug lowers λ to 3 for 75% of people and does nothing for 25%. A user has 2 colds. Let B = the drug is beneficial for him.

P(B | 2 colds) = 0.75 · e⁻³3²/2!0.75 · e⁻³3²/2! + 0.25 · e⁻⁵5²/2! = 0.75(0.2240)0.75(0.2240) + 0.25(0.0842) = 0.16800.1891 ≈ 0.8886.

HW8 problem 5: Clark and Diana. 400 questions; Clark errs with probability 0.0075, Diana 0.003. X_C ≈ Poisson(3), X_D ≈ Poisson(1.2). The sum of independent Poissons is Poisson with the summed mean (multiply the MGFs: e^3(et−1) e^1.2(et−1) = e^4.2(et−1)), so X_C + X_D ≈ Poisson(4.2) and P(X_C + X_D = 2) = e^−4.2(4.2)²/2 ≈ 0.1323.

Practice set, problem 17. X ~ Poisson(1), Y ~ Poisson(2), independent. P(max(X, Y) ≤ 1) = P(X ≤ 1 and Y ≤ 1) = P(X ≤ 1) P(Y ≤ 1) = (e⁻¹ + e⁻¹)(e⁻² + 2e⁻²) = 2e⁻¹ · 3e⁻² = 6e⁻³ ≈ 0.2987, in [0, 0.5). (Section 12.4 has the general max/min method.)

Two buses (slides). Bus 1 carries Poisson(4) passengers, bus 2 Poisson(5); you take bus 1 with probability 1/3. P(at least 2 passengers) = 13[1 − e⁻⁴(1 + 4)] + 23[1 − e⁻⁵(1 + 5)] = 13(0.9084) + 23(0.9596) ≈ 0.9425. Total probability over which bus, then a Poisson inside each case.

9.4 The Poisson process

Examples from the slides: earthquakes, deaths among an insurer's policy holders, shark attacks, buses arriving at a stop. The sample path N(t) is a staircase that goes up by 1 at each event.

Light bulbs (slides). Bulbs burn out at rate 7.2 per day. (1) Noon to 2 p.m. is 2/24 = 1/12 day, so N ~ Poisson(0.6): P(N > 2) = 1 − e^−0.6(1 + 0.6 + 0.18) ≈ 0.0231. (2) Next week: N ~ Poisson(50.4), P(N = 50) = e^−50.4 50.4⁵⁰/50! ≈ 0.0562. (If the calculator overflows on 50.4⁵⁰, compute ln of the whole thing: −50.4 + 50 ln 50.4 − ln 50! ≈ −2.878, then exponentiate.)

HW8 problem 6: I-40 accidents. 3 accidents per weekday, days independent. (1) P(no accidents Monday) = e⁻³ ≈ 0.0498. (2) Let p = P(exactly 2 on a given day) = e⁻³3²/2 ≈ 0.2240. The number of days (out of 5) with exactly 2 accidents is B(5, p), so P(exactly 4 such days) = (54) p⁴(1 − p) ≈ 5(0.2240)⁴(0.7760) ≈ 0.0098. A Binomial whose p comes from a Poisson.

HW8 problems 7–8: Diane's phone. Calls arrive at rate 2 per hour. (7) In 10 minutes, N ~ Poisson(2 · 1/6) = Poisson(1/3): P(at least one call) = 1 − e^−1/3 ≈ 0.2835. (8) P(no calls in t hours) = e^−2t, which equals 0.5 when t = (ln 2)/2 ≈ 0.3466 hours ≈ 20.8 minutes. Breaks of at least 20.8 minutes have P(no calls) ≤ 0.5 (the homework's literal wording, "at most 0.5"); shorter breaks have P(no calls) > 0.5. Either way the number to report is the threshold t = (ln 2)/2, found by taking ln of both sides.

Practice set, problem 11. Earthquakes at 4 per week; in 2 weeks N ~ Poisson(8), so P(N = 4) = e⁻⁸ 8⁴/4!: option (C). Options with e⁻⁴ forgot to multiply the rate by the length; option (D) divided instead of multiplying.

9.5 Check yourself

Deck 610. Uniform and Exponential random variables

The idea. Two continuous "X-men". The Uniform is "a point chosen at random from an interval": every sub-interval of the same length is equally likely. The Exponential is "the waiting time until the next event" when events occur at a constant rate; it is the continuous twin of the Poisson process and it has no memory, which makes it both strange and extremely useful.

10.1 The Uniform distribution on (a, b)

Mean and variance (slides). E(X) = ∫_a^b xb − a dx = b² − a²2(b − a) = a + b2. E(X²) = b³ − a³3(b − a) = a² + ab + b²3. Subtracting (a + b)²4 and simplifying gives (b − a)²12. For (0, 1): mean 1/2, variance 1/12, as in section 6.

Practice set, problem 19 (chocolate bar again). The toothpick is at p = 1/4 and the break point U is uniform on [0, 1]. Amy's piece has length g(U) = 1 − U if U < 1/4 (she gets [U, 1]) and g(U) = U if U > 1/4. P(g(U) ≥ 1/2) = P(U < 1/4 and 1 − U ≥ 1/2) + P(U ≥ 1/4 and U ≥ 1/2) = P(U < 1/4) + P(U ≥ 1/2) = 1/4 + 1/2 = 3/4, in (0.65, 1]. Draw the interval and mark where the piece is long; the answer is the total length of those regions.

Bus breakdown (slides, "slightly complicated"). A bus breaks down at a point X uniform on (0, 100) miles from city A. Service stations at 0, 50 and 100, or at 25, 50 and 75: which plan gives the smaller expected distance to the nearest station?

• Stations at 0, 50, 100: the distance D₁(x) is x on (0, 25], 50 − x on (25, 50], x − 50 on (50, 75], 100 − x on (75, 100]. By symmetry the four pieces contribute equally: E(D₁) = 4 ∫₀²⁵ x · 1100 dx = 4 · 312.5100 = 12.5 miles.
• Stations at 25, 50, 75: D₂(x) is 25 − x on (0, 25], then triangles of half-width 12.5 around 25, 50, 75, then x − 75 on (75, 100]. E(D₂) = 1100[312.5 + 4 · 78.125 + 312.5] = 9.375 miles.

The suggested plan is better. The method is the plug-in rule with a piecewise function: split the integral wherever the formula for the distance changes.

10.2 Why the Uniform matters: it generates every other distribution

Proof of (1): P(X ≤ x) = P(F⁻¹(U) ≤ x) = P(U ≤ F(x)) = F(x), because for a uniform P(U ≤ u) = u. Proof of (2): P(F(X) ≤ z) = P(X ≤ F⁻¹(z)) = F(F⁻¹(z)) = z. Punchline: a computer that can produce uniform random numbers can simulate any distribution: compute the cdf, draw U, output F⁻¹(U). For an Exponential, F⁻¹(u) = −ln(1 − u)/λ. The slides' "even cooler fact": the binary digits of one uniform are an infinite sequence of independent fair coin flips, so a single uniform contains infinitely many independent uniforms.

10.3 The Exponential distribution

Derivations (slides). F(a) = ∫₀^a λe^−λx dx = −e^−λx from 0 to a = 1 − e^−λa. E(X) = ∫₀^∞ x λe^−λx dx = 1/λ by parts (section 1.3). E(X²) = ∫₀^∞ x² λe^−λx dx = 2/λ², so Var(X) = 2/λ² − 1/λ² = 1/λ². MGF: ∫₀^∞ e^tx λe^−λx dx = λ ∫₀^∞ e^−(λ−t)x dx = λ/(λ − t), which needs λ − t > 0.

HW10 problem 9. Time to the next earthquake is Exponential with mean 2 years, so λ = 1/2. P(within one year) = 1 − e^−1/2 ≈ 0.3935. P(within ten years) = 1 − e⁻⁵ ≈ 0.9933.

Practice set, problem 25 (the median). Laptop lifetime Exponential with mean 36 months, λ = 1/36. The median a solves P(X ≤ a) = 1 − e^−a/36 = 1/2, so e^−a/36 = 1/2 and a = 36 ln 2 ≈ 24.95 months, in [20, 30). The median of an Exponential is always (ln 2)/λ ≈ 0.69 × the mean: the distribution is skewed right, so half the laptops die before 25 months even though the mean is 36.

Why it matters (slides). Waiting times (earthquakes, phone calls, hospital admissions); queueing theory (how businesses and internet routers behave); Markov processes (built on the memoryless property); and the Gamma distribution (sums of Exponentials) used in text mining and clustering.

10.4 The memoryless property

Proof (slides): P(X > s + t | X > s) = P(X > s + t and X > s)P(X > s) = P(X > s + t)P(X > s) = e^−λ(s+t)e^−λs = e^−λt = P(X > t). Conversely, if G(t) = P(X > t) satisfies G(s + t) = G(s)G(t), differentiating in s at s = 0 gives G′(t) = G′(0)G(t), whose only solutions with G(0) = 1 are G(t) = e^−λt with λ = −G′(0) > 0. That is the characterization.

Eva's hard drive (slides). Lifetime Exponential with mean 43,800 hours. She has used it for 3 years and is leaving on a 4380-hour trip. P(it survives the trip | it has survived 3 years) = P(X > 4380) = e^{−4380/43,800} = e^−0.1 ≈ 0.9048, by memorylessness; the three years of use are irrelevant. If the lifetime were not Exponential (a drive that wears out), the age would matter and you would have to compute P(X > s + t)/P(X > s) from the actual cdf.

HW10 problem 10. Repair time Exponential with rate 1 per hour. P(takes at least 10 hours | has exceeded 9 hours) = P(X ≥ 1) = e⁻¹ ≈ 0.3679. (Computing (e⁻¹⁰/e⁻⁹) directly gives the same thing.)

10.5 Connection to the Poisson process

So HW8 problem 8 (P(no calls in t hours) = e^−2t) and "the wait for the next call is Exponential(2)" are the same statement. With calls at 2 per hour, the expected wait is 1/2 hour and P(wait > 30 min) = e⁻¹ ≈ 0.3679.

10.6 Check yourself

Deck 711. Geometric distribution, joint distributions and linearity of expectation

The idea. Deck 7 does three things. It adds one more "famous" discrete distribution, the Geometric (trials until the first success). It starts what the slides call the most important aspect of the course: relationships between random variables, described by a joint distribution. And it proves the "super duper useful fact" that E(X + Y) = E(X) + E(Y) with no conditions at all, which, combined with indicator variables, computes expectations of very complicated counts in two lines (coat matching, coupon collecting, Homework 9).

11.1 The Geometric distribution

Where the formulas come from (slides). "First success on trial n" is n − 1 failures then a success, and the trials are independent. The probabilities add to 1 by the geometric series: p Σ_n≥1(1 − p)ⁿ⁻¹ = p/(1 − (1 − p)) = 1. "X > k" means the first k trials all failed. For the moments the slides use the MGF:

M_X(t) = Σ_k≥1 e^tk(1 − p)^k−1p = pe^t Σ_k≥1 [e^t(1 − p)]^k−1 = pe^t1 − (1 − p)e^t,

which is finite when (1 − p)e^t < 1, i.e. t < −ln(1 − p). Differentiating, M′(t) = pe^t(1 − (1 − p)e^t)², so E(X) = M′(0) = p/p² = 1/p. One more derivative gives E(X²) = M″(0) = (2 − p)/p², hence Var(X) = (2 − p)/p² − 1/p² = (1 − p)/p². (As for the Binomial and Poisson, direct summation gives the same results.) Intuition for the mean: a success happens one time in 1/p trials, so you wait 1/p trials on average.

Memoryless. P(X > m + k | X > m) = (1 − p)^m+k/(1 − p)^m = (1 − p)^k = P(X > k): having failed m times tells you nothing about the future, just like the Exponential.

Practice set, problem 6. Each kidney matches with probability 0.1 independently; X = number tested until a match. E(X) = 1/0.1 = 10, in [6, 12).

HW6 problem 8(1), revisited. P(Eva's first mistake is on order 11) = (1/2)¹⁰(1/2): the Geometric(1/2) pmf at n = 11.

HW9 problem 7: who gets heads first. Your coin has P(H) = 3/4, your friend's has 1/4; X and Y are the numbers of flips each needs, independent. P(X < Y)? Condition on the value of X:

P(X < Y) = Σ_k≥1 P(X = k) P(Y > k) = Σ_k≥1 (1/4)^k−1(3/4) · (3/4)^k = 916 Σ_k≥1 (3/16)^k−1 = 9/161 − 3/16 = 913 ≈ 0.6923.

First-step check: in a round where both flip, you win outright if you get H and your friend gets T (probability (3/4)(3/4) = 9/16); the round is a do-over if both get T ((1/4)(3/4) = 3/16); otherwise your friend has H and you have not beaten them. So p = 9/16 + (3/16)p, p = 9/13. Both methods are always available; use whichever you set up faster.

11.2 Joint distributions: cdf, pmf, marginals

The slides motivate joint distributions with pairs like (Netflix price, Amazon price), (cholesterol, blood pressure), (rainfall in NC, rainfall in VA): most of statistics (regression, time series, factor models) and most of probability (Markov chains, stochastic processes) is about collections of random variables and how they relate.

Extra context beyond the slides: the conditional pmf p_X|Y(x | y) = p_X,Y(x, y)/p_Y(y) is the multiplication rule in pmf form, so p_X,Y(x, y) = p_Y(y) p_X|Y(x | y). Homework 9 problem 2 and practice problem 22 describe one variable "given" the other and need exactly this.

Problem 1 from the slides: two dice, X = the larger value, Y = the sum. All 36 ordered pairs (d₁, d₂) are equally likely. For the maximum to be x and the sum to be y, one die must show x and the other y − x ≤ x. If y − x < x there are two ordered pairs, (x, y − x) and (y − x, x); if y = 2x there is one, (x, x). So

p(x, y) = 236 for x + 1 ≤ y ≤ 2x − 1, p(x, 2x) = 136, and 0 otherwise.

(Entries are in 36ths; blank = 0.) The row totals are the pmf of the maximum, (2x − 1)/36, and the column totals are the familiar pmf of the sum. Reading probabilities off the table: P(X = 4, Y = 7) = 2/36, P(Y = 7 | X = 4) = (2/36)/(7/36) = 2/7, P(X ≤ 2, Y ≤ 3) = F(2, 3) = 3/36.

A smaller table. Toss a fair coin three times; X = heads in the first two tosses, Y = total heads:

P(X = Y) = 1/8 + 2/8 + 1/8 = 1/2 (the third toss is tails); P(X = 1 | Y = 2) = (2/8)/(3/8) = 2/3. This table returns in section 12 as an example of dependent variables.

Practice set, problem 14 (sampling without replacement). 3 sophomores, 4 juniors, 5 seniors; 3 are chosen at random; X = sophomores chosen, Y = juniors chosen. Every triple is equally likely, so

P(X = x, Y = y) = (3x)(4y)(53−x−y)(123), and P(X = 1, Y = 1) = 3 · 4 · 5(123) = 60220 ≈ 0.2727: option (B).

Option (D), (3/12)(4/12), treats "one sophomore" and "one junior" as two independent single draws, which is wrong twice over: the draws are without replacement, and even with replacement the probability of one of each class in three draws would be the multinomial 3! · (3/12)(4/12)(5/12) ≈ 0.2083, not a bare product of two fractions.

HW9 problem 2: two doctors. H = 1 if Dr. Hibbert performs the n surgeries (probability p), else H = 0. Given H = 1, each surgery succeeds with probability a independently, so X | H = 1 ~ Binomial(n, a); given H = 0, X | H = 0 ~ Binomial(n, b). By the multiplication rule the joint pmf is

P(H = 1, X = k) = p (nk) a^k(1 − a)^n−k, P(H = 0, X = k) = (1 − p) (nk) b^k(1 − b)^n−k, k = 0, …, n.

The marginal of X is the sum of the two rows: a mixture of two Binomials. For E(X), condition on the doctor: E(X) = p · na + (1 − p) · nb = n[pa + (1 − p)b]. For E(X²), use E(Y²) = Var(Y) + (E Y)² for each Binomial: E(X²) = p[na(1 − a) + n²a²] + (1 − p)[nb(1 − b) + n²b²]. Then Var(X) = E(X²) − (E X)², which simplifies to n[pa(1 − a) + (1 − p)b(1 − b)] + n²p(1 − p)(a − b)²: the average within-doctor variance plus the extra variance from not knowing which doctor.

Practice set, problem 22 (a product of dependent variables). Shankar gets X questions with P(X = 0, 1, 2) = 0.2, 0.4, 0.4; each question is answered inadequately with probability 1/5 independently, so Y | X = x ~ Binomial(x, 1/5). P(XY ≤ 1): go case by case on X.

• X = 0: XY = 0 ≤ 1 always. Contributes 0.2.
• X = 1: Y ∈ {0, 1} so XY = Y ≤ 1 always. Contributes 0.4.
• X = 2: XY = 2Y ≤ 1 only if Y = 0, probability (4/5)² = 0.64. Contributes 0.4 × 0.64 = 0.256.

Total 0.856, in (0.6, 1.0]. When one variable's distribution is described "given" the other, the joint pmf is p_X(x) p_Y|X(y | x) and you sum over the cases.

11.3 Expectations from a joint pmf, and linearity

Proof (slides, equation (1)): Σ_x,y(x + y) p(x, y) = Σ_x x Σ_y p(x, y) + Σ_y y Σ_x p(x, y) = Σ_x x p_X(x) + Σ_y y p_Y(y) = E(X) + E(Y). Only the marginals enter, so dependence between X and Y is irrelevant; the n-variable version follows by induction from the two-variable case (in class: Y = X₂ + ··· + X_n). Variances do not add this freely (section 12).

Immediate payoff: a Binomial(n, p) is a sum of n Bernoulli(p) variables, so its mean is np with no computation. In the coin table above, E(Y) = E(X) + E(third toss) = 1 + 1/2 = 3/2.

11.4 The power of indicator random variables

Ten students (slides). Student i does well with probability p_i. With X_i = 1 if student i does well, the number doing well is X = Σ X_i and E(X) = Σ_i=1¹⁰ p_i. No independence was assumed.

Coat matching (slides). Thirty guests' coats are handed back at random, so each guest gets their own coat with probability 1/30. M = number of guests who get their own coat = Σ_i=1³⁰ I_i, and E(M) = 30 · (1/30) = 1. The indicators are dependent (if 29 guests have the right coat, so does the 30th), and the answer is exactly 1 for any number of guests.

The coupon collector problem (slides). Each cereal box contains one of 100 X-men cards, chosen uniformly and independently. X = number of boxes needed to collect all 100. Break the wait into pieces: T_i = the number of boxes bought while you own exactly i distinct cards, until a new one appears, i = 0, 1, …, 99. Each box is "new" with probability (100 − i)/100, so T_i is Geometric with p = (100 − i)/100 and E(T_i) = 100/(100 − i). Then X = T₀ + T₁ + ··· + T₉₉ and

E(X) = Σ_i=0⁹⁹ 100100 − i = 100 (1 + 12 + 13 + ··· + 1100) = 100 H₁₀₀ ≈ 518.74 boxes.

At $2.50 a box, Z = 2.5X and E(Z) = 2.5 · 518.74 ≈ $1,296.85. The slides' punchline: "E(sum of a bunch of random variables) = sum of the individual expectations" was the whole tool; the problem looks made up but shows up in computer science and genetics (see the course "Extras").

HW9 problem 1: five students, four colleges. Each of 5 students independently picks UNC, Duke, NC State or Hogwarts-NC with probabilities 0.6, 0.1, 0.1, 0.2. I_UNC is the indicator that exactly 2 of the 5 choose UNC, and so on; X = number of schools that get exactly 2 students.

1. The number choosing a given school is Binomial(5, p_school), so E(I_UNC) = P(exactly 2) = (52)(0.6)²(0.4)³ = 0.2304; E(I_Duke) = (52)(0.1)²(0.9)³ = 0.0729; E(I_NCS) = 0.0729; E(I_HNC) = (52)(0.2)²(0.8)³ = 0.2048.
2. X = I_UNC + I_Duke + I_NCS + I_HNC.
3. E(X) = 0.2304 + 0.0729 + 0.0729 + 0.2048 = 0.581.

HW9 problem 3: Halloween houses. 12 houses in a row, 7 give candy, every arrangement equally likely; X = number of adjacent pairs with different behavior. Let I_i = 1 if houses i and i + 1 differ, i = 1, …, 11. For any pair, P(different) = P(C then N) + P(N then C) = 712 · 511 + 512 · 711 = 70132 = 3566. So E(X) = 11 · 3566 = 356 ≈ 5.83. The pmf of X would be a nightmare; the expectation takes two lines.

Practice set, problem 20: sad schools. Four athletes choose independently among UNC, Duke, NC State with the given probabilities. Sad = number of schools that get nobody. E(Sad) = Σ_schools P(nobody picks it): UNC (0.5)(0.2)(0.7)(0.1) = 0.007; Duke (0.7)(0.9)(0.6)(0.95) = 0.3591; NC State (0.8)(0.9)(0.7)(0.95) = 0.4788. E(Sad) = 0.8449, in (0.6, 1.2]. Each probability is a product because the athletes decide independently; the three indicators are not independent, and again that does not matter.

11.5 The multinomial distribution

Why (as written in class): a specific sequence of outcomes with n_i of type i has probability Π p_i^n_i, and the multinomial coefficient counts such sequences (the "labeled groups" count from Midterm 1). With r = 2 it is the Binomial. Each X_i on its own is Binomial(n, p_i) (a trial is "type i" or not), and the count of a union of categories, X_i + X_j, is Binomial(n, p_i + p_j).

HW9 problem 4: three candidates. 35% Sauron, 45% Voldemort, 20% Saruman; sample 4 voters.

1. P(X_A = 0, X_V = 1, X_S = 3) = 4!0! 1! 3! (0.35)⁰(0.45)¹(0.2)³ = 4(0.45)(0.008) = 0.0144.
2. Y = X_A + X_S = number of non-Voldemort voters ~ Binomial(4, 0.55): E(Y) = 2.20, Var(Y) = 4(0.55)(0.45) = 0.99.

Practice set, problem 23: ramen. Five customers; P(2 miso, 2 tonkotsu, 1 shoyu) = 5!2! 2! 1! (0.5)²(0.3)²(0.1) = 30(0.25)(0.09)(0.1) = 0.0675, in (0.0025, 0.1]. (The unused category, shio, simply has n_i = 0.)

11.6 Check yourself

Deck 812. Independent random variables, covariance and correlation

The idea. Deck 8 first isolates the special case where one random variable carries no information about another: independence. It makes products behave (E[g(X)h(Y)] = E[g(X)]E[h(Y)]), makes sums behave (MGFs multiply, so Binomials and Poissons add), and makes variances add. The deck then returns to general, possibly dependent, variables and measures their linear relationship with covariance and correlation, which gives "super powerful formulas and rules for computing variances of sums".

12.1 Independence of random variables

Why condition 2 is enough (in class): P(X ∈ A, Y ∈ B) = Σ_x∈A Σ_y∈B p(x, y) = Σ_x∈A Σ_y∈B p_X(x)p_Y(y) = [Σ_x∈A p_X(x)][Σ_y∈B p_Y(y)] = P(X ∈ A)P(Y ∈ B). And X, Y are not independent as soon as there are sets A, B with P(X ∈ B | Y ∈ A) ≠ P(X ∈ B).

Final Exam 2011 problem (slides). Two balls are drawn without replacement from an urn with 5 white and 8 red balls. X₁ = 1 if the first ball is red (else 0), X₂ = 1 if the second is red. Joint pmf by the multiplication rule:

Both marginals are the same (the second ball is red with probability 8/13 too), but P(X₁ = 1)P(X₂ = 1) = (8/13)² = 64/169 ≈ 0.379 while P(X₁ = 1, X₂ = 1) = 14/39 ≈ 0.359. One cell fails, so X₁ and X₂ are dependent, as sampling without replacement should be.

Pet boarding: thinning a Poisson (slides). N ~ Poisson(λ) dogs arrive in a day; each independently is "crazy" with probability p. Cr = number of crazy puppies, No = number of normal dogs, Cr + No = N. For c, d ≥ 0, condition on N = c + d; given N, Cr is Binomial(N, p):

P(Cr = c, No = d) = P(N = c + d) P(Cr = c | N = c + d) = e^−λ λ^c+d(c + d)! · (c + d)!c! d! p^c(1 − p)^d = [e^−λp (λp)^cc!] · [e^−λ(1−p) (λ(1 − p))^dd!],

using e^−λ = e^−λpe^−λ(1−p). The joint pmf is a product of a Poisson(λp) pmf in c and a Poisson(λ(1 − p)) pmf in d, so Cr and No are independent Poissons with means λp and λ(1 − p). This is the "factorization test" in action, and it is surprising: the two counts share the same random N and are still independent.

Practice set, problem 13. p(x, y) = (x² + y²)/140 for x = 1, 2, 3 and y = −x, …, x. Dependent, for two separate reasons: the range of y depends on x, and x² + y² is a sum, not a product h(x)g(y). Option (B).

The coin table of section 11.2. X = heads in tosses 1–2, Y = total heads: p(0, 3) = 0 but p_X(0)p_Y(3) = (2/8)(1/8) ≠ 0. Dependent (Y ≥ X always).

HW9 problem 8: Bert and Ernie's sprints. X ~ Binomial(5, 2/3) (from its MGF), Y ~ Poisson(2), independent; Z = XY. E(Z) = E(X)E(Y) = (10/3)(2) = 20/3 ≈ 6.67. For the variance, Var(Z) = E(Z²) − (E Z)² with E(Z²) = E(X²)E(Y²) (independence again, applied to X² and Y²). E(X²) = Var(X) + (E X)² = 10/9 + 100/9 = 110/9; E(Y²) = 2 + 4 = 6. So E(Z²) = (110/9)(6) = 220/3 and Var(Z) = 220/3 − 400/9 = 260/9 ≈ 28.89.

HW10 problem 11. X ~ N(0, 1) and Y ~ Exponential(1) independent: E[|X|(Y + 1)²] = E|X| · E[(Y + 1)²]. E|X| = √(2/π) ≈ 0.7979 (section 13.3). E[(Y + 1)²] = E(Y²) + 2E(Y) + 1 = (1 + 1) + 2 + 1 = 5, using E(Y²) = Var(Y) + (E Y)² = 1 + 1. Answer: 5√(2/π) ≈ 3.989.

12.2 Sums of independent random variables

Why sums (slides): a sample of 100 Chapel Hill voters with X_i = 1 if voter i supports the candidate gives the total Σ X_i ~ Binomial(100, p); a study comparing two universities bases its conclusion on the sample averages (1/40) Σ X_i and (1/30) Σ Y_j. Two methods find the distribution of a sum.

• Binomials with the same p (warning-sign slide). X_i ~ Bin(n_i, p) independent: M_S(t) = Π (1 − p + pe^t)^n_i = (1 − p + pe^t)^{n₁+···+n_m}, so S ~ Bin(n₁ + ··· + n_m, p). (Different p's do not combine into a Binomial.)
• Poissons (warning-sign slide). X_i ~ Pois(λ_i) independent: M_S(t) = e^{(λ₁+···+λ_m)(et−1)}, so S ~ Pois(λ₁ + ··· + λ_m), agreeing with Method 1.
• Normals (section 13.2): N(μ₁, σ₁²) + N(μ₂, σ₂²) = N(μ₁ + μ₂, σ₁² + σ₂²).
• Sums of Geometrics are not Geometric and sums of Exponentials are not Exponential (their MGF products are not of the same form).

HW8 problem 5 was this: Poisson(3) + Poisson(1.2) = Poisson(4.2), so P(sum = 2) = e^−4.2(4.2)²/2 ≈ 0.1323. By Method 1 instead: P(0)P(2) + P(1)P(1) + P(2)P(0) with the two pmfs gives the same number, more slowly.

12.3 Maximum and minimum of independent variables

Extra context beyond the slides, needed for HW9 problem 6 and practice problem 17.

HW9 problem 6. Twenty rolls of a fair four-sided die; Y = the maximum. Values 1, 2, 3, 4. P(Y ≤ i) = (i/4)²⁰, so P(Y = i) = (i/4)²⁰ − ((i − 1)/4)²⁰: about 9 × 10⁻¹³, 9.5 × 10⁻⁷, 0.00317, 0.99683. With 20 rolls the maximum is almost surely 4.

Practice set, problem 17. P(max(X, Y) ≤ 1) = P(X ≤ 1)P(Y ≤ 1) = 2e⁻¹ · 3e⁻² = 6e⁻³ ≈ 0.2987 for independent Poisson(1) and Poisson(2).

Minimum of Exponentials. X ~ Exp(λ₁), Y ~ Exp(λ₂) independent: P(min > t) = e^−λ₁te^−λ₂t = e^{−(λ₁+λ₂)t}, so the minimum is Exponential(λ₁ + λ₂).

12.4 Covariance

Meaning (slides). Covariance signals the tendency of two variables to go up or down together relative to their means: positive (height and weight of a random person), negative (X up, Y tends down). Everything is random, so one can only talk about tendency. (For spurious correlations, the slides point to tylervigen.com.)

Uncorrelated but dependent (slides). X = −1, 0, 1 each with probability 1/3 and Y = X². E(X) = 0, E(Y) = 2/3, E(XY) = E(X³) = 0, so Cov(X, Y) = 0 − 0 · (2/3) = 0. Yet Y is a function of X. Covariance only detects linear association.

Three coins (slides). X = number of heads, Y = number of tails in 3 fair tosses. Since Y = 3 − X, Cov(X, Y) = Cov(X, 3 − X) = Cov(X, 3) − Cov(X, X) = −Var(X) = −3(1/2)(1/2) = −3/4.

Practice set, problem 16. Cov(4X − Y, X + Y) = 4 Cov(X, X) + 4 Cov(X, Y) − Cov(Y, X) − Cov(Y, Y) = 4 Var(X) + 3 Cov(X, Y) − Var(Y): option (C).

Practice set, problem 24: socks. 8 left socks are matched at random with the 8 right socks (a random permutation). I_red = 1 if the red pair is matched correctly, I_blue likewise. P(red correct) = 1/8; P(red and blue both correct) = (1/8)(1/7) = 1/56. Cov(I_red, I_blue) = E(I_redI_blue) − E(I_red)E(I_blue) = 1/56 − 1/64 = 1/448 ≈ 0.00223, in (0.001, 0.0025]. Slightly positive: a correct red match removes one wrong option for blue. (For indicators, E(I_AI_B) = P(A ∩ B), so Cov = P(A ∩ B) − P(A)P(B).)

HW9 problem 10 (stock prices), part 1. E(X) = 1, E(Y) = 3, E(Z) = 3; E(X²) = 2, E(Y²) = 12, E(Z²) = 12; E(XY) = 2, E(XZ) = 4, E(YZ) = 9. Then Var(X) = 2 − 1 = 1, Var(Y) = 12 − 9 = 3, Var(Z) = 3; Cov(X, Y) = 2 − 3 = −1, Cov(X, Z) = 4 − 3 = 1, Cov(Y, Z) = 9 − 9 = 0.

Extra context beyond the slides: multinomial counts are negatively correlated, Cov(X_i, X_j) = −n p_ip_j. In HW9 problem 4, Cov(X_A, X_V) = −4(0.35)(0.45) = −0.63.

12.5 Variance of a sum

The Binomial variance np(1 − p) is n copies of the Bernoulli variance p(1 − p), and the average of n independent copies has Var(X̄) = σ²/n (used in section 14).

HW10 problem 7. X₀, …, X₃ independent with variance σ²; Y₁ = X₁ + 3X₀, Y₂ = X₂ + 2X₁, Y₃ = X₃ − X₂. First simplify the sum: Y₁ + Y₂ + Y₃ = 3X₀ + 3X₁ + X₃ (the X₂ terms cancel). Then Var = 9σ² + 9σ² + σ² = 19σ²: C = 19, in [15, 20), option (D). Adding Var(Y₁) + Var(Y₂) + Var(Y₃) = 10 + 5 + 2 = 17 would be wrong, because the Y's share X's and are correlated.

HW9 problem 9: symmetrization. X′ and X″ are independent copies of X and E[(X′ − X″)²] = 20. Since E(X′ − X″) = 0, E[(X′ − X″)²] = Var(X′ − X″) = Var(X) + Var(X) = 2Var(X). So Var(X) = 10.

HW9 problem 10, part 3: the portfolio. R = X + 2Y + Z. Var(R) = Var(X) + 4Var(Y) + Var(Z) + 2[2Cov(X, Y) + Cov(X, Z) + 2Cov(Y, Z)] = 1 + 12 + 3 + 2[−2 + 1 + 0] = 14, so σ_R = √14 ≈ 3.7417. The negative covariance between X and Y reduces the risk; that is diversification.

12.6 Correlation

Why |ρ| ≤ 1 (in class). Standardize: U = (X − EX)/√Var(X) and V = (Y − EY)/√Var(Y), so E(U) = E(V) = 0, E(U²) = E(V²) = 1 and ρ = E(UV). For any number t, 0 ≤ E[(U + tV)²] = 1 + 2tρ + t². Taking t = −ρ gives 0 ≤ 1 − ρ², so −1 ≤ ρ ≤ 1. If ρ = −1 the choice t = 1 gives E[(U + V)²] = 0, so U = −V always, which unwinds to Y = −aX + b with a = √(Var Y/Var X) > 0: Fact 2. Fact 3 follows from Cov(aX + b, Y) = a Cov(X, Y) and Var(aX + b) = a²Var(X).

HW9 problem 10, parts 1–2. ρ(X, Y) = −1/(√1 √3) = −1/√3 ≈ −0.5774. ρ(Y, Z) = 0/(√3 √3) = 0: Y and Z are uncorrelated (from the given moments you cannot tell whether they are independent).

12.7 Check yourself

Deck 813. The Normal distribution

The idea. The Normal (Gaussian, "bell curve") is the distribution of standardized test scores, blood pressure and beta-carotene levels, the driving component of the Black–Scholes model in finance, and, by the central limit theorem (section 14), the approximate distribution of any sum or average of many independent terms. Every computation reduces to the standard Normal Z through standardization, and probabilities for Z come from its cdf Φ, which has no formula but is tabulated.

13.1 Definition, standardization and the table

Note 2: why the constant is 1/√(2πσ²) (in class). Let I = ∫_−∞^∞ e^−x2/2 dx. Then I² = ∫∫ e^−(x2+y2)/2 dx dy = ∫₀^2π∫₀^∞ e^−r2/2 r dr dθ = ∫₀^2π 1 dθ = 2π in polar coordinates, so I = √(2π), and the substitution x = μ + σz brings in the σ. Why Z has mean 0: zφ(z) is odd. Why variance 1: integrate by parts, or use the MGF below.

Example. Heights are N(70, 9) inches (σ = 3). P(X > 76) = 1 − Φ(2) = 0.0228. P(67 < X < 73) = Φ(1) − Φ(−1) = 0.8413 − 0.1587 = 0.6826. The 90th percentile is μ + 1.2816σ ≈ 73.8 inches, since Φ(1.2816) = 0.90.

13.2 The MGF, the moments, and sums of Normals

Note 4 (in class). M_X(t) = 1σ√(2π) ∫ e^tx e^{−(x−μ)2/(2σ2)} dx. Substitute x = μ + σz, dx = σ dz: M_X(t) = 1√(2π) ∫ exp(μt + σtz − z²/2) dz. Complete the square, σtz − z²/2 = −(z − σt)²/2 + σ²t²/2:

M_X(t) = e^{μt + σ2t2/2} · 1√(2π) ∫ e^{−(z−σt)2/2} dz = e^{μt + σ2t2/2},

because the remaining integrand is the density of a Normal shifted by σt and integrates to 1. For Z itself, M_Z(t) = e^t2/2.

Note 5 (in class). M′_X(t) = (μ + σ²t) M_X(t), so E(X) = M′(0) = μ. M″_X(t) = σ²M_X(t) + (μ + σ²t)²M_X(t), so E(X²) = σ² + μ² and Var(X) = σ². (The slides suggest also computing ∫ x f(x) dx and ∫ x² f(x) dx directly at home.) From M_Z, E(Z⁴) = 3, so Var(Z²) = 2.

Note 3 via the MGF. M_aX+b(t) = e^btM_X(at) = e^{bt + aμt + a2σ2t2/2} = e^{(aμ+b)t + (a2σ2)t2/2}, exactly the MGF of N(aμ + b, a²σ²). Standardizing is the case a = 1/σ, b = −μ/σ.

Example. X ~ N(2, 4) and Y ~ N(1, 5) independent. X − Y ~ N(1, 9), so P(X > Y) = P(X − Y > 0) = 1 − Φ(−1/3) = Φ(1/3) ≈ 0.6306.

13.3 E|Z| (HW10 problem 11)

E|Z| = ∫_−∞^∞ |z| φ(z) dz = 2 ∫₀^∞ z e^−z2/2√(2π) dz = 2√(2π) [−e^−z2/2]₀^∞ = 2√(2π) = √(2/π) ≈ 0.7979.

(The substitution u = z²/2 does the integral.) So the average distance of a standard Normal from 0 is about 0.8, less than its standard deviation 1, as it should be: E|Z| ≤ √E(Z²) = 1. HW10 problem 11 then gives E[|X|(Y + 1)²] = √(2/π) · 5 ≈ 3.989 (section 12.1).

13.4 Check yourself

Deck 814. Limit theorems: the CLT, the law of large numbers, and the inequalities

Where this sits. These are the last slides of deck 8, and sections 6–7 of the instructor's Lectures 5–8 study guide mark them as warning-sign material. (The syllabus had listed them under deck 9, but deck 9 itself is jointly continuous random variables, which is not on this midterm.)

The idea. Why does a sample of about 1100 people predict an election? How long must a simulation run? The answers come from limit theory. The law of large numbers says the average of many independent observations settles at the mean. The central limit theorem says the fluctuations around the mean are Normal, with size σ/√n, whatever the original distribution; it is why the Normal is everywhere and how we approximate Binomial probabilities. Markov's and Chebyshev's inequalities give guaranteed bounds from a mean alone or a mean and a variance, and Chebyshev is what proves the law of large numbers.

14.1 The central limit theorem

Idea of the proof (in class). Assume the MGF of X_i exists. Let U_i = (X_i − μ)/σ, so E(U_i) = 0, E(U_i²) = 1 and Z_n = (U₁ + ··· + U_n)/√n. A second-order Taylor expansion at s = 0 gives M_U(s) = 1 + E(U)s + E(U²)s²/2 + o(s²) = 1 + s²/2 + o(s²). By independence, M_Zn(t) = [M_U(t/√n)]ⁿ = [1 + t²/(2n) + o(1/n)]ⁿ → e^t2/2, the MGF of N(0, 1), using (1 + x/n)ⁿ → e^x. Fact: if the MGFs converge, the cdfs converge.

Example 1 (slides): a random walk. X_i = ±1 with probability 1/2 each (a stock price moving up or down by 1 each day). P(S₅₀ > 10)?

1. One step: E(X_i) = 0, E(X_i²) = 1, so Var(X_i) = 1.
2. By independence, E(S₅₀) = 0 and Var(S₅₀) = 50; by the CLT, S₅₀ ≈ N(0, 50).
3. P(S₅₀ > 10) ≈ P(Z > 10/√50) = P(Z > 1.41) = 1 − Φ(1.41) ≈ 1 − 0.9207 = 0.079.

Extra context beyond the slides: S₅₀ only takes even values, so "S₅₀ > 10" means S₅₀ ≥ 12 and a continuity correction uses P(W > 11) = 1 − Φ(11/√50) = 1 − Φ(1.56) ≈ 0.060, very close to the exact 0.0595. The uncorrected 0.079 is what the lecture computed; either is acceptable as a CLT approximation unless the problem asks for the correction.

Example 3 (slides): the astronomer. Measurements are iid with mean d (the true distance) and variance 4. How many are needed to be "reasonably sure" (say 95%) that the average is within ±0.5 light-year of d? The average X̄_n ≈ N(d, 4/n), so P(|X̄_n − d| ≤ 0.5) = P(|Z| ≤ 0.5/(2/√n)) = P(|Z| ≤ 0.25√n). For this to be 0.95 we need 0.25√n ≥ 1.96, i.e. n ≥ (7.84)² ≈ 61.5: about 62 measurements. (Chebyshev, in 14.4, gives the distribution-free but much more conservative n ≥ 320.)

HW10 problem 1. X_i = 1 or 2 with probabilities 1/3, 2/3. μ = 1/3 + 4/3 = 5/3; E(X_i²) = 1/3 + 8/3 = 3, so σ² = 3 − 25/9 = 2/9 and σ = √2/3. The CLT says the distribution of

X₁ + ··· + X_n − 5n/3√(2n/9) = X₁ + ··· + X_n − 5n/3(√2/3)√n

tends to the standard Normal. The blanks are nμ and σ√n.

14.2 Normal approximation to the Binomial and the continuity correction

The beer example (slides). Eva's PR firm tests 100 affirmed Bond-beer drinkers blind; X = number who prefer VP beer. In class the "worst case" p = 1/2 was used (most people cannot tell non-specialty beers apart), so X ~ Bin(100, 1/2): μ = np = 50, σ² = np(1 − p) = 25, σ = 5, and W ~ N(50, 25).

1. P(X > 40) ≈ P(W > 40.5) = P(Z > (40.5 − 50)/5) = P(Z > −1.9) = Φ(1.9) ≈ 0.9713.
2. P(40 ≤ X ≤ 60) ≈ P(39.5 < W < 60.5) = P(−2.1 < Z < 2.1) = Φ(2.1) − Φ(−2.1) = 2Φ(2.1) − 1 ≈ 0.9643.

(Exact Binomial values: 0.9716 and 0.9648. Without the correction the answers would be 0.9772 and 0.9545, noticeably worse.) Rule of thumb: the approximation is good when np(1 − p) is at least about 10.

Another example. 100 fair coin tosses; P(at least 60 heads) ≈ P(W > 59.5) = 1 − Φ(9.5/5) = 1 − Φ(1.9) ≈ 0.0287 (exact 0.0284).

14.3 The weak law of large numbers

Exercise on the slide. With X_i = I_Ai the indicator that event A happens on trial i (so μ = P(A)), the law says the observed proportion of trials on which A occurs converges to P(A). That is the frequency interpretation of probability, now a theorem, and the reason the London-bomb and Gretzky tables match the Poisson so well.

14.4 Markov's and Chebyshev's inequalities

Proof (in class): since X ≥ 0 and a > 0, X ≥ a · 1_{{X ≥ a}} always (if X ≥ a the right side is a ≤ X; otherwise it is 0 ≤ X). Take expectations: E(X) ≥ a P(X ≥ a); divide by a.

Proof (in class): Y = (X − μ)² is a non-negative random variable, so Markov gives P(Y ≥ a²) ≤ E(Y)/a² = Var(X)/a²; and {Y ≥ a²} is the same event as {|X − μ| ≥ a}. With a = kσ: at most 1/k² of any distribution lies k or more standard deviations from its mean (compare the Normal's 0.0456 at k = 2).

Example 2a (slides): the factory. Weekly production X has mean 50.

1. P(X > 75) ≤ P(X ≥ 75) ≤ 50/75 = 2/3 by Markov (production is non-negative).
2. If also Var(X) = 25: P(40 ≤ X ≤ 60) = P(|X − 50| ≤ 10) = 1 − P(|X − 50| > 10) ≥ 1 − P(|X − 50| ≥ 10) ≥ 1 − 25/100 = 3/4 by Chebyshev.

Nothing was assumed about the shape of the distribution; that is the point, and also why the bounds are loose.

HW10 problem 2. X ~ Poisson(100), so E(X) = Var(X) = 100. (a) Markov: P(X > 120) ≤ P(X ≥ 120) ≤ 100/120 ≈ 0.8333 (or, using the integer values, ≤ 100/121 ≈ 0.8264; both are valid bounds). (b) Chebyshev: {X > 120} is contained in {|X − 100| ≥ 20}, so P(X > 120) ≤ 100/400 = 0.25. The true probability is about 0.023.

The astronomer by Chebyshev. P(|X̄_n − d| ≥ 0.5) ≤ Var(X̄_n)/0.25 = (4/n)/0.25 = 16/n. For this to be at most 0.05 we need n ≥ 320, versus about 62 from the CLT: Chebyshev holds for every distribution but pays for it in sharpness.

14.5 Proof of the weak law (how Chebyshev does it)

Practice set, problem 18. The bound used in the proof is σ²/(ε²n): option (C). Options with n in the numerator get worse with more data, which cannot be right.

14.6 Check yourself

15. Exam strategy

15.1 Recognize the distribution from the story

Most questions are "identify the model, write its parameters, use one formula". Decide the model before touching the calculator.

15.2 Four questions before you compute

1. What is the random variable, and what values can it take? Write "X = number of …" or "X = time until …" and its range. Half of the practice set is answered correctly by this alone.
2. Discrete or continuous? Discrete: sums, endpoints matter, P(X = a) can be positive. Continuous: integrals, endpoints never matter, P(X = a) = 0.
3. Independent or not? Independence lets you multiply probabilities and MGFs and add variances. Means add regardless. If the problem says "given", "conditional on", or describes one variable in terms of another, there is dependence and you need total probability or a joint pmf.
4. Is the complement easier? "At least one", "more than two", "at most n − 1": compute the small side and subtract from 1.

15.3 Multiple-choice tactics

• Interval answers: compute to four decimals, then check whether the endpoints are included: (0.25, 0.40] includes 0.40, [0.5, 0.65) excludes 0.65. When your number lands within 0.005 of an endpoint, recompute with more precision before choosing.
• Formula answers: test the options at an easy value (p = 0, n = 1, t = 0). An MGF option must equal 1 at t = 0. A pmf option must give probabilities between 0 and 1 at the smallest and largest values. Options that add MGFs or multiply variances by a instead of a² are wrong on sight.
• Sanity limits: variances are never negative; a probability from a Poisson or Binomial with mean 2 should not be 0.9; E(number of successes) ≤ n; E(X) for an Exponential is 1/λ, not λ; the median of an Exponential is below its mean.
• Units: rate per hour times minutes is the most common silent error. Convert first.
• Time: about 6 minutes per question for 12 questions in 75 minutes. Do the recognizable ones first; MGF and covariance algebra questions are usually fastest.

15.4 The traps

16. Practice exams

Part A is the practice set posted for Midterm 2, with worked answers. Part B is a new set in the same style. Do each part in 75 minutes with only a calculator, then read every solution, including the ones you got right. The real exam will not just be variants of these, so focus on which concept each question uses.

16.1 Part A: the posted practice problems

Every Part A answer below opens with the one big concept the question is testing, then works the arithmetic in full and flags the traps the distractors are built from. Read the concept line first — if you can name it before you compute, you can handle the exam's version of the same question.

16.2 Part B: a second exam

17. One-page summary

Read this the night before and again right before the exam. The distribution table is the single most useful thing to have memorized.

17.1 The distributions

Sums of independent variables: Bin(n, p) + Bin(m, p) = Bin(n + m, p); Pois(λ) + Pois(μ) = Pois(λ + μ); N(μ₁, σ₁²) + N(μ₂, σ₂²) = N(μ₁ + μ₂, σ₁² + σ₂²). Minimum of Exp(λ₁) and Exp(λ₂) is Exp(λ₁ + λ₂). Poisson process: N(t) ~ Pois(λt); time to the first event ~ Exp(λ).

17.2 The formulas