M1 Statistics: Conditional Probability and Bayes' Theorem
Hello everyone! Welcome to the study notes for one of the most interesting topics in statistics: Conditional Probability and Bayes' Theorem. Don't worry if the names sound a bit scary. The ideas behind them are actually very logical and are used everywhere, from weather forecasting to medical diagnoses and even in the recommendations you get on Netflix!
In this chapter, we'll learn how to answer questions like, "What's the probability it will rain today, given that the sky is cloudy?" We're going to break it all down into simple, manageable steps. Let's get started!
1. What is Conditional Probability?
At its heart, conditional probability is the probability of an event happening, given that another event has already occurred. We are basically updating our probability based on new information.
A Quick Review: Basic Probability
Remember that the basic probability of an event A is:
$$P(A) = \frac{\text{Number of favourable outcomes for A}}{\text{Total number of possible outcomes}}$$Example: The probability of rolling a '4' on a standard six-sided die is $$P(\text{rolling a 4}) = \frac{1}{6}$$, because there is one '4' and six total sides.
The Core Idea: A Smaller Universe
Conditional probability just shrinks our "universe" (the official term is sample space). When we know that a certain event has already happened, we can ignore all the other outcomes that are no longer possible.
Analogy: Finding a specific student in a school.
Imagine your school has 1000 students. The probability of randomly picking a girl who is in Form 6 is $$P(\text{Form 6 Girl})$$.
Now, what if I give you some new information? "I have already picked a Form 6 student."
Suddenly, you can ignore all the students from Form 1 to Form 5! Your sample space has shrunk from 1000 students to just the Form 6 students. The probability of picking a girl given that the student is from Form 6 will be different and likely higher.
Notation and The Formula
We write the conditional probability of "Event A happening given that Event B has happened" as P(A|B).
The little vertical line `|` means "given". So, you read P(A|B) as "the probability of A, given B".
The formula that connects everything is:
$$P(A|B) = \frac{P(A \cap B)}{P(B)}$$Let's break that down:
P(A|B): This is what we want to find – the probability of A, given B.
$$P(A \cap B)$$: This is the probability of both A and B happening together (the intersection of A and B).
P(B): This is the probability of the "given" event B. This is our new, smaller sample space!
Memory Aid:
Think of the formula as: "The probability of what you want (A), given the condition (B), is the probability of BOTH things happening, divided by the probability of the GIVEN condition."
Step-by-Step Example using a Table
A group of 100 students were asked if they play basketball or football. The results are in the table below.
| Plays Football | Does Not Play Football | Total | |
|---|---|---|---|
| Plays Basketball | 20 | 25 | 45 |
| Does Not Play Basketball | 30 | 25 | 55 |
| Total | 50 | 50 | 100 |
Question: What is the probability that a student plays basketball, given that they play football?
Step 1: Define the events.
Let B = The student plays basketball.
Let F = The student plays football.
We want to find P(B|F).
Step 2: Find the probabilities from the table.
First, find the probability of the intersection, $$P(B \cap F)$$. This is the probability that a student plays both basketball AND football.
From the table, 20 students play both. So, $$P(B \cap F) = \frac{20}{100} = 0.2$$
Next, find the probability of the "given" event, P(F). This is the probability that a student plays football.
From the table, 50 students play football. So, $$P(F) = \frac{50}{100} = 0.5$$
Step 3: Apply the formula.
$$P(B|F) = \frac{P(B \cap F)}{P(F)} = \frac{0.2}{0.5} = 0.4$$So, the probability that a student plays basketball, given they play football, is 0.4.
An easier way to think about it (using the reduced sample space):
The question tells us the student "plays football". So we can completely ignore the 50 students who don't play football. Our new "total" is just the 50 football players.
Out of these 50 football players, how many also play basketball? The table says 20.
So, the probability is simply $$\frac{20}{50} = 0.4$$. It's the same answer!
Common Mistakes to Avoid:
• Don't mix up P(A|B) and P(B|A). They are usually different! (e.g., The probability of being a Form 6 student given you are a girl is not the same as the probability of being a girl given you are a Form 6 student).
• The denominator is ALWAYS the probability of the 'given' event. For P(A|B), the denominator is P(B).
Key Takeaway:
Conditional probability is just probability in a smaller world. The formula $$P(A|B) = \frac{P(A \cap B)}{P(B)}$$ is your main tool, but always try to think about the reduced sample space to build your intuition.
2. The Multiplication Rule & Tree Diagrams
Sometimes we need to find the probability of two things happening in a sequence. We can rearrange the conditional probability formula to help us.
The General Multiplication Rule
Starting from $$P(A|B) = \frac{P(A \cap B)}{P(B)}$$, if we multiply both sides by P(B), we get:
$$P(A \cap B) = P(B) \times P(A|B)$$This tells us the probability of A and B both happening is the probability of B happening, multiplied by the probability of A happening after B has already happened. This is super useful for problems involving "without replacement".
Visualizing with Tree Diagrams
Tree diagrams are your best friend for conditional probability problems. They help you organize all the information clearly.
Example: A bag contains 5 red balls and 3 blue balls. You draw two balls from the bag one after the other, without replacement. What is the probability that both balls are red?
Step 1: Draw the first set of branches for the first draw.
Total balls = 8.
Probability of drawing a Red ball first, $$P(R_1) = \frac{5}{8}$$
Probability of drawing a Blue ball first, $$P(B_1) = \frac{3}{8}$$
Step 2: Draw the second set of branches, making them conditional.
If the first ball was Red: Now there are only 7 balls left (4 Red, 3 Blue).
• The probability the second is Red, given the first was Red, is $$P(R_2 | R_1) = \frac{4}{7}$$
• The probability the second is Blue, given the first was Red, is $$P(B_2 | R_1) = \frac{3}{7}$$
If the first ball was Blue: Now there are only 7 balls left (5 Red, 2 Blue).
• The probability the second is Red, given the first was Blue, is $$P(R_2 | B_1) = \frac{5}{7}$$
• The probability the second is Blue, given the first was Blue, is $$P(B_2 | B_1) = \frac{2}{7}$$
Step 3: Multiply along the branches to find the probability of a path.
We want the probability of "Red first AND Red second", which is $$P(R_1 \cap R_2)$$.
Using the multiplication rule:
(A tree diagram would show this path visually!)
Key Takeaway:
Use the multiplication rule $$P(A \cap B) = P(A) \times P(B|A)$$ for sequential events. When in doubt, draw a tree diagram! It makes complex problems much clearer.
3. Bayes' Theorem: Flipping the Condition
This is the big one! It looks complicated, but the idea is simple. Bayes' theorem helps us "flip" conditional probability. Often, we know P(B|A), but what we really want to find is P(A|B).
The "Why": A Medical Test Analogy
Imagine there's a medical test for a rare disease.
- We might know the test's accuracy: the probability of a positive test given you have the disease. Let's call this P(Positive | Disease).
- But what does a patient really want to know after they get a positive result? They want to know the probability they have the disease given they got a positive test. This is P(Disease | Positive).
Notice how the condition is flipped! Bayes' theorem is the tool that lets us calculate this.
The Formula
The simple version of Bayes' Theorem is derived directly from the multiplication rule:
$$P(A|B) = \frac{P(B|A) \times P(A)}{P(B)}$$The tricky part is that we often don't know P(B) directly. We have to calculate it using the Law of Total Probability. If event A can happen or not happen (A'), then the probability of B is:
$$P(B) = P(B|A)P(A) + P(B|A')P(A')$$Putting it all together, we get the full version you'll use in problems:
$$P(A|B) = \frac{P(B|A)P(A)}{P(B|A)P(A) + P(B|A')P(A')}$$Don't panic! This formula is just the "path you want" divided by the "sum of all possible paths to the outcome". A tree diagram makes this super easy to see.
Step-by-Step Problem Solving with Bayes' Theorem
Two factories, A and B, produce light bulbs. Factory A produces 60% of the bulbs and Factory B produces 40%. It is known that 2% of bulbs from Factory A are defective, and 5% from Factory B are defective.
Question: If you pick a bulb at random and it is defective, what is the probability it came from Factory A?
Step 1: Define the events.
A = Bulb is from Factory A.
B = Bulb is from Factory B.
D = Bulb is defective.
We want to find P(A|D) - the probability it's from A, GIVEN it's defective.
Step 2: List the known probabilities (a tree diagram is perfect here!).
• P(A) = 0.60 (Factory A produces 60%)
• P(B) = 0.40 (Factory B produces 40%)
Now for the conditional ones:
• P(D|A) = 0.02 (Probability of defective, GIVEN from A)
• P(D'|A) = 0.98 (Probability of not defective, GIVEN from A)
• P(D|B) = 0.05 (Probability of defective, GIVEN from B)
• P(D'|B) = 0.95 (Probability of not defective, GIVEN from B)
Step 3: Identify the "path you want" for the numerator.
We want P(A|D). The path for "from A AND defective" is $$P(D|A)P(A)$$.
$$P(D|A)P(A) = (0.02) \times (0.60) = 0.012$$
Step 4: Find the total probability of the outcome for the denominator.
The outcome is "defective" (D). A bulb can be defective in two ways:
1. From Factory A AND defective: $$P(D|A)P(A) = 0.012$$
2. From Factory B AND defective: $$P(D|B)P(B) = (0.05) \times (0.40) = 0.020$$
The total probability of being defective, P(D), is the sum of these paths:
$$P(D) = 0.012 + 0.020 = 0.032$$
Step 5: Calculate the final answer.
$$P(A|D) = \frac{\text{Path you want}}{\text{Sum of all paths to outcome}} = \frac{P(D|A)P(A)}{P(D)}$$ $$P(A|D) = \frac{0.012}{0.032} = 0.375$$So, even though Factory A makes more bulbs, if you find a defective one, there's only a 37.5% chance it came from Factory A. This is because Factory B has a higher defect rate.
Key Takeaway:
Bayes' Theorem helps us update our beliefs based on new evidence. It flips the condition. The formula might look scary, but with a tree diagram, it's just:
(The probability of the specific path that got you the result) / (The sum of probabilities of ALL paths that could have gotten you that result).
Chapter Summary: Quick Review
Conditional Probability
- What it is: The probability of A happening, given B already happened.
- Notation: P(A|B)
- Formula: $$P(A|B) = \frac{P(A \cap B)}{P(B)}$$
Multiplication Rule
- What it is: Helps find the probability of a sequence of events.
- Formula: $$P(A \cap B) = P(A) \times P(B|A)$$
- Best Tool: Tree Diagrams!
Bayes' Theorem
- What it does: "Flips" the condition (finds P(A|B) when you know P(B|A)).
- Formula: $$P(A|B) = \frac{P(B|A)P(A)}{P(B)}$$
- Problem-Solving Tip: Always define your events, list your knowns, and use a tree diagram. The answer is always (Your Path) / (All Possible Paths).
You've got this! Practice is key, so work through some problems and try to draw a tree diagram whenever you can. Good luck!