Continuous Random Variables: S2 Statistics Notes
Welcome to the exciting world of Continuous Random Variables! If you’ve successfully tackled Discrete Random Variables, you’re halfway there. The main difference is that instead of counting outcomes (like rolling a die), we are now measuring them (like time or height). Since we can’t count, we switch our statistical tool kit from summation (\(\Sigma\)) to the powerful language of Calculus (integration and differentiation)!
Don't worry if integration feels tricky—this chapter is a fantastic application of your Pure Maths skills, showing you exactly why those integration rules matter.
1. Discrete vs. Continuous: The Key Difference
When studying random variables (RVs), we classify them based on the values they can take:
Discrete Random Variables (Review)
- Take a finite number of distinct, countable values (e.g., 0, 1, 2, 3).
- We use Probability Mass Functions (PMF), \(P(X=x)\), which assign a specific probability to each point.
- We use summation (\(\Sigma\)) to find the total probability or expectation.
Continuous Random Variables (CRVs)
- Can take any value within a specified range or interval (e.g., the time it takes for a machine to fail, the exact height of a student).
- Since there are infinitely many possible values between any two points, the probability of the variable taking one exact single value is zero.
- Crucial Result: \(P(X = a) = 0\). This means that \(P(a < X < b)\) is the same as \(P(a \le X \le b)\) for CRVs!
- We use integration (\(\int\)) to find the probability over an interval (the area under the curve).
Analogy: Imagine a marathon runner's finish time. The probability they finish at exactly 4 hours, 0 minutes, 0.00000000... seconds is practically zero, because time is continuous. We can only find the probability they finish between 4 hours and 4 hours 5 minutes.
Key Takeaway 1:
For Continuous RVs, probability is measured over an interval (an area), not at a single point. If you see \(P(X=k)\), the answer is always \(0\).
2. The Probability Density Function (PDF), \(f(x)\)
For a Continuous Random Variable \(X\), we use a function called the Probability Density Function (PDF), denoted by \(f(x)\).
What is \(f(x)\)?
The PDF describes how the probability is distributed over the range of possible values. It is not the probability itself, but rather the 'density' of probability at point \(x\).
Conditions for a Valid PDF
For \(f(x)\) to be a valid PDF, it must satisfy two fundamental conditions:
- Non-negativity: The probability density cannot be negative.
$$\text{i.e., } f(x) \ge 0 \text{ for all values of } x$$ - Total Probability (Area is 1): The total area under the PDF curve must equal 1.
$$\text{i.e., } \int_{-\infty}^{\infty} f(x) dx = 1$$ (In practice, since \(f(x)\) is usually defined over a specific interval \([a, b]\), this integral becomes \(\int_{a}^{b} f(x) dx = 1\)).
Calculating Probabilities using the PDF
To find the probability that \(X\) lies between two values, \(a\) and \(b\), we simply calculate the area under the PDF curve between those limits:
$$P(a < X < b) = \int_{a}^{b} f(x) dx$$
Step-by-Step: Finding an Unknown Constant, \(k\)
Many exam questions start by asking you to find an unknown constant \(k\) in a PDF defined over an interval \([a, b]\).
- Set up the integral: Use the total probability rule, \(\int_{a}^{b} k \cdot (\text{function of } x) dx = 1\).
- Integrate the function: Perform the integration with respect to \(x\).
- Apply the limits: Evaluate the definite integral using \(F(b) - F(a)\).
- Solve for \(k\): Set the result equal to 1 and solve the resulting equation for \(k\).
Quick Review: PDF
- \(f(x)\) is the probability density.
- Total area under \(f(x)\) must be \(1\).
- Probability is found by integration.
3. The Cumulative Distribution Function (CDF), \(F(x)\)
The Cumulative Distribution Function (CDF), denoted by \(F(x)\), tells us the total probability accumulated up to a specific value \(x\). It is defined as:
$$F(x) = P(X \le x)$$
The Calculus Relationship: PDF and CDF
This is where your Pure Maths connection is strongest! Since \(F(x)\) accumulates probability (area), it is the integral of the PDF, \(f(x)\):
$$F(x) = \int_{-\infty}^{x} f(t) dt$$
(We use the dummy variable \(t\) inside the integral so we don't confuse it with the upper limit \(x\).)
Conversely, the PDF is the rate of change of the CDF—it is its derivative:
$$f(x) = \frac{d}{dx} F(x)$$
Properties and Usage of the CDF
- CDF must start at 0 and end at 1:
- For the minimum value of \(X\), \(F(x) = 0\).
- For the maximum value of \(X\), \(F(x) = 1\).
- Calculating Interval Probability using CDF: This is often much faster than integrating the PDF.
$$P(a < X < b) = F(b) - F(a)$$
Finding Percentiles and Medians
A common task is finding a specific value, \(k\), corresponding to a given probability (a percentile or quartile). For example, finding the Median, \(m\).
- The median \(m\) is the value such that \(P(X \le m) = 0.5\).
- To find \(m\), you solve the equation \(F(m) = 0.5\), or, if you haven't found \(F(x)\) yet: $$ \int_{a}^{m} f(x) dx = 0.5 $$
Did you know? The median is a crucial measure of central tendency because it is not affected by extreme outliers, unlike the mean!
Key Takeaway 2:
The PDF and CDF are related by differentiation and integration. Use the PDF to find the CDF, and use the CDF to quickly find probabilities and percentiles.
4. Measures of Central Tendency and Spread
Just like Discrete RVs, we need to calculate the Mean (Expectation), Variance, and Standard Deviation for continuous variables. We just replace summation with integration.
The Mean or Expectation, \(E(X)\)
The mean, \(\mu\), is the long-run average of the RV. It represents the 'balance point' of the distribution.
$$E(X) = \mu = \int_{-\infty}^{\infty} x f(x) dx$$
Expectation of a Function, \(E(g(X))\)
If we are interested in the expected value of some function of \(X\) (e.g., \(X^2\), \(1/X\)), we use the general formula:
$$E(g(X)) = \int_{-\infty}^{\infty} g(x) f(x) dx$$
Common mistake: Remember to multiply the function \(g(x)\) by the density function \(f(x)\) before integrating.
The Variance, \(\text{Var}(X)\)
Variance measures the spread or dispersion of the data around the mean.
The definitional formula is \(\text{Var}(X) = E((X-\mu)^2)\). However, the formula you must use for calculation is:
$$\text{Var}(X) = E(X^2) - [E(X)]^2$$
To use this formula:
- Calculate \(E(X)\) first (as found above).
- Calculate \(E(X^2)\) using the formula for \(E(g(X))\) where \(g(x) = x^2\):
$$E(X^2) = \int_{-\infty}^{\infty} x^2 f(x) dx$$ - Substitute both results into the variance formula.
Standard Deviation, \(\sigma\)
The standard deviation is simply the square root of the variance, giving the spread in the original units of \(X\).
$$\sigma = \sqrt{\text{Var}(X)}$$
Key Takeaway 3:
Mean and Variance calculations involve integration. Always use the simplified formula \(\text{Var}(X) = E(X^2) - [E(X)]^2\) for variance.
5. Manipulating Continuous Random Variables
These rules govern how the mean and variance change when you transform a single variable, or combine two independent variables. They are identical to the rules for Discrete RVs.
5.1 Linear Transformations of a Single Variable
Let \(Y = aX + b\), where \(a\) and \(b\) are constants.
Expectation Rule
Multiplying by \(a\) and adding \(b\) shifts the mean in the same way:
$$E(aX + b) = aE(X) + b$$
Example: If the average score E(X) is 50, then the average score after multiplying by 2 and adding 10 is \(2(50) + 10 = 110\).
Variance Rule
Adding a constant (\(b\)) does not change the spread, but multiplying by \(a\) scales the spread by \(a^2\) (since variance is in squared units):
$$\text{Var}(aX + b) = a^2\text{Var}(X)$$
Example: If \(\text{Var}(X)=4\), then \(\text{Var}(3X - 5)\) is \(3^2 \times 4 = 36\).
5.2 Sums and Differences of Two Independent Variables
Let \(X\) and \(Y\) be two independent continuous random variables.
Expectation Rule (Additive)
The expectation of a sum or difference is the sum or difference of the individual expectations:
$$E(aX \pm bY) = aE(X) \pm bE(Y)$$
Variance Rule (Always Additive)
When you combine independent variables, the uncertainty (variance) always increases, regardless of whether you are summing or subtracting the variables. The variation from both variables contributes to the final spread.
$$\text{Var}(aX \pm bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$$
Memory Aid for Variance: The variance of a difference is still a sum! \(\text{Var}(X-Y) = \text{Var}(X) + \text{Var}(Y)\).
Think: subtracting two variables just makes the overall result even more spread out and unpredictable, so the variance must increase.
Key Takeaway 4:
Linear transformation rules are critical! Remember that expectation scales linearly, but variance scales by the coefficient squared, and variances always add when combining independent RVs.
Quick Review: Continuous Random Variables
This table summarizes the core formulas you need for calculations in this chapter:
Calculation Summary
| Concept | Formula |
| Total Area (Normalization) | \(\int_{-\infty}^{\infty} f(x) dx = 1\) |
| Probability in an Interval | \(P(a < X < b) = \int_{a}^{b} f(x) dx\) |
| CDF | \(F(x) = \int_{-\infty}^{x} f(t) dt\) |
| Mean \(E(X)\) | \(\int_{-\infty}^{\infty} x f(x) dx\) |
| Variance \(\text{Var}(X)\) | \(E(X^2) - [E(X)]^2\) |
| Linear Expectation | \(E(aX+b) = aE(X) + b\) |
| Linear Variance | \(\text{Var}(aX+b) = a^2\text{Var}(X)\) |
| Variance of Sum/Difference (Independent) | \(\text{Var}(aX \pm bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)\) |