The question: how do you describe the centre and spread of something that has not happened yet?
A random variable attaches a number to each outcome of an experiment.
Something random is about to happen, and the random variable is the number you read off once it has. Roll a die and $X$ is whatever comes up. Twelve applications land on your desk and $X$ is how many are incomplete. You stand at a bus stop and $X$ is how long you wait.
The thing people trip on: $X$ itself is not random, and it is not a variable in the algebra sense either. The randomness is in which outcome happens. $X$ is just the number we have agreed to attach to each one. Before the die lands, $X$ is a distribution. After it lands, $X$ is 4.
Capital letters for the random variable, lower case for a value it took. $\mathrm{P}(X = x)$ reads “the probability that the random variable takes the particular value $x$”.
Discrete — you can list the values. Number of defects, heads in ten flips, people in a queue. Here $\mathrm{P}(X = k)$ is a real, usable number.
Continuous — the values fill an interval. Time, weight, price. Here $\mathrm{P}(X = k) = 0$ for every single $k$, only intervals carry probability, and probability is area under a curve.
That fork decides which tool you reach for: discrete, add up probabilities; continuous, find an area. This chapter is discrete. Chapter 9 is continuous.
For a discrete random variable, the distribution is a list: every value, and its probability.
| $x$ | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| $\mathrm{P}(X = x)$ | 0.10 | 0.30 | 0.40 | 0.20 |
To be valid it must satisfy two conditions, and checking them is often a mark:
A weighted average of the values, weighted by how often each occurs. It is what you would average if you ran the experiment forever.
For the table above:
\[E(X) = 0(0.10) + 1(0.30) + 2(0.40) + 3(0.20) = 0 + 0.30 + 0.80 + 0.60 = 1.7\]Note what just happened: the expected value is 1.7, and $X$ can never be 1.7. The expected value is where the distribution balances, not a value to expect. A household can have 2.4 children on average and no household has 2.4 children.
The same idea as Chapter 3 — average squared distance from the centre — but weighted by probability instead of divided by $n$.
The two forms are not two definitions. The second follows from the first in four lines, and the derivation is worth doing once because every step is a rule you already have from this chapter.
Start from the definition, and write $(X-\mu)^2$ out:
\[V(X) = E\big[(X-\mu)^2\big] = E\big[X^2 - 2\mu X + \mu^2\big]\]Expectation of a sum is the sum of the expectations, so split it into three:
\[= E(X^2) - E(2\mu X) + E(\mu^2)\]Now use the two facts that do the work. $\mu$ is a constant, not a random variable — it is a fixed number once the distribution is fixed. So $2\mu$ comes outside the middle expectation, and $\mu^2$ is already constant, so its expectation is itself:
\[= E(X^2) - 2\mu\,E(X) + \mu^2\]Finally, $E(X)$ is $\mu$. Substitute:
\[= E(X^2) - 2\mu \cdot \mu + \mu^2 = E(X^2) - 2\mu^2 + \mu^2\] \[\boxed{\;V(X) = E(X^2) - \mu^2\;}\]Two things worth noticing.
The shortcut is faster because it needs one pass through the table instead of two. The definition requires $\mu$ before you can compute a single deviation; the shortcut lets you accumulate $\sum x\,\mathrm{P}(x)$ and $\sum x^2\mathrm{P}(x)$ together.
And it explains why $E(X^2) \neq [E(X)]^2$. The gap between them is exactly $V(X)$, which is non-negative — so $E(X^2)$ is always at least $[E(X)]^2$, with equality only when $X$ is a constant. In the example above, $3.7$ against $2.89$, and the difference $0.81$ is the variance.
The second form is the shortcut, and it is nearly always faster by hand. Compute $E(X^2)$ by squaring each value and weighting:
\[E(X^2) = 0^2(0.10) + 1^2(0.30) + 2^2(0.40) + 3^2(0.20) = 0 + 0.30 + 1.60 + 1.80 = 3.7\] \[V(X) = 3.7 - (1.7)^2 = 3.7 - 2.89 = 0.81 \qquad \sigma = \sqrt{0.81} = 0.90\]Both forms give the same answer. Use whichever the table in front of you makes easier, but know the shortcut — on an exam it saves a column.
These do a surprising amount of work, so look at what each says.
In the expectation law, both $a$ and $b$ come through. Scale everything and the average scales; shift everything and the average shifts.
In the variance law, the $b$ has vanished. Adding a constant slides the whole distribution along without stretching it, so the spread is untouched. Only the multiplier matters, and it arrives squared — which is why variance is in squared units and why we report $\sigma$ instead.
A depot hires out vehicles with the distribution above. It charges $50 per hire and has fixed daily costs of $20, so daily revenue net of costs is $R = 50X - 20$.
\(E(R) = 50(1.7) - 20 = 85 - 20 = 65\) \(V(R) = 50^2 (0.81) = 2500(0.81) = 2025 \qquad \sigma_R = \sqrt{2025} = 45\)
Raise the fixed costs to $100 and $E(R)$ falls by 80 while $V(R)$ does not move at all.
For two random variables, the covariance and correlation of Chapter 4 carry over unchanged in meaning:
\[\text{COV}(X,Y) = E\big[(X-\mu_x)(Y-\mu_y)\big] \qquad \rho = \frac{\text{COV}(X,Y)}{\sigma_x \sigma_y}\]And the rule that matters later:
\[V(X + Y) = V(X) + V(Y) + 2\,\text{COV}(X,Y)\]Variances add only when the covariance is zero — that is, when the two are uncorrelated. Chapter 11 leans on this to derive the standard error, so it is worth having it fixed now.
The snippets below assume the header and preamble from Chapter 1 — clear all, the log, and a cd to your own folder.
Stata is built for data, not for probability tables, so most of this chapter is pencil work. But you can compute expectations from a table directly:
clear
input x px
0 0.10
1 0.30
2 0.40
3 0.20
end
generate double xp = x*px
generate double x2p = x*x*px
summarize xp x2p, meanonly
quietly summarize xp
scalar EX = r(sum)
quietly summarize x2p
scalar EX2 = r(sum)
display "E(X) = " EX
display "E(X^2) = " EX2
display "V(X) = " EX2 - EX^2
display "SD(X) = " sqrt(EX2 - EX^2)
Note r(sum) rather than r(mean): an expectation is a weighted sum, and the
weights are already in the column.
An expectation is a weighted sum, so it is one dot product:
import numpy as np
v = np.array([0, 1, 2, 3])
p = np.array([0.10, 0.30, 0.40, 0.20])
assert np.isclose(p.sum(), 1) # always check this first
E = (v * p).sum() # 1.70
E2 = (v**2 * p).sum() # 3.70
V = E2 - E**2 # 0.81
print(E, V, np.sqrt(V)) # 1.70 0.81 0.90
That is the shortcut from the derivation above, written in two lines. Confirm it against the definition:
print(((v - E)**2 * p).sum()) # 0.81 -- the same number
The laws are worth checking too, because seeing them hold is more convincing than being told:
a, b = 50, 20
print(a*E + b, (a*v + b @ p if False else ((a*v + b) * p).sum())) # both 105.0
print(a**2 * V, (((a*v + b) - (a*E + b))**2 * p).sum()) # both 2025.0
Note the second line: adding b shifted the mean by 20 and left the variance
untouched, exactly as $V(aX+b) = a^2V(X)$ says.
Probabilities that do not sum to 1. Check before computing anything.
Expecting the expected value. $E(X) = 1.7$ when $X$ is a count of vehicles is correct and perfectly normal.
Forgetting to square the $a$ in $V(aX + b)$, or wrongly adding the $b$.
Using $E(X^2) = [E(X)]^2$. These are different, and the gap between them is the variance. Here $E(X^2) = 3.7$ while $[E(X)]^2 = 2.89$.
Assuming variances add. Only when the covariance is zero.
A random variable has this distribution:
| $x$ | 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|---|
| $\mathrm{P}(X=x)$ | 0.15 | 0.20 | 0.30 | 0.25 | 0.10 |
(a) Verify it is valid. (b) Find $E(X)$. (c) Find $V(X)$ and $\sigma$.
For the variable in question 1, let $Y = 20X + 5$. Find $E(Y)$, $V(Y)$ and $\sigma_Y$.
A distribution has $\mathrm{P}(X=0) = 0.2$, $\mathrm{P}(X=1) = 0.5$ and $\mathrm{P}(X=2) = p$. Find $p$, then find $E(X)$.
$E(X) = 8$ and $E(X^2) = 73$. Find $V(X)$ and $\sigma$. Then find $V(3X - 4)$.
Two product lines have $V(X) = 36$, $V(Y) = 64$ and $\text{COV}(X,Y) = 24$. (a) Find $V(X+Y)$. (b) What would it be if the two were uncorrelated? (c) Which arrangement would a risk-averse manager prefer, and why?
(a) All five lie in $[0,1]$ and they sum to $1.00$. Valid. (b) $E(X) = 1(0.15)+2(0.20)+3(0.30)+4(0.25)+5(0.10) = 0.15+0.40+0.90+1.00+0.50 = 2.95$. (c) $E(X^2) = 1(0.15)+4(0.20)+9(0.30)+16(0.25)+25(0.10) = 0.15+0.80+2.70+4.00+2.50 = 10.15$. So $V(X) = 10.15 - 2.95^2 = 10.15 - 8.7025 = 1.4475$ and $\sigma = 1.2031$.
$E(Y) = 20(2.95) + 5 = 64$. $V(Y) = 20^2(1.4475) = 579$. $\sigma_Y = 24.0624$. (Equivalently $\sigma_Y = 20 \times 1.2031$ — the multiplier passes straight through to the standard deviation, unsquared.)
Probabilities sum to 1, so $p = 1 - 0.2 - 0.5 = 0.3$. Then $E(X) = 0(0.2) + 1(0.5) + 2(0.3) = 1.1$.
$V(X) = 73 - 64 = 9$, so $\sigma = 3$. And $V(3X-4) = 3^2(9) = 81$ — the $-4$ does nothing.
(a) $36 + 64 + 2(24) = 148$. (b) $36 + 64 = 100$. (c) The uncorrelated arrangement. Expected total profit is unchanged either way, but its variance falls from 148 to 100 — the same return with less risk. That is diversification, and it is entirely a statement about the covariance term.
Ten more. Question 10 is the derivation itself — if you can reproduce it from a blank page you will never again wonder which of $E(X^2)$ and $E(X)^2$ gets subtracted from which.
The number of claims a policyholder files in a year, $X$, has
| $x$ | 0 | 1 | 2 | 3 |
|---|---|---|---|---|
| $\mathrm{P}(X=x)$ | 0.1 | 0.3 | 0.4 | 0.2 |
(a) Confirm this is a valid probability distribution. (b) Find $E(X)$. (c) Find $E(X^2)$, then $V(X)$ and the standard deviation.
Using the distribution in question 1, let $Y = 3X + 2$. (a) Find $E(Y)$ and $V(Y)$ without re-tabulating. (b) Find the standard deviation of $Y$. (c) Why does the $+2$ affect the mean but not the variance?
An insurer charges a premium of $50 and pays out $10,000 if a claim occurs, which happens with probability 0.003. (a) Find the expected profit per policy. (b) The insurer writes 20,000 such policies. Find the expected total profit. (c) Why is the expected value a sensible basis for pricing 20,000 policies but a poor guide to whether you should buy one?
$X$ and $Y$ have the joint distribution
| $Y=0$ | $Y=1$ | |
|---|---|---|
| $X=0$ | 0.2 | 0.1 |
| $X=1$ | 0.3 | 0.4 |
(a) Find the marginal distributions of $X$ and $Y$, and $E(X)$, $E(Y)$. (b) Find $E(XY)$ and hence $\text{COV}(X,Y)$. (c) Are $X$ and $Y$ independent? Show the check. (d) Find $V(X+Y)$.
A game pays $10 with probability 0.2, $2 with probability 0.5, and nothing otherwise. It costs $4 to play. (a) Find the expected net gain. (b) Would you play it repeatedly? Once?
$X$ has mean 20 and standard deviation 4. Find the mean and standard deviation of (a) $X + 10$, (b) $5X$, (c) $(X - 20)/4$.
Two independent random variables have $V(X) = 9$ and $V(Y) = 16$. (a) Find $V(X+Y)$ and $V(X-Y)$. (b) Now suppose instead that $\text{COV}(X,Y) = -6$. Recompute both. (c) In (b), is it possible for $V(X-Y)$ to exceed $V(X)+V(Y)$? Explain.
$S = X_1 + X_2 + \cdots + X_{25}$, where the $X_i$ are independent with mean 8 and standard deviation 3. (a) Find $E(S)$ and the standard deviation of $S$. (b) Find $E(\bar{X})$ and the standard deviation of $\bar{X}$. (c) Note that one of these grows with $n$ and one shrinks. Say which, and why.
A student computes $V(X) = -2$ and concludes the variable is “negatively variable”. What has gone wrong, and what is the most likely arithmetic slip?
Starting from $V(X) = E\big[(X-\mu)^2\big]$, derive $V(X) = E(X^2) - \big[E(X)\big]^2$, justifying each step.
(a) All four probabilities are between 0 and 1 and they sum to $0.1+0.3+0.4+0.2 = 1$. Valid. (b) $E(X) = 0(0.1) + 1(0.3) + 2(0.4) + 3(0.2) = 1.7000$. (c) $E(X^2) = 0(0.1) + 1(0.3) + 4(0.4) + 9(0.2) = 3.7000$, so $V(X) = 3.7 - 1.7^2 = 3.7 - 2.89 = 0.8100$ and the standard deviation is 0.9000. Note $E(X^2) = 3.70 \neq [E(X)]^2 = 2.89$; the gap between them is the variance.
(a) $E(Y) = 3(1.7) + 2 = 7.1000$ and $V(Y) = 3^2(0.81) = 7.2900$. (b) $\sqrt{7.29} = 2.7000$ — which is $3 \times 0.9$, as it must be. (c) Adding 2 shifts every outcome and the mean by the same amount, so every deviation from the mean is unchanged and the spread is untouched. Multiplying by 3 stretches the deviations by 3, so the standard deviation scales by 3 and the variance by $3^2$. Location and spread respond to different parts of a linear transformation.
(a) $50 - 0.003(10{,}000) = 50 - 30 = $20.0000$. (b) $20{,}000 \times 20 = $400{,}000$. (c) Because expectation is an average over many repetitions, and the insurer gets those repetitions — across 20,000 policies the realised total will land close to the expected one. You buy one policy once. For you the distribution is $50 lost with probability 0.997 and $9,950 gained with probability 0.003, and the $20 “expected” figure describes neither outcome. Insurance is rational for the buyer because of the shape of that distribution, not its mean.
(a) $X$: $\mathrm{P}(0) = 0.3$, $\mathrm{P}(1) = 0.7$, so $E(X) = 0.7000$ and $V(X) = 0.7 - 0.49 = 0.2100$. $Y$: $\mathrm{P}(0) = 0.5$, $\mathrm{P}(1) = 0.5$, so $E(Y) = 0.5000$ and $V(Y) = 0.2500$. (b) $XY$ is 1 only in the bottom-right cell, so $E(XY) = 0.4000$ and $\text{COV} = 0.4 - 0.7(0.5) = 0.0500$. (c) Not independent: $\mathrm{P}(X{=}1, Y{=}1) = 0.4$ but $\mathrm{P}(X{=}1)\mathrm{P}(Y{=}1) = 0.7(0.5) = 0.35$. One failing cell is enough. (d) $V(X+Y) = 0.21 + 0.25 + 2(0.05) = 0.5600$.
(a) $E(\text{payout}) = 10(0.2) + 2(0.5) + 0(0.3) = 2 + 1 = $3.0000$, so the expected net gain is $3 - 4 = -$1.0000$. (b) Not repeatedly — you lose a dollar a go on average, and over many plays that average is what you get. Once is a different question: a single play is a 20% chance of being $6 up against a 30% chance of being $4 down, and whether that trade appeals is about your tolerance for the spread, not about the mean.
(a) Mean 30, standard deviation 4 — unchanged. (b) Mean 100, standard deviation 20. (c) Mean 0, standard deviation 1. This is the $z$-score transformation from Chapter 3, and the general rule $E(aX+b) = aE(X)+b$, $\text{SD}(aX+b) = |a|\,\text{SD}(X)$ is what guarantees it always lands on mean 0 and standard deviation 1.
(a) Independent, so the covariance is zero and both are $9 + 16 = 25.0000$. (b) $V(X+Y) = 9 + 16 + 2(-6) = 13.0000$ and $V(X-Y) = 9 + 16 - 2(-6) = 37.0000$. (c) Yes, and it just did. Subtracting a negatively correlated variable adds variance, because the minus sign in $-2\text{COV}$ meets the minus sign in the covariance. Variances add; the covariance term is the only one whose sign can go either way, and it is where every interesting answer in this section comes from.
(a) $E(S) = 25(8) = 200$ and $V(S) = 25(9) = 225$, so $\text{SD}(S) = 15.0000$. (b) $E(\bar{X}) = 8$ and $\text{SD}(\bar{X}) = 3/\sqrt{25} = 0.6000$. (c) The standard deviation of the sum grows, as $\sqrt{n}\sigma$; the standard deviation of the mean shrinks, as $\sigma/\sqrt{n}$. Both follow from the same fact that variances add: a sum of 25 things accumulates 25 variances, while the mean divides that accumulated total by $25^2$. This is the result Chapter 11 is built on.
Variance is an expected squared deviation, so it is an average of non-negative numbers and cannot be negative. The likely slip is subtracting the wrong way round — computing $[E(X)]^2 - E(X^2)$ instead of $E(X^2) - [E(X)]^2$. A negative variance is always an arithmetic error, never a finding, and it is worth treating as a built-in alarm.
Expand the square inside the expectation: \(E\big[(X-\mu)^2\big] = E\big[X^2 - 2\mu X + \mu^2\big]\) Expectation is linear, so it passes through the sum: \(= E(X^2) - 2\mu E(X) + \mu^2\) The step that needs care is $E(\mu^2) = \mu^2$ and $E(2\mu X) = 2\mu E(X)$: $\mu$ is a fixed number, not a random variable, so it comes out of the expectation untouched. Now substitute $E(X) = \mu$: \(= E(X^2) - 2\mu^2 + \mu^2 = E(X^2) - \mu^2 = E(X^2) - \big[E(X)\big]^2\) Check it against question 1: $3.70 - 1.7^2 = 0.81$, which is what the deviation route gave. The identity is not a different definition of variance — it is the same quantity rearranged so you only need one pass through the table.