auto-update(nvim): 2025-02-10 17:57:19
This commit is contained in:
parent
cb17974289
commit
be53d8c61d
GPG key ID:
865658ED1FE61EC3
1 changed files with 384 additions and 0 deletions
|
|
@ -1642,3 +1642,387 @@ $F_z(x)$, we use the special $phi(x)$ and $Phi(x)$.
|
|||
- use technology (calculator)
|
||||
- use the standard normal probability table in the textbook
|
||||
]
|
||||
|
||||
= Lecture #datetime(day: 11, year: 2025, month: 2).display()
|
||||
|
||||
== Expectation
|
||||
|
||||
#definition[
|
||||
The expectation or mean of a discrete random variable $X$ is the weighted
|
||||
average, with weights assigned by the corresponding probabilities.
|
||||
|
||||
$
|
||||
E(X) = sum_("all" x_i) x_i dot p(x_i)
|
||||
$
|
||||
]
|
||||
|
||||
#example[
|
||||
Find the expected value of a single roll of a fair die.
|
||||
|
||||
- $X = "score" / "dots"$
|
||||
- $x = 1,2,3,4,5,6$
|
||||
- $p(x) = 1 / 6, 1 / 6,1 / 6,1 / 6,1 / 6,1 / 6$
|
||||
|
||||
$
|
||||
E[x] = 1 dot 1 / 6 + 2 dot 1 / 6 ... + 6 dot 1 / 6
|
||||
$
|
||||
]
|
||||
|
||||
== Binomial expected value
|
||||
|
||||
$
|
||||
E[x] = n p
|
||||
$
|
||||
|
||||
== Bernoulli expected value
|
||||
|
||||
Bernoulli is just binomial with one trial.
|
||||
|
||||
Recall that $P(X=1) = p$ and $P(X=0) = 1 - p$.
|
||||
|
||||
$
|
||||
E[X] = 1 dot P(X=1) + 0 dot P(X=0) = p
|
||||
$
|
||||
|
||||
Let $A$ be an event on $Omega$. Its _indicator random variable_ $I_A$ is defined
|
||||
for $omega in Omega$ by
|
||||
|
||||
$
|
||||
I_A (omega) = cases(1", if " &omega in A, 0", if" &omega in.not A)
|
||||
$
|
||||
|
||||
$
|
||||
E[I_A] = 1 dot P(A) = P(A)
|
||||
$
|
||||
|
||||
== Geometric expected value
|
||||
|
||||
Let $p in [0,1]$ and $X ~ "Geom"[ p ]$ be a geometric RV with probability of
|
||||
success $p$. Recall that the p.m.f. is $p q^(k-1)$, where prob. of failure is defined by $q := 1-p$.
|
||||
|
||||
Then
|
||||
|
||||
$
|
||||
E[X] &= sum_(k=1)^infinity k p q^(k-1) \
|
||||
&= p dot sum_(k=1)^infinity k dot q^(k-1)
|
||||
$
|
||||
|
||||
Now recall from calculus that you can differentiate a power series term by term inside its radius of convergence. So for $|t| < 1$,
|
||||
|
||||
$
|
||||
sum_(k=1)^infinity k t^(k-1) =
|
||||
sum_(k=1)^infinity dif / (dif t) t^k = dif / (dif t) sum_(k=1)^infinity t^k = dif / (dif t) (1 / (1-t)) = 1 / (1-t)^2 \
|
||||
therefore E[x] = sum^infinity_(k=1) k p q^(k-1) = p sum^infinity_(k=1) k q^(k-1) = p (1 / (1 - q)^2) = 1 / p
|
||||
$
|
||||
|
||||
== Expected value of a continuous RV
|
||||
|
||||
#definition[
|
||||
The expectation or mean of a continuous random variable $X$ with density
|
||||
function $f$ is
|
||||
|
||||
$
|
||||
E[x] = integral_(-infinity)^infinity x dot f(x) dif x
|
||||
$
|
||||
|
||||
An alternative symbol is $mu = E[x]$.
|
||||
]
|
||||
|
||||
$mu$ is the "first moment" of $X$, analogous to physics, its the "center of
|
||||
gravity" of $X$.
|
||||
|
||||
#remark[
|
||||
In general when moving between discrete and continuous RV, replace sums with
|
||||
integrals, p.m.f. with p.d.f., and vice versa.
|
||||
]
|
||||
|
||||
#example[
|
||||
Suppose $X$ is a continuous RV with p.d.f.
|
||||
|
||||
$
|
||||
f_X (x) = cases(2x", " &0 < x < 1, 0"," &"elsewhere")
|
||||
$
|
||||
|
||||
$
|
||||
E[X] = integral_(-infinity)^infinity x dot f(x) dif x = integral^1_0 x dot 2x dif x = 2 / 3
|
||||
$
|
||||
]
|
||||
|
||||
#example("Uniform expectation")[
|
||||
Let $X$ be a uniform random variable on the interval $[a,b]$ with $X ~
|
||||
"Unif"[a,b]$. Find the expected value of $X$.
|
||||
|
||||
$
|
||||
E[X] = integral^infinity_(-infinity) x dot f(x) dif x = integral_a^b x / (b-a) dif x \
|
||||
= 1 / (b-a) integral_a^b x dif x = 1 / (b-a) dot (b^2 - a^2) / 2 = underbrace((b+a) / 2, "midpoint formula")
|
||||
$
|
||||
]
|
||||
|
||||
#example("Exponential expectation")[
|
||||
Find the expected value of an exponential RV, with p.d.f.
|
||||
|
||||
$
|
||||
f_X (x) = cases(lambda e^(-lambda x)", " &x > 0, 0"," &"elsewhere")
|
||||
$
|
||||
|
||||
$
|
||||
E[x] = integral_(-infinity)^infinity x dot f(x) dif x = integral_0^infinity x dot lambda e^(-lambda x) dif x \
|
||||
= lambda dot integral_0^infinity x dot e^(-lambda x) dif x \
|
||||
= lambda dot [lr(-x 1 / lambda e^(-lambda x) |)_(x=0)^(x=infinity) - integral_0^infinity -1 / lambda e^(-lambda x) dif x] \
|
||||
= 1 / lambda
|
||||
$
|
||||
]
|
||||
|
||||
#example("Uniform dartboard")[
|
||||
Our dartboard is a disk of radius $r_0$ and the dart lands uniformly at
|
||||
random on the disk when thrown. Let $R$ be the distance of the dart from the
|
||||
center of the disk. Find $E[R]$ given density function
|
||||
|
||||
$
|
||||
f_R (t) = cases((2t)/(r_0 ^2)", " &0 <= t <= r_0, 0", " &t < 0 "or" t > r_0)
|
||||
$
|
||||
|
||||
$
|
||||
E[R] = integral_(-infinity)^infinity t f_R (t) dif t \
|
||||
= integral^(r_0)_0 t dot (2t) / (r_0^2) dif t \
|
||||
= 2 / 3 r_0
|
||||
$
|
||||
]
|
||||
|
||||
== Expectation of derived values
|
||||
|
||||
If we can find the expected value of $X$, can we find the expected value of
|
||||
$X^2$? More precisely, can we find $E[X^2]$?
|
||||
|
||||
If the distribution is easy to see, then this is trivial. Otherwise we have the
|
||||
following useful property:
|
||||
|
||||
$
|
||||
E[X^2] = integral_("all" x) x^2 f_X (x) dif x
|
||||
$
|
||||
|
||||
(for continuous RVs).
|
||||
|
||||
And in the discrete case,
|
||||
|
||||
$
|
||||
E[X^2] = sum_("all" x) x^2 p_X (x)
|
||||
$
|
||||
|
||||
In fact $E[X^2]$ is so important that we call it the *mean square*.
|
||||
|
||||
#fact[
|
||||
More generally, a real valued function $g(X)$ defined on the range of $X$ is
|
||||
itself a random variable (with its own distribution).
|
||||
]
|
||||
|
||||
We can find expected value of $g(X)$ by
|
||||
|
||||
$
|
||||
E[g(x)] = integral_(-infinity)^infinity g(x) f(x) dif x
|
||||
$
|
||||
|
||||
or
|
||||
|
||||
$
|
||||
E[g(x)] = sum_("all" x) g(x) f(x)
|
||||
$
|
||||
|
||||
#example[
|
||||
You roll a fair die to determine the winnings (or losses) $W$ of a player as
|
||||
follows:
|
||||
|
||||
$
|
||||
W = cases(-1", if the roll is 1, 2, or 3", 1", if the roll is a 4", 3", if the roll is 5 or 6")
|
||||
$
|
||||
|
||||
What is the expected winnings/losses for the player during 1 roll of the die?
|
||||
|
||||
Let $X$ denote the outcome of the roll of the die. Then we can define our
|
||||
random variable as $W = g(X)$ where the function $g$ is defined by $g(1) =
|
||||
g(2) = g(3) = -1$ and so on.
|
||||
|
||||
Note that $P(W = -1) = P(X = 1 union X = 2 union X = 3) = 1/2$. Likewise $P(W=1)
|
||||
= P(X=4) = 1/6$, and $P(W=3) = P(X=5 union X=6) = 1/3$.
|
||||
|
||||
Then
|
||||
$
|
||||
E[g(X)] = E[W] = (-1) dot P(W=-1) + (1) dot P(W=1) + (3) dot P(W=3) \
|
||||
= -1 / 2 + 1 / 6 + 1 = 2 / 3
|
||||
$
|
||||
]
|
||||
|
||||
#example[
|
||||
A stick of length $l$ is broken at a uniformly chosen random location. What is
|
||||
the expected length of the longer piece?
|
||||
|
||||
Idea: if you break it before the halfway point, then the longer piece has length
|
||||
given by $l - x$. If you break it after the halfway point, the longer piece
|
||||
has length $x$.
|
||||
|
||||
Let the interval $[0,l]$ represent the stick and let $X ~ "Unif"[0,l]$ be the
|
||||
location where the stick is broken. Then $X$ has density $f(x) = 1/l$ on
|
||||
$[0,l]$ and 0 elsewhere.
|
||||
|
||||
Let $g(x)$ be th length of the longer piece when the stick is broken at $x$,
|
||||
|
||||
$
|
||||
g(x) = cases(1-x", " &0 <= x < l/2, x", " &1/2 <= x <= l)
|
||||
$
|
||||
|
||||
Then
|
||||
$
|
||||
E[g(X)] = integral_(-infinity)^infinity g(x) f(x) dif x = integral_0^(l / 2) (l-x) / l dif x + integral_(l / 2)^l x / l dif x \
|
||||
= 3 / 4 l
|
||||
$
|
||||
|
||||
So we expect the longer piece to be $3/4$ of the total length, which is a bit
|
||||
pathological.
|
||||
]
|
||||
|
||||
== Moments of a random variable
|
||||
|
||||
We continue discussing expectation but we introduce new terminology.
|
||||
|
||||
#fact[
|
||||
The $n^"th"$ moment (or $n^"th"$ raw moment) of a discrete random variable $X$
|
||||
with p.m.f. $p_X (x)$ is the expectation
|
||||
|
||||
$
|
||||
E[X^n] = sum_k k^n p_X (k) = mu_n
|
||||
$
|
||||
|
||||
If $X$ is continuous, then we have analogously
|
||||
|
||||
$
|
||||
E[X^n] = integral_(-infinity)^infinity x^n f_X (x) = mu_n
|
||||
$
|
||||
]
|
||||
|
||||
The *deviation* is given by $sigma$ and the *variance* is given by $sigma^2$ and
|
||||
|
||||
$
|
||||
sigma^2 = mu_2 - (mu_1)^2
|
||||
$
|
||||
|
||||
$mu_3$ is used to measure "skewness" / asymmetry of a distribution. For
|
||||
example, the normal distribution is very symmetric.
|
||||
|
||||
$mu_4$ is used to measure kurtosis/peakedness of a distribution.
|
||||
|
||||
== Central moments
|
||||
|
||||
Previously we discussed "raw moments." Be careful not to confuse them with
|
||||
_central moments_.
|
||||
|
||||
#fact[
|
||||
The $n^"th"$ central moment of a discrete random variable $X$ with p.m.f. p_X
|
||||
(x) is the expected value of the difference about the mean raised to the
|
||||
$n^"th"$ power
|
||||
|
||||
$
|
||||
E[(X-mu)^n] = sum_k (k - mu)^n p_X (k) = mu'_n
|
||||
$
|
||||
|
||||
And of course in the continuous case,
|
||||
|
||||
$
|
||||
E[(X-mu)^n] = integral_(-infinity)^infinity (x - mu)^n f_X (x) = mu'_n
|
||||
$
|
||||
]
|
||||
|
||||
In particular,
|
||||
|
||||
$
|
||||
mu'_1 = E[(X-mu)^1] = integral_(-infinity)^infinity (x-mu)^1 f_X (x) dif x \
|
||||
= integral_(infinity)^infinity x f_X (x) dif x = integral_(-infinity)^infinity mu f_X (x) dif x = mu - mu dot 1 = 0 \
|
||||
mu'_2 = E[(X-mu)^2] = sigma^2_X = "Var"(X)
|
||||
$
|
||||
|
||||
#example[
|
||||
Let $Y$ be a uniformly chosen integer from ${0,1,2,...,m}$. Find the first and
|
||||
second moment of $Y$.
|
||||
|
||||
The p.m.f. of $Y$ is $p_Y (k) = 1/(m+1)$ for $k in [0,m]$. Thus,
|
||||
|
||||
$
|
||||
E[Y] = sum_(k=0)^m k 1 / (m+1) = 1 / (m+1) sum_(k=0)^m k \
|
||||
= m / 2
|
||||
$
|
||||
|
||||
Then,
|
||||
|
||||
$
|
||||
E[Y^2] = sum_(k=0)^m k^2 1 / (m+1) = 1 / (m+1) = (m(2m+1)) / 6
|
||||
$
|
||||
]
|
||||
|
||||
#example[
|
||||
Let $c > 0$ and let $U$ be a uniform random variable on the interval $[0,c]$.
|
||||
Find the $n^"th"$ moment for $U$ for all positive integers $n$.
|
||||
|
||||
The density function of $U$ is
|
||||
|
||||
$
|
||||
f(x) = cases(1/c", if" &x in [0,c], 0", " &"otherwise")
|
||||
$
|
||||
|
||||
Therefore the $n^"th"$ moment of $U$ is,
|
||||
|
||||
$
|
||||
E[U^n] = integral_(-infinity)^infinity x^n f(x) dif x
|
||||
$
|
||||
]
|
||||
|
||||
#example[
|
||||
Suppose the random variable $X ~ "Exp"(lambda)$. Find the second moment of $X$.
|
||||
|
||||
$
|
||||
E[X^2] = integral_0^infinity x^2 lambda e^(-lambda x) dif x \
|
||||
= 1 / (lambda^2) integral_0^infinity u^2 e^(-u) dif u \
|
||||
= 1 / (lambda^2) Gamma(2 + 1) = 2! / lambda^2
|
||||
$
|
||||
]
|
||||
|
||||
#fact[
|
||||
In general, to find teh $n^"th"$ moment of $X ~ "Exp"(lambda)$,
|
||||
$
|
||||
E[X^n] = integral^infinity_0 x^n lambda e^(-lambda x) dif x = n! / lambda^n
|
||||
$
|
||||
]
|
||||
|
||||
== Median and quartiles
|
||||
|
||||
When a random variable has rare (abnormal) values, its expectation may be a bad
|
||||
indicator of where the center of the distribution lies.
|
||||
|
||||
#definition[
|
||||
The *median* of a random variable $X$ is any real value $m$ that satisfies
|
||||
|
||||
$
|
||||
P(X >= m) >= 1 / 2 "and" P(X <= m) >= 1 / 2
|
||||
$
|
||||
|
||||
With half the probability on both ${X <= m}$ and ${X >= m}$, the median is
|
||||
representative of the midpoint of the distribution. We say that the median is
|
||||
more _robust_ because it is less affected by outliers. It is not necessarily
|
||||
unique.
|
||||
]
|
||||
|
||||
#example[
|
||||
Let $X$ be discretely uniformly distributed in the set ${-100, 1, 2, ,3, ..., 9}$ so $X$ has probability mass function
|
||||
$
|
||||
p_X (-100) = p_X (1) = dots.c = p_X (9)
|
||||
$
|
||||
|
||||
Find the expected value and median of $X$.
|
||||
|
||||
$
|
||||
E[X] = (-100) dot 1 / 10 + (1) dot 1 / 10 + dots.c + (9) dot 1 / 10 = -5.5
|
||||
$
|
||||
|
||||
While the median is any number $m in [4,5]$.
|
||||
|
||||
The median reflects the fact that 90% of the values and probability is in the
|
||||
range $1,2,...,9$ while the mean is heavily influenced by the $-100$ value.
|
||||
]
|
||||
|
|
|
|||
Loading…
Reference in a new issue