From be53d8c61da6c6ea55329d1b0b3b2461efcc7f68 Mon Sep 17 00:00:00 2001 From: Youwen Wu Date: Mon, 10 Feb 2025 17:57:19 -0800 Subject: [PATCH] auto-update(nvim): 2025-02-10 17:57:19 --- .../pstat-120a/course-notes/main.typ | 384 ++++++++++++++++++ 1 file changed, 384 insertions(+) diff --git a/documents/by-course/pstat-120a/course-notes/main.typ b/documents/by-course/pstat-120a/course-notes/main.typ index b6c7dca..40961eb 100644 --- a/documents/by-course/pstat-120a/course-notes/main.typ +++ b/documents/by-course/pstat-120a/course-notes/main.typ @@ -1642,3 +1642,387 @@ $F_z(x)$, we use the special $phi(x)$ and $Phi(x)$. - use technology (calculator) - use the standard normal probability table in the textbook ] + += Lecture #datetime(day: 11, year: 2025, month: 2).display() + +== Expectation + +#definition[ + The expectation or mean of a discrete random variable $X$ is the weighted + average, with weights assigned by the corresponding probabilities. + + $ + E(X) = sum_("all" x_i) x_i dot p(x_i) + $ +] + +#example[ + Find the expected value of a single roll of a fair die. + + - $X = "score" / "dots"$ + - $x = 1,2,3,4,5,6$ + - $p(x) = 1 / 6, 1 / 6,1 / 6,1 / 6,1 / 6,1 / 6$ + + $ + E[x] = 1 dot 1 / 6 + 2 dot 1 / 6 ... + 6 dot 1 / 6 + $ +] + +== Binomial expected value + +$ + E[x] = n p +$ + +== Bernoulli expected value + +Bernoulli is just binomial with one trial. + +Recall that $P(X=1) = p$ and $P(X=0) = 1 - p$. + +$ + E[X] = 1 dot P(X=1) + 0 dot P(X=0) = p +$ + +Let $A$ be an event on $Omega$. Its _indicator random variable_ $I_A$ is defined +for $omega in Omega$ by + +$ + I_A (omega) = cases(1", if " &omega in A, 0", if" &omega in.not A) +$ + +$ + E[I_A] = 1 dot P(A) = P(A) +$ + +== Geometric expected value + +Let $p in [0,1]$ and $X ~ "Geom"[ p ]$ be a geometric RV with probability of +success $p$. Recall that the p.m.f. is $p q^(k-1)$, where prob. of failure is defined by $q := 1-p$. + +Then + +$ + E[X] &= sum_(k=1)^infinity k p q^(k-1) \ + &= p dot sum_(k=1)^infinity k dot q^(k-1) +$ + +Now recall from calculus that you can differentiate a power series term by term inside its radius of convergence. So for $|t| < 1$, + +$ + sum_(k=1)^infinity k t^(k-1) = + sum_(k=1)^infinity dif / (dif t) t^k = dif / (dif t) sum_(k=1)^infinity t^k = dif / (dif t) (1 / (1-t)) = 1 / (1-t)^2 \ + therefore E[x] = sum^infinity_(k=1) k p q^(k-1) = p sum^infinity_(k=1) k q^(k-1) = p (1 / (1 - q)^2) = 1 / p +$ + +== Expected value of a continuous RV + +#definition[ + The expectation or mean of a continuous random variable $X$ with density + function $f$ is + + $ + E[x] = integral_(-infinity)^infinity x dot f(x) dif x + $ + + An alternative symbol is $mu = E[x]$. +] + +$mu$ is the "first moment" of $X$, analogous to physics, its the "center of +gravity" of $X$. + +#remark[ + In general when moving between discrete and continuous RV, replace sums with + integrals, p.m.f. with p.d.f., and vice versa. +] + +#example[ + Suppose $X$ is a continuous RV with p.d.f. + + $ + f_X (x) = cases(2x", " &0 < x < 1, 0"," &"elsewhere") + $ + + $ + E[X] = integral_(-infinity)^infinity x dot f(x) dif x = integral^1_0 x dot 2x dif x = 2 / 3 + $ +] + +#example("Uniform expectation")[ + Let $X$ be a uniform random variable on the interval $[a,b]$ with $X ~ + "Unif"[a,b]$. Find the expected value of $X$. + + $ + E[X] = integral^infinity_(-infinity) x dot f(x) dif x = integral_a^b x / (b-a) dif x \ + = 1 / (b-a) integral_a^b x dif x = 1 / (b-a) dot (b^2 - a^2) / 2 = underbrace((b+a) / 2, "midpoint formula") + $ +] + +#example("Exponential expectation")[ + Find the expected value of an exponential RV, with p.d.f. + + $ + f_X (x) = cases(lambda e^(-lambda x)", " &x > 0, 0"," &"elsewhere") + $ + + $ + E[x] = integral_(-infinity)^infinity x dot f(x) dif x = integral_0^infinity x dot lambda e^(-lambda x) dif x \ + = lambda dot integral_0^infinity x dot e^(-lambda x) dif x \ + = lambda dot [lr(-x 1 / lambda e^(-lambda x) |)_(x=0)^(x=infinity) - integral_0^infinity -1 / lambda e^(-lambda x) dif x] \ + = 1 / lambda + $ +] + +#example("Uniform dartboard")[ + Our dartboard is a disk of radius $r_0$ and the dart lands uniformly at + random on the disk when thrown. Let $R$ be the distance of the dart from the + center of the disk. Find $E[R]$ given density function + + $ + f_R (t) = cases((2t)/(r_0 ^2)", " &0 <= t <= r_0, 0", " &t < 0 "or" t > r_0) + $ + + $ + E[R] = integral_(-infinity)^infinity t f_R (t) dif t \ + = integral^(r_0)_0 t dot (2t) / (r_0^2) dif t \ + = 2 / 3 r_0 + $ +] + +== Expectation of derived values + +If we can find the expected value of $X$, can we find the expected value of +$X^2$? More precisely, can we find $E[X^2]$? + +If the distribution is easy to see, then this is trivial. Otherwise we have the +following useful property: + +$ + E[X^2] = integral_("all" x) x^2 f_X (x) dif x +$ + +(for continuous RVs). + +And in the discrete case, + +$ + E[X^2] = sum_("all" x) x^2 p_X (x) +$ + +In fact $E[X^2]$ is so important that we call it the *mean square*. + +#fact[ + More generally, a real valued function $g(X)$ defined on the range of $X$ is + itself a random variable (with its own distribution). +] + +We can find expected value of $g(X)$ by + +$ + E[g(x)] = integral_(-infinity)^infinity g(x) f(x) dif x +$ + +or + +$ + E[g(x)] = sum_("all" x) g(x) f(x) +$ + +#example[ + You roll a fair die to determine the winnings (or losses) $W$ of a player as + follows: + + $ + W = cases(-1", if the roll is 1, 2, or 3", 1", if the roll is a 4", 3", if the roll is 5 or 6") + $ + + What is the expected winnings/losses for the player during 1 roll of the die? + + Let $X$ denote the outcome of the roll of the die. Then we can define our + random variable as $W = g(X)$ where the function $g$ is defined by $g(1) = + g(2) = g(3) = -1$ and so on. + + Note that $P(W = -1) = P(X = 1 union X = 2 union X = 3) = 1/2$. Likewise $P(W=1) + = P(X=4) = 1/6$, and $P(W=3) = P(X=5 union X=6) = 1/3$. + + Then + $ + E[g(X)] = E[W] = (-1) dot P(W=-1) + (1) dot P(W=1) + (3) dot P(W=3) \ + = -1 / 2 + 1 / 6 + 1 = 2 / 3 + $ +] + +#example[ + A stick of length $l$ is broken at a uniformly chosen random location. What is + the expected length of the longer piece? + + Idea: if you break it before the halfway point, then the longer piece has length + given by $l - x$. If you break it after the halfway point, the longer piece + has length $x$. + + Let the interval $[0,l]$ represent the stick and let $X ~ "Unif"[0,l]$ be the + location where the stick is broken. Then $X$ has density $f(x) = 1/l$ on + $[0,l]$ and 0 elsewhere. + + Let $g(x)$ be th length of the longer piece when the stick is broken at $x$, + + $ + g(x) = cases(1-x", " &0 <= x < l/2, x", " &1/2 <= x <= l) + $ + + Then + $ + E[g(X)] = integral_(-infinity)^infinity g(x) f(x) dif x = integral_0^(l / 2) (l-x) / l dif x + integral_(l / 2)^l x / l dif x \ + = 3 / 4 l + $ + + So we expect the longer piece to be $3/4$ of the total length, which is a bit + pathological. +] + +== Moments of a random variable + +We continue discussing expectation but we introduce new terminology. + +#fact[ + The $n^"th"$ moment (or $n^"th"$ raw moment) of a discrete random variable $X$ + with p.m.f. $p_X (x)$ is the expectation + + $ + E[X^n] = sum_k k^n p_X (k) = mu_n + $ + + If $X$ is continuous, then we have analogously + + $ + E[X^n] = integral_(-infinity)^infinity x^n f_X (x) = mu_n + $ +] + +The *deviation* is given by $sigma$ and the *variance* is given by $sigma^2$ and + +$ + sigma^2 = mu_2 - (mu_1)^2 +$ + +$mu_3$ is used to measure "skewness" / asymmetry of a distribution. For +example, the normal distribution is very symmetric. + +$mu_4$ is used to measure kurtosis/peakedness of a distribution. + +== Central moments + +Previously we discussed "raw moments." Be careful not to confuse them with +_central moments_. + +#fact[ + The $n^"th"$ central moment of a discrete random variable $X$ with p.m.f. p_X + (x) is the expected value of the difference about the mean raised to the + $n^"th"$ power + + $ + E[(X-mu)^n] = sum_k (k - mu)^n p_X (k) = mu'_n + $ + + And of course in the continuous case, + + $ + E[(X-mu)^n] = integral_(-infinity)^infinity (x - mu)^n f_X (x) = mu'_n + $ +] + +In particular, + +$ + mu'_1 = E[(X-mu)^1] = integral_(-infinity)^infinity (x-mu)^1 f_X (x) dif x \ + = integral_(infinity)^infinity x f_X (x) dif x = integral_(-infinity)^infinity mu f_X (x) dif x = mu - mu dot 1 = 0 \ + mu'_2 = E[(X-mu)^2] = sigma^2_X = "Var"(X) +$ + +#example[ + Let $Y$ be a uniformly chosen integer from ${0,1,2,...,m}$. Find the first and + second moment of $Y$. + + The p.m.f. of $Y$ is $p_Y (k) = 1/(m+1)$ for $k in [0,m]$. Thus, + + $ + E[Y] = sum_(k=0)^m k 1 / (m+1) = 1 / (m+1) sum_(k=0)^m k \ + = m / 2 + $ + + Then, + + $ + E[Y^2] = sum_(k=0)^m k^2 1 / (m+1) = 1 / (m+1) = (m(2m+1)) / 6 + $ +] + +#example[ + Let $c > 0$ and let $U$ be a uniform random variable on the interval $[0,c]$. + Find the $n^"th"$ moment for $U$ for all positive integers $n$. + + The density function of $U$ is + + $ + f(x) = cases(1/c", if" &x in [0,c], 0", " &"otherwise") + $ + + Therefore the $n^"th"$ moment of $U$ is, + + $ + E[U^n] = integral_(-infinity)^infinity x^n f(x) dif x + $ +] + +#example[ + Suppose the random variable $X ~ "Exp"(lambda)$. Find the second moment of $X$. + + $ + E[X^2] = integral_0^infinity x^2 lambda e^(-lambda x) dif x \ + = 1 / (lambda^2) integral_0^infinity u^2 e^(-u) dif u \ + = 1 / (lambda^2) Gamma(2 + 1) = 2! / lambda^2 + $ +] + +#fact[ + In general, to find teh $n^"th"$ moment of $X ~ "Exp"(lambda)$, + $ + E[X^n] = integral^infinity_0 x^n lambda e^(-lambda x) dif x = n! / lambda^n + $ +] + +== Median and quartiles + +When a random variable has rare (abnormal) values, its expectation may be a bad +indicator of where the center of the distribution lies. + +#definition[ + The *median* of a random variable $X$ is any real value $m$ that satisfies + + $ + P(X >= m) >= 1 / 2 "and" P(X <= m) >= 1 / 2 + $ + + With half the probability on both ${X <= m}$ and ${X >= m}$, the median is + representative of the midpoint of the distribution. We say that the median is + more _robust_ because it is less affected by outliers. It is not necessarily + unique. +] + +#example[ + Let $X$ be discretely uniformly distributed in the set ${-100, 1, 2, ,3, ..., 9}$ so $X$ has probability mass function + $ + p_X (-100) = p_X (1) = dots.c = p_X (9) + $ + + Find the expected value and median of $X$. + + $ + E[X] = (-100) dot 1 / 10 + (1) dot 1 / 10 + dots.c + (9) dot 1 / 10 = -5.5 + $ + + While the median is any number $m in [4,5]$. + + The median reflects the fact that 90% of the values and probability is in the + range $1,2,...,9$ while the mean is heavily influenced by the $-100$ value. +]