Definition

\(X\) is a continuous random variable if there exists a nonnegative function defined for all \(x \in \Re\) having the property for any (measurable) set of real numbers \(B\),

\[ \begin{eqnarray} \Pr(X \in B) & = & \int_{B} f_X(x)dx \end{eqnarray} \]

• Non-negative meaning \(f(x)\) is never negative
• \(\Pr(X \in B)\): probability that \(X\) is is an element of \(B\)
• We’ll call \(f(\cdot)\) the probability density function (PDF) for \(X\)

The probability that the value of \(X\) falls within an interval is

\[\Pr (a \leq X \leq b) = \int_a^b f_X(x) dx\]

and can be interpreted as the area under the graph of the PDF. For any single value \(a\), we have \(\Pr (X = a) = \int_a^a f_X(x) dx = 0\). Note that to qualify as a PDF, a function \(f_X\) must be nonnegative, i.e. \(f_X(x) \geq 0\) for every \(x\), and must also have the normalization property

\[\int_{-\infty}^{\infty} f_X(x) dx = \Pr (-\infty \leq X \leq \infty) = 1\]

Example: Uniform Random Variable

\(X \sim \text{Uniform}(0,1)\)

\[ f_X(x) = \left\{ \begin{array}{ll} c & \quad \text{if } 0 \leq x \leq 1 \\ 0 & \quad \text{otherwise} \end{array} \right. \]

for some constant \(c\). The constant can be determined from the normalization property

\[1 = \int_{-\infty}^{\infty} f_X(x)dx = \int_0^1 c dx = c \int_0^1 dx = c\]

so that \(c=1\).

data_frame(x = seq(-.5, 1.5, by = .01),
           y = dunif(x)) %>%
  ggplot(aes(x, y)) +
  geom_step() +
  labs(x = expression(x),
       y = expression(f[X](x)))

\[ \begin{eqnarray} \Pr(X \in [0.2, 0.5]) & = & \int_{0.2}^{0.5} 1 dx \nonumber \\ & = & X |^{0.5}_{0.2} \nonumber \\ & = & 0.5 - 0.2 \nonumber \\ & = & 0.3\nonumber \end{eqnarray} \]

\[ \begin{eqnarray} \Pr(X \in [0, 1] ) & = & \int_{0}^{1} 1 dx \nonumber \\ & = & X |^{1}_{0} \nonumber \\ & = & 1 - 0 \nonumber \\ & = & 1 \nonumber \end{eqnarray} \]

\[ \begin{eqnarray} \Pr(X \in [0.5, 0.5]) & = & \int_{0.5}^{0.5} 1dx \nonumber \\ & = & X|^{0.5}_{0.5} \nonumber \\ & = & 0.5 - 0.5 \nonumber \\ & = & 0 \nonumber \end{eqnarray} \]

\[ \begin{eqnarray} \Pr(X \in \{[0, 0.2]\cup[0.5, 1]\}) & = & \int_{0}^{0.2} 1dx + \int_{0.5}^{1} 1dx \nonumber \\ & = & X_{0}^{0.2} + X_{0.5}^{1} \nonumber \\ & = & 0.2 - 0 + 1 - 0.5 \nonumber \\ & = & 0.7 \nonumber \end{eqnarray} \]

More generally, the PDF of a uniform random variable has the form

\[ f_X(x) = \left\{ \begin{array}{ll} \frac{1}{b-a} & \quad \text{if } a \leq x \leq b \\ 0 & \quad \text{otherwise} \end{array} \right. \]

• To summarize
  • \(\Pr(X = a) = 0\)
  • \(\Pr(X \in (-\infty, \infty) ) = 1\)
  • If \(F\) is antiderivative of \(f\), then \(\Pr(X \in [c,d]) = F(d) - F(c)\) (Fundamental theorem of calculus)

Expectation

The expected value of a continuous random variable \(X\) is defined as

\[\E[X] = \int_{-\infty}^{\infty} x f_X(x) dx\]

This is similar to the discrete case except instead of a summation operation, we use integration to calculate the expected value. If \(X\) is a continuous random variable with a given PDF, any real-valued function \(Y = g(X)\) is also a random variable. The mean of \(g(X)\) satisfies the expected value rule:

\[\E[g(X)] = \int_{-\infty}^{\infty} g(x) f_X(x) dx\]

• The \(n\)th moment is defined as \(\E[X^n]\), the expected value of the random variable \(X^n\)
• The variance, denoted by \(\Var(X)\) is defined as the expected value of the random variable \((X - \E[X])^2 = \E[X^2] - (\E[X])^2\)

Exponential random variable

An exponential random variable has a PDF of the form

\[ f_X(x) = \left\{ \begin{array}{ll} \lambda e^{-\lambda x} & \quad \text{if } x \geq 0 \\ 0 & \quad \text{otherwise} \end{array} \right. \]

where \(\lambda\) is a positive parameter characterizing the PDF.

exp_pdf <- function(rate){
  data_frame(x = seq(0, 20, by = .05),
           pmf = dexp(x = x, rate = rate)) %>%
  ggplot(aes(x, pmf)) +
  geom_line() +
  labs(title = "Exponential PDF",
       subtitle = bquote(lambda == .(rate)),
       x = expression(x),
       y = expression(f[X] (x)))
}

exp_pdf(1) +
  exp_pdf(3) +
  exp_pdf(7)

\[\E[X] = \frac{1}{\lambda}, \quad \Var(X) = \frac{1}{\lambda^2}\]

It is frequently used to model phenomena of a continuous nature such as the time between arrivals (e.g. time between customers arriving at a restaraunt) and the distance between occurrences (e.g. distance between defects in a plate glass window). It is closely associated with the Poisson discrete random variable, which we will return to later.

Example: uniform distribution

• Suppose \(X \sim Uniform(0,1)\), then

\[ \begin{eqnarray} F_X(x) & = & P(X\leq x) \\ & = & 0 \text{, if $x< 0$ } \\ & = & 1 \text{, if $x >1$ } \\ & = & x \text{, if $x \in [0,1]$} \end{eqnarray} \]

unif_pdf <- function(a, b){
  data_frame(x = seq(a - 1, b + 1, by = .01),
             pdf = dunif(x = x, min = a, max = b)) %>%
    ggplot(aes(x, pdf)) +
    geom_line() +
    labs(title = "Uniform PDF",
         subtitle = bquote(a == .(a) ~ b == .(b)),
         x = expression(x),
         y = expression(f[X] (x)))
}

unif_cdf <- function(a, b){
  data_frame(x = seq(a - 1, b + 1, by = .01),
             cdf = punif(q = x, min = a, max = b)) %>%
    ggplot(aes(x, cdf)) +
    geom_line() +
    labs(title = "Uniform CDF",
         subtitle = bquote(a == .(a) ~ b == .(b)),
         x = expression(x),
         y = expression(F[X] (x)))
}

unif_pdf(0, 1) +
  unif_cdf(0, 1)

Properties of CDFs

• \(F_X\) is monotonically nodecreasing – if \(x \leq y\), then \(F_X(x) \leq F_X(y)\)
• \(F_X(x)\) tends to \(0\) as \(x \rightarrow -\infty\), and to \(1\) as \(x \rightarrow \infty\)
• \(F_X(x)\) is a continuous function of \(x\)

• If \(X\) is continuous, the PDF and CDF can be obtained from each other by integration or differentiation

  \[F_X(x) = \int_{-\infty}^x f_X(t) dt, \quad f_X(x) = \frac{dF_X}{dx} (x)\]

Expected value/variance of normal distribution

• \(Z\) is a standard normal distribution if

  \[ \begin{eqnarray} Z & \sim & \text{Normal}(0,1) \nonumber \end{eqnarray} \]

• We’ll call the cumulative distribution function of \(Z\),

  \[ \begin{eqnarray} F_{Z}(x) & = & \frac{1}{\sqrt{2\pi} }\int_{-\infty}^{x} \exp(-z^2/2) dz \end{eqnarray} \]

• Suppose \(Z \sim \text{Normal}(0,1)\)

  • \(Y = 2Z + 6\)

  • \(Y \sim \text{Normal}(6, 4)\)

    data_frame(x = seq(-5, 15, by = .05),
               Z = dnorm(x),
               Y = dnorm(x, mean = 6, sd = 2)) %>%
      gather(norm, val, -x) %>%
      ggplot(aes(x, val, color = norm)) +
      geom_line() +
      scale_color_brewer(type = "qual") +
      labs(x = NULL,
           y = "Density height",
           color = NULL) +
      theme(legend.position = "bottom")

    

• Scale/Location: If \(Z \sim N(0,1)\), then \(X = aZ + b\) is,

  \[ \begin{eqnarray} X & \sim & \text{Normal} (b, a^2) \nonumber \end{eqnarray} \]

• Assume we know:

  \[ \begin{eqnarray} \E[Z] & = & 0 \\ \Var(Z) & = & 1 \end{eqnarray} \]

• This implies that, for \(Y \sim \text{Normal}(\mu, \sigma^2)\)

  \[ \begin{eqnarray} \E[Y] & = & \E[\sigma Z + \mu] \\ & = & \sigma \E[Z] + \mu \nonumber \\ & = & \mu \nonumber \\ \Var(Y) & = & \Var(\sigma Z + \mu) \\ & = & \sigma^2 \Var(Z) + \Var(\mu) \\ & = & \sigma^2 + 0 \\ & =& \sigma^2 \end{eqnarray} \]

This illustrates a key property of normal random variables. If \(X\) is a normal random variable with mean \(\mu\) and variance \(\sigma^2\), and if \(a \neq 0, b\) are scalars, then the random variable

\[Y = aX + b\]

is also normal, with mean and variance

\[\E[Y] = a\mu + b, \quad \Var(Y) = a^2 \sigma^2\]

Why rely on the standard normal distribution

• Normal distribution is commonly used in statistical analysis
  • Many random variables can be approximated with the normal distribution
• Standardizing this makes it easier to make comparisons across variables with different ranges/variances
  • Example - GRE scores
  • Verbal and quant scores have different variability
  • Cannot easily determine if a 159 on the verbal is better or worse than a 159 on the quant
  • The standard normal distribution is unitless, so any random variable can be compared with another random variable
• Saves time on the calculus
  • Don’t need to recalculate the integral when calculating cumulative probability based on the unique \(\mu\) and \(\sigma^2\)
  • Instead, do this once and store the info in a lookup table
  • Made calculating probabilities (relatively) easy before computers

Example: Support for President Obama

Suppose we are interested in modeling presidential approval.

• Let \(Y\) represent random variable: proportion of population who “approves job president is doing”
• Individual responses (that constitute proportion) are independent and identically distributed (sufficient, not necessary) and we take the average of those individual responses
• Observe many responses (\(N\rightarrow \infty\))
• Then (by Central Limit Theorm) \(Y\) is Normally distributed, or

\[ \begin{eqnarray} Y & \sim & \text{Normal}(\mu, \sigma^2) \\ f_Y(y) & = & \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{(y-\mu)^2}{2\sigma^2} \right) \end{eqnarray} \]

• Suppose \(\mu = 0.39\) and \(\sigma^2 = 0.0025\)
• \(\Pr(Y\geq 0.45)\) (What is the probability it isn’t that bad?) ?

\[ \begin{eqnarray} \Pr(Y \geq 0.45) & = & 1 - \Pr(Y \leq 0.45 ) \\ & = & 1 - \Pr(0.05 Z + 0.39 \leq 0.45) \\ & = & 1 - \Pr(Z \leq \frac{0.45-0.39 }{0.05} ) \\ & = & 1 - \frac{1}{\sqrt{2\pi} } \int_{-\infty}^{6/5} \exp(-z^2/2) dz \\ & = & 1 - F_{Z} (\frac{6}{5} ) \\ & = & 0.1150697 \end{eqnarray} \]

Conditioning a random variable on an event

The conditional PDF of a continuous random variable \(X\), given an event \(A\) with \(\Pr (A) > 0\), is defined as a nonnegative function \(f_{X|A}\) that satisfies

\[\Pr (X \in B | A) = \int_B f_{X|A}(x) dx\]

for any subset \(B\) of the real line. By letting \(B\) be the entire real line, we obtain the normalization property

\[\int_{-\infty}^{\infty} f_{X|A} (x) dx = 1\]

so that \(f_{X|A}\) is a legitimate PDF. Based on the definition of conditional probabilities, when we condition on an event of the form \({X \in A}\) with \(\Pr (X \in A) > 0\) we get

\[\Pr (X \in B | X \in A) = \frac{\Pr (X \in B, X \in A)}{\Pr (X \in A)} = \frac{\int_{A \cap B} f_X(x) dx}{\Pr (X \in A)}\]

Therefore the conditional PDF is

\[ f_{X | X \in A} (x) = \left\{ \begin{array}{ll} \frac{f_X (x)}{\Pr (X \in A)} & \quad \text{if } x \in A \\ 0 & \quad \text{otherwise} \end{array} \right. \]

Within the conditioning set, the conditional PDF has exactly the same shape as the unconditional one, except that it is scaled by the constant factor \(1 / \Pr (X \in A)\), so that \(f_{X | \{X \in A\}}\) integrates to 1. Thus, the conditional PDF is similar to an ordinary PDF, except that it refers to a new universe in which the event \(\{ X \in A \}\) is known to have occurred.

data_frame(x = seq(-3, 3, by = .01),
           pdf = dnorm(x = x),
           pdf_cond = ifelse(x > -1 & x < 1, pdf / (pnorm(1) - pnorm(-1)), 0)) %>%
  gather(pdf, value, -x) %>%
  ggplot(aes(x, value, color = pdf)) +
  geom_line() +
  scale_color_brewer(type = "qual", labels = c(expression(f[X](x)), expression(f[X*"|"*X %in% A](x)))) +
  labs(title = "Standard normal random variable",
       x = expression(X),
       y = "PDF",
       color = NULL) +
  theme(legend.position = "bottom")

Weather forecasting

According to British weather forecasters, the average monthly rainfall in London during the month of June is \(\mu = 2.09\) inches. Assume the monthly precipitation is a normally-distributed random variable with a standard deviation of \(\sigma = 0.48\) inches.

pnorm(q = 2.5, mean = 2.09, sd = 0.48) - pnorm(q = 1.5, mean = 2.09, sd = 0.48)

## [1] 0.694

pnorm(q = 1, mean = 2.09, sd = 0.48)

## [1] 0.0116

qnorm(p = .95, mean = 2.09, sd = 0.48)

## [1] 2.88

Booking travel (redux)

Book4Less.com is an online travel website that offers competitive prices on airline and hotel bookings. During a typical weekday, the website averages 10 visits per minute.

1 - pexp(q = 18, rate = 1/6)

## [1] 0.0498

dpois(x = 0, lambda = 3)

## [1] 0.0498