Bernoulli R.V

${\displaystyle P_{X}(x)={\begin{cases}1-p&x=0\\p&x=1\\0&o.w.\end{cases}}\!\,}$

${\displaystyle 0\leq p\leq 1\!\,}$ success probability

Geometric Random Variable

Number of trials until (and including) a success for an underlying Bernoulli

${\displaystyle P_{X}(x)=p(1-p)^{x-1}\quad x=1,2,3,\ldots \!\,}$

Binomial R.V

"# of successes in n trials"

${\displaystyle P_{x}(x)={n \choose x}p^{x}(1-p)^{n-x}\quad x=0,1,\ldots n\!\,}$

Pascal R.V

"number of trials until (and including) the kth success with an underlying Bernoulli"

${\displaystyle P_{X}(x)={x-1 \choose k-1}p^{k}(1-p)^{x-k}\quad x=k,k+1,k+2,\ldots \!\,}$

where ${\displaystyle {x-1 \choose k-1}\!\,}$ is ${\displaystyle k-1\!\,}$ successes in ${\displaystyle x-1\!\,}$ trials

Note: Pascal ${\displaystyle X=Y_{1}+Y_{2}+\ldots +Y_{k}\!\,}$ where ${\displaystyle Y_{1},Y_{2},\ldots Y_{k}\!\,}$ are geometric R.V.

Note: K=1 Pascal=Geometric

Discrete Uniform R.V.

${\displaystyle P_{X}(x)={\begin{cases}{\frac {1}{b-a+1}}&x=a,a+1,\ldots ,b\\0&{\mbox{otherwise}}\end{cases}}\!\,}$

Poisson R.V.

${\displaystyle P_{X}(x)=e^{-\alpha }{\frac {\alpha ^{x}}{x!}}\quad x=0,1,2\ldots \!\,}$

(Exercise) limiting case of binomial with ${\displaystyle n\rightarrow \infty ,p\rightarrow 0,np=\alpha \!\,}$

PMF is a complete model for a random variable

Example

Flip the coins ${\displaystyle X=\!\,}$# of H

${\displaystyle P_{X}(x)={\begin{cases}{\frac {1}{4}}&x=0\\{\frac {1}{2}}&x=1\\{\frac {1}{2}}&x=2\\0&{\mbox{otherwise}}\end{cases}}\!\,}$

${\displaystyle {\begin{aligned}P[X\leq 0]&=P[X=0]\\P[X\leq 0.5]&=P[X=0]\\P[X\leq 1]&=\underbrace {P[X=0]} _{\frac {1}{4}}+\underbrace {P[X=1]} _{\frac {1}{2}}\\\end{aligned}}\!\,}$

Properties of CDF

• a) ${\displaystyle F_{X}(-\infty )=0\Leftarrow F_{X}(-\infty )=P[X\leq -\infty ]=0\!\,}$

${\displaystyle F_{X}(\infty )=1\Leftarrow F_{X}(\infty )=P[X\leq \infty ]=1\!\,}$

"starts at 0 and ends at 1"

• b) For all ${\displaystyle x'\geq x\!\,}$, ${\displaystyle F_{X}(x')\geq F_{X}(x)\!\,}$

"non-decreasing in x"

${\displaystyle {\begin{aligned}F_{X}(x')&\quad F_{X}(x)\\P[X\leq x']&\quad P[X\leq x]\\P[s:X(s)\leq x']&\quad P[s:X(s)\leq x]\\\{s:X(s)\leq x\}&\subset \{s:X(s)\leq x'\}\\P[X\leq x]&\leq P[X\leq x']\\F_{X}(x)&\leq F_{X}(x')\end{aligned}}\!\,}$

• c) For all ${\displaystyle x,x'\!\,}$

${\displaystyle P[x\leq X\leq x']=F_{X}(x')-F_{X}(x)\!\,}$

"probabilities can be found by difference of the CDF"

${\displaystyle \{s:X(x)\leq x'\}=\{s:X(x)\leq x\}\cup \{s:x\leq X(s)\leq x'\}\!\,}$

${\displaystyle P[X\leq x']=P[X\leq x]+P[x\leq X\leq x']\!\,}$

• d) For all ${\displaystyle x\!\,}$,

${\displaystyle \lim _{\epsilon \rightarrow 0}F_{X}(x+\epsilon )=F_{X}(x)\!\,}$

"CDF is right continuous"

• e) For ${\displaystyle x_{i}\in S_{X}\!\,}$

${\displaystyle F_{X}(x_{i})-F_{X}(x_{i}-\epsilon )=P_{X}(x_{i})\!\,}$

"For a discrete random variable, there is a jump (discontinuity) in the CDF at each value ${\displaystyle x_{i}\in S_{X}\!\,}$. This jump equals ${\displaystyle P_{X}(x_{i})\!\,}$

${\displaystyle P[x_{i}\leq x]-P[x_{i}\leq x_{i}-\epsilon ]=P[x_{i}-\epsilon <x<x_{i}]=P[X=x_{i}]\!\,}$

• f) ${\displaystyle F_{X}(x)=F_{X}(x_{i})\!\,}$ for all ${\displaystyle x_{i}\leq x\leq x_{i+1}\!\,}$

"Between two jumps the CDF is constant"

${\displaystyle {\begin{aligned}P[X\leq x]&=P[X\leq x_{i}\cup x_{i}<X\leq x]\\&=P[X\leq x_{i}+\underbrace {P[x_{i}<X\leq x]} _{0}\\&=F_{X}(x_{i})\end{aligned}}}$

• g) ${\displaystyle P[X>x]=1-\underbrace {F_{X}(x)} _{P[X\leq x]}\!\,}$

Example

T: arrival of a partical

${\displaystyle S_{T}=\{t:0\leq t<\infty \}\!\,}$

V: voltage

${\displaystyle S_{V}=\{v:-\infty <v<\infty \}\!\,}$

${\displaystyle \theta \!\,}$: angle

${\displaystyle S_{\theta }=\{\theta :0\leq \theta \leq 2\pi \}\!\,}$

${\displaystyle X\!\,}$: distance

${\displaystyle S_{X}=\{X:0\leq x\leq 1\}\!\,}$

${\displaystyle P[x\in A]={\frac {1}{n}}\rightarrow 0\!\,}$

No PMF, ${\displaystyle P[X=x]=0\forall x\!\,}$

Theorem

For any random variable (continuous or discrete)

• a) ${\displaystyle F_{X}(-\infty )=0\quad F_{X}(\infty )=1\!\,}$

• b) ${\displaystyle F_{X}(x)\!\,}$ is nondecreasing in ${\displaystyle X\!\,}$

• c) ${\displaystyle P[x<X\leq x']=F_{X}(x')-F_{X}(x)\!\,}$

• d) ${\displaystyle F_{X}(x)\!\,}$ is right continuous

Example

${\displaystyle S_{X}=[0,1]\!\,}$

${\displaystyle P_{[}x\in A]=P[x\ inB]\!\,}$ where A, B are intervals of the same length contained in [0,1]

${\displaystyle P[X\leq 1]=1\Leftrightarrow F_{X}(1)=1\!\,}$

${\displaystyle P[X\leq 0]=0\Leftrightarrow F_{X}(0)=0\!\,}$

${\displaystyle P[x_{1}<x<x_{2}]=P[0<x<x_{2}-x_{1}]\!\,}$

(exercise)${\displaystyle F_{X}(x_{2})-F_{X}(x_{1})=F_{X}(x_{2}-x_{1})\rightarrow F_{x}(x)=x\!\,}$

Probability Density Function (PDF)

${\displaystyle f_{X}(x)={\frac {dF_{x}(x)}{dx}}\!\,}$

discrete: PMF <--> CDF (sum/difference)

continuous <---> (derivative/integral)

Theorem: Properties of PDF

• a) ${\displaystyle f_{X}(x)\geq 0\!\,}$ (${\displaystyle F_{X}(x)\!\,}$ is nondecreasing)

• b) ${\displaystyle F_{X}(x)=\int _{\infty }^{x}f_{X}(x)dx\quad (F_{X}(\infty )=0)\!\,}$

• c) ${\displaystyle \int _{-\infty }^{\infty }f_{X}(x)dx=1\quad (F_{x}(\infty )=1)\!\,}$

Theorem

${\displaystyle {\begin{aligned}P[x_{1}\leq X\leq x_{2}]&=F_{X}(x_{2})-F_{X}(x_{1})\\&=\int _{-\infty }^{x_{2}}f_{X}(x)dx-\int _{-\infty }^{x_{1}}f_{X}(x)dx\\&=\int _{x_{1}}^{x_{2}}f_{X}(x)\end{aligned}}\!\,}$

Some useful continuous Random Variables