Probability review for warm up

Probability Review

In this section, let's go through a quick review about probability theory. The best way to refresh ourselves is to read a few notes and go straight to problem exercises. From your EEE 137 we can summarize probability as a mathematical term which we use to investigate properties of mathematical models of chance phenomena. It is also a generalized notion of weights whereby we weigh events to see how likely they are to occur. In most cases probability is dependent on the relative frequency of events, while some look at fractions based on sets. Let's review some important properties and definitions then proceed immediately to practice exercises.

Basic Properties of Probability

Suppose we have events $A$ and $B$ which are subsets of a sample space $S$ (i.e., $A,B\subset S$ ). Let $P(A)$ be the probability that event $A$ happens, and $P(B)$ be the probability that event $B$ happens. Then some of the basic properties follow:

1. $0\leq P(A)\leq 1$ and $0\leq P(B)\leq 1$
2. $P(S)=1$ and $P(\varnothing )=0$. The $\varnothing$ is the null or empty set.
3. $P(A\cap B)=P(A)+P(B)$ if and only if $A$ and $B$ are disjoint.
4. $P(A-B)=P(A)-P(B)$ if $B\subseteq A$ and vice versa.
5. $P({\bar {A}})=1-P(A)$ and $P({\bar {B}})=1-P(B)$
6. $P(A)\leq P(B)$ whenever $A\subseteq B$ and vice versa.
7. $P(A\cup B)=P(A)+P(B)-P(A\cap B)$. This can be extended to $N$ sets or variables. This one is left as an exercise for you.
8. Let's say $C=\{E_{1},E_{2},E_{3}...,E_{n}\}$ is a partition of $S$ then $\sum _{j=1}^{n}P(E_{j})=1$.

Principle of Symmetry

Let $S$ be a finite sample space with outcomes or events $\{E_{1},E_{2},E_{3},...,E_{n}\}$ which all $E_{i}$ are physically identical or objects having the same properties and characteristics. In this case we have:

$P(E_{1})=P(E_{2})=P(E_{3})=...=P(E_{n})$.

Subjective Probabilities

These are often expressed in terms of odds. For example, suppose a betting site is offering odds of $x$ to $y$ on Team Secret beating Team TSM. This means out of the total $x+y$ equally valued coins, the better is willing to bet $x$ of them that Team Secret beats Team TSM. So if the outcome $s$ is the event that Team Secret beats Team TSM then we have:

$P(s)={\frac {x}{x+y}}$

Relative Frequency

Suppose we are monitoring a particular outcome $x$ and we observe that $x$ occurs in $r$ out of $n$ experiments (or repetitions). We define the relative frequency of $x$ based on $n$ experiments as:

$P(x)={\frac {r}{n}}$

Conditional Probability

In a nutshell, conditional probability is to gain in information about an event that leads to a change in its probability. Suppose an event $A$ happens with probability $P(A)$. However, when event $B$ happens, this influences the probability of $A$ such that we have $P(A|B)$ as the conditional probability of $A$ happening given $B$ occurred. Mathematically we know this as:

$P(A|B)={\frac {P(A\cap B)}{P(B)}}$

There are some interesting formulas to take note of. For example, if $A$, $B$, and $C$ are some events in a sample space, then we can compute the intersection of all events as:

$P(A\cap B\cap C)=P(C|A\cap B)P(B|A)P(A)$

File:Partition set.PNG

Figure 1: A sample space S cut into several E partitions. Blue is event A, red is event B, green is event C, gray is event D, and orange is event E.

The pattern carries over and you could generalize this. Here's another interesting formula: Suppose we have a partition set $Y=\{E_{1},E_{2},E_{3},...E_{n}\}$ of some sample space $S$ with each $P(E_{i})\neq 0$. In other words, we just cut the sample space $S$ into several partitions. Let event $A\ \epsilon \ Y$. In other words, event $A$ is just a part of the partition $Y$. We have:

$P(A)=\sum _{i=1}^{n}P_{E_{i}}(A)P(E_{i})$

Figure 1 visualizes this formula. The entire space is $S$ and we cut it into several $E_{i}$ components. The larger color-coded chunks are different events. Suppose the blue chunk is event $A$.