Probability review for warm up

Probability Review

In this section, let's go through a quick review about probability theory. The best way to refresh ourselves is to read a few notes and go straight to problem exercises. From your EEE 137 we can summarize probability as a mathematical term which we use to investigate properties of mathematical models of chance phenomena. It is also a generalized notion of weights whereby we weigh events to see how likely they are to occur. In most cases probability is dependent on the relative frequency of events, while some look at fractions based on sets. Let's review some important properties and definitions then proceed immediately to practice exercises.

Basic Properties of Probability

Suppose we have events $A$ and $B$ which are subsets of a sample space $S$ (i.e., $A,B\subset S$ ). Let $P(A)$ be the probability that event $A$ happens, and $P(B)$ be the probability that event $B$ happens. Then some of the basic properties follow:

1. $0\leq P(A)\leq 1$ and $0\leq P(B)\leq 1$
2. $P(S)=1$ and $P(\varnothing )=0$. The $\varnothing$ is the null or empty set.
3. $P(A\cap B)=P(A)+P(B)$ if and only if $A$ and $B$ are disjoint.
4. $P(A-B)=P(A)-P(B)$ if $B\subseteq A$ and vice versa.
5. $P({\bar {A}})=1-P(A)$ and $P({\bar {B}})=1-P(B)$
6. $P(A)\leq P(B)$ whenever $A\subseteq B$ and vice versa.
7. $P(A\cup B)=P(A)+P(B)-P(A\cap B)$. This can be extended to $N$ sets or variables. This one is left as an exercise for you.
8. Let's say $C=\{E_{1},E_{2},E_{3}...,E_{n}\}$ is a partition of $S$ then $\sum _{j=1}^{n}P(E_{j})=1$.

Principle of Symmetry

Let $S$ be a finite sample space with outcomes or events $\{E_{1},E_{2},E_{3},...,E_{n}\}$ which all $E_{i}$ are physically identical or objects having the same properties and characteristics. In this case we have:

$P(E_{1})=P(E_{2})=P(E_{3})=...=P(E_{n})$.

Subjective Probabilities

These are often expressed in terms of odds. For example, suppose a betting site is offering odds of $x$ to $y$ on Team Secret beating Team TSM. This means out of the total $x+y$ equally valued coins, the better is willing to bet $x$ of them that Team Secret beats Team TSM. So if the outcome $s$ is the event that Team Secret beats Team TSM then we have:

$P(s)={\frac {x}{x+y}}$

Relative Frequency

Suppose we are monitoring a particular outcome $x$ and we observe that $x$ occurs in $r$ out of $n$ experiments (or repetitions). We define the relative frequency of $x$ based on $n$ experiments as:

$P(x)={\frac {r}{n}}$

Conditional Probability

Figure 1: Simple Venn diagram showing the fractional parts for computing $P(A|B)$

In a nutshell, conditional probability is to gain in information about an event that leads to a change in its probability. Suppose an event $A$ happens with probability $P(A)$. However, when event $B$ happens, this influences the probability of $A$ such that we have $P(A|B)$ as the conditional probability of $A$ happening given $B$ occurred. Mathematically we know this as:

$P(A|B)={\frac {P(A\cap B)}{P(B)}}$

Figure 1 shows a Venn diagram that can visualize this. The $P(A|B)$ is the relative probability of event $A$ happening with respect to event $B$ happening. Therefore, it justifies that we first need to get $P(A\cap B)$ then dividing this by $P(B)$. It's just a matter of shifting the scope of event $A$ happening with respect to the entire space $S$ to the scope of event $A$ happening with respect to the space of event $B$. We can rearrange the equation to get:

$P(A\cap B)=P(A|B)P(B)$

We can extend this concept to three or more sets. For example, if $A$, $B$, and $C$ are some events in a sample space, then we can compute the intersection of all events as:

$P(A\cap B\cap C)=P(C|A\cap B)P(B|A)P(A)$

Figure 2: Simple partition example. The entire space $S$ is cut into different $E_{i}$ parts. The event $A$ is a subset of the space and it constitutes different fractions of $E_{i}$ components.

Here's another interesting formula: Suppose we have a partition set $Y=\{E_{1},E_{2},E_{3},...E_{n}\}$ of some sample space $S$ with each $P(E_{i})\neq 0$. In other words, we just cut the sample space $S$ into several partitions. Let event $A\ \epsilon \ Y$. In other words, event $A$ is just a part of the partition $Y$. We have:

$P(A)=\sum _{i=1}^{n}P(A|E_{i})P(E_{i})$

Figure 2 visualizes this formula. The entire space is $S$ and we cut it into several $E_{i}$ components. Suppose event $A$ is also part of the entire space $S$, then it's simply the sum of all $P(E_{i}\cap A)=P(A|E_{i})P(E_{i})$ components. You might wonder why the summation is from $i=1$ up to $i=n$ when we could just get the probabilities where it only matters? You are correct to think that we only need to get those that have a contribution to $P(A)$ but the equation is generalized to include all partitions. For example, $P(E_{6}\cap A)=0$ because event $E_{6}$ and event $A$ can never happen. So it's okay to generalize the formula to include all events even when their intersections don't happen at all.