Probability Review I

Notions of Probability

Let us look at two notions of the probability of an event happening:

The Frequentist Probability

Given $N$ distinct possible events, $\{x_{1},x_{2},\ldots ,x_{N}\}$, with the following assumptions:

• No two events can occur simultaneously, and
• The events occur with frequencies $\{n_{1},n_{2},\ldots ,n_{N}\}$,

Then the probability of an event $x_{i}$ is given by:

$P\left(x_{i}\right)={\frac {n_{i}}{\sum _{j=1}^{N}n_{j}}}$

(1)

From this definition, we can see that:

$\sum _{i=1}^{N}P\left(x_{i}\right)=1$

(2)

The Observer Relative Probability

Probability is an assertion about the belief that a specific observer has of the occurrence of a specific event. Thus, here, two different observers may assign different probabilities to the same event or phenomenon. Additionally, the probability assigned to an event is likely to change as the observer learns more about an event, or if certain aspects of the event changes.

We can think of the observer relative probability of an event to be an approximation to the frequentist probability, and additionally, we can view new knowledge as stepping stones towards better estimates of the relative frequencies of an event.

Probability Basics

Recall that for two events, $A$ and $B$:

$P\left({\bar {A}}\right)=1-P\left(A\right)$

(3)

$P\left(A\cup B\right)=P\left(A\right)+P\left(B\right)-P\left(A\cap B\right)$

(4)

Note that if $P\left(A\cap B\right)=P\left(A,B\right)=0$, then $A$ and $B$ are mutually exclusive.

Conditional Probability

We define $P\left(A\mid B\right)$ as the probability of $A$ occurring given that we know $B$ occurred. Thus, the joint probability of $A$ and $B$ is given by:

$P\left(A,B\right)=P\left(A\mid B\right)\cdot P\left(B\right)$

(5)

Since $P\left(A,B\right)=P\left(B,A\right)$, we arrive at Bayes' Theorem:

$P\left(A\mid B\right)\cdot P\left(B\right)=P\left(B\mid A\right)\cdot P\left(A\right)$

(6)

Or equivalently:

$P\left(A\mid B\right)={\frac {P\left(B\mid A\right)\cdot P\left(A\right)}{P\left(B\right)}}$

(7)

If $P\left(A\mid B\right)=P\left(A\right)$, then $A$ and $B$ are independent. From Bayes' Theorem, we get $P\left(B\mid A\right)=P\left(B\right)$. Thus, for independent events,

$P\left(A,B\right)=P\left(A\right)\cdot P\left(B\right)$

(8)