161-A1.1

Let's look at a few applications of the concept of information and entropy.

Student Grading

How much information can we get from a single grade? Note that the maximum information occurs when all the grades have equal probability.

• For Pass/Fail grades, the possible outcomes are: $\{P,F\}$ with probabilities $\{{\tfrac {1}{2}},{\tfrac {1}{2}}\}$. Thus,

$H\left(P\right)=\sum _{i=1}^{n}p_{i}\cdot \log _{2}\left({\frac {1}{p_{i}}}\right)={\frac {1}{2}}\cdot \log _{2}\left(2\right)+{\frac {1}{2}}\cdot \log _{2}\left(2\right)=1\,\mathrm {bit}$

(1)

• For grades = $\{1.0,2.0,3.0,4.0,5.0\}$ with probabilities $\{{\tfrac {1}{5}},{\tfrac {1}{5}},{\tfrac {1}{5}},{\tfrac {1}{5}},{\tfrac {1}{5}}\}$, we get:

$H\left(P\right)=\sum _{i=1}^{n}p_{i}\cdot \log _{2}\left({\frac {1}{p_{i}}}\right)=5\cdot {\frac {1}{5}}\cdot \log _{2}\left(5\right)=2.32\,\mathrm {bit}$

(2)