161-A1.1

Let's look at a few applications of the concept of information and entropy.

Student Grading

How much information can we get from a single grade? Note that the maximum information occurs when all the grades have equal probability.

• For Pass/Fail grades, the possible outcomes are: $\{P,F\}$ with probabilities $\{{\tfrac {1}{2}},{\tfrac {1}{2}}\}$. Thus,

$H\left(P\right)=\sum _{i=1}^{n}p_{i}\cdot \log _{2}\left({\frac {1}{p_{i}}}\right)={\frac {1}{2}}\cdot \log _{2}\left(2\right)+{\frac {1}{2}}\cdot \log _{2}\left(2\right)=1\,\mathrm {bit}$

(10)