Difference between revisions of "161-A1.1"

Revision as of 23:07, 13 September 2020

Let's look at a few applications of the concept of information and entropy.

How much information can we get from a single grade? Note that the maximum information occurs when all the grades have equal probability.

For Pass/Fail grades, the possible outcomes are: $\{P,F\}$ with probabilities $\{{\tfrac {1}{2}},{\tfrac {1}{2}}\}$ . Thus,

H\left(P\right)=\sum _{i=1}^{n}p_{i}\cdot \log _{2}\left({\frac {1}{p_{i}}}\right)={\frac {1}{2}}\cdot \log _{2}\left(2\right)+{\frac {1}{2}}\cdot \log _{2}\left(2\right)=1\,\mathrm {bit}

(10)

Revision as of 23:06, 13 September 2020 (view source) Louis Alarcon (talk \| contribs) ← Older edit		Revision as of 23:07, 13 September 2020 (view source) Louis Alarcon (talk \| contribs) Newer edit →
Line 5:		Line 5:
	* For Pass/Fail grades, the possible outcomes are: <math>\{P, F\}</math> with probabilities <math>\{\tfrac{1}{2}, \tfrac{1}{2}\}</math>. Thus,		* For Pass/Fail grades, the possible outcomes are: <math>\{P, F\}</math> with probabilities <math>\{\tfrac{1}{2}, \tfrac{1}{2}\}</math>. Thus,

−	{{NumBlk\|::\|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = \frac{1}{2}\cdot \log_2\left(2\right) + \frac{1}{2}\cdot \log_2\left(2\right) = 1\mathrm{bit}</math>\|{{EquationRef\|10}}}}	+	{{NumBlk\|::\|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = \frac{1}{2}\cdot \log_2\left(2\right) + \frac{1}{2}\cdot \log_2\left(2\right) = 1\,\mathrm{bit}</math>\|{{EquationRef\|10}}}}