Let's look at a few applications of the concept of information and entropy.
Student Grading
How much information can we get from a single grade? Note that the maximum information occurs when all the grades have equal probability.
- For Pass/Fail grades, the possible outcomes are: with probabilities . Thus,
-
|
|
(1)
|
- For grades = with probabilities , we get:
-
|
|
(2)
|