Difference between revisions of "161-A1.1"

From Microlab Classes
Jump to navigation Jump to search
Line 1: Line 1:
−
Let's look at a few applications of the concept of information and entropy.
+
* Activity: '''Who is Claude Shannon?'''
 +
* '''Instructions:''' In this activity, you are tasked to
 +
** Read two articles on Claude Shannon.
 +
** Write a short (1-page) report.
 +
* Should you have any questions, clarifications, or issues, please contact your instructor as soon as possible.  
  
−
== Surprise! The Unexpected Observation ==
+
== Articles ==
−
Information can be thought of as the amount of ''surprise'' at seeing an event. Note that a highly probable outcome is not surprising. Consider the following events:
+
* John Horgan, ''Claude Shannon: Tinkerer, Prankster, and Father of Information Theory'', IEEE Spectrum, 2016 ([https://spectrum.ieee.org/tech-history/cyberspace/claude-shannon-tinkerer-prankster-and-father-of-information-theory link])
 +
* Graham P. Collins, ''Claude E. Shannon: Founder of Information Theory'', Scientific American, October 14, 2002 ([https://www.scientificamerican.com/article/claude-e-shannon-founder/ link])
  
−
{| class="wikitable"
+
== Report Guide ==
−
|-
+
Based on the the papers (not limited to the two papers above) you have read, and any other resource that is available to you, write a short (1-page) report on the possible applications of information theory in the field of computer engineering.
−
! Event
 
−
! Probability
 
−
! Information (Surprise)
 
−
|-
 
−
|Someone tells you <math>1=1</math>.
 
−
|<math>1</math>
 
−
|<math>\log_2\left(1\right) = 0</math>
 
−
|-
 
−
| You got the wrong answer on a 4-choice multiple choice question.
 
−
|<math>\frac{3}{4}</math>
 
−
|<math>\log_2\left(\frac{4}{3}\right)=0.415\,\mathrm{bits}</math>
 
−
|-
 
−
| You guessed correctly on a 4-choice multiple choice question.
 
−
|<math>\frac{1}{4}</math>
 
−
|<math>\log_2\left(4\right)=2\,\mathrm{bits}</math>
 
−
|-
 
−
| You got the correct answer in a True or False question.
 
−
|<math>\frac{1}{2}</math>
 
−
|<math>\log_2\left(2\right)=1\,\mathrm{bit}</math>
 
−
|-
 
−
| You rolled a seven on rolling a pair of dice.
 
−
|<math>\frac{6}{36}</math>
 
−
|<math>\log_2\left(6\right)=2.58\,\mathrm{bits}</math>
 
−
|-
 
−
| Winning the Ultra Lotto 6/58 jackpot.
 
−
|<math>\frac{1}{40400000}</math>
 
−
|<math>\log_2\left(40400000\right)=25.27\,\mathrm{bits}</math>
 
−
|-
 
−
|}
 
  
−
== Student Grading ==
+
=== Submission ===
−
How much information can we get from a single grade? Note that the maximum entropy occurs when all the grades have equal probability.
+
Submit your report via email before proceeding to Module 2.
−
* For Pass/Fail grades, the possible outcomes are: <math>\{\mathrm{P}, \mathrm{F}\}</math> with probabilities <math>\{\tfrac{1}{2}, \tfrac{1}{2}\}</math>. Thus,
 
−
 
 
−
{{NumBlk|::|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = \frac{1}{2}\cdot \log_2\left(2\right) + \frac{1}{2}\cdot \log_2\left(2\right) = 1\,\mathrm{bit}</math>|{{EquationRef|1}}}}
 
−
 
 
−
* For grades = <math>\{1.00, 2.00, 3.00, 4.00, 5.00\}</math> with probabilities <math>\{\tfrac{1}{5}, \tfrac{1}{5}, \tfrac{1}{5}, \tfrac{1}{5}, \tfrac{1}{5}\}</math>, we get:
 
−
 
 
−
{{NumBlk|::|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = 5\cdot \frac{1}{5}\cdot \log_2\left(5\right) = 2.32\,\mathrm{bits}</math>|{{EquationRef|2}}}}
 
−
 
 
−
* For grades = <math>\{1.00, 1.50, 2.00, 2.50, 3.00, 4.00, 5.00\}</math> with probabilities <math>\{\tfrac{1}{7}, \tfrac{1}{7}, \tfrac{1}{7}, \tfrac{1}{7}, \tfrac{1}{7}, \tfrac{1}{7}, \tfrac{1}{7}\}</math>, we have:
 
−
 
 
−
{{NumBlk|::|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = 7\cdot \frac{1}{7}\cdot \log_2\left(7\right) = 2.81\,\mathrm{bits}</math>|{{EquationRef|3}}}}
 
−
 
 
−
* If we have all the possible grades <math>\{1.00, 1.25, 1.50, 1.75, 2.00, 2.25, 2.50, 2.75, 3.00, 4.00, 5.00, \mathrm{INC}, \mathrm{DRP}, \mathrm{LOA}\}</math> with probabilities <math>\{\tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}, \tfrac{1}{14}\}</math>, we have:
 
−
 
 
−
{{NumBlk|::|<math>H\left(P\right)=\sum_{i=1}^n p_i\cdot \log_2\left(\frac{1}{p_i}\right) = 14\cdot \frac{1}{14}\cdot \log_2\left(14\right) = 3.81\,\mathrm{bits}</math>|{{EquationRef|4}}}}
 

Revision as of 00:58, 14 September 2020

  • Activity: Who is Claude Shannon?
  • Instructions: In this activity, you are tasked to
    • Read two articles on Claude Shannon.
    • Write a short (1-page) report.
  • Should you have any questions, clarifications, or issues, please contact your instructor as soon as possible.

Articles

  • John Horgan, Claude Shannon: Tinkerer, Prankster, and Father of Information Theory, IEEE Spectrum, 2016 (link)
  • Graham P. Collins, Claude E. Shannon: Founder of Information Theory, Scientific American, October 14, 2002 (link)

Report Guide

Based on the the papers (not limited to the two papers above) you have read, and any other resource that is available to you, write a short (1-page) report on the possible applications of information theory in the field of computer engineering.

Submission

Submit your report via email before proceeding to Module 2.

Retrieved from "https://classes.up-microlab.org/index.php?title=161-A1.1&oldid=1215"