Difference between revisions of "Channeling your inner capacity"
Ryan Antonio (talk | contribs) |
Ryan Antonio (talk | contribs) |
||
| Line 1: | Line 1: | ||
== Channel Basics== | == Channel Basics== | ||
| − | When Shannon developed his theory of information, it was in the context of improving communication. His goal was to determine if there was a way to maximize the transmission rates over some noisy channel. In this module, we will learn more about the channel and some formal terminologies that we will | + | When Shannon developed his theory of information, it was in the context of improving communication. His goal was to determine if there was a way to maximize the transmission rates over some noisy channel. In this module, we will learn more about the channel and some formal terminologies that we will use in the succeeding modules. The GIF below recalls the story of Bob and Alice. Bob wants to send a message to Alice across some channel. Let's simplify the scenario. Let's say Bob just wants to send a love letter to Alice through some wireless (satellite based) channel. |
[[File:Bob alice nodencode.gif|center]] | [[File:Bob alice nodencode.gif|center]] | ||
| − | [[File:Simple channel.PNG| | + | [[File:Simple channel.PNG|500px|thumb|left|Figure 1: Simplified communication model]] |
| + | |||
| + | |||
Figure 2 shows a simplified model of their communication. Bob is the '''source'''. Sometimes we use the term '''sender''' or '''transmitter''' for the source. The medium where the message goes through is the '''channel''' (in this case the wireless channel). Alice is the '''receiver''' who probably gets the correct message at the other end of the channel. Let's take a look at each component formally. | Figure 2 shows a simplified model of their communication. Bob is the '''source'''. Sometimes we use the term '''sender''' or '''transmitter''' for the source. The medium where the message goes through is the '''channel''' (in this case the wireless channel). Alice is the '''receiver''' who probably gets the correct message at the other end of the channel. Let's take a look at each component formally. | ||
| − | === The | + | |
| + | |||
| + | |||
| + | === The Source === | ||
| + | |||
| + | The source is the component that produces messages or objects sent over a channel. We represent the source as a random variable <math> S </math> which contains the outcomes <math> \{s_1, s_2, s_3, ... , s_n \} </math> and each outcome has its own probability distribution <math>\{P(s_1), P(s_2), P(s_3), ... , P(s_n) \}</math>. The random variable <math> S </math> is also called the '''source alphabet''' and the outcomes are the '''symbols''' of the source alphabet. A combinational sequence of outcomes forms the '''message''' that we send over the channel. Consider the following examples: | ||
| + | |||
| + | * We can let the source alphabet <math> S </math> be the English alphabet where the symbols are all the letters from a to z including the space. Suppose we want to send the message "the quick brown fox jumps over the lazy dog". All letters of the English alphabet has a probability distribution associated with it. The second programming exercises did the exact same method for English, German, French, and Tagalog languages. | ||
| + | |||
| + | * We can let <math> S </math> be the binary alphabet where the symbols are <math>\{ 1, 0\} </math>. The message could be streams of 1s and 0s that could mean something. For example, a message could be a sequence of events when a light is on or off. It could also be an indicator for sending decimal values in the form of binary digits. | ||
| + | |||
| + | * In biology, we can let <math> S </math> be the DNA bases whose symbols are <math> \{A,C,G,T\} </math>. <math> A </math> is adanine, <math> C </math> is cytosine, <math> G </math> is guanine, and <math> T </math> is thymine. These symbols combine to form DNA sequences that are messages to instruct what kind of protein synthesis should a cell do. | ||
| + | |||
| + | Make sure to understand carefully what source alphabets, symbols, and messages mean. | ||
==Binary Symmetric Channels== | ==Binary Symmetric Channels== | ||
Revision as of 23:34, 25 February 2022
Contents
Channel Basics
When Shannon developed his theory of information, it was in the context of improving communication. His goal was to determine if there was a way to maximize the transmission rates over some noisy channel. In this module, we will learn more about the channel and some formal terminologies that we will use in the succeeding modules. The GIF below recalls the story of Bob and Alice. Bob wants to send a message to Alice across some channel. Let's simplify the scenario. Let's say Bob just wants to send a love letter to Alice through some wireless (satellite based) channel.
Figure 2 shows a simplified model of their communication. Bob is the source. Sometimes we use the term sender or transmitter for the source. The medium where the message goes through is the channel (in this case the wireless channel). Alice is the receiver who probably gets the correct message at the other end of the channel. Let's take a look at each component formally.
The Source
The source is the component that produces messages or objects sent over a channel. We represent the source as a random variable which contains the outcomes and each outcome has its own probability distribution . The random variable is also called the source alphabet and the outcomes are the symbols of the source alphabet. A combinational sequence of outcomes forms the message that we send over the channel. Consider the following examples:
- We can let the source alphabet be the English alphabet where the symbols are all the letters from a to z including the space. Suppose we want to send the message "the quick brown fox jumps over the lazy dog". All letters of the English alphabet has a probability distribution associated with it. The second programming exercises did the exact same method for English, German, French, and Tagalog languages.
- We can let be the binary alphabet where the symbols are . The message could be streams of 1s and 0s that could mean something. For example, a message could be a sequence of events when a light is on or off. It could also be an indicator for sending decimal values in the form of binary digits.
- In biology, we can let be the DNA bases whose symbols are . is adanine, is cytosine, is guanine, and is thymine. These symbols combine to form DNA sequences that are messages to instruct what kind of protein synthesis should a cell do.
Make sure to understand carefully what source alphabets, symbols, and messages mean.
