# An Introduction to Information Theory

Communication 101: Information Theory Made REALLY SIMPLE by Perry Marshall

Claude Shannon’s 1948 paper “A Mathematical Theory of Communication” is the paper that made the digital world we live in possible. Scientific American called it “The Magna Carta of the Information Age.”

Shannon defined modern digital communication and determined things like how much information can be transmitted over a telephone line, the effects of noise on the signal, and the measures you have to take to get a perfect signal on the other end. It made the Internet possible.

Trouble is, it’s tough reading – college level material for engineers and math geeks. HOWEVER Shannon’s concepts are simple and easy to explain. In just a few minutes you’ll understand Shannon’s concepts and you’ll see that any 7th grader can easily grasp them.

2. Say communication is not expensive, but error prone. In particular, each bit $i$ of your message is erroneously flipped with some known probably $p$, and all the errors are independent. Then the question is, how can one encode their messages to as to guarantee (with high probability) the ability to decode any sent message? This is called the noisy coding problem.