Entropy in information theory measures the uncertainty or unpredictability in a message or data source. It quantifies the average amount of information contained in each message received, reflecting how much "surprise" or new information is expected on average when sampling from that source
. When transmitting information, entropy sets a theoretical lower bound on how much data must be sent to convey the message without loss. However, real-world transmissions often include redundant bits -extra bits beyond the minimum needed to represent the information. These redundant bits serve important purposes:
- Error detection and correction: Redundancy allows the receiver to detect and correct errors caused by noise or interference in the communication channel. For example, the Voyager 2 spacecraft sends one redundant bit for every two bits of information, and the Hubble Space Telescope sends three redundant bits for every bit of information to ensure data integrity despite the noisy environment of space
- Robustness against mutations in DNA: Human DNA contains almost twice as much information as needed to code for all substances produced in the body. This redundancy in the genetic code-known as codon degeneracy-means multiple codons can encode the same amino acid. This reduces the impact of mutations, as some changes in the DNA sequence do not alter the resulting protein (silent mutations), thereby protecting organisms from harmful effects
In summary, entropy relates to the minimal amount of information needed to represent data, but redundancy is intentionally added in both biological systems (DNA) and engineered systems (space probes) to enhance reliability and error resilience. This redundancy ensures accurate transmission and interpretation of information despite noise, errors, or mutations, ultimately preserving the integrity and functionality of the transmitted message or genetic code.
