Probability & Statistics

Learn Information & Coding Theory

Quantifying and transmitting information: entropy, joint and conditional entropy, mutual information and KL divergence, the source-coding theorem and Huffman codes, channel capacity, Shannon's noisy-channel coding theorem, error-correcting and Hamming codes, and Kolmogorov complexity.

Free to start · adaptive placement finds your level · reviews timed so it stays learned.

What you'll learn

15 lessons in Information & Coding Theory

Entropy: measuring informationJoint & conditional entropyMutual information & KL divergenceSource coding & the entropy boundHuffman & prefix codesChannel capacityThe noisy-channel coding theoremError-correcting codesThe Hamming codeKolmogorov complexityDifferential entropyThe AEP & typical sequencesThe Gaussian channelThe data-processing inequalityRate–distortion theory
How Erudia teaches

Built to be understood — and remembered.

Every idea is taught with motivation and a worked example before the drills, and an FSRS spaced-repetition engine schedules each review for the moment just before you'd forget it. A short placement check finds what you already know, so you start Information & Coding Theory exactly where it's useful.

Related Probability & Statistics subjects