Probability & Statistics

Learn Information & Coding Theory

Quantifying and transmitting information: entropy, joint and conditional entropy, mutual information and KL divergence, the source-coding theorem and Huffman codes, channel capacity, Shannon's noisy-channel coding theorem, error-correcting and Hamming codes, and Kolmogorov complexity.

Free to start · adaptive placement finds your level · reviews timed to your own forgetting.

What you'll learn

30 lessons in Information & Coding Theory

Entropy: measuring informationJoint & conditional entropyMutual informationKL divergenceSource coding & the entropy boundHuffman & prefix codesChannel capacityThe noisy-channel coding theoremError-correcting codesThe Hamming codeKolmogorov complexityDifferential entropyThe AEP & typical sequencesThe Gaussian channelThe data-processing inequalityRate–distortion theoryFano's inequalityUniversal compressionEntropy ratePerplexityChecksums and cyclic redundancy checksReed–Solomon codesBurst errors and interleavingConvolutional codes and the trellisErasure channels and fountain codesPolar codesDistributed source codingError exponents in hypothesis testingMin-entropy and guessingShearer's inequality and the entropy method
How Erudia teaches

Built to be understood — and remembered.

Every idea is taught with motivation and a worked example before the drills, and an FSRS spaced-repetition engine schedules each review for the day its model predicts you would forget it. A short placement check finds what you already know, so you start Information & Coding Theory exactly where it's useful.

Related Probability & Statistics subjects