Information Theory, Inference, and Learning Algorithms

by David MacKay

Originally published as «INFORMATION THEORY, INFERENCE, AND LEARNING ALGORITHMS.» (UND)

Information theory and inference, taught together in this exciting textbook, lie at the heart of many important areas of modern technology - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics and cryptography. The book introduces theory in tandem with applications. Information theory is taught alongside practical communication systems such as arithmetic coding for data compression and sparse-graph codes for error-corre...