
Textbooks
The declaration of the copyright is at the bottom of this page. Please, don't hesitate to contact me at if you have any questions or if you need more information.
This textbook is thought to be an easytoread introduction to coding and information theory for students at the freshman level or for nonengineering major students. The required math background is minimal: simple calculus and probability theory on highschool level should be sufficient.
Link to Cambridge University Press:
List of Typos and Corrections:
I use these lecture notes in my course Information Theory, which is a graduate course in the first year. The notes intend to be an introduction to information theory covering the following topics:
 Definitions in information theory: entropy, mutual information, relative entropy, etc.
 Data compression: lossless source coding including Shannontype coding, Shannon coding, Fano coding, Huffman coding, Tunstall coding, arithmetic coding, Elias–Willems coding, and Lempel–Ziv coding.
 Karush–Kuhn–Tucker conditions.
 Gambling and horse betting.
 Data transmission: coding theorem for discrete memoryless channels, computing capacity, convolutional codes, polar codes.
 Joint source and channel coding: information transmission theorem, transmission above capacity.
 Gaussian channel: differential entropy, channel coding theorem and joint source and channel coding theorem for the Gaussian channel, bandlimited channels, parallel Gaussian channels.
 Asymptotic Equipartition Property and weak typicality.
 Short introduction to cryptography.
 Review of Gaussian random variables, vectors, and processes.
Download current version:
 Stefan M. Moser: “Information Theory (Lecture Notes)” (version 6.5 from 24 August 2019, PDF), 6th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
 Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. If you find typos, errors, or if you have any comments about these notes, I'd be very happy to hear them! Write to . Thanks!
I use these lecture notes in my course Advanced Topics in Information Theory, which is an advanced graduate course. Based on the theory introduced in the introductory notes Information Theory, it continues to explore the most important results concerning data compression and reliable communication over a communication channel, including multipleuser communication and lossy compression schemes. The course covers the following topics:
 Method of types.
 Large deviation theory (Sanov's theorem, conditional limit theorem).
 Strong typicality.
 Hypothesis testing, Chernoff–Stein Lemma.
 Parameter estimation, Fisher information, CramÃ©r–Rao Bound. (New!)
 Duality, capacity with costs, capacity with feedback.

 Independence and causality: causal interpretations.
 Error exponents for information transmission.
 Contexttree weighting algorithm. (New!)
 Rate distortion theory.
 Error exponents in rate distortion theory.
 Multiple description.
 Rate distortion with sideinformation (Wyner–Ziv).
 Distributed lossless data compression (Slepian–Wolf).
 Multipleaccess channel (MAC).
 Transmission of correlated sources over a MAC.
 Channels with noncausal sideinformation (Gel'fand–Pinsker).
 Broadcast channel.
 Multipleaccess channel (MAC) with common message.
 Discrete memoryless networks and cutset bound.
 Interference channel.
Download current version (New 4th Edition!):
 Stefan M. Moser: “Advanced Topics in Information Theory (Lecture Notes)” (version 4.0 from 24 August 2019, PDF), 4th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2019.
 Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. If you find typos, errors, or if you have any comments about these notes, I'd be very happy to hear them! Write to . Thanks!
Copyright
You are welcome to use the IT and ATIT lecture notes for yourself, for teaching, or for any other noncommercial purpose. If you use extracts from these lecture notes, please make sure that their origin is shown. The author assumes no liability or responsibility for any errors or omissions.
 __ __ / ____ Stefan M. Moser
[] ____ /__\ /__ Senior Scientist, ETH Zurich, Switzerland
__   _ / / Adjunct Professor, National Chiao Tung University, Taiwan
/ \ [] \ _ / \/ Web: http://moserisi.ethz.ch/
Last modified: Sat Aug 24 16:15:43 CEST 2019
