
Textbooks
The declaration of the copyright is at the bottom of this page. Please, don't hesitate to contact me at if you have any questions or if you need more information.
This textbook is thought to be an easytoread introduction to coding and information theory for students at the freshman level or for nonengineering major students. The required math background is minimal: simple calculus and probability theory on highschool level should be sufficient.
Link to Cambridge University Press:
List of Typos and Corrections:
I use these lecture notes in my course Information Theory, which is a graduate course in the first year. The notes intend to be an introduction to information theory covering the following topics:
 Definitions in information theory: entropy, mutual information, relative entropy, etc.
 Data compression: lossless source coding including Shannontype coding, Shannon coding, Fano coding, Huffman coding, Tunstall coding, arithmetic coding, Elias–Willems coding, and Lempel–Ziv coding.
 Karush–Kuhn–Tucker conditions.
 Gambling and horse betting.
 Data transmission: coding theorem for discrete memoryless channels, computing capacity, convolutional codes, polar codes.
 Joint source and channel coding: information transmission theorem, transmission above capacity.
 Gaussian channel: differential entropy, channel coding theorem and joint source and channel coding theorem for the Gaussian channel, bandlimited channels, parallel Gaussian channels.
 Asymptotic Equipartition Property and weak typicality.
 Short introduction to cryptography.
 Review of Gaussian random variables, vectors, and processes.
Download current version (New 6th Edition!):
 Stefan M. Moser: “Information Theory (Lecture Notes)” (version 6.2 from 22 July 2018, PDF), 6th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
 Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. If you find typos, errors, or if you have any comments about these notes, I'd be very happy to hear them! Write to . Thanks!
I use these lecture notes in my course Advanced Topics in Information Theory, which is an advanced graduate course. Based on the theory introduced in the introductory notes Information Theory, it continues to explore the most important results concerning data compression and reliable communication over a communication channel, including multipleuser communication and lossy compression schemes. The course covers the following topics:
 Method of types.
 Large deviation theory (Sanov's theorem, conditional limit theorem).
 Strong typicality.
 Hypothesis testing, Chernoff–Stein Lemma.
 Error exponents for information transmission.
 Cost constraints and feedback (work in progress).
 Rate distortion theory.
 Error exponents in rate distortion theory.
 Multiple description.
 Rate distortion with sideinformation (Wyner–Ziv).
 Distributed lossless data compression (Slepian–Wolf).
 Multipleaccess channel (MAC).
 Transmission of correlated sources over a MAC.
 Channels with noncausal sideinformation (Gel'fand–Pinsker).
 Broadcast channel.
 Multipleaccess channel (MAC) with common message.
 Discrete memoryless networks and cutset bound.
 Interference channel.
Download current version (New 3rd Edition!):
 Stefan M. Moser: “Advanced Topics in Information Theory (Lecture Notes)” (version 3.0 from 19 February 2018, PDF), 3rd edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
 Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. If you find typos, errors, or if you have any comments about these notes, I'd be very happy to hear them! Write to . Thanks!
Copyright
You are welcome to use the IT and ATIT lecture notes for yourself, for teaching, or for any other noncommercial purpose. If you use extracts from these lecture notes, please make sure that their origin is shown. The author assumes no liability or responsibility for any errors or omissions.
 __ __ / ____ Stefan M. Moser
[] ____ /__\ /__ Senior Researcher & Lecturer, ETH Zurich, Switzerland
__   _ / / Adj. Professor, National Chiao Tung University (NCTU), Taiwan
/ \ [] \ _ / \/ Web: http://moserisi.ethz.ch/
Last modified: Sun Jul 22 11:09:48 CEST 2018
