## TextbooksThe declaration of the copyright is at the bottom of this page. Please, don't hesitate to contact me at if you have any questions or if you need more information. ## A Student's Guide to Coding and Information TheoryThis textbook is thought to be an easy-to-read introduction to coding and information theory for students at the freshman level or for non-engineering major students. The required math background is minimal: simple calculus and probability theory on high-school level should be sufficient. Link to Cambridge University Press: - Stefan M. Moser, Po-Ning Chen:
*“A Student's Guide to Coding and Information Theory,”*Cambridge University Press, January 2012. ISBN: 978–1–107–01583–8 (hardcover) and 978–1–107–60196–3 (paperback).
## List of Typos and Corrections:- On p. 15, in Footnote 1: Unfortunately, the way we have written the statement, it is not true. For a finite number of events to be independent, one needs that for
**any**subset of events it holds that - Unfortunately, Lemma 5.22 does not hold. Instead Eq. (5.68) should read
## Information TheoryI use these lecture notes in my course - Definitions in information theory: entropy, mutual information, relative entropy, etc.
- Data compression: lossless source coding including Shannon-type coding, Shannon coding, Fano coding, Huffman coding, Tunstall coding, arithmetic coding, Elias–Willems coding, and
*Lempel–Ziv coding (new chapter!)*. - Karush–Kuhn–Tucker conditions.
- Gambling and horse betting.
- Data transmission: coding theorem for discrete memoryless channels, computing capacity, convolutional codes,
*polar codes (new chapter!)*, error exponents. - Joint source and channel coding: information transmission theorem, transmission above capacity.
- Gaussian channel: differential entropy, channel coding theorem and joint source and channel coding theorem for the Gaussian channel, bandlimited channels, parallel Gaussian channels.
- Asymptotic Equipartition Property and weak typicality.
- Short introduction to cryptography.
- Review of Gaussian random variables, vectors, and processes.
Download current version - Stefan M. Moser:
*“Information Theory (Lecture Notes)”*(version 5.3 from 17 March 2017, PDF), 5th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Department of Electrical & Computer Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2017. - Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. ## Advanced Topics in Information TheoryI use these lecture notes in my course - Method of types.
- Large deviation theory (Sanov's theorem, conditional limit theorem).
- Strong typicality.
- Rate distortion theory.
- Error exponents in rate distortion theory.
- Multiple description.
- Rate distortion with side-information (Wyner–Ziv).
- Distributed lossless data compression (Slepian–Wolf).
- Multiple-access channel (MAC).
- Transmission of correlated sources over a MAC.
- Channels with noncausal side-information (Gel'fand–Pinsker).
- Broadcast channel.
- Multiple-access channel (MAC) with common message.
- Discrete memoryless networks and cut-set bound.
- Interference channel
*(new chapter!)*.
Download current version - Stefan M. Moser:
*“Advanced Topics in Information Theory (Lecture Notes)”*(version 2.9 from 20 March 2017, PDF), 2nd edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Department of Electrical & Computer Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2013. - Teacher's material: all figures and tables (PDF).
These notes are still undergoing corrections and improvements. You are welcome to use the IT and ATIT lecture notes for yourself, for teaching, or for any other noncommercial purpose. If you use extracts from these lecture notes, please make sure that their origin is shown. The author assumes no liability or responsibility for any errors or omissions.
-||- _|_ _|_ / __|__ Stefan M. Moser Last modified: Thu Mar 30 07:25:26 UTC+8 2017 |