Back to overview over all lectures...
Information Theory II
|
Prof. Amos Lapidoth | Prof. Stefan M. Moser | |
Office: | ETF E107 | ETF E104 |
Phone: | 044 632 51 92 | 044 632 36 24 |
E-mail: |
Robert Graczyk
Office: ETF E104
Phone: 044 632 28 99
E-mail:
Note that usually the lecture is from 3 to 5 and the exercise hours are from 1 to 3.
We use the book by Thomas M. Cover and Joy A. Thomas:
T.M. Cover, J.A. Thomas, Elements of Information Theory,
2nd Edition, John Wiley & Sons, Inc., New York, NY, USA.
(Here is a link to an
electronic copy of it.)
Stefan M. Moser: Information Theory (Lecture Notes), 6th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
Stefan M. Moser: Advanced Topics in Information Theory (Lecture Notes), 3rd edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
A. El Gamal, Y.-H. Kim, Network Information Theory, Cambridge University Press
G. Kramer, Topics in Multi-User Information Theory, available here.
Printed copies of the exercises will be handed out in class. See time table below.
Oral exam (30 min.).
None.
The lecture is held in English.
W | Date | Topic | Handouts | Exercise | Solutions | Lecturer |
1 | 21 Feb. | Differential entropy, max-entropy principle, relative entropy and mutual information for continuous RVs | Infos Contents | — | — | Stefan M. Moser |
2 | 28 Feb. | AEP for continuous RVs: volume, properties of (jointly) typical sets; setup of Gaussian channel: average power constraint, information capacity of Gaussian channel | Exercise 1 | Solutions 1 | Stefan M. Moser | |
3 | 7 Mar. | Gaussian channel, coding theorem: intuition, achievability, and converse | Exercise 2 | Solutions 2 | Stefan M. Moser | |
4 | 14 Mar. | Multiple-Access Channel: channel model; achievable rate region, intuition, time-sharing; coding-theorem: achievability | Exercise 3 | Solutions 3 | Amos Lapidoth | |
5 | 21 Mar. | Multiple-Access Channel: introduction of the Q-region and equivalence to time-sharing (no redundant pentagon constraints, Caratheodory); coding-theorem: converse | Exercise 4 | Solutions 4 | Stefan M. Moser | |
6 | 28 Mar. | Broadcast Channel: channel model; (physically and stochastically) degraded BCs; capacity region depends on marginals only; achievabilty via superposition-coding; converse for degraded BCs | Handout 1 | Exercise 5 | Solutions 5 | Stefan M. Moser |
7 | 4 Apr. | Channels with States: introduction, examples; Gel'fand-Pinsker problem: achievability | Exercise 6 | Solutions 6 | Stefan M. Moser | |
8 | 11 Apr. | Gel'fand-Pinsker problem: converse; examples: Writing on Dirty Paper, Channels that Sometimes Get Stuck | Exercise 7 | Solutions 7 | Stefan M. Moser | |
9 | 18 Apr. | Method of types: introduction, number and size of type classes, probability of observing a type; large devitation theory:2 Sanov's theorem; universal source coding | Exercise 8 | Solutions 8 | Amos Lapidoth | |
— | 25 Apr. | holidays | — | — | ||
10 | 2 May | Minimizing relative entropy subject to equality constraints; Stein's Lemma; Chernoff Information | Exercise 9 | Solutions 9 | Amos Lapidoth | |
11 | 9 May | Capacity-Redundancy Theorem; Properties of Bayesian Mixture Codes; Normalized Maximum Likelihood Codes Minimize Worst-Case Individual Sequence Redundancy | Exercise 10 | Solutions 10 | Amos Lapidoth | |
12 | 16 May | d/2*log(n) redundancy; Laplace method of integration; Context-Tree Weighting algorithm | Handout 2 | Exercise 11 | Solutions 11 | Amos Lapidoth |
13 | 23 May | Guessing and Rényi entropy | Handout 3 | Exercise 12 | Solutions 12 | Robert Graczyk |
14 | 30 May | holiday | — | — |
-||- _|_ _|_ / __|__ Stefan M. Moser
[-] --__|__ /__\ /__ Senior Researcher & Lecturer, ETH Zurich, Switzerland
_|_ -- --|- _ / / Adj. Professor, National Yang Ming Chiao Tung University (NCTU), Taiwan
/ \ [] \| |_| / \/ Web: https://moser-isi.ethz.ch/
Last modified: Thu Jul 14 06:08:52 CEST 2022