Back to overview over all lectures...
Information Theory II
|
Prof. Amos Lapidoth | Prof. Stefan M. Moser | |
Office: | ETF E107 | ETF E104 |
Phone: | 044 632 51 92 | 044 632 36 24 |
E-mail: |
Robert Graczyk
Office: ETF E104
Phone: 044 632 28 99
E-mail:
We use the book by Thomas M. Cover and Joy A. Thomas:
T.M. Cover, J.A. Thomas, Elements of Information Theory,
2nd Edition, John Wiley & Sons, Inc., New York, NY, USA.
(Here is a link to an
electronic copy of it.)
Stefan M. Moser: Information Theory (Lecture Notes), 6th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2018.
Stefan M. Moser: Advanced Topics in Information Theory (Lecture Notes), 4th edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Institute of Communications Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2019.
A. El Gamal, Y.-H. Kim, Network Information Theory, Cambridge University Press
G. Kramer, Topics in Multi-User Information Theory, available here.
Printed copies of the exercises will be handed out in class. See time table below.
Oral exam (30 min.).
None.
The lecture is held in English.
W | Date | Topic | Handouts | Exercise | Solutions | Lecturer |
1 | 25 Feb. | Differential entropy, relative entropy and mutual information for continuous RVs, max-entropy principle (IT: 16.1–5, 3.4; CT: 8.1, 8.3–6, 12.1) | Infos Contents | — | — | Stefan M. Moser |
2 | 4 Mar. | AEP for continuous RVs, typical sets, volume, properties, jointly typical sets, Gaussian channel (IT: 20.13, 17; CT: 8.2, 9.1–2) | Exercise 1 | Solutions 1 | Stefan M. Moser | |
3 | 11 Mar. | Gaussian channel: coding theorem (intuition, achievability, and converse), joint source and channel coding (IT: 17, 15; ATIT: 10.8; CT: 9.1-2) | Exercise 2 | Solutions 2 | Stefan M. Moser | |
4 | 18 Mar. | Multiple-Access Channel: channel model; rate region, time sharing, convex hull; coding-theorem: achievability (ATIT: 15; CT: 15.3) | Exercise 3 | Solutions 3 | Amos Lapidoth | |
5 | 25 Mar. | Multiple-Access Channel: q-pentagons, Q-region, equivalence to time-sharing (no redundant pentagon constraints, Caratheodory); coding-theorem: converse (ATIT: 15; CT: 15.3) | Handout 1 | Exercise 4 | Solutions 4 | Amos Lapidoth |
6 | 1 Apr. | Broadcast channel: setup, degraded BC, achievability by superposition coding, converse, Gaussian BC (ATIT: 18; CT: 15.6) | Exercise 5 | Solutions 5 | Stefan M. Moser | |
— | 8 Apr. | holidays | — | — | ||
7 | 15 Apr. | Channels with States: introduction, examples; Gel'fand-Pinsker problem: achievability (ATIT: 17, 4.3 [Markov Lemma]) | Exercise 6 | Solutions 6 | Stefan M. Moser | |
8 | 22 Apr. | Gel'fand-Pinsker problem: converse; examples: Writing on Dirty Paper, Channels that Sometimes Get Stuck (ATIT: 17) | Exercise 7 | Solutions 7 | Stefan M. Moser | |
9 | 29 Apr. | Method of types: introduction, number and size of type classes, probability of observing a type; large deviation theory: universal source coding, Sanov's theorem (CT: 11; ATIT: 2,3) | Handout 2 | Exercise 8 | Solutions 8 | Amos Lapidoth |
10 | 6 May | Minimizing relative entropy subject to equality constraints; Stein's Lemma; Chernoff Information (CT: 11; ATIT: 3,5) | Handout 3 | Exercise 9 | Solutions 9 | Amos Lapidoth |
— | 13 May | holiday | — | — | ||
11 | 20 May | Soft covering: proof of direct part | Handout 4 | Exercise 10 | Solutions 10 | Amos Lapidoth |
12 | 27 May | Wiretap channel | Handout 5 | Exercise 11 | Solutions 11 | Amos Lapidoth |
13 | 3 Jun. | Guessing and Rényi entropy | Exercise 12 | Solutions 12 | Robert Graczyk |
-||- _|_ _|_ / __|__ Stefan M. Moser
[-] --__|__ /__\ /__ Senior Researcher & Lecturer, ETH Zurich, Switzerland
_|_ -- --|- _ / / Adj. Professor, National Yang Ming Chiao Tung University (NCTU), Taiwan
/ \ [] \| |_| / \/ Web: https://moser-isi.ethz.ch/
Last modified: Thu Jul 14 05:49:16 CEST 2022