Back to overview over all lectures...

Information Theory II
Spring 2015


⇒ to time table and download of class material

Aim

This course builds on Information Theory I. It introduces additional topics in single-user communication, connections between Information Theory and Statistics, and Network Information Theory.

The course has two objectives: to introduce the students to the key information theoretic results that underlie the design of communication systems and to equip the students with the tools that are needed to conduct research in Information Theory.

Contents

  • Method of Types, strong typicality
  • Large deviation theory
  • Rate distortion theory
  • The multiple description problem
  • Rate distortion with side-information: Wyner-Ziv
  • Distributed lossless data compression: Slepian-Wolf
  • The multiple-access channel (MAC)
  • The broadcast channel (BC)
  • Channels with noncausal side-information: Gelfand-Pinsker
  • Discrete memoryess networks, cut-set bound
  • The interference channel

Prerequisites

  • Undergraduate studies
  • Solid foundation in probability and calculus
  • Pleasure with mathematics
  • Information Theory I

Instructor

Prof. Stefan M. Moser
ETF E104
phone: 044 632 36 24
e-mail:

Teaching Assistant

Annina Bracher
Office: ETF E105
Phone: 044 632 28 99
e-mail:

Time and Place

  • Lecture: Thursday, 13:15–15:00, ETZ E6
  • Exercise: Thursday, 15:15–17:00 ETZ E6

Textbook

We use the book by Stefan M. Moser:
Stefan M. Moser: Advanced Topics in Information Theory (Lecture Notes), 2nd edition, Signal and Information Processing Laboratory, ETH Zürich, Switzerland, and Department of Electrical & Computer Engineering, National Chiao Tung University (NCTU), Hsinchu, Taiwan, 2013.

Supplementary Notes

A. El Gamal, Y.-H. Kim, Network Information Theory, Cambridge University Press

G. Kramer, Topics in Multi-User Information Theory, available here.

T.M. Cover, J.A. Thomas, Elements of Information Theory, 2nd Edition, John Wiley & Sons, Inc., New York, NY, USA, available here.

Exercises

Printed copies of the exercises will be handed out in class. See time table below.

Examination

Oral exam (30 min.).

Testat requirements

None.

Special Remarks

The lecture is in English.

Time Table

Note that some linked documents in this table can only be downloaded within ETHZ!

W Date Topic Handouts Exercise Solutions Lecturer
1 19 Feb. Chapter 2: Method of Types Syllabus Exercise 1 Solutions 1 Stefan M. Moser
2 26 Feb. Chapter 3: Large Deviation Theory: Sanov's Theorem, Pythagoras for Relative Entropy, Pinsker Inequality   Exercise 2 Solutions 2 Stefan M. Moser
3 5 Mar. Chapter 3: Conditional Limit Theorem; Chapter 4: Strongly Typical Sets   Exercise 3 Solutions 3 Stefan M. Moser
4 12 Mar. Chapter 4: Strongly Typical Sets; Chapter 5: Rate Distortion Theory   Exercise 4 Solutions 4 Stefan M. Moser
5 19 Mar. Chapter 5: Rate Distortion Theory   Exercise 5 Solutions 5 Stefan M. Moser
6 26 Mar. Chapter 5: Rate Distortion Theory; Chapter 8: Rate Distortion with Side-Information (Wyner–Ziv)   Exercise 6 Solutions 6 Stefan M. Moser
7 2 Apr. Chapter 8: Rate Distortion with Side-Information (Wyner–Ziv); Chapter 10: The Multiple-Access Channel   Exercise 7 Solutions 7 Stefan M. Moser
9 Apr. holidays    
8 16 Apr. Chapter 10: The Multiple-Access Channel   Exercise 8 Solutions 8 Stefan M. Moser
9 23 Apr. no lecture, only exercise   Exercise 9 Solutions 9
10 30 Apr. no lecture, only exercise   Exercise 10 Solutions 10
11 7 May Chapter 13: The Broadcast Channel   Exercise 11 Solutions 11 Stefan M. Moser
12 14 May holiday    
13 21 May Chapter 13: The Broadcast Channel; Chapter 12: Channels with Noncausal Side-Information   Exercise 12 Solutions 12 Stefan M. Moser
14 28 May Chapter 13: The Broadcast Channel; Chapter 12: Channels with Noncausal Side-Information   Stefan M. Moser

-||-   _|_ _|_     /    __|__   Stefan M. Moser
[-]     --__|__   /__\    /__   Senior Researcher & Lecturer, ETH Zurich, Switzerland
_|_     -- --|-    _     /  /   Adj. Professor, National Yang Ming Chiao Tung University (NCTU), Taiwan
/ \     []  \|    |_|   / \/    Web: https://moser-isi.ethz.ch/


Last modified: Mon May 15 08:02:50 UTC+8 2017