• Home
• About me
• Publica-
  tions
• IT-Scripts
• Lectures
• Software/
  Manuals
• Links
• Private
•  
•  
• 
  
•  
•  
• 
• 

Advanced Topics in Information Theory
Autumn 2006/2007

News

• Class Evaluation: The class evaluation takes place on January 3 during the normal lecture hours. I would very much appreciate your feedback, so please come to class at this date! Thanks!
• Final Exam: The final exam will take place on
  • Monday, January 8, 15:40-18:30 (Note that this is one hour longer than usual!)
  Regulations:
  • open book: any book is allowed
  • not allowed are: any telecommunication devices like mobile phones, any laptop with wireless capabilities, any "friend", or any other help from outside...
  • covered material: everything covered in class
• Mid-Term Exam: The mid-term exam will take place on
  • Monday, November 6, 15:40-18:30 (Note that this is one hour longer than usual!)
  Regulations:
  • open book: any book is allowed
  • not allowed are: any telecommunication devices like mobile phones, any laptop with wireless capabilities, any "friend", or any other help from outside...
  • covered material: everything covered in class until October 30 (not including error exponents)
• Note: On October 9 there will be no lecture!

Instructor

Stefan M. Moser
Engineering Building IV, Room 727
phone: 03-571 21 21 ext. 54548
e-mail:

Teaching Assistant

Zheng Yan-Xiu
Engineering Building IV, Room 811
phone: 03-571 21 21 ext. 54571
e-mail: <non2000.cm88g@nctu.edu.tw>

Time and Place

The course is scheduled for 4 hours per week:

• Monday, 15:40--17:30, Engineering Building IV, Room 111
• Wednesday, 15:40--17:30, Engineering Building IV, Room 111

Course Objectives

This course is an advanced course in information theory. Based on the theory we have learned in the course Information Theory we will continue to explore the most important results concerning data compression and reliable communication over a communication channel: mainly we will concentrate on multiple-user communication and lossy compression schemes. The course will cover approximately the following topics:

• Maximum entropy
• Methods of types
• Rate distortion theory (lossy compression)
• Multiple-users channels:
  • Multiple-access channel
  • Broadcast channel
  • Relay channel
  • Interference channel
• Gel'fand-Pinsker problem: channels with random parameters known at the transmitter
• Correlated source encoding (Slepian-Wolf)
• Information theory and the stock market

We hope that a student who finishes the course will be able to better understand the principles underlying all state-of-the-art communication systems and the difficulties encountered when designing and trying to improve them.

Prerequisites

• Probability
• Information Theory

Textbook

The Course will be mainly be based on

• Thomas M. Cover and Joy A. Thomas: "Elements of Information Theory," Wiley, 1991.

Further references and recommended readings:

• Robert G. Gallager: "Information Theory and Reliable Communication," Wiley, 1968.
• Raymond W. Yeung: "A First Course in Information Theory," Kluwer Academic Publishers, 2005.
• Imre Csiszár, János Körner: "Information Theory: Coding Theorems for Discrete Memoryless Systems", 3rd edition, Akademiai Kiado, Budapest.

Grading

The exercises are an essential part of this lecture and we will spend a considerable amount of time in discussing and solving them during class. There will be one exercise every week consisting of a about four to six problems. The time in class will not be sufficient to solve all problems, i.e., the students are asked to finish the problems at home. For the understanding of the course and also as a preparation for the mid-term and final exam we highly recommend to solve the exercises! Since the material of this course is rather demanding by itself, we have decided not to further challenge the students with additional tasks like, e.g., a presentation of a paper. We hope that the saved time will be used instead for solving the exercises, going over the notes, and reading the textbook!

Your grade will be an average of

• your homework (15%)
• the midterm exam (35%)
• the final exam (50%)

The grade of your homework will not be based on the correctness of your answers, but rather the effort you show in trying to solve them. To pass the course there is the additional condition that at least 10 exercises have to be handed in.

This course is worth 3 credits.

Time Table

Special Remarks

The lecture will be held in English.

-||-   _|_ _|_     /    __|__   Stefan M. Moser
[-]     --__|__   /__\    /__   Senior Scientist, ETH Zurich, Switzerland
_|_     -- --|-    _     /  /   Adjunct Professor, National Yang Ming Chiao Tung University, Taiwan
/ \     []  \|    |_|   / \/    Web: https://moser-isi.ethz.ch/