Back to overview over all lectures...
Advanced Topics in Information Theory
Pythagoras and Relative Entropy:
A Glimpse into Large Deviation Theory
28 April 2014
News
If there is something to be announce, it will be published here...
Course Abstract
The method of types is a powerful tool that has been successfully used in information theory for many years already and that has very interesting applications in advanced probability theory. In this course we will focus on one or two rather exotic such applications. We will start with a quick review of types and their properties and then explore the connection to Large Deviation Theory and in particular focus on Sanov's Theorem.
While I claim boldly that quite likely you will never actually be using Sanovâ€™s Theorem in your future career, I still hope to motivate you to have a closer look at it. It is a gorgeous example of how an information theoretic quantity (i.e., the relative entropy) shows up unexpectedly in probability theory and gives the correct answer to a probabilistic problem. This connection then serves us as a path to a deeper understanding of the actual meaning of relative entropy. In particular, we will be discussing the famous Pythagorean Theorem in the context of relative entropies.
Depending on the time we might also have a quick look at the Conditional Limit Theorem.
Prerequisites
 Probability
 Information Theory
Instructor
Prof. Stefan M. Moser
Engineering Building IV, ITLab 716A
email:
Time and Place
The course will take place on:
 Monday, 28 April, 2014, 14:20–16:20 in Engineering Building IV, Room 824 (ED824)
Handouts
The participants will receive a handout that is an extract of my lecture notes:
Exercises
I will distribute some exercise problems for the interested student. Solutions will then be available a week later for download here.
Special Remarks
The lecture will be held in English.
 __ __ / ____ Stefan M. Moser
[] ____ /__\ /__ Senior Researcher & Lecturer, ETH Zurich, Switzerland
__   _ / / Adj. Professor, National Chiao Tung University (NCTU), Taiwan
/ \ [] \ _ / \/ Web: http://moserisi.ethz.ch/
Last modified: Mon Apr 14 15:49:01 UTC+8 2014
