Back to overview over all lectures...

Probability
Spring 2009


⇒ to time table and download of class material

News

  • Final Exam: The final exam will take place on
    • Tuesday, 23 June, 15:40–18:30 (Note that this is TWO hours longer than usual!)
    • Location: Engineering Building IV, Room B21 (ED021)
    Regulations:
    • open book: any book is allowed
    • not allowed are: any telecommunication devices like mobile phones, any laptop with wireless capabilities, any "friend", or any other help from outside...
    • covered material: everything covered in class except Gaussian stochastic processes and ergodicity
  • Change of Lab: Note that the lab of this class' teaching assistant has changed to 716A!
  • Mid-Term Exam: The mid-term exam will take place on
    • Thursday, 23 April, 10:10–13:00 (Note that this is one hour longer than usual!)
    Regulations:
    • open-book: any book, notes, and exercises are allowed
    • not allowed are: any telecommunication devices like mobile phones, any laptop with wireless capabilities, any "friend", or any other help from outside...
    • covered material: everything covered in class until 9 April, i.e., Chapter 1 to 3 and Chapter 4 up to (and including) 4.4 (sum of random number of independent random variables)
  • TA-class: Every Tuesday evening, from 19:00–21:00, there will be a repetition class with the TA of this course. New Location: ED301.
  • Textbook: There are two versions available of the textbook. The new second edition is a hardcover version including two more chapters that won't be covered in this class, i.e., you can without problems buy the much cheaper first edition.

Course Objectives

This course is an introduction to probability. Its goal is to give the students a profound knowledge about probability theory and some of its important applications. We will cover the following subjects:

  • Sample Space and Probability
    • probabilistic models and conditional probability
    • total probability and Bayes' Rule
    • independence
  • Discrete Random Variables (RV)
    • probability mass function (PMF)
    • transforming RVs
    • expectations
    • joint PMF
    • conditioning and independence
  • General Random Variables
    • probability density function (PDF) and cumulative distribution function (CDF)
    • joint PDF
    • Gaussian RVs
    • conditioning
    • transforming RVs
    • sum of random number of independent RVs
    • least squares estimation
  • Stochastic Processes
    • Bernoulli process
    • Poisson process
    • Markov chains
  • Limit Theorems
    • Markov and Chebyshev inequalities
    • weak and strong law of large numbers
    • convergence
    • ergodicity
    • central limit theorem

For more detail see the time table below.

We believe that this course is essential for any engineer and we very much hope that a student who finishes the course will feel comfortable with the theory and can apply it.

Prerequisites

The following lectures/topics are recommended:

  • basic math from high-school

Instructor

Prof. Stefan M. Moser
Engineering Building IV, Office 727
phone: 03-571 21 21 ext. 54548
e-mail:

Teaching Assistant

In case you would like to discuss some questions in Chinese, you may contact the TA of this class:

  • Lin Hsuan-Yin
    e-mail:
    Office: Engineering Building IV, Lab 716A (ED716A)
    Phone: 03-571 21 21 ext. 54630
    Office hours: Tuesday, 13:00–15:00, and Thursday, 13:00–15:00

Time and Place

The course is scheduled for 3 hours per week:

  • Tuesday, 15:40–16:30 (G), Engineering Building IV, Room 303 (ED303)
  • Thursday, 10:10–12:00 (CD), Engineering Building IV, Room 303 (ED303)

The course starts on Tuesday, 24 February, and finishes on Thursday, 25 June.

Additionally there will be a TA-class on

  • Tuesday, 19:00–21:00, Engineering Building IV, Room 301 (ED301)

Office Hours

NCTU requests that every teacher offers two hours per week where students may come to ask questions:

  • Thursday, 13:00–15:00, Engineering Building IV, Office 727

However, we would like to encourage you to show up in the teacher's or teaching assistant's office at any time in case you have questions about the class or related subjects. Moreover, we are always available during and after classes.

Textbook

Dimitri P. Bertsekas, John N. Tsitsiklis: "Introduction to Probability," Athena Scientific, Massachusetts, 2002.

For certain topics there will be additional handouts during classes.

Exercises

Every week, an exercise will be distributed in class and also made available online for download. This exercise will consist of several problems that need to be solved at home and handed in during the class of the following week. A model solution will be distributed in class and made available online afterwards.

We believe the exercises to be extremely important and crucial to the understanding of the course. They also serve as a preparation for the mid-term and final exams and we therefore highly recommend to solve them. To pass the course you need to hand in at least 10 exercises.

Exams

There will be one mid-term and one final exam. Both exams are going to last three hours and be open-book. Details about the covered material will be published in due time.

Grading

The grade will be an average of

  • the homework (15%),
  • the mid-term exam (35%), and
  • the final exam (50%).

The grade of the homework will not be based on the correctness of the answers, but rather on the effort the student shows in trying to solve them. This course is worth 3 credits.

Special Remarks

The lecture will be held in English.

Time Table

W Date Topic Handouts Exercise (due on) Solutions Comments
1 24 Feb. Introduction, set theory, probabilistic models Syllabus Exercise 1 (3 Mar.)    
  26 Feb. Conditional probability, total probability theorem, Bayes' rule    
 
2 3 Mar. Independence   Exercise 2 (10 Mar.)    
  5 Mar. Counting, discrete RV: PMF, functions of RVs, expectations     Solutions 1  
3 10 Mar. Mean, variance, joint PMF   Exercise 3 (17 Mar.)    
  12 Mar. Conditional PMF, independent RVs     Solutions 2  
4 17 Mar. Continuous RVs: PDF, CDF   Exercise 4 (24 Mar.)    
  19 Mar. Gaussian RV, conditioning, joint PDF     Solutions 3  
5 24 Mar. Joint PDF, Bayes' rule   Exercise 5 (31 Mar.)    
  26 Mar. Derived distributions     Solutions 4  
6 31 Mar. Derived distributions   Exercise 6 (7 Apr.)    
  2 Apr. No lecture (Holiday)    
 
7 7 Apr. Transform (moment generating function)   Exercise 7 (14 Apr.) Solutions 5  
  9 Apr. Conditional variance, sum of random number of independent RVs, covariance and correlation     Solutions 6  
8 14 Apr. Covariance and correlation, MMSEE   Exercise 8 (28 Apr.)    
  16 Apr. MMSEE, LMMSEE, covariance matrices Handout about Gaussian RVs   Solutions 7  
9 21 Apr. Covariance matrices, multivariate Gaussian distribution  
Solutions 8 (short version)  
  23 Apr. Mid-Term Exam    
ATTENTION: This is a 3 hours exam: 10:10–13:00
10 28 Apr. Discussion exam, multivariate Gaussian distribution   Exercise 9 (5 May)    
  30 Apr. Multivariate Gaussian distribution, stochastic processes, stationarity     Solutions 8  
11 5 May Stationarity, Bernoulli process   Exercise 10 (12 May)    
  3 May Bernoulli process, Poisson process     Solutions 9  
12 12 May Poisson process   Exercise 11 (19 May)    
  14 May Poisson process, Markov process     Solutions 10  
13 19 May Markov process: steady state and stationarity   Exercise 12 (26 May)    
  21 May Markov process: steady state and stationarity     Solutions 11  
14 26 May Markov process: steady state and stationarity   Exercise 13 (2 Jun.)    
  28 May No lecture (Holiday)    
 
15 2 Jun. Markov process: long-term frequency interpretation,   Exercise 14 (9 Jun.) Solutions 12  
  4 Jun. Markov process: short-term transient behavior     Solutions 13  
16 9 Jun. Limit Theorems: Markov Inequality, Chebyshev Inequality, Chernoff bound, Jensen's bound, convergence in probability   Exercise 15 (16 Jun.)
Exercise 16 (18 Jun.)
   
  11 Jun. Strong law of large numbers, almost sure convergence, Borel-Cantelli lemma, Central limit theorem     Solutions 14 Class evaluation online until June 26
17 16 Jun. Ergodicity & stationarity  
   
  18 Jun. Ergodicity & stationarity     Solutions 15,
Solutions 16
 
18 23 Jun. Final Exam  
  ATTENTION: This is a 3 hours exam: 15:40–18:30 and in different location (ED021)!
  25 Jun. Discussion of final exam    
 

-||-   _|_ _|_     /    __|__   Stefan M. Moser
[-]     --__|__   /__\    /__   Senior Researcher & Lecturer, ETH Zurich, Switzerland
_|_     -- --|-    _     /  /   Adj. Professor, National Chiao Tung University (NCTU), Taiwan
/ \     []  \|    |_|   / \/    Web: http://moser-isi.ethz.ch/


Last modified: Fri Jun 26 08:33:26 UTC+8 2009