**Syllabus**: pdf

**Outline:**

- Jan 6: Motivation (slides)/ Discrete Time Markov chains (Markov property, transition matrix, notes)
- Jan 8: Stationary distribution, transition diagram, Chapman-Kolmogorov equations (notes)
- Jan 10: Classification of states: accessibility, communicating classes, period (notes)
- Jan 13: Transient and recurrent states (notes)
- Jan 17: Transient and recurrent states, Gambler’s ruin problem (notes)
- Jan 20: Gambler’s ruin problem, Recurrence of random walk on Z (notes)

– Some references on difference equations: for the method with order 2, you can follow this link; to go deeper: classical reference (book), or this link, which makes the connection with linear algebra.

– On the Stirling formula: see this link for a derivation, or for more complete expansion, here - Jan 22: Recurrence of random walk on Z^d (notes)
- Jan 24: Recurrence of random walk on Z^d (notes)
- Jan 27: Limiting Probabilities (notes)
- Jan 29: Limiting Probabilities, doubly stochastic matrix, Time reversibility (notes)
- Jan 31: Time reversibility (notes)
- Feb 3: Time reversibility (Kolmogorov’s criterion), Ehrenfest chain (notes)
- Feb 5: Ehrenfest chain, branching process (notes)
- Feb 7: Branching process, generating function (notes)
- Feb 10: Branching process (notes)
- Feb 12: Poisson Process (Introduction, Exponential Distribution) (notes)
- Feb 14: Exponential distribution and no memory property (notes)
- Feb 24: Minimum and sum of exponential r.v.’s (notes)
- Feb 26: Sum of exponential r.v.’s, first description of Poisson Process (notes)
- Feb 28: Poisson Process (pmf, stationary and increment properties, second definition) (notes)
- Mar 2: Poisson Process (equivalency of the two definitions) (notes)
- Mar 4: Superposition and thinning of Poisson process (notes)
- Mar 6: Conditional arrival times (notes)
- Mar 9: Conditional arrival times, sampling (notes)
- Mar 11: Sampling, Continuous time Markov chain (notes)
- Mar 13: Birth death process (notes)
- Mar 16: From now on we move to online course (see Canvas)
- April 3: Special lecture on modeling of COVID-19: deterministic modeling / stochastic modeling (~27.MB, to watch video at the end, open with acrobat reader)
- Week 9-10 course material notes
- Slides for April 10 session on Collaborate Ultra: pdf
- Slides for April 14 session on Collaborate Ultra: pdf

**Homework:**

- Homework 1 (date due: Jan 17) pdf, solution
- Homework 2 (date due: Jan 24) pdf, solution
- Homework 3 (date due: Jan 31) pdf, solution
- Homework 4 (date due: Feb 7) pdf , solution
- Homework 5 (date due: March 2) pdf, solution
- Homework 6 (date due: March 9) pdf, solution
- Homework 7 (date due: March 19) pdf, solution , grading
- Homework 8 (date due: March 27) pdf, solution
- Homework 9 (date due April 3) pdf, solution

**Midterm 1:**

**Midterm 2:**

- Sample past midterms with solutions: midterm_sample2
- Cancelled (see Canvas)