**Syllabus**: pdf

**Outline:**

- January 6: Motivation (slides)/ Discrete Time Markov chains (Markov property, transition matrix, notes)
- January 8: Stationary distribution, transition diagram, Chapman-Kolmogorov equations (notes)
- January 10: Classification of states: accessibility, communicating classes, period (notes)
- January 13: Transient and recurrent states (notes)
- January 17: Transient and recurrent states, Gambler’s ruin problem (notes)
- January 20: Gambler’s ruin problem, Recurrence of random walk on Z (notes)

– Some references on difference equations: for the method with order 2, you can follow this link; to go deeper: classical reference (book), or this link, which makes the connection with linear algebra.

– On the Stirling formula: see this link for a derivation, or for more complete expansion, here - January 22: Recurrence of random walk on Z^d (notes)
- January 24: Recurrence of random walk on Z^d (notes)
- January 27: Limiting Probabilities (notes)

**Homework:**