Mental Models 4 Life

Here is a short (and somewhat unusual) course on statistical machine learning that I have delivered multiple times over the last few years.

1. Introduction to Statistical Learning Theory
2. Bayesian Probability Theory
3. Sequence Prediction and Data Compression
4. Bayesian Networks

In designing this course, I have deliberately steered away from the usual practice of giving students a (long) list of commonly used algorithms and use cases. Instead, the course places more emphasis on some of the central principles of statistical machine learning, including the limits of learnability, and MAP and Bayesian optimal estimators. I believe a good understanding of these basic principles will equip students with an ability to understand the vast majority of machine learning algorithms in the literature, many of which can be readily understood as computational approximations to the theoretically optimal learning algorithms proposed by the basic principles.