CSCI 5525 - Advanced Machine Learning
Lecture details and list of relevant reading materials/references for Advanced Machine Learning course (CSCI - 5525) offered at Computer Science and Engineering department, University of Minnesota, Fall 2025.
- Instructor: Aryan Deshwal
- Time/Location: 02:30 PM ‑ 03:45 PM TTh
Relevant Textbooks
PRML: Pattern recognition and Machine Learning, Chris Bishop
Deep Learning: Foundations and Concepts, Chris Bishop and Hugh Bishop
GPML: Gaussian Processes for Machine Learning, Carl Edward Rasmussen and Christopher K. I. Williams
PML: Probabilistic Machine Learning: Advanced Topics, Kevin Murphy
| Lecture | Topic | Reading Materials/References |
|---|---|---|
| 1 | Introduction to Machine Learning and its History | -- Additional Resources: Bernhard Schölkopf's excellent ML history lecture at MLSS 2020 |
| 2 | Introduction to Probabilistic Modeling with Linear Regression | PRML Chapter 1-3, GPML Chapter 1 |
| 3 | Gaussian Processes and Kernel Methods | GPML Chapter 2-4
-- Additional Resources: Dive into Deep Learning Chapter 18 on GPs Kanagawa et al., Gaussian Processes and Kernel Methods: A Review on Connections and Equivalences GpyTorch |
| 4 | Generalization, Model selection and Occam's razor | GPML Chapter 5 -- Additional Resources: David Mackay's book Chapter 28 Lotfi et al., Bayesian Model Selection, the Marginal Likelihood, and Generalization |
| 5 | Bayesian Decision Theory | PRML Chapter 1.5
-- Additional Resources: Roman Garnett's Bayesian Optimization Chapter 4 and 5 |
| 6 | Neural Networks: Learning Representations | Bishop Deep Learning Chapter 6 |
| 7 | Backpropagation and Automatic Differentiation | Bishop Deep Learning Chapter 8
-- Additional Resources: Andrej Karpathy's post on backpropagation |
| 8 | Stochastic Optimization | Bishop Deep Learning Chapter 7 |
| 9 | Training tips for Neural Networks |
Deep Learning Tuning Playbook
Andrej Karpathy's A Recipe for Training Neural Networks Experiment Management |
| 10 | Attention and Transformers | Bishop Deep Learning Chapter 12
-- Additional Resources: Richard Turner's An Introduction to Transformers Simon Institute's Workshop on Transformers Edelman et al., Self-Attention Inductive Bias |
| 11 | Deep Latent Variable Models: Variational Autoencoders | Murphy PML (Advanced Topics) Chapter 21 |
| 12 | Diffusion Models | Murphy PML (Advanced Topics) Chapter 25
-- Additional Resources: Lai et al., The Principles of Diffusion Models Turner et al., Denoising Diffusion Probabilistic Models in Six Simple Steps History of Diffusion Interview |
| 13 | Autoregressive Models | Murphy PML (Advanced Topics) Chapter 22
-- Additional Resources: Alec Radford's Language Models Lecture |
| 14 | Reinforcement Learning (RL): Exploration/Exploitation Tradeoff | Murphy PML (Advanced Topics) Chapter 22 |
| 14 | Policy based RL | Murphy PML (Advanced Topics) Chapter 22
-- Additional Resources: Spinning up in Deep RL Huang et al., The 37 Implementation Details of Proximal Policy Optimization |
| 15 | Bayesian Optimization | BoTorch Tutorials |