Stats 306B: Methods for Applied Statistics: Unsupervised Learning

Lester Mackey, Stanford University, Spring 2014
Announcements

Course Schedule

Topics Lecture Notes Reading Problem Set Solutions
Mon. 3/31 Unsupervised vs. Supervised Learning; Clustering with k-means and k-medoids Lecture 1 Slides ESL 14.1, 14.3 (except 14.3.7, 14.3.12)
Optional: k-means++, Gap statistic, kd-trees
- -
Weds. 4/2 Gaussian Mixture Models; Expectation-Maximization Lecture 2 Scribed Notes
Lecture 2 Slides
ESL 6.8, 8.5, 14.3.7
Mixture modeling chapter, EM chapter
- -
Mon. 4/7 Expectation-Maximization; General Mixture Modeling Lecture 3 Scribed Notes Mixture modeling chapter, EM chapter
Optional: Original EM paper
- -
Weds. 4/9 Discrete Hidden Markov Models Lecture 4 Slides
Lecture 4 Scribed Notes
HMM chapter Homework 1, Data Homework 1 Solutions
Mon. 4/14 Discrete HMMs; Hierarchical Clustering Lecture 5 Scribed Notes HMM chapter, ESL 14.3.12 - -
Weds. 4/16 Hierarchical Clustering; Spectral Clustering Lecture 6 Scribed Notes
Lecture 6 Slides
ESL 14.3.12, 14.5.3, Spectral clustering tutorial
Optional: Minimax linkage
- -
Mon. 4/21 Spectral Clustering; Linear Dimensionality Reduction via Principal Component Analysis Lecture 7 Slides
Lecture 7 Scribed Notes
Spectral clustering tutorial, ESL 14.5.3, 14.5.1, 3.5.1
Optional: Normalized cuts
- -
Weds. 4/23 Principal Component Analysis; Kernel PCA Lecture 8 Slides
Lecture 8 Scribed Notes
ESL 14.5.1, 3.5.1, 14.5.4
Kernel PCA
Homework 2, Data Homework 2 Solutions
Mon. 4/28 Kernel PCA; Factor Analysis Lecture 9 Slides
Lecture 9 Scribed Notes
ESL 14.7.1, Multivariate Gaussian chapter, Factor analysis chapter - -
Weds. 4/30 Factor Analysis; Linear Gaussian State-Space Models and Kalman Filtering Lecture 10 Scribed Notes Factor analysis chapter, State-space models chapter
Optional: Probabilistic PCA
- -
Mon. 5/5 Linear Gaussian SSMs Lecture 11 Scribed Notes State-space models chapter - -
Weds. 5/7 SSMs; Independent Component Analysis; Canonical Correlation Analysis Lecture 12 Slides
Lecture 12 Scribed Notes
ESL 14.7, 3.7, ICA Homework 3, Data Homework 3 Solutions
Mon. 5/12 CCA; Sparse Unsupervised Learning Lecture 13 Scribed Notes ESL 3.7, 14.5.5, Exact and greedy sparse PCA - -
Weds. 5/14 Sparse Unsupervised Learning Lecture 14 Scribed Notes
Lecture 14 Slides
ESL 14.5.5, DSPCA
Optional: Deflation methods, Sparse clustering
Practice Midterm Questions Practice Midterm Solutions
Mon. 5/19 Unsupervised Deep Learning Lecture 15 Slides
Lecture 15 Scribed Notes
Representation learning
Optional: Deep learning
- -
Weds. 5/21 In-class Midterm - - Midterm Midterm Solutions
Mon. 5/26 Memorial Day - No Class - - - -
Weds. 5/28 Learning with Missing Data Lecture 16 Scribed Notes ESL 9.6 - -
Mon. 6/2 Unsupervised Learning with Missing Data Lecture 17 Scribed Notes Matrix factorization, Nuclear norm heuristic
Optional: Alternating minimization theory, Weighted trace norm
- -
Weds. 6/4 Final Project Presentations - Sequoia Hall Courtyard - - - -


  • ESL = The Elements of Statistical Learning

Final Project Deadlines

See the final project page for details on each milestone.

Weds. 4/23 Abstract Due
Weds. 5/7 Progress Report Due
Weds. 6/4 In-class Presentation
Thurs. 6/5 Final Report Due (by 11:59PM)