18-799S: Statistical Discovery and Learning

All lecture notes and course information are now hosted on Google Site 18799F11, the mailing list for the class is 18799f11@googlegroups.com

Course Description

Statistical Discovery and Learning studies the question "how can we build computer programs that automatically improve their performance through experience?"   This includes learning to perform many types of tasks based on many types of experience.  For example, it includes robots learning to better navigate based on experience gained by roaming their environments, medical decision aids that learn to predict which therapies work best for which diseases based on data mining of historical health records,  and speech recognition systems that lean to better understand your speech based on experience listening to you.  This course is designed to give students a thorough grounding in the methods, theory, mathematics and algorithms needed to do research and applications in machine learning. The topics of the course draw from machine learning, classical statistics, data mining, Bayesian statistics and information theory and other areas.
This course is project-oriented and is intended to give students abundant hands-on experience with different machine learning algorithms. Students who have already taken CS 10-701/15-781 Machine Learning should not take this course.
The contents of the course, including the Course Schedule, Grading Policy, and Late Homework Policy, are subject to change.


Joy Ying Zhang (joy+@cs.cmu.edu) and Ole J. Mengshoel (ole.mengshoel@sv.cmu.edu)
Guest instructors: Jason Lohn (Jason.lohn@sv.cmu.edu), Abe Ishihara (abe.ishihara@sv.cmu.edu)


Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with maturity in mathematics and computer science can catch up and fully participate.
Course projects require students to have programming skills in either C/C++, Java or a similar programming languages.




The midterm and final exams will be open book and open notes.  Computers will not be allowed.

Course Schedule (26 lectures in total, including midterm and final presentations)

  1. Course overview, final course project introduction [Joy/Ole]
  2. Introduction to Probability and Statistics [Joy]
  3. Density Estimation  and Confidence Intervals [Joy]
  4. Cross Validation and Regression [Joy]

* Sign up project

  1. Maximum Likelihood Estimation and Estimation Bias [Joy]
  2. Decision Tree and Information Gain [Ole]
  3. Generative vs. Discriminative Models, Bayes Classifiers [Ole]
  4. Logistic Regression and Regularization [Joy]

* Project proposal, short presentation, QA and discussion (bring hand outs for feedback)

  1. K-Means and Hierarchical Clustering [Joy]
  2. Markov Models and Hidden Markov Models [Ole]
  3. Mixture Models, EM Algorithm [Ole]
  4. Bayesian Network I [Ole]
  5. Bayesian Network II [Ole]
  6. Neural Networks (Abe Ishihara)
  7. Instance-based Learning [Joy]
  8. Midterm Exam
  9. Optimization [Ole]
  10. Evolutionary Computation (Jason Lohn)
  11. Support Vector Machines [Joy]
  12. Learning from Labeled and Unlabeled Data [Joy]
  13. Boosting (Ole)
  14. Reinforcement and Active Learning  [Ole]
  15. Imagine processing and learning –tentative (Ara)
  16. Final project presentation I
  17. Final project presentation II
  18. Final project presentation III
  19. Final project presentation IV


Final Projects

Students are expected to form small teams to work on one of the final course projects proposed by faculty members using techniques covered in this course or closely related techniques. At the end of the project, each team needs to write a final project report in the style of a conference paper.

Cours Projects from the Past