CS 6347: Statistical Methods in AI and ML
Spring 2023Course Info
Where: ECSN 2.112
When: F, 10:00am-12:45pm
Instructor: Nicholas Ruozzi
Office Hours: W. 1pm-2pm, and by appointment in ECSS 3.409
TA: TBD
Office Hours: TBD
Grading: problem sets (70%), final project (25%), class participation & extra credit (5%)
Prerequisites: some familiarity with basic probability, linear algebra, and introductory machine learning (helpful, but not required).
Schedule & Lecture Slides
Week | Dates | Topic | Readings |
1 | Jan. 20 | Introduction & Basic Probability Bayesian Networks | K&F: Ch. 1 & 2Basic Probability BN Notes |
2 | Jan. 27 | More BNs: D-separation Markov Random Fields | K&F: Ch. 3, 4, and 9Octave (free version of MATLAB) MRF Notes |
3 | Feb. 3 | Variable Elimination & BP | K&F: 13.1-13.5, A.5.3Boyd: Ch. 5.1-5.5 |
4 | Feb. 10 | Approx. MAP EstimationMAP LP | Approximate MAP Notes K&F 11.1-11.2, 11.5Sections 1-3 of this paper |
5 | Feb. 17 | Variational Methods | |
6 | Feb. 24 | Intro to Sampling | K&F 12.1-12.3 |
7 | March 3 | Markov Chain Monte Carlo Intro to Machine Learning | K&F: 17.1-17.4 |
8 | Mar. 10 | MLE for BNs and Log-Linear Models | K&F: 20.1-20.5 | 9 | Mar. 24 | MLE for CRFs |
10 | Mar. 31 | Alternatives to MLE Expectation Maximization | K&F: 19.1-19.2Box 17.E |
11 | April 7 | Hidden Markov Models Structure Learning | K&F: 19.1-19.2, 20.6Box 17.E |
Problem Sets
All problem sets will be available on the eLearning site and are to be turned in there. See the homework guidelines below for the homework policies.
Textbooks & References
This semster, online notes in book form will (hopefully) be available for each lecture. In addition, the following textbook is suggested:
- Probabilistic Graphical Models: Principles and Techniques, by Daphne Koller and Nir Friedman.
- Modeling and Reasoning with Bayesian Networks, by Adnan Darwiche.
- Machine Learning: a Probabilistic Perspective, by Kevin Murphy.
Homework Guidelines*
We expect you to try solving each problem set on your own. However, when being stuck on a problem, I encourage you to collaborate with other students in the class, subject to the following rules:
- You may discuss a problem with any student in this class, and work together on solving it. This can involve brainstorming and verbally discussing the problem, going together through possible solutions, but should not involve one student telling another a complete solution.
- Once you solve the homework, you must write up your solutions on your own, without looking at other people's write-ups or giving your write-up to others.
- In your solution for each problem, you must write down the names of any person with whom you discussed it. This will not affect your grade.
- Do not consult solution manuals or other people's solutions from similar courses - ask the course staff, we are here to help!
*adpated from David Sontag