MCS 548 - Mathematical Theory of Artificial Intelligence
University of Illinois - Chicago
Fall 2018

This class will focus on the foundations of machine learning theory. Example topics include inductive inference, query learning, PAC learning and VC theory, Occam's razor, online learning, boosting, support vector machines, bandit algorithms, statistical queries, Rademiacher complexity, and neural networks.

This course is represented on the computer science prelim.

Basic Information

Syllabus: pdf
Time and Location: M-W-F 1:00pm - 1:50pm, Lincoln Hall (LH) 104
Instructor Contact Information: Lev Reyzin, SEO 418, (312)-413-3745,
Required Textbook: Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. Foundations of Machine Learning (available online via UIC library)
Optional Textbook: Shai Shalev-Shwartz and Shai Ben-David. Understanding Machine Learning: From Theory to Algorithms
Office Hours: M 10:00-10:50am, W 11:00-11:50am

Exam Dates

Final Exam: Monday December 10, 1:00-3:00pm in LH 104

Projects and Presentations

Problem Sets

Lectures and Readings

Note: lectures will have material not covered in the readings.

Lecture 1 (8/27/18)
covered material: intro to the course, preview of learning models
reading: section 7 of Computing Machinery and Intelligence by Turing (1950)

Lecture 2 (8/29/18)
covered material: positive and negative results for learning in limit from text and informant
reading: Language Idenification in the Limit by Gold (1967)
optional reading: Theorem 13.5 from 13.4.1 of Mohri et al.

Lecture 3 (8/31/18)
covered material: efficient exact learning, membership and equivalence queries, L* algorithm of exact learning of regular languages
reading: 13.3.3 Mohri et al.
next: continuing with L*

Lecture 4 (9/5/18)
covered material: continuing L* algorithm, relating EQ and MQ with learning in the limit
reading: 13.1 and 13.2 of Mohri et al.
optional reading: Learning Regular Sets from Queries and Counterexamples by Angluin (1987)
next: PAC (section 2.1)

Lecture 5 (9/7/18)
covered meterial: PAC learning of axis-aligned regtangles; PAC guarantees for finite realizable case
reading: 2.1 and 2.2 of Mohri et al.; A Theory of the Learnable by Valiant (1984)
optional reading: Occam's Razor by Blumer et al. (1986)

Lecture 6 (9/8/18)
covered material: PAC guarantees for inconsistent and agnostic learning reduction from query learning to PAC + MQ
reading: 2.3, 2.4.1, and 13.3.2 of Mohri et al.

Lecture 7 (9/14/18)
covered material: introduction to Rademacher complexity
reading beginning of 3.1

Lecture 8 (9/17/18)
covered material: Rademacher generalization bounds
reading finish 3.1
optional reading: Rademacher Penalties and Structural Risk Minization by Koltchinskii (2001)