TABLE OF CONTENTS
1. Introduction
We ask a simple yet theoretically challenging question:
How can we trust systems driven by data?
This semester (Spring 2027) will be the third offering of the course, and we are broadening our target audience.
- This course is research-domain-agnostic and largely self-contained, and students from all backgrounds and departments are welcome.
- Juniors/seniors are encouraged to enroll, although it is a graduate-level course. Undergraduates have done wonderful work in previous offerings.
- The only expected background is a basic understanding of linear algebra and probability.
As data-driven systems, including those powered by AI, are increasingly deployed in high-consequence applications, mathematically understanding and predicting their behavior is becoming more important than ever. We explore the open problems behind this challenge.
While this course is not tied to any particular application, examples of such systems include:
- Space robots adapting to new environments
- Neural nets monitoring Earth from satellite imagery
- Self-driving cars navigating traffic with semantic cues
- AI trading bots making investment decisions
- LLMs debugging code using human feedback
- VLMs learning to play video games
- Robots w/ VLAs assisting patients with daily activities.
The figures above illustrate representative applications from our research group.
2. Course Format
We primarily use lecture slides like these. Our past lecture materials are partially available here for reference (https://hirotsukamoto.com/contraction/).
3. Topics
At various levels of detail, this course would cover:
- Static Estimation: How Can We Determine Static Unknowns from Data?
- Brief Review of Probability & Statistics
- Least Squares
- Minimum Variance Estimation
- Maximum Likelihood Estimation
- Parametric Learning: How Can We Learn Static Unknowns from Data?
- Gradient Decent
- Stochastic Gradient Descent
- Learning Convergence
- Regret Bounds in Online Learning
- Robust Estimation: How Can We Robustly Determine Dynamic Unknowns from Data?
- Robust Lyapunov Theorem
- Nonlinear Filtering
- Contraction Theory
- Partial Contraction
- Statistical Inference: How Much Can We Trust Conclusions Drawn from Limited Data?
- Probably Approximately Correctness (PAC)
- PAC-Bayes
- Scenario Approach
- Conformal Inference
- Statistical Robustness: How Can We Trust Data-Driven Dynamical Systems?
- Robust Constraint Satisfaction
- Conformal Lyapunov
- Conformal contraction
- Introduction to Statistical Adaptation: How Can We Trust Continually Changing Data-Driven Systems?
- Adaptive Statistical Inference
- Adaptive Estimation/Control
- Certified Online Adaptation
- Examples & Open Problems
4. Assignments & Grading
There will be up to five assignments, depending on our progress. I’m tempted to make the midterm and final take-home trusting you, but I may have to make them proctored because of AI. They will be posted on and submitted through Canvas. Collaboration is strongly encouraged for assignments, but is not allowed for exams.
I’ll also ask you to submit a one-page research proposal as your final project. I’ll do my best to give you feedback on whether your idea could lead to a paper. To encourage ambitious ideas, you’ll get full credit just for submitting it. The grades will be weighted as follows:
- Assignments: 50%
- Midterm: 20%
- Final: 20%
- Final Project: 10%.
5. Recommended Textbooks
There are no required textbooks, but some of the course topics are based on the following:
- Optimal Estimation of Dynamic Systems, John L. Crassidis and John L. Junkins
- Algorithmic Learning in a Random World, Vladimir Vovk, Alexander Gammerman, and Glenn Shafer
- Applied Nonlinear Control, Jean-Jacques E. Slotine and Weiping Li
- Nonlinear Systems, Hassan K. Khalil.
6. Use of Generative AI
I am personally in favor of using generative AI as a personal learning assistant. However, you must always be mindful of their hallucinations. At least for now, you should still be the one to judge whether an AI-generated response is correct or not, exactly as you would when using search engines. Thus, for homework assignments, I do not care how you reach the solution, as long as you derive it yourself. While it is impossible to perfectly prevent every way of cheating with these tools, at a minimum, please do not prompt generative AI to “solve my homework assignment.” I trust your integrity, and I hope everyone will use these tools to learn efficiently, not merely to pass this course, the latter of which would be just a waste of your time.
The use of generative AI is prohibited on exams. Violations of this policy could result in immediate failure of the course.
7. Honor Code
The details can be found here (https://studentcode.illinois.edu/article1/part4/1-401/). There is also a good summary of the honor code here (https://siebelschool.illinois.edu/academics/honor-code) from the School of Computing and Data Science. As mentioned earlier, students are encouraged to discuss coursework with peers, provided all collaboration is appropriately acknowledged and cited; all submitted work must be your own, and suspected academic integrity violations (including plagiarism) may be reported through the Faculty Academic Integrity Reporting (FAIR: https://history.illinois.edu/resources/faculty-resources/reporting-academic-integrity-infractions) system and may result in a zero on the assignment or, in some cases, failure of the course.
8. Student Support
This course is committed to a safe, inclusive learning environment; concerns involving racism, discrimination, or other offensive behavior may be raised with the course instructor or reported to Campus Belonging (https://access.illinois.edu/compliance/accommodations-support/campus-belonging/). Students seeking disability-related accommodations can contact the instructor and Disability Resources and Educational Services (DRES: https://dres.illinois.edu/); DRES also offers academic screening for undiagnosed conditions. The University will reasonably accommodate religious observances; students should review this syllabus early and notify the instructor of any conflicts through Student Affairs (https://odos.illinois.edu/resources/students/religious-observances). Finally, faculty and staff must report disclosures of sexual misconduct to the Title IX Office (https://access.illinois.edu/title-ix/office/), while confidential resources and additional reporting information are available on We Care (https://wecare.illinois.edu/).
9. Instructor Information
