Optimization lies at the very heart of machine learning driving the success of modern artificial intelligence. This course offers introductory lectures on the fundamental ideas in mathematical optimization for machine learning. Students will also gain practical experience by performing a mini group project throughout the course.
Code CSED490Y
Term Spring 2022
Audience UG and PG students at POSTECH
Lectures Mondays and Wednesdays 11am-12:15pm (on Zoom)
Office hours Thursdays 5-6pm (by appointment)
Online PLMS
Instructor Namhoon Lee (namhoonlee@postech.ac.kr)
TA Jinseok Chung (jinseokchung@postech.ac.kr)
CA Dongyun Kim (dykim97@postech.ac.kr)
Week 01
Introduction
[Mon 21 Feb]
[Wed 23 Feb]
Week 02
Basics of machine learning and optimization
[Mon 28 Feb]
[Wed 2 Mar]
Week 03
Convex set
[Mon 7 Mar]
[Wed 9 Mar H]
Week 04
Convex function and gradient descent
[Mon 14 Mar]
[Wed 16 Mar]
Week 05
Gradient descent
[Mon 21 Mar H]
[Wed 23 Mar]
Week 06
Subgradient method
[Mon 28 Mar]
[Wed 30 Mar]
Week 07
Projected gradient descent
[Mon 4 Apr]
[Wed 6 Apr]
Week 08
Proximal gradient descent
[Mon 11 Apr H]
[Wed 13 Apr]
Week 09
Stochastic gradient descent
[Mon 18 Apr]
[Wed 20 Apr]
Week 10
Midterm exam
[Mon 25 Apr: midterm]
[Wed 27 Apr]
Week 11
Accelerated methods
[Mon 2 May]
[Wed 4 May]
Week 12
Variance reduced methods
[Mon 9 May]
[Wed 11 May: rg
6,
4,
3,
7,
10]
Week 13
Second order methods
[Mon 16 May]
[Wed 18 May: rg
1,
5,
9,
11,
8]
Week 14
Distributed optimization
[Mon 23 May]
[Wed 25 May: rg
2,
12,
13]
Week 15
Project presentations
[Mon 30 May: project
13,
12,
2,
8,
11,
9]
[Wed 1 Jun H]
Week 16 Final exam Project presentations
[Mon 6 Jun H]
[Wed 8 Jun: project
5,
1,
10,
7,
3,
4,
6]
day date: slides H: holidays/no class
The purpose is to get hands-on experience by actually implementing optimization algorithms for some machine learning models and practical applications. Students are encouraged to team up with up to three students and conduct an empirical study of the topic of their interests. By trying to reproduce expected performance, students can obtain insights into the inner workings of the model under investigation.
Participation 5%
Quizzes 15%
Midterm exam 40% (offline; date/time/location to be announced)
Final project 40% (= 5% proposal + 10% presentation + 25% final report)
I don't know much about optimization or ML. Can I still take this course? Yes.
My math skills are quite rusty. Can I still take this course? Yes.
Is it okay if I prefer working alone for the project? Yes (if you are certain).
Do I need a prior research experience? No.
As an undergrad, do I need to worry about graduate students in the same classroom? No.