The primary goal of this course is to provide ideas and analysis for convex optimization problems that arise frequently in many scientific and engineering disciplines. This includes first-order methods for both unconstrained and constrained optimization problems, duality theory and dual-based methods, and possibly some modern methods for large-scale optimization problems. The course also includes assignments on theory and exercises.
Code CSED700H or AIGS700H
Term Fall 2023
Audience PG and UG students at POSTECH
Lectures Tuesdays and Thursdays 9:30-10:45am (Room 102 in Eng bldg Ⅱ)
Office hours Wednesdays 5-6pm (by appointment)
Online PLMS
Instructor Namhoon Lee (namhoonlee@postech.ac.kr)
TA Jinhwan Nam (njh18@postech.ac.kr)
1
Introduction
2
Convex sets
3
Convex functions
4
Convex optimization problems
5
Duality
6
Gradient methods
7
Proximal gradient methods
8
Accelerated gradient methods
9
Second-order methods
10
Stochastic optimization
11
Dual-based optimization
12
Constrained optimization
13
Large-scale optimization
14
Nonconvex optimization
supplementary: logistics, mathematical background
TBA
This course will frequently borrow materials from multiple sources including but not limited to the following:
(book) Convex Optimization by Stephen Boyd and Lieven Vandenberghe
(book) Convex Optimization: Algorithms and Complexity by Sébastien Bubeck
(book) Numerical Optimization by Jorge Nocedal and Stephen J. Wright
(lecture) Convex Optimization by Ryan Tibshirani
(lecture) Convex Optimization by Stephen Boyd
(lecture) Optimization Methods for Large-Scale Systems by Lieven Vandenberghe
(lecture) Optimization Algorithms by Constantine Caramanis
(lecture) First-Order Optimization Algorithms for Machine Learning by Mark Schmidt