Research statement

My research focuses on solving large-scale machine learning problems. This challenge arises frequently in modern machine learning concerning a lot of data and complex learning models. Some examples include understanding the convergence and generalization characteristics of artificial neural networks and developing efficient and robust optimization methods in stochastic and distributed settings. I pursue this goal by investigating theoretical and applied aspects of the principles and practices in optimization as much as by studying the science of deep learning. My vision is to engage in the broad field of artificial intelligence and push the frontiers of the state of the art in machine learning research.

Publications

[Google Scholar profile]

• Pruning Neural Networks with Velocity-Constrained Optimization
  Donghyun Oh*, Jinseok Chung*, Namhoon Lee
  NeurIPS 2023 Workshop on Optimization for Machine Learning
• The Effects of Overparameterization on Sharpness-Aware Minimization: An Empirical and Theoretical Analysis
  Sungbin Shin*, Dongyeop Lee*, Maksym Andriushchenko, Namhoon Lee
  arXiv preprint; ICML 2023 Workshop on High-dimensional Learning Dynamics
• On the Effectiveness of Sharpness-Aware Minimization with Large Mini-batches
  Jinseok Chung, Seonghwan Park, Jaeho Lee, Namhoon Lee
  ICML 2023 Workshop on High-dimensional Learning Dynamics
• FedFwd: Federated Learning without Backpropagation
  Seonghwan Park, Dahun Shin, Jinseok Chung, Namhoon Lee
  ICML 2023 Workshop on Federated Learning and Analytics in Practice
• Semi-supervised Concept Bottleneck Models
  Jeeon Bae, Sungbin Shin, Namhoon Lee
  ICML 2023 Workshop on Artificial Intelligence and Human-Computer Interaction
• A Closer Look at the Intervention Procedure of Concept Bottleneck Models
  Sungbin Shin, Yohan Jo, Sungsoo Ahn, Namhoon Lee
  ICML 2023; NeurIPS 2022 Workshop on Trustworthy and Socially Responsible ML
• Almost Sure Last Iterate Convergence of Sharpness-Aware Minimization
  Kyunghun Nam, Jinseok Chung, Namhoon Lee
  ICLR 2023 Tiny Papers
• JaxPruner: A Concise Library for Sparsity Research
  Joo Hyung Lee, Wonpyo Park, Nicole Mitchell, Jonathan Pilault, Johan Obando-Ceron, Han-Byul Kim, Namhoon Lee, Elias Frantar, Yun Long, Amir Yazdanbakhsh, Shivani Agrawal, Suvinay Subramanian, Xin Wang, Sheng-Chun Kao, Xingyao Zhang, Trevor Gale, Aart Bik, Woohyun Han, Milen Ferev, Zhonglin Han, Hong-Seok Kim, Yann Dauphin, Gintare Karolina Dziugaite, Pablo Samuel Castro, Utku Evci
  CPAL 2024 (Oral); ICLR 2023 Workshop on Sparsity in Neural Networks
• MaskedKD: Efficient Distillation of Vision Transformers with Masked Images
  Seungwoo Son, Namhoon Lee, Jaeho Lee
  ICLR 2023 Workshop on Sparsity in Neural Networks
• Meta-Learning Sparse Implicit Neural Representations
  Jaeho Lee*, Jihoon Tack*, Namhoon Lee, Jinwoo Shin
  NeurIPS 2021; SNN 2021
• Toward Efficient Deep Learning with Sparse Neural Networks
  Namhoon Lee
  Ph.D. Thesis, University of Oxford
• Optimal Mini-Batch Size for Stochastic Gradient Methods
  Namhoon Lee, Philip H. S. Torr, Richard Hartley
  ICLR 2021 Workshop on Science and Engineering of Deep Learning
• Understanding the Effects of Data Parallelism and Sparsity on Neural Network Training
  Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr, Martin Jaggi
  ICLR 2021
• Data Parallelism in Training Sparse Neural Networks
  Namhoon Lee, Philip H. S. Torr, Martin Jaggi
  ICLR 2020 Workshop on Practical ML
• A Signal Propagation Perspective for Pruning Neural Networks at Initialization
  Namhoon Lee, Thalaiyasingam Ajanthan, Stephen Gould, Philip H. S. Torr
  ICLR 2020 (Spotlight)
• SNIP: Single-shot Network Pruning based on Connection Sensitivity
  Namhoon Lee, Thalaiyasingam Ajanthan, Philip H. S. Torr
  ICLR 2019
• Learn to Pay Attention
  Saumya Jetley, Nicholas Lord, Namhoon Lee, Philip H. S. Torr
  ICLR 2018
• Synthesizing a Scene-Specific Pedestrian Detector and Pose Estimator for Static Video Surveillance
  Hironori Hattori*, Namhoon Lee*, Vishnu N. Boddeti, Fares Beainy, Kris M. Kitani, Takeo Kanade
  IJCV 2018
• DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents
  Namhoon Lee, Wongun Choi, Paul Vernaza, Christopher B. Choy, Philip H. S. Torr, Manmohan Chandraker
  CVPR 2017 (Spotlight)
• Forecasting Interactive Dynamics of Pedestrians with Fictitious Play
  Wei-Chiu Ma, De-An Huang, Namhoon Lee, Kris M. Kitani
  CVPR 2017
• Modeling of Dynamic Environments for Visual Forecasting of American Football Plays
  Namhoon Lee (committee: Kris M. Kitani, Martial Hebert, Sebastian Scherer)
  M.S. Thesis, Carnegie Mellon University, Dec 2015
• Predicting Wide Receiver Trajectories in American Football
  Namhoon Lee, Kris M. Kitani
  WACV 2016 (Oral)

Talks

• Effects of overparameterization on sharpness-aware minimization.
  Chung-Ang University (2023).
• Human-interpretable AI with concept bottleneck models.
  Korean Society of Artificial Intelligence in Medicine (2023).
• Honouring the 2018 ACM Turing Award Laureates.
  KAIST Turing lecture commentary (2023).
• Toward efficient deep learning with sparse neural networks.
  KAIST (2021), Samsung Research (2021), UNIST (2021), POSTECH (2021), IPIU rising researchers (2021), Oxford University (2020).
• A signal propagation perspective for pruning neural networks at initialization.
  ICLR spotlight (2020).
• On the compression of random neural networks: A signal propagation perspective.
  Hanyang University (2020), École Polytechnique Fédérale de Lausanne (2019).
• Efficient Neural Network Compression.
  Microsoft Research Redmond (2019), Australian National University (2019).
• DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents.
  Wayve (2019), FiveAI (2017), CVPR spotlight (2017).