题　目：Sharp Characterization and Control of Global Dynamics of SGDs with Heavy Tails

嘉　宾：Xingyu Wang, Postdoctoral Researcher, University of Amsterdam

主持人：唐卓栋 助理教授 上海交通大学安泰经济与管理学院

时　间：2025年12月4日（周四）14:00-15:30

地　点：安泰楼A511室

内容简介：

The empirical success of deep learning is often attributed to the mysterious ability of stochastic gradient descents (SGDs) to avoid sharp local minima in the loss landscape, as sharp minima are believed to lead to poor generalization. To unravel this mystery and potentially further enhance such capability of SGDs, it is imperative to go beyond the traditional local convergence analysis and obtain a comprehensive understanding of SGDs’ global dynamics within complex non-convex loss landscapes. In this talk, we characterize the global dynamics of SGDs through the heavy-tailed large deviations and local stability framework. This framework systematically characterizes the rare events in heavy-tailed dynamical systems; building on this, we characterize intricate phase transitions in the first exit times, which leads to the heavy-tailed counterparts of the classical Freidlin-Wentzell and Eyring-Kramers theories. Moreover, applying this framework to SGD, we reveal a fascinating phenomenon in deep learning: by injecting and then truncating heavy-tailed noises during the training phase, SGD can almost completely avoid sharp minima and hence achieve better generalization performance for the test data.

演讲人简介：

Xingyu Wang is a postdoc researcher at the Department of Quantitative Economics at the University of Amsterdam. He earned his Ph.D. from the Department of Industrial Engineering and Management Sciences at Northwestern University. His research interests lie in applied probability, machine learning, and stochastic simulation. His research has been recognized with awards including second place in the 2023 George Nicholson Student Paper Competition (INFORMS) and the 2025 Nemhauser Best Dissertation Prize (Northwestern University).

欢迎广大师生参加！