题　目：Efficient Data-driven Methods for Robust Sequential Decision Making

嘉　宾：Yan Li，Incoming Assistant Professor，Texas A&M University

主持人：江浦平 助理教授 上海交通大学安泰经济与管理学院

时　间：2024年6月27日（周四）14:00-15:30pm

地　点：安泰楼A503室

内容简介：

Markov decision processes involve a decision-maker seeking the optimal policy to sequentially interact with an environment in order to complete a certain task. While such a paradigm has brought substantial empirical successes, it is also known that the optimized policy often exhibits brittle robustness against potential environment disturbances. Hence when exploring its applications in critical domains such as healthcare engineering, it is imperative to ensure robust and trustworthy decisions are being made. In this talk, we attempt to tackle this problem under the framework of robust Markov decision processes, formulated as a minimax game between the decision-maker and an adversarially changing environment. We reveal the underlying pseudo-convexity of the non-convex, non-smooth objective of the decision-maker. Consequently, we introduce a novel first-order method that achieves optimal iteration and sample complexities for the first time in the literature. While constructing the first-order information, we address the problem of evaluating the worst-case performance of a given policy. In particular, we exploit the dynamic nature of this non-concave problem and propose a globally convergent, model-free method with optimal performances. We also discuss its natural variant capable of incorporating function approximation to handle large-scale problems, thereby addressing an important yet unresolved question for robust Markov decision processes.

演讲人简介：

Yan Li is a PhD student at the School of Industrial and Systems Engineering, Georgia Institute of Technology. He will be joining the Department of Industrial and Systems Engineering, Texas A&M University as an assistant professor starting Fall 2024. His research centers around the algorithmic and theoretical foundations of data science. Much of his recent effort has been devoted to sequential decision-making problems and their robust counterparts. His research has been recognized by the Alice and John Jarvis Ph.D. Student Research award, the Margaret and Stephen Kendrick Research Excellence award, and the IDEaS-TRIAD fellowship from Georgia Tech.

欢迎广大师生参加！