Title: Is This Predictor More Informative than Another? A Decision-Theoretical Comparison

Speaker: Yiding Feng, Assistant Professor, The Hong Kong University of Science and Technology

Host: Zhiyu Zeng, Assistant Professor, Antai College of Economics and Management, Shanghai Jiao Tong University

Time: 14:00‑15:30, Tuesday, September 22, 2026

Venue: Room A507, Pao Siu‑loong Library, Antai College of Economics and Management, Shanghai Jiao Tong University

Brief introduction of the content: 

In many real-world applications, a model provider provides probabilistic forecasts to downstream decision-makers who use them to make decisions under diverse payoff objectives. The provider may have access to multiple predictive models, each potentially miscalibrated, and must choose which model to deploy in order to maximize the usefulness of predictions for downstream decisions. A central challenge arises: how can the provider meaningfully compare predictors when neither is guaranteed to be well-calibrated, and when the relevant decision tasks may differ across users and contexts?

To answer this, our first contribution introduces the notion of the informativeness gap between any two predictors, defined as the maximum normalized payoff advantage one predictor offers over the other across all decision-making tasks. Our framework strictly generalizes several existing notions including U-Calibration [Kleinberg et al., 2023] and Calibration Decision Loss [Hu and Wu, 2024], and it recovers Blackwell informativeness [Blackwell, 1951, 1953] as a special case when both predictors are perfectly calibrated. Our second contribution is a dual characterization of the informativeness gap. We show that this measure satisfies natural desiderata: it is complete and sound, and it can be estimated sample-efficiently in the prediction-only access setting. We complement our theory with experiments on LLM-based forecasters in real-world prediction tasks, showing that the informativeness gap offers a more decision-relevant alternative to traditional metrics, and provides a principled lens for evaluating how ad hoc calibration post-processing affects downstream decision usefulness.

Speaker's profile:

Yiding Feng is an Assistant Professor in the Department of Industrial Engineering and Decision Analytics at the Hong Kong University of Science and Technology (HKUST). Prior to joining HKUST, he was a Principal Researcher at the University of Chicago Booth School of Business and a Postdoctoral Researcher at Microsoft Research New England. He received his Ph.D. in Computer Science from Northwestern University in 2021 and his B.S. from the ACM Honors Class at Shanghai Jiao Tong University in 2016.

His research interests lie at the intersection of operations research, economics and computation, and theoretical computer science. His work has been published in leading journals such as Management Science, Operations Research and Quantitative Economics, as well as top theoretical computer science and EconCS conferences including STOC, FOCS, SODA, EC, and WINE. He was a recipient of the INFORMS Auctions and Market Design (AMD) Michael H. Rothkopf Junior Researcher Paper Prize (Second Place) and the APORS Young Researcher Best Paper Award.

Welcome all!