Principled Exploration in Sequential Decision Making

ABSTRACT

Interactive Machine Learning (IML) possesses the unique capability to harness feedback from interactions, making it indispensable in a wide array of real-world sequential decision-making applications. However, a significant challenge, known as the "exploitation and exploration dilemma," prominently arises within sequential decision-making. In this context, learners must not only exploit current information but also explore to uncover potential knowledge for long-term gains. Despite decades of research yielding a rich landscape of algorithms, frameworks, and theories for effectively utilizing collected data to train machine learning models, a fundamental question has remained largely unaddressed: How can IML models systematically make principled exploration for long-term benefits, alongside the full exploitation of current data? In this report, I will motivate the exploration strategies by human principles in sequential decision-making and present our research efforts in developing principled exploration strategies in IML systems with theoretical performance guarantees and real-world applications.

SPEAKER BIO

Yikun Ban is a Ph.D. candidate in the Department of Computer Science at the University of Illinois at Urbana–Champaign. He is a member of DAIS (Data and Information Systems) Research Lab. He received his M.CS. degree from Peking University in 2019 and B.Eng. degree from Wuhan University in 2016. His research interests lie in multi-armed bandits/Reinforcement Learning to design and develop principled exploration strategies in sequential decision-making. He has published more than 10 papers at top conferences in Machine Learning and Data Mining (e.g., NeurIPS, ICLR, KDD, WWW, AAAI) and has been a reviewer or program committee member of mainstream machine learning journals and conferences. He was an applied scientist intern at Amazon Web Service, and his research works have been powering primary applications in Amazon and Instacart.