Multi-Armed Bandit Algorithms and Applications

The Hong Kong University of Science and Technology (Guangzhou)

Data Science and Analytics Thrust

PhD Qualifying Examination

By Ms. Suizi HUANG

Abstract

The multi-armed bandit (MAB) problem is a fundamental problem in decision theory and operations research. It involves a sequential decision-making process in order to maximize the expected reward in the long term. The main challenge is to balance the exploration of new options and the exploitation of known options. In recent years, the MAB problem has gained significant attention in both academia and industry due to its strong ability to make decisions under uncertainty. One of the extensions to the traditional MAB problem is the contextual bandit (CB) problem, where additional contextual information is available to help with the decision-making process. In this survey, we provide a comprehensive overview of different algorithms that have been developed for both non-contextual and contextual MAB problems by discussing the strengths and weaknesses of each algorithm and providing insights into the theoretical guarantees. We also present various real-life applications and provide the potential future directions.

PQE Committee

Chairperson: Prof. Xiaowen CHU

Prime Supervisor: Prof Wenjia WANG

Co-Supervisor: Prof Xinzhou GUO

Examiner: Prof Wei ZENG