OFFLINE REINFORCEMENT LEARNING

The Hong Kong University of Science and Technology (Guangzhou)

Data Science and Analytics Thrust

PhD Thesis Proposal Examination

By Ms. Jing ZHANG

Abstract

Reinforcement learning (RL) has achieved remarkable success in various manipulation and goal-conditioned tasks. However, the high cost of interaction has limited the potential of RL. Offline reinforcement learning focuses on learning valuable policies from pre-collected data, thereby extending the application of RL to numerous fields. Nevertheless, the absence of online interaction poses significant obstacles to the performance of offline RL algorithms. The primary challenges in learning a policy from a static dataset involve distribution shifts and extrapolation errors during policy exploration. And these problems have made most RL methods struggle in offline scenarios. This proposal aims to provide a concise overview of the key challenges encountered in offline RL and offers a brief analysis of the crucial aspects that need to be addressed. Furthermore, we will discuss potential solutions to overcome these challenges and explore perspectives on unresolved issues in the field.

TPE Committee

Chairperson: Prof. Qiong LUO

Prime Supervisor: Prof Wenjia WANG

Co-Supervisor: Prof Molong DUAN

Examiner: Prof Yanlin ZHANG