Exploiting Unavailable Multimodal Data at Test Time to Enhance Healthcare Models: A Survey

The Hong Kong University of Science and Technology (Guangzhou)

Data Science and Analytics Thrust

PhD Qualifying Examination

By Mr. Lin CHEN

Abstract

Benefiting from the complementary information that multimodal data provide, utilizing multimodal data has emerged as a promising approach to enhance the performance of healthcare models. However, in real-world scenarios, the collection of multimodal data is often limited by challenges such as complexity, cost, and privacy concerns, making it difficult to gather comprehensive multimodal datasets. This limitation restricts the use of models based on multimodal data. This survey reviews research in the healthcare field on the utilization of multimodal data to enhance models when complete multimodal datasets are inaccessible during testing. We categorize these methods into four types: masked missing modalities, modality generation, knowledge transfer, and data augmentation. Additionally, this survey discusses the advantages and limitations of these methods, as well as future research directions. By providing a comprehensive overview of current strategies and identifying gaps in existing research, this paper aims to guide future efforts to overcome the challenges associated with the utilization of multimodal data in healthcare settings, thereby enhancing the robustness and adaptability of healthcare models.

PQE Committee

Chairperson: Prof. Nan TANG

Prime Supervisor: Prof Kaishun WU

Co-Supervisor: Prof Yingcong CHEN

Examiner: Prof Yanlin ZHANG