A Survey on LLM Routing: Objectives,Methods, and Evaluation

The Hong Kong University of Science and Technology (Guangzhou)

Data Science and Analytics Thrust

PhD Qualifying Examination

By Mr. ZENG, Jiahao

Abstract

Large language models (LLMs) are now used in many applications, but they often vary substantially across key dimensions such as capability, cost, and latency. As a result, using a single fixed model for all inputs is often suboptimal. Simple queries may be answered well by smaller and cheaper models, while complex queries may require stronger models with better reasoning ability or domain knowledge. LLM routing addresses this problem by adaptively selecting the most suitable model for each input according to the needs of the application. This survey provides a structured review of LLM routing as a decision-making problem over heterogeneous models. We discuss how routing systems define their objectives, how they make modelselection decisions, and how they are evaluated in existing benchmarks. We also discuss open challenges that remain underexplored. By organizing existing work around objectives, methods, and evaluation, this survey aims to clarify the current design space of LLM routing and identify future opportunities for building more efficient and flexible LLM systems.

PQE Committee