On June 26, 2026, Dr. Richard Tai-Chiu Hsung from the Department of Computer Science at Hong Kong Chu Hai College, along with MSAAI student Tianyuan Chen, presented their latest research findings at 2026 IEEE 7th International Conference on Artificial Intelligence and Electromechanical Automation (AIEA2026).
- Tianyuan Chen, Richard Tai-Chiu Hsung*, Harris Sik-Ho Tsang, Wai-Lun Lo, Tony Yu-Lin Zhu, Xiaoxing Yang, Billy Chiu and Walter Lam, “Choice-Head Distillation for Dental Multiple-Choice Question Answering,” 2026 IEEE 7th International Conference on Artificial Intelligence and Electromechanical Automation, Shenzhen, China, June 26-28, 2026. (AIEA2026)
Medical LLMs perform well on benchmarks but are costly to deploy in real-world settings, especially for dental multiple-choice QA (e.g., CMExam). Existing distillation methods often still train on the full vocabulary distribution. We propose Choice-Head distillation, which transfers the teacher’s answer-option distribution only, enabling lighter and decision-space distillation compatible with black-box API teachers. Experiments on CMExam (991-question full test set) use DeepSeek-V3 as the teacher and Qwen2.5-7B/14B as students on a resplit. In multiple runs, students match or exceed teachers: the best 14B model achieves 89.10% accuracy (vs. 87.18% teacher), with a three-seed mean of 88.67%, indicating consistent gains. Results also suggest benefits from a subsequent hard-label fine-tuning stage. Overall, for structured medical multiple-choice tasks, choice-level distillation is computationally lighter, easier to deploy, and works with black-box teachers.
The Hong Kong Chu Hai College remains committed to supporting students’ research endeavors, encouraging publication in international conferences and journals. More are coming!
Fig.1 – Our presentation session at AIEA 2026..
