Fanqi Wan (万凡琦)

Hello! My name is Fanqi Wan, and I am a final-year MS student at Sun Yat-sen University, advised by Prof. Xiaojun Quan. Before this, I received my Bachelor's degree from Xi'an Jiaotong University. Previously, I conducted my internship at Tencent AI Lab (2023.03-2024.05), where I am mentored by Dr. Xinting Huang and Dr. Wei Bi. Now, I am currently conducting my internship at ByteDance Doubao (Seed) LLM Team (2024.06-Now).

Email / CV(en/zh) / Google Scholar / Semantic Scholar / GitHub / HF / X

My main research interests focused on deep learning for natural language generation. Previously, my work primarily focused on dialogue systems. After the emergence of large language models (LLMs), my research direction shifted towards efficient alignment (e.g., developing LLMs for specific domains, mitigating hallucinations of LLMs, self-improving of LLMs) and knowledge fusion (e.g., combining the strengths of LLMs with diverse structures/scales/functionarities). Representative papers are highlighted.

We propose FuseChat, an extended framework of FuseLLM to integrate the collective knowledge and individual strengths of multiple structure- and scale-varied chat LLMs into a more powerful chat LLM. FuseChat-7B is comparable to the larger Mixtral-8x7B-Instruct and and approaches GPT-3.5-Turbo-1106 on MT-Bench.

We introduce a novel approach that enhances the fusion process by incorporating both the training and inference modes from multiple structurally diverse LLMs.

We propose FuseLLM to create a unified model that combines the distinctive strengths of multiple structurally diverse LLMs. FuseLLM-7B surpasses Llama-2-7B on 12 benchmarks, including commonsense, reasoning, question-answering, and code generation.

We introduce self-evolution fine-tuning for policy optimization, which eliminates the need for annotated data samples during alignment.

We introduce Knowledge Consistent Alignment to verify and minimize the knowledge inconsistency between external knowledge in the alignment data and the intrinsic knowledge embedded in foundation LLMs, thus mitigating hallucinations before alignment.

We propose a novel approach to enhance the domain-specific instruction coverage by utilizing LLMs to explore the domain space from both breadth and depth automatically. Explore-Instruct outperforms Self-Instruct in three specific domains.

We introduce maximal marginal likelihood for retriever training to address the retrieval-generation misalignment in end-to-end task-oriented dialogue systems.

We propose a multi-grained knowledge retriever and introduce a novel distillation objective for retriever training. MAKER achieves SOTA performance on MultiWOZ 2.1 and CamRest with both condensed KB and full KB.

We propose a training-free structured pruning approach for fine-grained LLMs pruning.

We propose a novel method for zero-shot personality detection in a multi-turn dialogue manner.

We propose a novel method that incorporates cluster information into contrastive learning for unsupervised sentence representation learning.

MS Student in Computer Science, Sun Yat-sen University (2022.9-now).

Bachelor of Automation, Xi'an Jiaotong University (2018.9-2022.6).

Research Intern at LLM Team, ByteDance Doubao (Seed). (2024.6-now).

Research Intern at NLPC, Tencent AI Lab, supervised by Dr. Xinting Huang and Dr. Wei Bi. (2023.3-2024.5).

Commercial Projects on E-Commerce Platforms, Vipshop, supervised by Dr. Rui Wang. (2022.4-2023.1).

2nd Prize on 2023 Xingzhi Cup Deep Learning Model Interpretability Task. (Team Leader) [Task]

2nd Prize on 2022 IFLYTEK AI Developer Competition Paper Abstracts Classification Task. (Team Leader) [Task]

3nd Prize on 2022 Ali Lingjie E-commerce Search Algorithm Competition. (Team Leader) [Task]

NeurIPS Reviewer, 2024-.

ICLR Reviewer, 2025-.

AISTATS Reviewer, 2025-.

National Scholarship, Sun Yat-sen University, 2023.

Outstanding Award for Tencent AI Lab Rhino-Bird Focused Research Program, Tencent, 2023.

Excellent Graduate Students, Xi'an Jiaotong University, 2022.

National Scholarship, Xi'an Jiaotong University, 2018.

Website's code is from Jon Barron.