|
Quzhe Huang
Email:huangquzhe [AT] pku [DOT] edu [DOT] cn
|
|
- Comprehensive Experience in LLM Training: I have had the opportunity to participate in projects covering the full training procedure of LLMs, including Pretraining, SFT, and RLHF. In many of these projects, I have taken on a leadership role.
- Specialization in Domain-Specific LLMs: I developed the first Chinese Legal LLM, Lawyer LLaMA. I am particularly interested in how domain experts can collaborate with LLMs to enhance the models' domain-specific reasoning skills.
- Interest in Multimodal LLMs: I have contributed to the development of a unified language and vision pretraining framework. And I hope the LLMs could benefit more from vision information (maybe in the future).
|
|
|
Let Real Users Decide: Evaluate Role-Play Chatbot with User Simulator (Under Review)
Quzhe Huang, Kun Xu, Zhao Zhang, Tian Zhang, Liwei Chen, et, al.
TL:DR: We conduct large-scale human evaluation for Role-Play Conversation and propose an user-simulator based evaluation framework as a proxy for expected human preference.
| paper |
|
|
|
Automating Legal Concept Interpretation with LLMs: Retrieval, Generation, and Evaluation (Under Review)
Kangcheng Luo*, Quzhe Huang*, Cong Jiang, Yansong Feng
TL:DR: To free the legal experts from vague concept inpterpretation, we emulate legal doctrinal method and introduce a novel framework, ATRIE, using LLMs to AuTomatically Retrieve concept-related information, Interpret legal concepts, and Evaluate generated interpretations.
| paper |
|
|
|
JUREX-4E: Juridical Expert-Annotated Four-Element Knowledge Base for Legal Reasoning (Under Review)
Huanghai Liu*, Quzhe Huang*, Qingjing Chen, Yiran Hu, Jiayu Ma, Yun Liu, Weixin Shen, Yansong Feng
TL:DR: We publish an expert-annotated Four-Element Knowledge Base and prove its efficiency in improving the legal reasoning ability of LLMs.
| paper | Dataset |
|
|
|
Unlocking the Potential of Model Merging for Low-Resource Languages (Findings of EMNLP 2024)
Mingxu Tao*, Chen Zhang*, Quzhe Huang*, Tianyao Ma, Songfang Huang, Dongyan Zhao, Yansong Feng
TL:DR: Apply model merging to develop task-solving LLMs for lowresource languages without SFT data in the target languages.
| paper |
|
|
|
Harder Tasks Need More Experts: Dynamic Routing in MoE Models (ACL 2024)
Quzhe Huang*, Zhenwei An*, Nan Zhuang*, Mingxu Tao, Chen Zhang, Yang Jin, Kun Xu, Liwei Chen, Songfang Huang, Yansong Feng
TL:DR: We present a dynamic routing mechanism for Mixture of Experts(MoE) models that improves efficiency and performance by adjusting the number of activated experts based on input difficulty.
| paper | code |
|
|
|
Lawyer LLaMA: Enhancing LLMs with Legal Knowledge (Arxiv)
Quzhe Huang, Mingxu Tao, Zhenwei An, Chen Zhang, Cong Jiang, Zhibin Chen, Zirui Wu and Yansong Feng
TL:DR: We have designed a framework to adapt a general LLM into a specific domain and built a legal model, Lawyer LLaMA, based on the framework.
| paper | code & data |
|
|
|
Video-LaVIT: Unified Video-Language Pre-training with Decoupled Visual-Motional Tokenization (ICML 2024)
Yang Jin, Zhicheng Sun, Kun Xu, Liwei Chen, Hao Jiang, Quzhe Huang, Chengru Song, Yuliang Liu, Di Zhang, Yang Song, Kun Gai, Yadong Mu
TL:DR: Our proposed framework efficiently decomposes videos into keyframes and temporal motions, enabling unified generative pre-training for videos, images, and text.
| paper | code | website |
|
|
|
LaVIT: Unified Language-Vision Pretraining with Dynamic Discrete Visual Tokenization (ICLR 2024)
Yang Jin, Kun Xu, Kun Xu, Liwei Chen, Chao Liao, Jianchao Tan, Quzhe Huang, Bin Chen, Chenyi Lei, An Liu, Chengru Song, Xiaoqiang Lei, Di Zhang, Wenwu Ou, Kun Gai, Yadong Mu
TL:DR: Our method introduces a visual tokenizer that translates images into discrete tokens, allowing LaVIT to handle and generate multi-modal content seamlessly within a unified generative learning paradigm.
| paper | code |
|
|
|
Probing Multimodal Large Language Models for Global and Local Semantic Representations (COLING 2024)
Mingxu Tao, Quzhe Huang, Kun Xu, Liwei Chen, Yansong Feng, Dongyan Zhao
TL:DR: The topmost layers of multimodal LLMs may excessively focus on local information, leading to a diminished ability to encode global information.
| paper | code |
|
|
|
More than Classification: A Unified Framework for Event Temporal Relation Extraction (ACL 23)
Quzhe Huang, Yutong Hu, Shengqi Zhu, Yansong Feng, Chang Liu, Dongyan Zhao
TL:DR: Determining the temporal relation between events based on their start and end time points, instead of treating this problem as a simple classification task.
| paper |
|
|
|
Do Charge Prediction Models Learn Legal Theory? (Findings of EMNLP 22)
Zhenwei An*, Quzhe Huang*, Cong Jiang, Yansong Feng, Dongyan Zhao
TL:DR: An empirical study of whether charge prediction models make judgments corresponds to the decision logic of human judges.
| paper |
|
|
|
Does Recommend-Revise Produce Reliable Annotations? An Analysis on Missing Instances in DocRED (ACL 22)
Quzhe Huang, Shibo Hao, Yuan Ye, Shengqi Zhu, Yansong Feng, Dongyan Zhao
TL:DR: An analysis of recommend-revise annotation scheme, where existing models or knowledge bases are used to recommend candidate instances and annotators revise the recommendations. We find that in the revision stage, annotators cannot supplement adequate missing instances, which might cause bias in the constructed dataset and the models trained on the new data
| paper | code & data |
|
|
|
Three Sentences Are All You Need: Local Path Enhanced Document Relation Extraction (ACL 21)
Quzhe Huang, Shengqi Zhu, Yansong Feng, Yuan Ye, Yuxuan Lai, Dongyan Zhao
TL:DR: Empirically showed that determining the relation between entities in long texts only requires limited evidence, and proposed a method based on co-reference and multi-hop reasoning to select evidence sentences for document-level RE.
| paper | code |
|
|
|
Exploring Distantly-Labeled Rationales in Neural Network Models (ACL 21)
Quzhe Huang, Shengqi Zhu, Yansong Feng, Dongyan Zhao
TL:DR: Proposed a method to prevent the model from ignoring the potential important non-rationale words and not
distinguishing the importance of different rationale words, when incorporating human rational.
| paper |
|
Service
- ACL Rollings, area chair, 2023.10
- ACL Rollings, reviewer, since 2021.9
- ACL, reviewer, 2022,2023
- EMNLP, reviewer, 2022,2023
- COLING, reviewer, 2022,2023
- EACL, reviewer, 2023
- AAAI, reviewer, 2023
|
Awards
- President Scholarship, Peking University, 2022
- Uniqlo Scholarship, Peking University, 2017
- Panasonic Scholarship, Peking University, 2016
|
| Contact
Wangxuan Institute of Computer Technology, Peking University
No. 128 Zhongguancun North Street,
Haidian District, Beijing, 100871
huangquzhe[AT]pku[DOT]edu[DOT]cn
|
|
