Zheng Lian
Research
(* Equal contribution; † Corresponding author)
MERBench: A Unified Evaluation Benchmark for Multimodal Emotion Recognition
Zheng Lian, Licai Sun, Yong Ren, Hao Gu, Haiyang Sun, Lan Chen, Bin Liu, Jianhua Tao
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2026 | [paper] EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?
Zheng Lian, Licai Sun, Lan Chen, Haoyu Chen, Zebang Cheng, Fan Zhang, Ziyu Jia, Ziyang Ma, Fei Ma, Xiaojiang Peng, Jianhua Tao
ICLR, 2026 | [paper] MME-Emotion: A Holistic Evaluation Benchmark for Emotional Intelligence in Multimodal Large Language Models
Fan Zhang, Zebang Cheng, Chong Deng, Haoxuan Li, Zheng Lian, Qian Chen, Huadai Liu, Wen Wang, etc.
ICLR, 2026 | [paper] AStar: Boosting Multimodal Reasoning with Automated Structured Thinking
Jinyang Wu, Mingkuan Feng, Guocheng Zhai, Shuai Zhang, Zheng Lian, Fangrui Lv, Pengpeng Shao, Ruihan Jin, Zengqi Wen, Jianhua Tao
AAAI (Oral), 2026 | [paper] QuMAB: Query-based Multi-annotator Behavior Pattern Learning
Liyun Zhang, Zheng Lian, Hong Liu, Takanori Takebe, Yuta Nakashima
AAAI (Oral), 2026 | [paper] SimLabel: Similarity-Weighted Semi-supervision for Multi-annotator Learning with Missing Labels
Liyun Zhang, Zheng Lian, Hong Liu, Takanori Takebe, Yuta Nakashima
AAAI, 2026 | [paper] IRNet: Iterative Refinement Network for Noisy Partial Label Learning
Zheng Lian, Mingyu Xu, Lan Chen, Licai Sun, Bin Liu, Lei Feng, Jianhua Tao
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025 | [paper] AffectGPT: A New Dataset, Model, and Benchmark for Emotion Understanding with Multimodal Large Language Models
Zheng Lian, Haoyu Chen, Lan Chen, Haiyang Sun, Licai Sun, Yong Ren, Zebang Cheng, Bin Liu, Rui Liu, Xiaojiang Peng, Jiangyan Yi, Jianhua Tao
ICML, 2025 (Oral, Top 1%) | [paper] OV-MER: Towards Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun, Haoyu Chen, Lan Chen, Hao Gu, Zhuofan Wen, Shun Chen, Siyuan Zhang, Hailiang Yao, Bin Liu, Rui Liu, Shan Liang, Ya Li, Jiangyan Yi, Jianhua Tao
ICML, 2025 | [paper] MER 2025: When affective computing meets large language models
Zheng Lian, Rui Liu, Kele Xu, Bin Liu, Xuefei Liu, Yazhou Zhang, Xin Liu, Yong Li, Zebang Cheng, Haolin Zuo, Ziyang Ma, Xiaojiang Peng, Xie Chen, Ya Li, Erik Cambria, Guoying Zhao, Björn W Schuller, Jianhua Tao
ACM Multimedia, 2025 | [paper] SarcasmBench: Towards Evaluating Large Language Models on Sarcasm Understanding
Yazhou Zhang, Chunwang Zou, Zheng Lian†, Prayag Tiwari†, Jing Qin
IEEE Transactions on Affective Computing (IEEE TAFFC), 2025 | [paper] Are MLMs Trapped in the Visual Room?
Yazhou Zhang, Chunwang Zou, Qimeng Liu, Lu Rong, Ben Yao, Zheng Lian†, Qiuchi Li†, Peng Zhang, Jing Qin
PRCV (Best Paper Mention Award), 2025 | [paper] Exploring the Use of Large Language Models and Interpretable Features for Explainable Speech Emotion Recognition
Qifei Li, Yingming Gao, Yuhua Wen, Yingying Zhou, Zheng Lian, Bin Liu, Zhengqi Wen, Jianhua Tao, Ya Li
IEEE Journal of Selected Topics in Signal Processing, 2025 | [paper] REFED: A Subject Real-time Dynamic Labeled EEG-fNIRS Synchronized Recorded Emotion Dataset
Xiaojun Ning, Jing Wang, Zhiyang Feng, Tianzuo Xin, Shuo Zhang, Shaoqi Zhang, Zheng Lian, Yi Ding, Youfang Lin, Ziyu Jia
NeurIPS (D&B Track), 2025 | [paper] MMAR: A Challenging Benchmark for Deep Reasoning in Speech, Audio, Music, and Their Mix
Ziyang Ma, Yinghao Ma, Yanqiao Zhu, Chen Yang, Yi-Wen Chao, Ruiyang Xu, Wenxi Chen, Yuanzhe Chen, Zhuo Chen, Jian Cong, Kai Li, Keliang Li, Siyou Li, Xinfeng Li, Xiquan Li, Zheng Lian, Yuzhe Liang, Minghao Liu, etc.
NeurIPS (D&B Track), 2025 | [paper] Hardness-Aware Dynamic Curriculum Learning for Robust Multimodal Emotion Recognition with Missing Modalities
Rui Liu, Haolin Zuo, Zheng Lian, Hongyu Yuan, Qi Fan
ACM Multimedia, 2025 | [paper] ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection
Hao Gu, Jiangyan Yi, Chenglong Wang, Jianhua Tao, Zheng Lian, Jiayi He, Yong Ren, Yujie Chen, Zhengqi Wen
ACM Multimedia, 2025 | [paper] Listen, Watch, and Learn to Feel: Retrieval-Augmented Emotion Reasoning for Compound Emotion Generation
Zhuofan Wen, Zheng Lian, Shun Chen, Hailiang Yao, Longjiang Yang, Bin Liu, Jianhua Tao
ACL (Findings), 2025 | [paper] MEIJU-The 1st Multimodal Emotion and Intent Joint Understanding Challenge
Rui Liu, Xiaofen Xing, Zheng Lian, Haizhou Li, Björn W Schuller, Haolin Zuo
ICASSP, 2025 | [paper] Adversarial Training and Gradient Optimization for Partially Deepfake Audio Localization
Siding Zeng, Jiangyan Yi, Jianhua Tao, Jiayi He, Zheng Lian, Shan Liang, Chuyuan Zhang, Yujie Chen, Xiaohui Zhang
ICASSP, 2025 | [paper] Explainable Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun, Hao Gu, Zhuofan Wen, Siyuan Zhang, etc.
Arxiv, 2024 | [paper] MER 2024: Semi-Supervised Learning, Noise Robustness, and Open-Vocabulary Multimodal Emotion Recognition
Zheng Lian, Haiyang Sun, Licai Sun, Zhuofan Wen, Siyuan Zhang, Shun Chen, Hao Gu, etc.
Proceedings of the 2nd International Workshop on Multimodal and Responsible Affective Computing, 2024 | [paper] GPT-4V with Emotion: A Zero-shot Benchmark for Generalized Emotion Recognition
Zheng Lian, Licai Sun, Haiyang Sun, Kang Chen, Zhuofan Wen, Hao Gu, Bin Liu, Jianhua Tao
Information Fusion, 2024 | [paper] SVFAP: Self-supervised Video Facial Affect Perceiver
Licai Sun, Zheng Lian†, Kexin Wang, Yu He, Mingyu Xu, Haiyang Sun, Bin Liu†, Jianhua Tao†
IEEE Transactions on Affective Computing (IEEE TAFFC), 2024 | [paper] Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning
Zebang Cheng, Zhi-Qi Cheng, Jun-Yan He, Jingdong Sun, Kai Wang, Yuxiang Lin, Zheng Lian, Xiaojiang Peng, Alexander Hauptmann
NeurIPS, 2024 | [paper] HiCMAE: Hierarchical Contrastive Masked Autoencoder for Self-Supervised Audio-Visual Emotion Recognition
Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao
Information Fusion, 2024 | [paper] Pseudo Labels Regularization for Imbalanced Partial-Label Learning
Mingyu Xu, Zheng Lian, Bin Liu, Zerui Chen, Jianhua Tao
ICASSP, 2024 | [paper] NLoPT: N-gram Enhanced Low-Rank Task Adaptive Pre-training for Efficient Language Model Adaption
Hao Gu, Jiangyan Yi, Zheng Lian, Jianhua Tao, Xinrui Yan
LREC-COLING, 2024 | [paper] Contrastive Learning based Modality-Invariant Feature Acquisition for Robust Multimodal Emotion Recognition with Missing Modalities
Rui Liu, Haolin Zuo, Zheng Lian, Björn W. Schuller, Haizhou Li
IEEE Transactions on Affective Computing (IEEE TAFFC), 2024 | [paper] MFSN: Multi-perspective Fusion Search Network For Pre-training Knowledge in Speech Emotion Recognition
Haiyang Sun, Fulin Zhang, Yingying Gao, Zheng Lian, Shilei Zhang, Junlan Feng
Interspeech, 2024 | [paper] IERP 2024: Induced Emotion Recognition with Personality Characteristics Challenge 2024
Cong Cai, Shan Liang, Xuefei Liu, Kang Zhu, Zhenhua Cheng, Zheng Lian, Bin Liu
ISCSLP, 2024 | [paper] Learning Noise-Robust Joint Representation for Multimodal Emotion Recognition under Incomplete Data Scenarios
Qi Fan, Haolin Zuo, Rui Liu, Zheng Lian, Guanglai Gao
MRAC@ACM Multimedia, 2024 | [paper] Social Perception Prediction for MuSe 2024: Joint Learning of Multiple Perceptions
Zhuofan Wen, Hailiang Yao, Shun Chen, Haiyang Sun, Mingyu Xu, Licai Sun, Zheng Lian, Bin Liu, Fengyu Zhang, Siyuan Zhang, Jianhua Tao
MuSe@ACM Multimedia (Winner), 2024 | [paper] DPP: A Dual-Phase Processing Method for Cross-Cultural Humor Detection
Shun Chen, Hailiang Yao, Mingyu Xu, Zhuofan Wen, Haiyang Sun, Licai Sun, Zheng Lian, Bin Liu, Fengyu Zhang, Siyuan Zhang, Jianhua Tao
MuSe@ACM Multimedia (Winner), 2024 | [paper] GCNet: Graph Completion Network for Incomplete Multimodal Learning in Conversation
Zheng Lian, Lan Chen, Licai Sun, Bin Liu, Jianhua Tao
Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2023 | [paper] MER 2023: Multi-label Learning, Modality Robustness, and Semi-Supervised Learning
Zheng Lian, Haiyang Sun, Licai Sun, Kang Chen, Mingyu Xu, Kexin Wang, etc.
ACM Multimedia (Organize Grand Challenge), 2023 | [paper] MRAC'23: 1st International Workshop on Multimodal and Responsible Affective Computing
Zheng Lian, Erik Cambria, Guoying Zhao, Björn W Schuller, Jianhua Tao
ACM Multimedia (Organize Workshop), 2023 | [paper] ALIM: Adjusting Label Importance Mechanism for Noisy Partial Label Learning
Mingyu Xu*, Zheng Lian*, Lei Feng, Bin Liu, Jianhua Tao
NeurIPS, 2023 | [paper] EmotionNAS: Two-stream Architecture Search for Speech Emotion Recognition
Haiyang Sun*, Zheng Lian*, Bin Liu, Ying Li, Licai Sun, Cong Cai, Jianhua Tao, Meng Wang, Yuan Cheng
Interspeech, 2023 | [paper] VRA: Variational Rectified Activation for Out-of-distribution Detection
Mingyu Xu, Zheng Lian†, Bin Liu, Jianhua Tao
NeurIPS, 2023 | [paper] MAE-DFER: Efficient Masked Autoencoder for Self-supervised Dynamic Facial Expression Recognition
Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao
ACM Multimedia, 2023 | [paper] Efficient multimodal transformer with dual-level feature restoration for robust multimodal sentiment analysis
Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao
IEEE Transactions on Affective Computing (IEEE TAFFC), 2023 | [paper] Integrating VideoMAE based model and Optical Flow for Micro-and Macro-expression Spotting
Ke Xu, Kang Chen, Licai Sun, Zheng Lian, Bin Liu, Gong Chen, Haiyang Sun, Mingyu Xu, Jianhua Tao
ACM Multimedia, 2023 | [paper] PIRNet: Personality-enhanced Iterative Refinement Network for Emotion Recognition in Conversation
Zheng Lian, Bin Liu, Jianhua Tao
IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2022 | [paper] SMIN: Semi-supervised Multi-modal Interaction Network for Conversational Emotion Recognition
Zheng Lian, Bin Liu, Jianhua Tao
IEEE Transactions on Affective Computing (IEEE TAFFC), 2022 | [paper] AMOA: Global acoustic feature enhanced modal-order-aware network for multimodal sentiment analysis
Ziming Li, Yan Zhou, Weibo Zhang, Yaxin Liu, Chuanpeng Yang, Zheng Lian, Songlin Hu
COLING, 2022 | [paper] Multimodal temporal attention in sentiment analysis
Yu He, Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao, Meng Wang, Yuan Cheng
MuSe@ACM Multimedia (Winner), 2022 | [paper] DECN: Dialogical Emotion Correction Network for Conversational Emotion Recognition
Zheng Lian, Bin Liu, Jianhua Tao
Neurocomputing, 2021 | [paper] CTNet: Conversational Transformer Network for Emotion Recognition
Zheng Lian, Bin Liu, Jianhua Tao
IEEE/ACM Transactions on Audio, Speech, and Language Processing (TASLP), 2021 | [paper] Towards Fine-Grained Prosody Control for Voice Conversion
Zheng Lian, Rongxiu Zhong, Zhengqi Wen, Bin Liu, Jianhua Tao
Proceedings of the 12th International Symposium on Chinese Spoken Language Processing (ISCSLP), 2021 | [paper] Investigation of multimodal features, classifiers and fusion methods for emotion recognition
Zheng Lian, Ya Li, Jianhua Tao, Jian Huang
National Conference Man-Machine Speech Communication, NCMMSC, 2021 | [paper] Multimodal Cross-and Self-Attention Network for Speech Emotion Recognition
Licai Sun, Bin Liu, Jianhua Tao, Zheng Lian
ICASSP, 2021 | [paper] Multimodal sentiment analysis based on recurrent neural network and multimodal attention
Cong Cai, Yu He, Licai Sun, Zheng Lian, Bin Liu, Jianhua Tao, Mingyu Xu, Kexin Wang
MuSe@ACM Multimedia (Winner), 2021 | [paper] Multimodal emotion recognition and sentiment analysis via attention enhanced recurrent model
Licai Sun, Mingyu Xu, Zheng Lian, Bin Liu, Jianhua Tao, Meng Wang, Yuan Cheng
MuSe@ACM Multimedia (Winner), 2021 | [paper] Conversational Emotion Recognition Using Self-Attention Mechanisms and Graph Neural Networks
Zheng Lian, Jianhua Tao, Bin Liu, Jian Huang, Zhanlei Yang, Rongjun Li
Interspeech, 2020 | [paper] Context-Dependent Domain Adversarial Neural Network for Multimodal Emotion Recognition
Zheng Lian, Jianhua Tao, Bin Liu, Jian Huang, Zhanlei Yang, Rongjun Li
Interspeech, 2020 | [paper] ARVC: An Auto-Regressive Voice Conversion System Without Parallel Training Data
Zheng Lian, Zhengqi Wen, Xinyong Zhou, Songbai Pu, Shengkai Zhang, Jianhua Tao
Interspeech, 2020 | [paper] CASIA voice conversion system for the voice conversion challenge 2020
Zheng Lian, Jianhua Tao, Zhengqi Wen, Rongxiu Zhong
VCC@Interspeech (Winner), 2020 | [paper] Expression analysis based on face regions in real-world conditions
Zheng Lian, Ya Li, Jianhua Tao, Jian Huang, Mingyue Niu
International Journal of Automation and Computing, 2020 | [paper] Multi-modal continuous dimensional emotion recognition using recurrent neural network and self-attention mechanism
Licai Sun*, Zheng Lian*, Jianhua Tao, Bin Liu, Mingyue Niu
MuSe@ACM Multimedia (Winner), 2020 | [paper] Learning Utterance-level Representations with Label Smoothing for Speech Emotion Recognition
Jian Huang, Jianhua Tao, Bin Liu, Zheng Lian
Interspeech, 2020 | [paper] Multimodal Transformer Fusion for Continuous Emotion Recognition
Jian Huang, Jianhua Tao, Bin Liu, Zheng Lian, Mingyue Niu
ICASSP, 2020 | [paper] Multimodal Spatiotemporal Representation for Automatic Depression Level Detection
Mingyue Niu, Jianhua Tao, Bin Liu, Jian Huang, Zheng Lian
IEEE Transactions on Affective Computing (IEEE TAFFC), 2020 | [paper] Unsupervised Representation Learning with Future Observation Prediction for Speech Emotion Recognition
Zheng Lian, Jianhua Tao, Bin Liu, Jian Huang
Interspeech, 2019 | [paper] Conversational Emotion Analysis via Attention Mechanisms
Zheng Lian, Jianhua Tao, Bin Liu, Jian Huang
Interspeech, 2019 | [paper] Discriminative video representation with temporal order for micro-expression recognition
Mingyue Niu, Jianhua Tao, Ya Li, Jian Huang, Zheng Lian
ICASSP, 2019 | [paper] Speech emotion recognition via contrastive loss under siamese networks
Zheng Lian, Ya Li, Jianhua Tao, Jian Huang
Joint Workshop on Affective Social Multimedia Computing and Multi-Modal Affective Computing, 2018 | [paper] End-to-End Continuous Emotion Recognition from Video Using 3D ConvLSTM Networks
Jian Huang, Ya Li, Jianhua Tao, Zheng Lian, Jiangyan Yi
ICASSP, 2018 | [paper] Speech Emotion Recognition from Variable-Length Inputs with Triplet Loss Function
Jian Huang, Ya Li, Jianhua Tao, Zheng Lian
Interspeech, 2018 | [paper] Multimodal continuous emotion recognition with data augmentation using recurrent neural networks
Jian Huang, Ya Li, Jianhua Tao, Zheng Lian, Mingyue Niu, Minghao Yang
AVEC@ACM Multimedia (2nd Place), 2018 | [paper] Continuous multimodal emotion prediction based on long short term memory recurrent neural network
Jian Huang, Ya Li, Jianhua Tao, Zheng Lian, Zhengqi Wen, Minghao Yang, Jiangyan Yi
AVEC@ACM Multimedia (2nd Place), 2017 | [paper]
|