Publications

2026

1. TIOT

   A Hierarchical Test Platform for Vision Language Model (VLM)-Integrated Real-World Autonomous Driving

   Yupeng Zhou,  Can Cui, Juntong Peng, Zichong Yang, Juanwu Lu, Jitesh Panchal, Bin Yao, and Ziran Wang

   ACM Transactions on Internet of Things (TIOT) , 2026

   Website
2. arXiv

   ICR-Drive: Instruction Counterfactual Robustness for End-to-End Language-Driven Autonomous Driving

   Kaiser Hamid,  Can Cui, and Nade Liang

   In arXiv preprint (arXiv) , 2026

   arXiv
3. arXiv

   Post-Training in End-to-End Autonomous Driving

   Ruining Yang, Muxing Wang, Yixiao Chen, Tongfei Guo, Yi Xu,  Can Cui, Zichong Yang, Yitian Zhang, Ziran Wang, Yun Fu, and Lili Su

   In arXiv preprint (arXiv) , 2026

   arXiv
4. arXiv

   OmniV2X: A Generative Foundation Planner for Efficient End-to-End Cooperative Driving

   Juntong Peng, Juanwu Lu, Yupeng Zhou,  Can Cui, Yaobin Chen, and Ziran Wang

   In arXiv preprint (arXiv) , 2026

   arXiv
5. arXiv

   Efficient and Explainable End-to-End Autonomous Driving via Masked Vision-Language-Action Diffusion

   Jiaru Zhang, Manav Gagvani,  Can Cui, Juntong Peng, Ruqi Zhang, and Ziran Wang

   In arXiv preprint (arXiv) , 2026

   arXiv

2025

1. P-IEEE

   LLM4AD: Large Language Models for Autonomous Driving – Concept, Review, Benchmark, Experiments, and Future Trends

   Can Cui, Yunsheng Ma, Sung-Yeon Park, Zichong Yang, Yupeng Zhou, Juanwu Lu, Juntong Peng, Jiaru Zhang, Ruqi Zhang, Lingxi Li, Yaobin Chen, Jitesh H. Panchal, Amr Abdelraouf, Rohit Gupta, Kyungtae Han, and Ziran Wang

   In Proceedings of the IEEE (P-IEEE) , 2025

   arXiv
2. EMNLP

   DASR: Distributed Adaptive Scene Recognition-A Multi-Agent Cloud-Edge Framework for Language-Guided Scene Detection

   Can Cui, Yongkang Liu, Seyhan Ucar, Juntong Peng, Ahmadreza Moradipari, Maryam Khabazi, and Ziran Wang

   In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track (EMNLP) , 2025

   Website
3. arXiv

   Generative AI for Autonomous Driving: Frontiers and Opportunities

   Yuping Wang, Shuo Xing, Cui Can, Renjie Li, Hongyuan Hua, Kexin Tian, Zhaobin Mo, Xiangbo Gao, Keshu Wu, Sulong Zhou, Hengxu You, Juntong Peng, Junge Zhang, Zehao Wang, Rui Song, Mingxuan Yan, Walter Zimmer, Xingcheng Zhou, Peiran Li, Zhaohan Lu, Chia-Ju Chen, Yue Huang, Ryan A Rossi, Lichao Sun, Hongkai Yu, Zhiwen Fan, Frank Hao Yang, Yuhao Kang, Ross Greer, Chenxi Liu, Eun Hak Lee, Xuan Di, Xinyue Ye, Liu Ren, Alois Knoll, Xiaopeng Li, Shuiwang Ji, Masayoshi Tomizuka, Marco Pavone, Tianbao Yang, Jing Du, Ming-Hsuan Yang, Hua Wei, Ziran Wang, Yang Zhou, Jiachen Li, and Zhengzhong Tu

   In arXiv preprint (arXiv) , 2025

   arXiv
4. ICCV

   NuPlanQA: A Large-Scale Dataset and Benchmark for Multi-View Driving Scene Understanding in Multi-Modal Large Language Models

   Sung-Yeon Park,  Can Cui, Yunsheng Ma, Ahmadreza Moradipari, Rohit Gupta, Kyungtae Han, and Ziran Wang

   In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) , 2025

   Website
5. WACV

   Position: Prospective of Autonomous Driving-Multimodal LLMs World Models Embodied Intelligence AI Alignment and Mamba

   Yunsheng Ma, Wenqian Ye,  Can Cui, Haiming Zhang, Shuo Xing, Fucai Ke, Jinhong Wang, Chenglin Miao, Jintai Chen, Hamid Rezatofighi, Zhen Li, Guangtao Zheng, Chao Zheng, Tianjiao He, Manmohan Chandraker, Burhaneddin Yaman, Xin Ye, Hang Zhao, and Xu Cao

   In Proceedings of the Winter Conference on Applications of Computer Vision (WACV) , 2025

   Website
6. arXiv

   ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving

   Can Cui, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Zichong Yang, Prashanth Sankaranarayanan, Jiaru Zhang, Ruqi Zhang, and Ziran Wang

   In arXiv preprint (arXiv) , 2025

   arXiv
7. arXiv

   SIMSplat: Predictive Driving Scene Editing with Language-Aligned 4D Gaussian Splatting

   Sung-Yeon Park, Adam Lee, Juanwu Lu,  Can Cui, Luyang Jiang, Rohit Gupta, Kyungtae Han, Ahmadreza Moradipari, and Ziran Wang

   In arXiv preprint (arXiv) , 2025

   arXiv
8. arXiv

   FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling

   Kaiser Hamid,  Can Cui, Khandakar Ashrafi Akbar, Ziran Wang, and Nade Liang

   In arXiv preprint (arXiv) , 2025

   arXiv
9. IROS

   On-Board Vision-Language Models (VLMs) for Personalized Motion Control of Autonomous Vehicles

   Can Cui, Zichong Yang, Yupeng Zhou, Juntong Peng, Sung-Yeon Park, Cong Zhang, Yunsheng Ma, Xu Cao, Wenqian Ye, Yiheng Feng, Jitesh Panchal, Lingxi Li, Yaobin Chen, and Ziran Wang

   In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IROS) , 2025

   Website

2024

1. ITSC

   Personalized Autonomous Driving with Large Language Models: Field Experiments

   Can Cui, Zichong Yang, Yupeng Zhou, Yunsheng Ma, Juanwu Lu, Lingxi Li, Yaobin Chen, Jitesh Panchal, and Ziran Wang

   In 2024 IEEE 27th International Conference on Intelligent Transportation Systems (ITSC) (ITSC) , 2024

   Website
2. CVPR

   LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs

   Yunsheng Ma*,  Can Cui*, Xu Cao*, Wenqian Ye, Peiran Liu, Juanwu Lu, Amr Abdelraouf, Rohit Gupta, Kyungtae Han, Aniket Bera, James M. Rehg, and Ziran Wang

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024

   Website
3. ITSM

   Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of IEEE Intelligent Transportation Systems Magazine (ITSM) , 2024

   arXiv
4. WACV

   Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2024

   arXiv
5. WACV

   A Survey on Multimodal Large Language Models for Autonomous Driving

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, Yang Zhou, Kaizhao Liang, Jintai Chen, Juanwu Lu, Zichong Yang, Kuei-Da Liao, Tianren Gao, Erlong Li, Kun Tang, Zhipeng Cao, Tong Zhou, Ao Liu, Xinrui Yan, Shuqi Mei, Jianguo Cao, Ziran Wang, and Chao Zheng

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2024

   Website
6. CVPR

   MAPLM: A Real-World Large-Scale Vision-Language Benchmark for Map and Traffic Scene Understanding

   Xu Cao, Tong Zhou, Yunsheng Ma, Wenqian Ye,  Can Cui, Kun Tang, Zhipeng Cao, Kaizhao Liang, Ziran Wang, James M Rehg, and Chao Zheng

   In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2024

   Website
7. EMNLP Findings

   Learning Autonomous Driving Tasks via Human Feedbacks with Large Language Models

   Yunsheng Ma, Xu Cao, Wenqian Ye,  Can Cui, Kai Mei, and Ziran Wang

   In Findings of the Association for Computational Linguistics: EMNLP 2024 (EMNLP Findings) , 2024

   Website
8. 

   Quantifying Uncertainty in Motion Prediction with Variational Bayesian Mixture

   Juanwu Lu,  Can Cui, Yunsheng Ma, Aniket Bera, and Ziran Wang

   CVPR 2024 , 2024

   Website

2023

1. T-IV

   REDFormer: Radar Enlightens the Darkness of Camera Perception with Transformers

   Can Cui, Yunsheng Ma, Juanwu Lu, and Ziran Wang

   In Proceedings of the IEEE Transactions on Intelligent Vehicles (T-IV) , 2023

   Abs Website

   Enhancing the accuracy and reliability of perception systems in automated vehicles is critical, especially under varying driving conditions. Unfortunately, the challenges of adverse weather and low-visibility conditions can seriously degrade camera performance, introducing significant risks to vehicle safety. To address these concerns, in this study, we introduce a novel transformer-based 3D object detection model named ‘REDFormer’. By exploiting bird’s-eye-view camera-radar fusion, the REDFormer offers a more practical and financially viable solution for tackling low-visibility conditions. We validate our model using the comprehensive nuScenes dataset, incorporating camera images, multi-radar point clouds, weather information, and time-of-day data. In comparative analyses, our model surpasses state-of-the-art benchmarks in both classification and detection accuracy. An in-depth ablation study further elucidates the individual contributions of each model component in overcoming the challenges posed by weather and lighting conditions. Experimental results specifically highlight the model’s significant performance improvements, demonstrating a 31.31% increase in accuracy under rainy conditions and a 46.99% enhancement during nighttime scenarios, affirming REDFormer’s potential as a robust and cost-effective solution for automated vehicles.

2. WACV

   MACP: Efficient Model Adaptation for Cooperative Perception

   Yunsheng Ma, Juanwu Lu,  Can Cui, Sicheng Zhao, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2023

   Abs

   Vehicle-to-vehicle (V2V) communications have greatly enhanced the perception capabilities of connected and automated vehicles (CAVs) by enabling information sharing to “see through the occlusions", resulting in significant performance improvements. However, developing and training complex multi-agent perception models from scratch can be expensive and unnecessary when existing single-agent models show remarkable generalization capabilities. In this paper, we propose a new framework termed MACP, which equips a single-agent pre-trained model with cooperation capabilities. We approach this objective by identifying the key challenges of shifting from single-agent to cooperative settings, adapting the model by freezing most of its parameters and adding a few lightweight modules. We demonstrate in our experiments that the proposed framework can effectively utilize cooperative observations and outperform other state-of-the-art approaches in both simulated and real-world cooperative perception benchmarks while requiring substantially fewer tunable parameters with reduced communication costs.

3. SEC

   Human-Autonomy Teaming on Autonomous Vehicles with Large Language Model-Enabled Human Digital Twins

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of the ACM/IEEE Symposium on Edge Computing (SEC) , 2023

   Abs

   The development of autonomous vehicles is dramatically reshaping the transportation landscape, bringing new challenges and opportunities in human-machine interaction. As autonomous vehicles evolve, understanding and responding to human intent becomes significant, and therefore require new ways of human-autonomy teaming. A human digital twin (HDT) is a virtual representation of an individual driver, capturing their preferences, behaviors, and physiological states, enabling machines to better understand and predict human needs and responses. In this paper, we explore how large language models (LLMs), like GPT-4 and LLaMA, together with HDTs are changing the way humans team up with autonomous vehicles. These LLMs help make our conversations with vehicles more natural and intuitive. By pairing them in HDTs, we can get real-time feedback and smarter responses. This combination offers not just easier control but also safer driving experiences. We will break down how this works, why it matters, and what we might expect in the future.

4. ITSC

   Radar Enlighten the Dark: Enhancing Low-Visibility Perception for Automated Vehicles with Camera-Radar Fusion

   Can Cui, Yunsheng Ma, Juanwu Lu, and Ziran Wang

   IEEE International Conference on Intelligent Transportation Systems (ITSC) , 2023

   arXiv