Publications

2023

1. under review

   Receive, Reason, and React: Drive as You Say with Large Language Models in Autonomous Vehicles

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In reviewing (under review) , 2023

   arXiv
2. under review

   Drive as You Speak: Enabling Human-Like Interaction with Large Language Models in Autonomous Vehicles

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In reviewing (under review) , 2023

   arXiv
3. T-IV

   REDFormer: Radar Enlightens the Darkness of Camera Perception with Transformers

   Can Cui, Yunsheng Ma, Juanwu Lu, and Ziran Wang

   In Proceedings of the IEEE Transactions on Intelligent Vehicles (T-IV) , 2023

   Abs

   Enhancing the accuracy and reliability of perception systems in automated vehicles is critical, especially under varying driving conditions. Unfortunately, the challenges of adverse weather and low-visibility conditions can seriously degrade camera performance, introducing significant risks to vehicle safety. To address these concerns, in this study, we introduce a novel transformer-based 3D object detection model named ‘REDFormer’. By exploiting bird’s-eye-view camera-radar fusion, the REDFormer offers a more practical and financially viable solution for tackling low-visibility conditions. We validate our model using the comprehensive nuScenes dataset, incorporating camera images, multi-radar point clouds, weather information, and time-of-day data. In comparative analyses, our model surpasses state-of-the-art benchmarks in both classification and detection accuracy. An in-depth ablation study further elucidates the individual contributions of each model component in overcoming the challenges posed by weather and lighting conditions. Experimental results specifically highlight the model’s significant performance improvements, demonstrating a 31.31% increase in accuracy under rainy conditions and a 46.99% enhancement during nighttime scenarios, affirming REDFormer’s potential as a robust and cost-effective solution for automated vehicles.

4. WACV

   MACP: Efficient Model Adaptation for Cooperative Perception

   Yunsheng Ma, Juanwu Lu,  Can Cui, Sicheng Zhao, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) , 2023

   Abs

   Vehicle-to-vehicle (V2V) communications have greatly enhanced the perception capabilities of connected and automated vehicles (CAVs) by enabling information sharing to “see through the occlusions", resulting in significant performance improvements. However, developing and training complex multi-agent perception models from scratch can be expensive and unnecessary when existing single-agent models show remarkable generalization capabilities. In this paper, we propose a new framework termed MACP, which equips a single-agent pre-trained model with cooperation capabilities. We approach this objective by identifying the key challenges of shifting from single-agent to cooperative settings, adapting the model by freezing most of its parameters and adding a few lightweight modules. We demonstrate in our experiments that the proposed framework can effectively utilize cooperative observations and outperform other state-of-the-art approaches in both simulated and real-world cooperative perception benchmarks while requiring substantially fewer tunable parameters with reduced communication costs.

5. SEC

   Human-Autonomy Teaming on Autonomous Vehicles with Large Language Model-Enabled Human Digital Twins

   Can Cui, Yunsheng Ma, Xu Cao, Wenqian Ye, and Ziran Wang

   In Proceedings of the ACM/IEEE Symposium on Edge Computing (SEC) , 2023

   Abs

   The development of autonomous vehicles is dramatically reshaping the transportation landscape, bringing new challenges and opportunities in human-machine interaction. As autonomous vehicles evolve, understanding and responding to human intent becomes significant, and therefore require new ways of human-autonomy teaming. A human digital twin (HDT) is a virtual representation of an individual driver, capturing their preferences, behaviors, and physiological states, enabling machines to better understand and predict human needs and responses. In this paper, we explore how large language models (LLMs), like GPT-4 and LLaMA, together with HDTs are changing the way humans team up with autonomous vehicles. These LLMs help make our conversations with vehicles more natural and intuitive. By pairing them in HDTs, we can get real-time feedback and smarter responses. This combination offers not just easier control but also safer driving experiences. We will break down how this works, why it matters, and what we might expect in the future.

6. ITSC

   Radar Enlighten the Dark: Enhancing Low-Visibility Perception for Automated Vehicles with Camera-Radar Fusion

   Can Cui, Yunsheng Ma, Juanwu Lu, and Ziran Wang

   IEEE International Conference on Intelligent Transportation Systems (ITSC) , 2023

   arXiv