Gao, Ruiyuan | 高瑞元

Ph.D in CSE of CUHK.

CUHK, Sha Tin, N.T.
Hong Kong, China

I am PhD candidate at CURE lab of CUHK. My supervisor is Prof. Qiang Xu.

Before joining CUHK, I worked with Prof.Hailong Yang and Prof.Xianglong Liu at Beihang University, Beijing, and received a B.E. degree in computer science and technology from Shenyuan Honors College in 2020.

My current research interests span data generation, including generative world models and synthetic data for perception tasks; and trustworthy AI, including adversarial attack/defence and AI privacy.

I am looking for jobs starting from Fall, 2025. Please do not hesitate to contact me through email!

Email: rygao.me [at] gmail.com

News

Mar 24, 2025 I join NVIDIA Spatial Intelligence Lab as a Research Intern, working with Huan Ling and Sanja Fidler on NVIDIA Cosmos.
Dec 7, 2024 We release all the checkpoints for MagicDrive and the final checkpoint for MagicDrive-V2. Enjoy the open-source!
Oct 29, 2024 Three papers (Non-Cross Diffusion, TrackDiffusion, CODA-LM) are accepted to WACV 2025!
Jun 21, 2024 MagicDrive supports Pangu Large Model and appears at HDC2024! [More Details]
Jun 1, 2024 Based on MagicDrive and CODA-LM, we hold the “Multimodal Perception and Comprehension of Corner Cases in Autonomous Driving” workshop (W-CODA2024) @ECCV24. Paper submission starts now, and challenge submission will start soon. Stay tuned!

Talks

Sep 11, 2024 “3D Geometry in Data Synthesis for Autonomous Driving”, Li Auto Inc.
Jul 1, 2024 "3D Geometry in Data Synthesis for Autonomous Driving", Autonomous Intelligence Lab, Westlake University. [More Details]
Jan 18, 2024 “MagicDrive - 基于3D几何控制的自动驾驶街景数据生成”, TechBeat, [online]

Autonomous Driving

(*) denotes equal contribution.
  1. arXiv
    MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Control
    Ruiyuan Gao,  Kai Chen,  Bo Xiao, Lanqing Hong, Zhenguo Li, and Qiang Xu
    arXiv preprint arXiv:2411.13807 2024
    arXiv PDF Web GitHub Repo stars
  2. arXiv
    MagicDrive3D: Controllable 3D Generation for Any-View Rendering in Street Scenes
    Ruiyuan Gao,  Kai Chen,  Zhihao Li, Lanqing Hong, Zhenguo Li, and Qiang Xu
    arXiv preprint arXiv:2405.14475 2024
    arXiv PDF Web GitHub Repo stars
  3. WACV
    Automated Evaluation of Large Vision-Language Models on Self-driving Corner Cases
    Kai Chen*,  Yanze Li*, Wenhua Zhang*, Yanxin Liu, Pengxiang Li, Ruiyuan Gao,  Lanqing Hong, Meng Tian, Xinhai Zhao, Zhenguo Li, and others
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025
    arXiv PDF Web
  4. ICLR
    MagicDrive: Street View Generation with Diverse 3D Geometry Control
    Ruiyuan Gao*,  Kai Chen*,  Enze Xie,  Lanqing Hong, Zhenguo Li, Dit-Yan Yeung,  and Qiang Xu
    In International Conference on Learning Representations (ICLR) 2024
    arXiv Poster Web PR VIDEO (中文) GitHub Repo stars
  5. TNNLS
    Boost 3-D Object Detection via Point Clouds Segmentation and Fused 3-D GIoU-L_1 Loss
    Yaran Chen, Haoran Li, Ruiyuan Gao,  and Dongbin Zhao
    IEEE Transactions on Neural Networks and Learning Systems 2020
    HTML

Generative Models

(*) denotes equal contribution.
  1. WACV
    TrackDiffusion: Tracklet-Conditioned Video Generation via Diffusion Models
    Pengxiang Li*, Kai Chen*,  Zhili Liu*, Ruiyuan Gao,  Lanqing Hong, Dit-Yan Yeung,  Huchuan Lu, and Xu Jia
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025
    arXiv PDF Web GitHub Repo stars
  2. WACV
    Non-Cross Diffusion for Semantic Consistency
    Ziyang Zheng*, Ruiyuan Gao*,  and Qiang Xu
    In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) 2025
    arXiv PDF
  3. CVPR
    DetDiffusion: Synergizing Generative and Perceptive Models for Enhanced Data Generation and Perception
    Yibo Wang*, Ruiyuan Gao*,  Kai Chen*,  Kaiqiang Zhou, Yingjie Cai,  Lanqing Hong, Zhenguo Li, Lihui Jiang, Dit-Yan Yeung,  Qiang Xu,  and Kai Zhang
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
    arXiv PDF

Robustness and AI Safety

(*) denotes equal contribution.
  1. CVPR
    MMA-Diffusion: MultiModal Attack on Diffusion Models
    Yijun Yang,  Ruiyuan Gao,  Xiaosen Wang, Tsung-Yi Ho, Nan Xu, and Qiang Xu
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
    arXiv PDF GitHub Repo stars
  2. ICCV
    DiffGuard: Semantic Mismatch-Guided Out-of-Distribution Detection using Pre-trained Diffusion Models
    Ruiyuan Gao,  Chenchen Zhao, Lanqing Hong, and Qiang Xu
    In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) 2023
    arXiv PDF Poster GitHub Repo stars
  3. ECCV
    Out-of-Distribution Detection with Semantic Mismatch Under Masking
    Yijun Yang,  Ruiyuan Gao,  and Qiang Xu
    In European Conference on Computer Vision (ECCV) 2022
    arXiv PDF Code
  4. NDSS
    What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction
    Yijun Yang,  Ruiyuan Gao,  Yu Li,  Qiuxia Lai, and Qiang Xu
    In Network and Distributed System Security Symposium (NDSS) 2022
    arXiv PDF Code
  5. CCGrid
    PriPro: Towards Effective Privacy Protection on Edge-Cloud System running DNN Inference
    Ruiyuan Gao,  Hailong Yang,  Shaohan Huang, Ming Dun, Mingzhen Li, Zerong Luan, Zhongzhi Luan, and Depei Qian
    In 2021 IEEE/ACM 21st International Symposium on Cluster, Cloud and Internet Computing (CCGrid) 2021
    HTML

Other Selected Papers

(*) denotes equal contribution. See more at full list.
  1. TCYB
    ModuleNet: Knowledge-Inherited Neural Architecture Search.
    Yaran Chen*, Ruiyuan Gao*,  Fenggang Liu, and Dongbin Zhao
    IEEE transactions on cybernetics 2021
    HTML