Mengdi Xu

I am a postdoc at Stanford Vision and Learning Lab (SVL) working with Prof. Jiajun Wu and Prof. Fei-Fei Li. I received my PhD from Carnegie Mellon University with Best PhD Dissertation Award, advised by Prof. Ding Zhao (SafeAI Lab). I have spent wonderful summers at Google DeepMind Robotics, MIT-IBM Watson AI Lab, and Toyota Research Institute. Previously, I received a master's degree in Machine Learning from Machine Learning Department at CMU, and a master's degree in Robotics from LCSR at JHU, supervised by Prof. Gregory S. Chirikjian. I received a B.E. from Tsinghua University.

Perspective students: If you are interested in robot learning and have a strong background in machine learning and robotics, feel free to reach out and I am happy to chat!

Email  /  CV (Mar 2024)  /  Google Scholar  /  Github  /  X

profile photo

Research Highlights

I am broadly interested in building scalable, adaptable, and reliable robots that seamlessly interact with humans in daily activities. My current research highlights the following perspectives:

  • In-context robot learning with few-shot demonstrations and robot foundation models.
  • Unsupervised learning to discover task/skill structures in dynamics, visual geometries, and agent policies.
  • Distributionally robust RL that balances performance and robustness when facing task uncertainties.

News

2024/08 - One paper got accepted to JMLR.
2024/07 - Invited talk at CMU LeCAR Lab.
2024/06 - Joined Stanford Vision and Learning Lab (SVL) as a postdoctoral researcher.
2024/05 - We are organizing the RSS Workshop on Lifelong Robot Learning: Generalization, Adaptation, and Deployment with Large Models.
2024/05 - We are organizing the RSS Pioneers 2024 Workshop. See you in Delft!
2024/05 - I successfully passed my thesis defense. Many thanks to my thesis committee!
2024/03 - One paper got accepted to NAACL 2024.
2023/10 - A new arXiv preprint RoboTool.
2023/09 - One paper got accepted to NeurIPS 2023.
2023/08 - Two papers got accepted to CoRL 2023, and COVERS was accepted for an oral presentation.
2023/08 - Selected as EECS Rising Star.
2023/05 - Selected as RSS Pioneer.
2023/02 - Selected as Rising Star in Computational and Data Sciences.


Selected Publications

A full list of publications is here. (* indicates equal contribution.)

Few-shot Generalization

 Creative Robot Tool Use with Large Language Models
Mengdi Xu*, Peide Huang*, Wenhao Yu*, Shiqi Liu, Xilun Zhang, Yaru Niu, Tingnan Zhang, Fei Xia, Jie Tan, Ding Zhao
CoRL 2023 Workshop on Language and Robot Learning: Language as Grounding
[paper] [webpage] [MLD Blog] [TechXplore]

Hyper-Decision Transformer for Efficient Online Policy Adaptation
Mengdi Xu, Yuchen Lu, Yikang Shen, Shun Zhang, Ding Zhao, Chuang Gan
The Eleventh International Conference on Learning Representations (ICLR), 2023
[paper] [webpage]

Prompting Decision Transformer for Few-shot Policy Generalization
Mengdi Xu, Yikang Shen, Shun Zhang, Yuchen Lu, Ding Zhao, Josh Tenenbaum, Chuang Gan
Thirty-ninth International Conference on Machine Learning (ICML), 2022
[paper] [webpage] [code]

Efficient Adaptation

 Continual Vision-based Reinforcement Learning with Group Symmetries
Shiqi Liu*, Mengdi Xu*, Peide Huang, Yongkang Liu, Kentaro Oguchi, Ding Zhao
Conference on Robot Learning (CoRL), 2023 (oral, 6.6%)
[paper] [webpage] [code]

Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processes
Mengdi Xu, Wenhao Ding, Jiacheng Zhu, Zuxin Liu, Baiming Chen, Ding Zhao
Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS), 2020
[paper] [code]

What Went Wrong? Closing the Sim-to-Real Gap via Differentiable Causal Discovery
Peide Huang, Xilun Zhang*, Ziang Cao*, Shiqi Liu*, Mengdi Xu, Wenhao Ding, Jonathan Francis, Bingqing Chen, Ding Zhao
Conference on Robot Learning (CoRL), 2023
[paper] [webpage]

Curriculum Reinforcement Learning using Optimal Transport via Gradual Domain Adaptation
Peide Huang, Mengdi Xu, Jiacheng Zhu, Laixi Shi, Fei Fang, Ding Zhao,
The 36th Conference on Neural Information Processing Systems, (NeurIPS), 2022
[paper] [code]

Embodied Executable Policy Learning with Language-based Scene Summarization
Jielin Qiu*, Mengdi Xu*, William Han*, Seungwhan Moon, Ding Zhao
2024 Annual Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)
ICML 2023 Workshop on Interactive Learning with Implicit Human Feedback (spotlight)
[paper]

Functional Optimal Transport: Map Estimation and Domain Adaptation for Functional data
Jiacheng Zhu*, Aritra Guha*, Dat Do*, Mengdi Xu, XuanLong Nguyen, Ding Zhao
JMLR: Journal of Machine Learning Research
AAAI OT-SDM 2022 workshop (spotlight)
[paper] [code]

Robust and Safe Robot Learning

 Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables
Mengdi Xu, Peide Huang, Yaru Niu, Visak Kumar, Jielin Qiu, Chao Fang, Kuan-Hui Lee, Xuewei Qi, Henry Lam, Bo Li, Ding Zhao
The 26th International Conference on Artificial Intelligence and Statistics, (AISTATS), 2023
[paper] [webpage]

Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event Sampling
Mengdi Xu, Peide Huang, Fengpei Li, Jiacheng Zhu, Xuewei Qi, Kentaro Oguchi, Zhiyuan Huang, Henry Lam, Ding Zhao
IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2022
[paper]

Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Training
Peide Huang, Mengdi Xu, Fei Fang, Ding Zhao
International Joint Conference on Artificial Intelligence (IJCAI), 2022
[paper]

Context-Aware Safe Reinforcement Learning for Non-Stationary Environments
Baiming Chen, Zuxin Liu, Jiacheng Zhu, Mengdi Xu, Wenhao Ding, Liang Li, Ding Zhao
IEEE International Conference on Robotics and Automation (ICRA), 2021
[paper]

Delay-Aware Model-Based Reinforcement Learning for Continuous Control
Baiming Chen, Mengdi Xu, Liang Li, Ding Zhao
Neurocomputing, 2021
[paper] [code]

CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios
Wenhao Ding, Mengdi Xu, Ding Zhao
International Conference on Robotics and Automation (ICRA), 2020
[paper] [code]

Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizability
Mengdi Xu*, Zuxin Liu*, Peide Huang*, Wenhao Ding, Zhepeng Cen, Bo Li, Ding Zhao
Preprint, under review
[paper]

Services

Co-organizing the RSS 2024 Workshop on Lifelong Robot Learning: Generalization, Adaptation, and Deployment with Large Models
Co-organizing RSS Pioneers Workshop 2024
Breakout session leader at 3rd Women in Machine Learning Un-Workshop, ICML 2022
Mentor of 2020 CMU Robotics Institute Summer Scholars Program (RISS)
Conference Reviewer: NeurIPS, ICML, ICLR, CoRL, AISTATS, ACL, ICRA, ICCV, ECCV, CVPR, AAAI, L4DC, ICASSP
Journal Reviewer: RA-L, T-ITS, T-IV





Website template from Jon Barron.