Yicheng Luo

About

I am a final-year Ph.D candidate at UCL AI Centre supervised by Marc Deisenroth and Edward Grefenstette.

I am interested in developing algorithms that allow AI agents to learn from a diverse set of offline data sources. Currently, I am focusing on enabling reinforcement learning agents to learn from demonstrations across different domains and embodiements.

In 2020, I graduated with first-class honors from the Department of Computing, Imperial College London. For my thesis, I worked on probabilistic program analysis with Antonio Filieri.

As an undergraduate at Imperial College, I worked on Gaussian processes, reinforcement learning, generative models and probabilistic programming languages. In 2019, I was a Research Engineering intern at Google DeepMind working on conditional generative models.

Publications

H-GAP: Humanoid Control with a Generalist Planner
Zhengyao Jiang, Yingchen Xu, Nolan Wagener, Yicheng Luo, Michael Janner, Edward Grefenstette, Tim Rocktäschel, Yuandong Tian
International Conference on Learning Representations (ICLR) 2024 (Spotlight) | arXiv:2312.02682 | Code

Optimal Transport for Offline Imitation Learning
Yicheng Luo, Zhengyao Jiang, Samuel Cohen, Edward Grefenstette, Marc Peter Deisenroth
International Conference on Learning Representations (ICLR) 2023 (Notable 25%) | arXiv:2303.13971 | Code

Safe Trajectory Sampling in Model-Based Reinforcement Learning
Sicelukwanda Zwane, Denis Hadjivelichkov, Yicheng Luo, Yasemin Bekiroglu, Dimitrios Kanoulas, Marc Peter Deisenroth
International Conference on Automation Science and Engineering (CASE) 2023

ChessGPT: Bridging Policy Learning and Language Modeling
Xidong Feng, Yicheng Luo, Ziyan Wang, Hongrui Tang, Mengyue Yang, Kun Shao, David Mguni, Yali Du, Jun Wang
Neural Information Processing Systems (NeurIPS) 2023 | arXiv:2306.09200

Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
Yicheng Luo, Jackie Kay, Edward Grefenstette, Marc Peter Deisenroth
Reinforcement Learning and Decision Making (RLDM) 2022 | arXiv:2303.17396

SYMPAIS: SYMbolic Parallel Adaptive Importance Sampling for Probabilistic Program Analysis
Yicheng Luo, Antonio Filieri, Yuan Zhou
ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC/FSE) 2021 | arXiv:2010.05050 | Code

Adversarial synergy graph model for predicting game outcomes in human basketball
Somchaya Liemhetcharat, Yicheng Luo
AAMAS Workshop on Adaptive and Learning Agents 2015

Applying the Synergy Graph Model to Human Basketball
Somchaya Liemhetcharat, Yicheng Luo
International Conference on Autonomous Agents and Multiagent Systems 2015

Software

I enjoy building tools and libraries for research. I think open-source, high-quality, reproducible research softwares are fundamental to the progress of machine learning.

Here are some of open-source libraries that I create and maintain.

Here are some of them:

• lxm3: an XManager launch backend for HPC clusters such as SGE and Slurm. This allows easy pacakging and deployment of your research project on a HPC cluster to run thousands of experiments.
• corax: library for reinforcement learning algorithms in JAX. Implements a large number of state-of-the-art RL/IL algorithms.
• d4rl-slim: a minimal library for working with D4RL environments that uses DeepMind’s MuJoCo binding instead of mujoco-py.
• jam: collection of machine learning models in JAX (Flax/Haiku). Includes vision models such as ResNet, ConvNeXt, MVP, NFNet and R3M and pretrained weights.

I maintain some forks of existing libraries and tools that include useful fixes and improvements. You may wish to give them a try.

• mujoco-py. The v2.1.2.14-patch branch fixes some issues with the original mujoco-py package such that it runs in containerized environments like Singularity/Apptainer. There are also experimental manylinux wheels in the release page that can be directly installed.

• D4RL: fork of D4RL that removes pybullet dependencies and direct git dependency (mjrl).