Justin (Zhaocong) Yuan

I am a MASc student in Reinforcement Learning (RL) and Robotics at the University of Toronto, supervised by Prof. Angela Schoellig at Dynamic Systems Lab (DSL), also part of Vector Institute and UofT Robotics Institute.

I received my BASc degree from Engineering Science (Robotics), UofT. Before joining DSL, I interned in Apple Siri team in Seattle and Nvidia Toronto AI lab led by Prof. Sanja Fidler. I also spent time as a research student at Data-Driven Decision Making Lab led by Prof. Scott Sanner.

I'm generally interested in machine learning, reinforcement learning, and robotics. My current research focus is on safe RL and transfer learning in RL (specifically in Sim-to-Real applications).

Email  /  CV  /  LinkedIn  /  Google Scholar  /  Github

News & Activities

• New: IROS 2022 presentation of the safe-control-gym paper.
• Oct 2022, Paper accepted to NeurIPS 2022 Workshops (Distribution Shifts: Connecting Methods and Applications & Progress and Challenges in Building Trustworthy Embodied AI).
  DistShift / TEA
• Aug 2022, Vector AI Engineering Blog by Catherine Glossop on using safe-control-gym to benchmark RL robustness.
  blog
• May 2022, ICRA 2022 Workshop on Releasing Robots into the Wild: Simulations, Benchmarks, and Deployment (co-organizer).
  website / videos
• Apr 2022, UofT AER1517 guest lecture (co-speaker).
  
• Mar 2022, Vector Institute Industry Workshop (co-speaker).
  
• Dec 2021, NeurIPS 2021 Workshop on Deployable Decision Making in Embodied Systems (committee team).
  website / videos (from NeurIPS page)
• Nov 2021, UCSD guest lecture on Safe Learning in Robotics (co-speaker).
  
• Sep 2021, IROS 2021 Workshop on Safe Real-World Robot Autonomy (live session team, volunteer).
  website / videos

University of Toronto (Supervisor: Angela P. Schoellig )

University of Toronto (Supervisor: Sanja Fidler ), graduated with High Honours

I worked as a deep learning intern and focused on synthetic data generation for computer vision tasks in autonomous driving. I also worked on trajectory prediction, graph neural networks, and distribution matching on videos.

I worked on learning-to-rank problems for the Siri pipeline, experimented with both tradition supervised learning techniques and deep learning sequence models.

I worked as a research intern and focused on image classification, text classification using deep learning models such as CNN, RNN and attention networks.

We leverage safe-control-gym to benchmark robust RL methods (specifically robust adversarial RL) against common disturbances in a robotics setting.

We propose safe-control-gym as a benchmark suite for safe-learning in robotics. It implements several PyBullet-based benchmark environments and control algorithms from traditional control, safe-learning control, safe RL, robust RL.

RL transfer learning baselines are recently added.

We conduct an extensive review on safe-learning-based methods in robotics, and provides a formulation of safe-learning-control to bridge between control theory and reinforcement learning. We also show safe control examples to highlight the need for a safety benchmark.

We propose Meta-Sim, which learns a generative model of synthetic scenes with the help of a graphics engine. It minimizes the distribution gap between synthetic images and real images. Experiments show that Meta-Sim can greatly improve scene quality and help in downstream task training.

We provide the full description of safe-control-gym and use it to perform in-depth benchmarks over RL baselines regarding three aspects of safety in robot control. We also propose useful practices to design safe agents by looking at their respective ablations.

We investigate the emergent behaviors in Multi-agent Reinforcement Learning (MARL) with the OpenAI MPE environment. From experiments, we discover that meaningful team collaboration and communication protocols can be learned.

Many thanks to Jon Barron's awesome template!
Other template references: Abhishek Kar, Tingwu Wang, Stuart Geiger