Past Event! Note: this event has already taken place.
Seminar – Reinforcement Learning for High-Dimensional Problems: From PDE Control to Model Learning
January 30, 2020 at 11:00 AM to 12:00 PM
| Location: | 5345 Herzberg Laboratories |
| Contact Email: | cuids@carleton.ca |
| Contact Phone: | (613) 520-2600 ext. 8751 |
Data Science Distinguished Speaker Seminar Series
Refreshments will be provided
Abstract
Many real-world decision-making problems can be formulated as a reinforcement learning (RL) problem with high-dimensional state and action spaces. Due to the curse of dimensionality, however, solving these problems can be extremely challenging unless one exploits their intrinsic regularities. The regularities of an RL problem might be in the value function, policy, or the model of the environment.
In the first part of the talk, I introduce the problem of data-driven control of Partial Differential Equations (PDE), which has many industrial applications, as an example of a high-dimensional RL problem. I formulate it as an RL problem and empirically show that methods that benefit from regularities of the value function can solve such a problem.
In the second part of the talk, I focus on model-based RL (MBRL). I argue that conventional model learning approaches that are based on learning a good predictive model of the environment might be an overkill for MBRL. I discuss a new type of regularity of RL problems that arises due to the interaction of the dynamics of the environment and the value function, and introduce the Value-Aware Model Learning framework, which benefits from this regularity.
About the Speaker
Amir-massoud Farahmand is a faculty member, research scientist, and Canada CIFAR AI Chair at the Vector Institute in Toronto, Canada. He is also an assistant professor (status) at the Department of Computer Science, University of Toronto, with a cross-appointment at the Department of Mechanical and Industrial Engineering. His research interests are in reinforcement learning and machine learning with a focus on developing theoretically-sound algorithms for challenging industrial problems. He received his PhD from the University of Alberta in 2011, followed by postdoctoral fellowships at McGill University (2011–2014) and Carnegie Mellon University (CMU) (2014). Prior to joining the Vector Institute in 2018, he worked as a principal research scientist at Mitsubishi Electric Research Laboratories (MERL) in Cambridge, USA for three years.
Amir-massoud received an NSERC Postdoctoral Fellowship (2012–2014) and the University of Alberta’s Department of Computing Science Ph.D. Outstanding Thesis Award for the period of 2011–2012. His work has been published in top machine learning and AI (JMLR, MLJ, NeurIPS, ICML, ICLR, AISTATS, AAAI, IJCAI), control engineering (IEEE TAC, CDC, ACC), and robotics (IROS and ICRA) venues.