Machine Learning in Robot Motion Planning

Maxim Likhachev, Carnegie Mellon University

Learning in Heuristic Search-based Planning

In this talk I will first briefly go over different ways how learning can be integrated into Search-based Planning. I will then talk in more details about our recent and ongoing work on speeding up Search-based Planning from experiences, our work on learning planning dimensions based on demonstrations and our ongoing work on integrating offline skill learning into Search-based Planning. I will mostly utilize examples from mobile manipulation to illustrate our work.

Jeannette Bohg, Stanford University

Is Motion Planning Overrated?

I present a fully-integrated system that emphasises the importance of continuous, real-time perception and its tight integration with reactive motion generation methods for robotic grasping and manipulation in a dynamic and uncertain environment. We extensively evaluated this system and compared against baselines in manipulation scenarios that exhibit either challenging workspace geometry or a dynamic environment. We found that pure feedback control brings us surprisingly far. But one can also easily construct scenarios that demand look-ahead over a longer time horizon. This system does not rely on any machine learning but is purely based on optimisation and model-based approaches. Nevertheless, these findings have implications for what kind of problems in motion generation we may want to address through machine learning. I will specifically focus on relaxing some of the assumptions that are made in the above manipulation system. I will present our most recent work on combining learned and analytical models within a model-predictive controller. The controller consumes a sequence of images as input and outputs an optimised sequence of actions over a certain time horizon. We show how this combination addresses the limitation of covariate shift found in purely data-driven methods.

David Hsu, National University of Singapore

Robot Decision-Making under Uncertainty: From Data to Actions

Planning and learning are two primary approaches to robot intelligence. Planning enables us to reason about the consequence of immediate actions far into the future, but it requires accurate world models, which are often difficult to acquire in practice. Policy learning circumvents the need for models and learns from data a direct mapping from robot perceptual inputs to actions. However, without models, it is much more difficult to generalize and adapt learned policies to new contexts. In this talk, I will present our recent work on robust robot decision-making under uncertainty through planning, through learning, most importantly by integrating planning and learning. This integration (i) improves planning by learning a model optimized for a specific planning algorithm and (ii) improves learning by incorporating the planning algorithm as a structure prior. It seamlessly fuses model-based planning and model-free learning.

Marc Toussaint, University of Stuttgart

What can we learn from demonstrations?

Aleksandra Faust, Google Brain

Robot Navigation: From Abilities to Capabilities

Robot sensors, geometry, dynamics, and even type and quality of actuators define robot’s motion abilities. The abilities determine how the basic motion skills, such as reaching a goal or following paths without collisions, look for a particular robot. Robots with noiser sensors might move slower, while smaller robots might be able to fit through more narrow spaces. The basic motion skills can be learned through exploration and interaction with environments, independently from specific navigation tasks and environments. Once learned, the robot can use them as building blocks for more complex navigation capabilities in different environments. In this talk, we present learning robust, basic motion skills, short-distance goal navigation and path following motions, which take noisy sensor observations as inputs and output wheel velocities. Next, we examine building up from the basic motion skills. We discuss navigation through indoor buildings from floor maps, and navigation by following natural language instructions.

Siddhartha Srinivasa, University of Washington

The Experienced Piano Movers Problem: New Piano. New Home. Same Mover

Andrey Kolobov, Microsoft Research

Dealing with Dead Ends in Goal-Directed Reinforcement Learning

In many reinforcement learning scenarios, the agent strives to achieve a goal by ending up in a desirable state. Learning a good policy to get to it, in itself an involved problem due to world dynamics being generally unknown to the agent at the start, can be further complicated by the presence of dead ends --- states from which reaching the goal is impossible. Running into a dead end in the physical world means, at best, a premature end of an agent's mission, as in the case of Mars rover Spirit that got stuck in unexpectedly treacherous terrain. Even in simulated environments dead ends cause serious computational issues, slowing exploration and causing RL algorithms oblivious of them to produce unsafe policies. This talk will discuss principled optimization objectives that take dead ends into account, survey methods for maximizing them when MDP dynamics are known, and pose research questions about dealing with dead ends in model-free settings.

Byron Boots, Georgia Institute of Technology

Machine Learning for Planning and Control

Sanjiban Choudhury

University of Washington

Debadeepta Dey

Microsoft Research

Siddhartha Srinivasa

University of Washington

Marc Toussaint

University of Stuttgart

Byron Boots

Georgia Institute of Technology