Human-Flow-Aware Long-Term Mobile Robot Task Planning Based on Hierarchical Reinforcement Learning

Yuchen Liu, Luigi Palmieri, Ilche Georgievski, and Marco AielIo
IEEE Robotics and Automation Letters

Abstract

The difficulty in finding long-term planning policies for a mobile robot increases when operating in crowded and dynamic environments. State-of-the-art approaches do not consider cues of human-robot-shared dynamic environments. Aiming to fill this gap, we present a novel Human-Flow-Aware Guided Hierarchical Dyna-Q (HA-GHDQ) algorithm, which solves long-term robot task planning problems by using human motion patterns encoded in Maps of Dynamics (MoDs). To tackle the complexity of long-term robot operation in dynamic environments, we propose a combination of symbolic planning and Hierarchical Reinforcement Learning (HRL) that generates robot policies considering cost information derived from MoDs. We evaluated HA-GHDQ in a factory environment with two simulation and one real-world datasets to complete a transportation-and-assembly task. Our approach outperforms the baselines with respect to sample efficiency and final plan quality. Moreover, we show that it is more adaptable and robust against environmental changes than the baselines.

@InProceedings{SchaeferICRA2023,
title = { A Benchmark for Multi-Robot Planning in Realistic, Complex and Cluttered Environments },
author = {Schaefer, Simon and Palmieri, Luigi and Heuer, Lukas and Dillmann, Rüdiger and Koenig, Sven and  Kleiner, Alexander},
@ARTICLE{LiuRAL2023,  
author={Liu, Yuchen and Palmieri, Luigi and Georgievski, Ilche and Aiello, Marco},
journal={IEEE Robotics and Automation Letters}, 
title={Human-Flow-Aware Long-Term Mobile Robot Task Planning Based on Hierarchical Reinforcement Learning}, 
year={2023},
volume={8},
number={7},
pages={4068-4075},
doi={10.1109/LRA.2023.3280816}}