Monte-Carlo Policy-Tree Decision Making

University of Michigan APRIL Laboratory, 2022

PDF thumbnail
(PDF, 5.4 MB )

Abstract

In this paper, we propose Monte-Carlo Policy-Tree Decision Making (MCPTDM), an uncertainty-aware framework for high-variance planning problems with multiple dynamic agents. Planning when surrounded by multiple uncertain dynamic agents is hard because we cannot be certain of either the initial states or the future actions of those agents, leading to an exponential explosion in possible futures. Many important real-world problems, such as autonomous driving, fit this model. To address these difficulties, we combine Multi-policy Decision Making (MPDM) and Monte Carlo tree search (MCTS) and perform policy tree search with marginal action cost (MAC) estimation and repeated belief particles. We first design a synthetic experiment to evaluate these novel improvements in isolation. Then we evaluate the complete framework in a self-driving car simulation experiment and compare it against MPDM and Efficient Uncertainty-aware Decision Making (EUDM) methods. We release our complete source code for replicating our experiments and results.


bibtex

@techreport{haggenmiller2022mcptdm,
    AUTHOR     = {Acshi Haggenmiller and Edwin Olson},
    TITLE      = {{Monte-Carlo} Policy-Tree Decision Making},
    INSTITUTION = {University of Michigan APRIL Laboratory},
    YEAR       = {2022},
    MONTH      = {March},
}