A Formal Approach to Discrepancy Generation for Systematic Testing of Execution Monitoring Algorithms in Simulation

Successful plan generation for autonomous systems is necessary but not sufficient to guarantee reach a goal state by an execution of the plan, since various discrepancies between the expected state and the observed state may occur during the plan execution (e.g., due to unexpected exogenous events, changes in the goals, or failure of robot parts) and these discrepancies may lead to plan failures. For that reason, these systems should be equipped with execution monitoring algorithms so that they can autonomously recover from such discrepancies. Before execution monitoring algorithms are deployed on autonomous systems, comprehensive testing and simulation is needed to evaluate their performances and to understand their applicability. With this motivation, we introduce formal methods for discrepancy generation with respect to the plan being executed and by utilizing feasibility checks of robotic actions, and we propose a novel generic algorithm for simulation of execution monitoring algorithms that enables systematic testing of them in simulation. We illustrate an application of our methods on an execution monitoring algorithm that involves guided replanning and diagnosis, in the context of some service robotics scenarios that utilize multiple bi-manual mobile manipulators.