Continuous Visual World Modeling for Autonomous Robot Manipulation

Service robots need to handle a variety of everyday manipulation tasks to accomplish household chores such as cooking and cleaning. Successful execution of these tasks is highly dependent on how reliable the robot perceives its environment through noisy sensing. We present a visual world modeling system for service robots to generate and maintain accurate models of their environments for continuous scenarios. This system is designed to provide a generic platform for both humanoid and ground manipulation robots using different types of vision sensors and algorithms. In our particular implementation, visual data processed by different perception algorithms are used for building and continuously updating a world model of the environment. We evaluate our system on a variety of object manipulation scenarios and show that the system produces consistent perception outcomes suitable for different manipulation tasks.