An Activity Recognition Application based on Markov Decision Process Through Fish-Eye Camera

In this paper, we propose an artificial intelligence based solution to determine the number of customers in restaurants and automatize the production level arrangements. We employ computer vision techniques that are applied to the fish-eye cameras in fast-food restaurants in order to predict the number of customers in the restaurants, average waiting times and eventually determining the production level in the kitchen. We developed a people activity recognition framework based on Markov Decision Process (MDP) in a dynamic restaurant environment via fish-eye camera streams. The main purpose of the paper is to understand the activity knowledge of customers according to the movement sequence. Our framework infers the intention of each person from consequent movements of the person. The consequent actions of MDP are input parameters of the value function, which is solved by dynamic programming. After analyzing results of Faster-RCNN and YOLO on fish-eye images, we compared our Faster-RCNN-MDP framework with a rule based inference algorithm, which could not achieve a logical knowledge understanding due to detection noise of the fish-eye camera. However, our approach reached up to significant performance in accurately calculating the number of customers and average waiting time of customers in fast food restaurants.