Region based multi-stream convolutional neural networks for collective activity recognition

Collective activity recognition, which analyses the behavior of groups of people in videos, is an important goal of video surveillance systems. In this paper, we focus on collective activity recognition problem and propose a new multi-stream convolutional neural network architecture that utilizes information extracted from multiple regions. The proposed method is the first work that uses a multi-stream network and multiple regions in this problem. Various strategies to fuse multiple spatial and temporal streams are explored. We evaluate the proposed method on two benchmark datasets, the Collective Activity Dataset and the Volleyball Dataset. Our experimental results show that the proposed method improves collective activity recognition performance when compared to the state-of-the-art approaches. (C) 2019 Elsevier Inc. All rights reserved.