Video anomaly detection with autoregressive modeling of covariance features

In this paper, we propose three different methods for anomaly detection in surveillance videos based on modeling of observation likelihoods. By means of the methods we propose, normal (typical) events in a scene are learned in a probabilistic framework by estimating the features of consecutive frames taken from the surveillance camera. The proposed methods are based on long short-term memory (LSTM) and linear regression. To decide whether an observation sequence (i.e., a small video patch) contains an anomaly or not, its likelihood under the modeled typical observation distribution is thresholded. An anomaly is decided to be present if the threshold is exceeded. Due to its effectiveness in object detection and action recognition applications, covariance features are used in this study to compactly reduce the dimensionality of the shape and motion cues of spatiotemporal patches obtained from the video segments. The two most successful methods are based on the final state vector of LSTM and support vector regression applied to mean covariance features and achieve an average performance of up to 0.95 area under curve on benchmark datasets.