Visual object tracking by using ranking loss and spatial-temporal features

This paper introduces a novel two-stream deep neural network tracker for robust object tracking. In the proposed network, we use both spatial and temporal features and employ a novel loss function called ranking loss. The class confidence scores coming from the two-stream (spatial and temporal) networks are fused at the end for final decision. Using ranking loss in the proposed tracker enforces the networks to learn giving higher scores to the candidate regions that frame the target object better. As a result, the tracker returns more precise bounding boxes framing the target object, and the risk of tracking error accumulation and drifts are largely mitigated when the proposed network architecture is used with a simple yet effective model update rule. We conducted extensive experiments on six different benchmarks, including OTB-2015, VOT-2017, TC-128, DTB70, NfS and UAV123. Our proposed tracker achieves the state-of-the-art results on the most of the tested challenging tracking datasets. Especially, our results on the OTB-2015, DTB70, NfS and TC-128 datasets are very promising.