Temporal stabilization of video object segmentation for 3D-TV applications

Our aim is to insert depth information into an existing 2D video sequence to provide content for 3D-TV applications, which we try to achieve through segmentation of the objects in the given 2D video sequence. To this effect, we present a method for temporal stabilization of video object segmentation algorithms for 3D-TV applications. First, two quantitative measures to evaluate temporal stability without ground-truth are discussed. Then, a pseudo-3D curve evolution method, which spatio-temporally stabilizes the estimated segmentation of a video object is introduced. Temporal stability is achieved by re-distributing existing object segmentation errors such that they will be less disturbing when the scene is rendered and viewed in 3D. Our starting point is the hypothesis that if making segmentation errors is inevitable, these errors should be made in a temporally consistent way for 3D-TV applications. This hypothesis is supported by the experiments, which show that there is significant improvement in segmentation quality both in terms of the objective quantitative measures and in terms of the viewing comfort in subjective perceptual tests. Therefore, it is possible to increase the perceptual object segmentation quality without increasing the actual segmentation accuracy. (C) 2004 Elsevier B.V. All rights reserved.