Sequence Matching Based Automatic Retake Detection Framework for Rushes Video

Abstract

Automatically selecting the important content from rushes video is a challenging task due to the difficulty in eliminating raw data, such as useless content and redundant content. Redundancy elimination is difficult since repetitive segments, which are takes of the same scene, usually have different lengths and motion patterns. In this work, a new methodology is proposed to determine retakes in rushes video. The video is divided into shots by the proposed automatic shot boundary detection using local singular value decomposition and k-means clustering. Shots that contain the useless contents were eliminated by our proposed technique integrated a near duplicated key frame (NDK) algorithm. The local features of each remaining frames were extracted using the scale-invariant feature transform (SIFT) algorithm. The similarity between consecutive frames was calculated using SIFT matching and then converted into a string. The given strings were then concatenated into a longer string sequence to use as a shot representation. The similarity between each pair of sequences was evaluated using the longest common subsequence algorithm. Our method was evaluated in direct comparison with the conventional technique. Overall, when evaluated across the TRECVID 2007 and 2008 data sets that represent diverse styles of rushes videos, the proposed methodology provided a higher degree of accuracy in the detection of retakes in rushes videos.