Video Summarization using Spatio-Temporal Features by Detecting Representative Content based on Supervised Deep Learning

Abstract

Video Summarization is the approach to generate the compact version of video keeping relevant content intact and eliminating redundancy. In this work, a frame- work has been proposed which makes use of the spatial and temporal features with self attention from the video sequences to identify the representative con- tent by generating temporal proposals and supervised learning from the data manually created by humans or users. Existing Supervised methods don't deal with the temporal interest and its consistency. For that temporal uniformity is also necessary which can be addressed by predicting the temporal proposals of the video segment. The proposed work treats it as temporal action detection which predicts importance score and location of the segments simultaneously by developing the anchor based method which generates anchors of varying lengths to identify interesting proposals. Moreover the extensive quantitative and qualitative analysis on TVSumm and SumMe datasets augmented with OVP and YouTube datasets justify the e ectiveness of the method.