Temporal Consistent Video Manipulation : Facial Expression Transfer video to video

Abstract

Facial expression transfer is an important problem in computer vision community due to its applicability in human computer interaction, facial animation, etc. Most of the existing approaches mainly deal with images. However, expression transfer to videos yet to be explored. The extension of existing image based approaches to videos often leads to flickering artifacts in the result. This is due to the temporal inconsistency that may have been produced due to the independent processing of frames. In this paper, we address this problem by proposing a combination of loss functions in associated with a deep learning based method. Image reconstruction loss takes care of spatial domain consistency. The flow and warp losses address the temporal consistency. By maintaining consistency spatially as well as temporally, our method produce visually plausible results. The efficacy of our method is further explored in different problems like video dehazing, and video colorization method.