SinGAN-GIF: Learning a Generative model from a single GIF

Rajat Arora Yong Jae Lee

Paper Video


Abstract

We propose SinGAN-GIF which is an extension of SinGAN to short video snippets, often referred to as GIFs after the file format they are usually distributed in. Our method learns the distribution of both the image patches as well as their motion pattern. We do so by using a pyramid of 3D convolutional networks along with an image and a video discriminator. We show that though generative video models struggle to generate convincing results, our framework provides a good alternative to harness the power of GANs for various applications, working directly on video frames in entirety instead of working frame by frame.


Random Samples

Original

Samples

HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon


Changing Aspect Ratio

HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon
HTML5 Icon HTML5 Icon HTML5 Icon HTML5 Icon


Video Super Resolution (Vid 4 dataset)


Original

SuperRes



Original

SuperRes



Original

SuperRes



Original

SuperRes



Motion Transfer

Original Video

Image

Output

HTML5 Icon HTML5 Icon HTML5 Icon

Video Editing

Translated Video

Corrected

HTML5 Icon HTML5 Icon

Video Slo Mo

Original

Random Slow Sample

HTML5 Icon HTML5 Icon

Acknowledgements

This template has been partially borrowed from Phillip Isola and Richard Zhang for a colorful ECCV project; the code can be found here.