Pixel-wise conditioned generative adversarial networks for image synthesis and completion

Research output: Contribution to journalArticle

Authors

  • Cyprien Ruffino
  • Romain Christian Hérault
  • Eric Laloy
  • Gilles Gasso

Institutes & Expert groups

  • INSA Rouen - Normandie Université

Documents & links

Abstract

Generative Adversarial Networks (GANs) have proven successful for unsupervised image generation. Several works have extended GANs to image inpainting by conditioning the generation with parts of the image to be reconstructed. Despite their success, these methods have limitations in settings where only a small subset of the image pixels is known beforehand. In this paper we investigate the effectiveness of conditioning GANs when very few pixel values are provided. We propose a modelling framework which results in adding an explicit cost term to the GAN objective function to enforce pixel-wise conditioning. We investigate the influence of this regularization term on the quality of the generated images and the fulfillment of the given pixel constraints. Using the recent PacGAN technique, we ensure that we keep diversity in the generated samples. Conducted experiments on FashionMNIST show that the regulariza- tion term effectively controls the trade-offbetween quality of the generated images and the conditioning. Experimental evaluation on the CIFAR-10 and CelebA datasets evidences that our method achieves accu- rate results both visually and quantitatively in term of Fréchet Inception Distance, while still enforcing the pixel conditioning. We also evaluate our method on a texture image generation task using fully- convolutional networks. As a final contribution, we apply the method to a classical geological simulation application.

Details

Original languageEnglish
Pages (from-to)1-13
Number of pages13
JournalNeurocomputing
DOIs
Publication statusPublished - 20 Apr 2020

Keywords

  • Deep generative models, Generative adversarial networks, Conditional GAN

ID: 6875091