Literature on GANs
14 Jan 2019GANs are a popular types of generative models that attemps to approximate a probabilty distribution. It does so by setting up a game between 2 agents, one that is tasked with generating samples (the generator), and one that is tasked with deciding whether samples generated are real or fake (the discriminator). The tyical setup uses deep nets as both generator and discriminator, and each have different cost functions. They are trained by alternating optimization steps on each.
One criticism is that the distribution learned by GANs have a very-low support, and therefore only learn a low-dimensional subset of the target distribution. This is discussed in Do GANs actually learn the distribution? An empirical study.
There exists different architecture for GANs. In Are GANs Created Equal? A Large-Scale Study, the authors compare different GANs and find no consistent, meaningful difference among those. They also propose some ways to compare GANs; another paper in that direction is Assessing Generative Models via Precision and Recall.
This article discusses a few workaround to some of the biggest pitfalls of GANs, namely:
- mode collapse
- non-convergence (oscillation)
- vanishing gradient (when discriminator is over-confident)
Unfortunately, I find most of the solutions to be ad-hoc solutions, like duct tape, not really trying to address the underlying problems. This article still has some merit, at least as a nice concise summary of the current state of research (e.g., summarize all cost functions used in GANs).
[ML deeplearning gan