AI從頭學（）：Generative Adversarial Nets

AI從頭學（）：Generative Adversarial Nets

2017/05/23

前言：

施工中...

Summary：

Generative Adversarial Nets (GAN) [1] 自2014年推出以來，引 AI 界起很大的熱潮。GAN 的概念，是由 generative net (GN) 跟 discriminative net (DN) 相互對抗，最後 DN 不再能分辨 GN 生成的圖片是真是假，GN 就成功了（能產生以假亂真的圖片）。Adversarial 的觀念是新的，而 generative 跟 discriminative 的觀念則已超過十年 [2]。

有關 GAN 的簡單介紹，可以參考 [3], [4]，較深入的討論，則可參考 [5]-[10]。[11], [12] 則有視覺化的訓練可以參考。

Log likelihood 是學習 GAN 的基礎 [13]-[16]。另外我們可以參考其他的論文來瞭解 GN [17]-[22]。最後則提供徹底掌握 GAN 所需的資料 [23]-[26]。

其實，以上資料並不足以徹底掌握 GAN。Wasserstein GAN [27]-[31] 才是完備的 GAN。而 Kullback–Leibler divergence [32] 與 Jensen–Shannon divergence [33] 算是基礎。

-----

Outline：

1. Formula
2. Generative Net
3. Deep Generative Models

本文重點有三：

1. GAN 公式
2. 生成網路構造
3. 瞭解 GAN 所需之相關資料

-----

Fig. 1.1a. Backpropagate derivatives through generative processes, p. 2 [1].

Fig. 1.1b. Random variable and probability distribution, p. 57 [23].

Fig. 1.1c. Expectation, p. 60 [23].

Fig. 1.1d. Normal distribution, also known as the Gaussian distribution, p. 63 [23].

-----

Fig. 1.2a. D and G play the following two-player minimax game with value function V (G;D), p. 3 [1].

Fig. 1.2b. The model can then be trained by maximizing the log likelihood, p. 2 [1].

Fig. 1.2c. Decomposition into the positive phase and negative phase of learning, p. 608 [23].

Fig. 1.3. Generative adversarial nets are trained by simultaneously updating the discriminative distribution, p. 4 [1].

Fig. 1.4. Minibatch stochastic gradient descent training of generative adversarial nets, p.4 [1].

Fig. 2.1a. DCGAN generator used for LSUN scene modeling, p. 4 [17].

Fig. 2.1b. A 100 dimensional uniform distribution Z, p. 4 [17].

Fig. 2.2. The architecture of the generator in Style-GAN, p. 324 [18].

Fig. 2.3. Text-conditional convolutional GAN architecture, p. 4 [19].

Fig. 2.4. A deconvnet layer (left) attached to a convnet layer (right), p. 822 [20].

Fig. 3.1. Deep generative models, p. vi [23].

Fig. 3.2. Deep learning taxonomy, p. 492 [24].

Fig. 3.3. Chapters 16-19, p. 671 [23].

Fig. 3.4. From section 3.14 to chapter 16, p. 560 [23].

Fig. 4.1. Fully-observed models [6].

Fig. 4.2. Transformation models [6].

Fig. 4.3. Latent bariable models [6].

Fig. 5.1. Probabilistic modeling of natural images, p. 563 [23], p. 8 [26].

Fig. 5.2. An illustration of the slow mixing problem in deep probabilistic models, p. 604 [23].

Fig. 5.3. Positive phase and negative phase, p. 611 [23].

Fig. 5.4. The KL divergence is asymmetric, p. 76 [23].



-----

References

1 GAN

[1] 2014_Generative adversarial nets

[2] 2007_Generative or discriminative, getting the best of both worlds

-----

2 GAN Internet

[3] 生成对抗式网络(Generative Adversarial Networks) – LHY's World
http://closure11.com/%E7%94%9F%E6%88%90%E5%AF%B9%E6%8A%97%E5%BC%8F%E7%BD%91%E7%BB%9Cgenerative-adversarial-networks/

[4] 能根據文字生成圖片的GAN，深度學習領域的又一新星 GigCasa 激趣網
http://www.gigcasa.com/articles/465963

[5] 深度学习与生成式模型 - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52512459

[6] 生成式对抗网络GAN研究进展（一） - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52537114

[7] 生成式对抗网络GAN研究进展（二）——原始GAN - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52549409

[8] 生成式对抗网络GAN研究进展（三）——条件GAN - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52555083

[9] 生成式对抗网络GAN研究进展（四）——Laplacian Pyramid of Adversarial Networks，LAPGAN - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52562851

[10] 生成式对抗网络GAN研究进展（五）——Deep Convolutional Generative Adversarial Nerworks，DCGAN - Solomon1558的专栏 - 博客频道 - CSDN.NET
http://blog.csdn.net/solomon1558/article/details/52573596

[11] An introduction to Generative Adversarial Networks (with code in TensorFlow) – AYLIEN
http://blog.aylien.com/introduction-generative-adversarial-networks-code-tensorflow/

[12] Adverarial Nets
http://cs.stanford.edu/people/karpathy/gan/

-----

3 log likelihood

[13] 2009_Deep Boltzmann machines

[14] Likelihood function - Wikipedia
https://en.wikipedia.org/wiki/Likelihood_function

[15] 1.4 - Likelihood & LogLikelihood _ STAT 504
https://onlinecourses.science.psu.edu/stat504/node/27

[16] Chapter 18 Confronting the Partition Function
http://www.deeplearningbook.org/contents/partition.html

-----

4 Generator

[17] 2016_Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks

[18] 2016_Generative image modeling using style and structure adversarial networks

[19] 2016_Generative adversarial text to image synthesis

[20] 2014_Visualizing and understanding convolutional networks

[21] 2011_Adaptive deconvolutional networks for mid and high level feature learning

[22] 2016_A guide to convolution arithmetic for deep learning

-----

5 Goodfellow

[23] 2016_Deep Learning
https://github.com/HFTrader/DeepLearningBook/raw/master/DeepLearningBook.pdf

[24] 2016_Practical Machine Learning

[25] 2009_Learning multiple layers of features from tiny images

[26] 2011_Unsupervised models of images by spike-and-slab RBMs

-----

6 Goodfellow

[27] 2016_NIPS 2016 Tutorial, Generative Adversarial Networks

-----

7 Wasserstein GAN

[28] 令人拍案叫绝的Wasserstein GAN - 知乎专栏
https://zhuanlan.zhihu.com/p/25071913

[29] 生成式对抗网络GAN有哪些最新的发展，可以实际应用到哪些场景中？ - 知乎
https://www.zhihu.com/question/52602529/answer/158727900

[30] 2017_Towards principled methods for training generative adversarial networks

[31] 2017_Wasserstein GAN

[32] 2017_ Improved training of Wasserstein GANs

[33] Kullback–Leibler divergence - Wikipedia
https://en.wikipedia.org/wiki/Kullback%E2%80%93Leibler_divergence

[34] Jensen–Shannon divergence - Wikipedia
https://en.wikipedia.org/wiki/Jensen%E2%80%93Shannon_divergence