AI 從頭學（一八）：Convolutional Neural Network

AI 從頭學（一八）：Convolutional Neural Network

2017/03/22

有關 Convolutional Neural Network (CNN) [1]，我建議先瞭解一下 BP Algorithm [2]，然後再看我寫的兩篇 LeNet 的文章 [3], [4]，Goodfellow 的 CNN 並不容易看懂 [5]。雖然之前已經有相關的研究 [6]-[10]，但 Yann LeCun 才算開啟現代 CNN 研究的先河 [11]-[15] 。

-----

Fig. Evolution（圖片來源：Pixabay）

-----

在 LeNet-5 [13] 之後，CNN 又有持續的演進，更多層，準確率更高，硬體需求也更高。包含2012 AlexNet [16]-[19]、2013 ZFNet [20], [21]、2014 GoogLeNet [22]、2014 VGGNet [23]、2015 ResNet [24]-[28]、2016 DenseNet [29]、2017 SqueezeNet [30] 等等。

以上的CNNs，在 [31] 有簡潔明瞭的介紹，參考圖1。本文以推薦這本書為主要目的，有心於深度學習的讀者，強力建議購買。精華部分也有英文版 [32], [33]。想直接看論文者，我已經幫忙查好了！：）CNN的推進，可以參考圖2。

[31] 的後半部介紹了 R-CNN 系列，R 是 Region 的意思。R-CNN 可以從圖片中選出幾個物體，用框框標示後，加以識別。R-CNN 與時間相關的 RNN 結合後，可以辨識影片中的物體，此項技術可以應用於自駕車！

Fig. 1. 深度學習: Caffe 之經典模型詳解與實戰 [31]。

-----

Fig. 2. Evolution of depth, p. 33, [34].

-----

出版說明：

2019/10/18

每個人有其理解世界的方式。Top-down 是先有全觀，再求細部理解。Bottom-up 則是由一堆零組件開始。

關於深度學習，如果你過去線性代數跟統計沒有學好，還是可以直接進入。首先去理解一個網路：網路的輸入、輸出是什麼？網路的結構是什麼？如何優化網路的參數？一堆網路是如何演進？

重點中的重點是要經常問自己「什麼」與「為什麼」！

-----

References

[1] Convolutional neural network - Wikipedia
https://en.wikipedia.org/wiki/Convolutional_neural_network

[2] AI從頭學（九）：Back Propagation
http://hemingwang.blogspot.tw/2017/02/aiback-propagation.html

[3] AI從頭學（一二）：LeNet
http://hemingwang.blogspot.tw/2017/03/ailenet.html

[4] AI從頭學（一三）：LeNet - F6
http://hemingwang.blogspot.tw/2017/03/ailenet-f6.html

[5] 9 Convolutional Networks
http://www.deeplearningbook.org/contents/convnets.html 

◎ 1983 Neocognitron

[6] 1974_A biological approach to pattern recognition

[7] 1975_Cognitron, A self-organizing multilayered neural network

[8] 1982_Neocognitron, A new algorithm for pattern recognition tolerant of deformations and shifts in position

[9] 1983_Neocognitron, a neural network model for a mechanism of visual pattern recognition

[10] 1988_Neocognitron, A hierarchical neural network capable of visual pattern recognition

◎ 1998 LeNet

[11] 1994_Word level training of a handwritten word recognizer base on convolutional neural networks

[12] 1997_Reading checks with multilayer graph transformer networks

[13] 1998_Gradient-Based Learning Applied to Document Recognition

[14] 2009_What is the best multi-stage architecture for object recognition

[15] 2012_Convolutional neural networks applied to house numbers digit classification

◎ 2012 AlexNet

[16] 2009_Learning multiple layers of features from tiny images

[17] 2012_Imagenet classification with deep convolutional neural networks

◎ Dropout

[18] 2013_Regularization of neural networks using dropconnect

[19] 2014_Dropout, a simple way to prevent neural networks from overfitting

◎ 2013 ZFNet

[20] 2013_Stochastic pooling for regularization of deep convolutional neural networks

[21] 2014_Visualizing and understanding convolutional networks

◎ 2014 GoogLeNet

[22] 2015_Going deeper with convolutions

◎ 2014 VGGNet

[23] 2015_Very deep convolutional networks for large-scale image recognition

◎ 2015 ResNet

[24] 2015_Deep residual learning for image recognition

[25] 2016_Deep residual learning for image recognition

[26] 2016_Identity mappings in deep residual networks

[27] 2016_Resnet in resnet, Generalizing residual architectures

[28] Torch  Training and investigating Residual Nets
http://torch.ch/blog/2016/02/04/resnets.html

◎ 2016 DenseNet

[29] 2016_Densely connected convolutional networks

◎ 2017_SqueezeNet

[30] 2017_SqueezeNet, AlexNet-level accuracy with 50x fewer parameters and 0.5 MB model size

[31] 書名：深度學習: Caffe 之經典模型詳解與實戰，ISBN：7121301180，作者：樂毅，出版社：電子工業，出版日期：2016-09-30.

[32] The 9 Deep Learning Papers You Need To Know About (Understanding CNNs Part 3)
https://adeshpande3.github.io/adeshpande3.github.io/The-9-Deep-Learning-Papers-You-Need-To-Know-About.html

[33] 9 Key Deep Learning Papers, Explained
http://www.kdnuggets.com/2016/09/9-key-deep-learning-papers-explained.html

[34] Convnets.pdf
http://www.cs.toronto.edu/~guerzhoy/411/lec/W06/convnets.pdf