-----

1.1 CNN

◎ LeNet。第一個「成功」應用於 MNIST 手寫字元集的 CNN。
LeCun, Yann, et al. "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86.11 (1998): 2278-2324.

◎ AlexNet 跟 VGGNet 都引用的論文。第一篇使用 GPU 的 CNN？本篇加寬失敗，VGGNet 加深成功。
Ciresan, Dan C., et al. "Flexible, high performance convolutional neural networks for image classification." IJCAI Proceedings-International Joint Conference on Artificial Intelligence. Vol. 22. No. 1. 2011.

◎ AlexNet

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012.

◎ ZFNet。微調 AlexNet 的參數。特徵可視化。

Zeiler, Matthew D., and Rob Fergus. "Visualizing and understanding convolutional networks." European conference on computer vision. Springer, Cham, 2014.

◎ VGGNet。conv5 分解成兩個 conv3，然後不斷加深。

Simonyan, Karen, and Andrew Zisserman. "Very deep convolutional networks for large-scale image recognition." arXiv preprint arXiv:1409.1556 (2014).

◎ NIN。GoogLeNet 的靈感來源。1 x 1 convolution。

Lin, Min, Qiang Chen, and Shuicheng Yan. "Network in network." arXiv preprint arXiv:1312.4400 (2013).

◎ GoogLeNet 的架構，Inception 的靈感來源。

Arora, Sanjeev, et al. "Provable bounds for learning some deep representations." International Conference on Machine Learning. 2014.

◎ GoogLeNet。

Szegedy, Christian, et al. "Going deeper with convolutions." Proceedings of the IEEE conference on computer vision and pattern recognition. 2015.

◎ GoogLeNet 系列。Inception v2。BN。

Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International Conference on Machine Learning. 2015.

◎ GoogLeNet 系列。Inception v3。

Szegedy, Christian, et al. "Rethinking the inception architecture for computer vision." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

◎ GoogLeNet 系列。Inception v4。結合 ResNet。

Szegedy, Christian, et al. "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning." AAAI. 2017.

◎ 太深了，梯度傳不下去，於是有了 highway。

Srivastava, Rupesh Kumar, Klaus Greff, and Jürgen Schmidhuber. "Highway networks." arXiv preprint arXiv:1505.00387 (2015).

◎ ResNet。繼承 VGGNet 的架構。

He, Kaiming, et al. "Deep residual learning for image recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2016.

◎ ResNet 的理論基礎。

He, Kaiming, et al. "Identity mappings in deep residual networks." European Conference on Computer Vision. Springer International Publishing, 2016.

◎ DenseNet。

Huang, Gao, et al. "Densely connected convolutional networks." arXiv preprint arXiv:1608.06993 (2016).

-----

1.2 RNN

◎ LSTM

Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780.

-----

1.3 Overview

◎ 如果你懂 CNN 與 RNN 之後，這篇可以看一下。如果不懂，還是可以看，領略一下 Deep Learning 在應用面的「威力」。

LeCun, Yann, Yoshua Bengio, and Geoffrey Hinton. "Deep learning." Nature 521.7553 (2015): 436-444.

◎ 這本「很棒」，也「有點難」。讀論文的時候，可以來參考相關的章節。

Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016.

https://github.com/HFTrader/DeepLearningBook/raw/master/DeepLearningBook.pdf

◎ 偏「歷史性」的回顧。不過如果你不懂 CNN、RNN 等，會不知所云。

Schmidhuber, Jürgen. "Deep learning in neural networks: An overview." Neural networks 61 (2015): 85-117.

-----

1.4 ML

◎ Top 10

Wu, Xindong, et al. "Top 10 algorithms in data mining." Knowledge and information systems 14.1 (2008): 1-37.

◎ Hot

Chen, Tianqi, and Carlos Guestrin. "XGBoost: A scalable tree boosting system." Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining. ACM, 2016.

----

2.1. R-CNN

◎ Pilot of R-CNN。

Szegedy, Christian, Alexander Toshev, and Dumitru Erhan. "Deep neural networks for object detection." Advances in neural information processing systems. 2013.

◎ R-CNN

Girshick, Ross, et al. "Rich feature hierarchies for accurate object detection and semantic segmentation." Proceedings of the IEEE conference on computer vision and pattern recognition. 2014.

◎ Fast R-CNN。

Girshick, Ross. "Fast R-CNN." Proceedings of the IEEE international conference on computer vision. 2015.

◎ Faster R-CNN。

Ren, Shaoqing, et al. "Faster R-CNN: Towards real-time object detection with region proposal networks." Advances in neural information processing systems. 2015.

◎ YOLO。

Redmon, Joseph, et al. "You only look once: Unified, real-time object detection." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016.

◎ SSD。

Liu, Wei, et al. "SSD: Single shot multibox detector." European conference on computer vision. Springer, Cham, 2016.

◎ Mask R-CNN。

He, Kaiming, et al. "Mask R-CNN." arXiv preprint arXiv:1703.06870 (2017).

-----

2.2 GAN

◎ GAN。

Goodfellow, Ian, et al. "Generative adversarial nets." Advances in neural information processing systems. 2014.

◎ Wasserstein GAN - 1。

Arjovsky, Martin, and Léon Bottou. "Towards principled methods for training generative adversarial networks." arXiv preprint arXiv:1701.04862 (2017).

◎ Wasserstein GAN - 2。

Arjovsky, Martin, Soumith Chintala, and Léon Bottou. "Wasserstein gan." arXiv preprint arXiv:1701.07875 (2017).

◎ Wasserstein GAN - 3。

Gulrajani, Ishaan, et al. "Improved training of wasserstein gans." arXiv preprint arXiv:1704.00028 (2017).

◎

Creswell, Antonia, et al. "Generative Adversarial Networks: An Overview." arXiv preprint arXiv:1710.07035 (2017).

◎
Wang, Kunfeng, et al. "Generative adversarial networks: introduction and outlook." IEEE/CAA Journal of Automatica Sinica 4.4 (2017): 588-598.

◎
How Generative Adversarial Nets and its variants Work: An Overview of GAN
https://arxiv.org/pdf/1711.05914.pdf

-----

3. Reinforcement Learning

◎ RL 的經典書。
Sutton, Richard S., and Andrew G. Barto. Reinforcement learning: An introduction. (2016): 424.
http://incompleteideas.net/sutton/book/bookdraft2016sep.pdf

-----

3.1 Game

◎ DQN
Mnih, Volodymyr, et al. "Human-level control through deep reinforcement learning." Nature 518.7540 (2015): 529-533.

◎ A3C
Mnih, Volodymyr, et al. "Asynchronous methods for deep reinforcement learning." International Conference on Machine Learning. 2016.

◎ UNREAL
Jaderberg, Max, et al. "Reinforcement learning with unsupervised auxiliary tasks." arXiv preprint arXiv:1611.05397 (2016).

-----

3.2 Go

◎ 1994

Schraudolph, Nicol N., Peter Dayan, and Terrence J. Sejnowski. "Temporal difference learning of position evaluation in the game of Go." Advances in Neural Information Processing Systems. 1994.

◎ 1996

Enzenberger, Markus. "The integration of a priori knowledge into a Go playing neural network." URL: http://www. markus-enzenberger. de/neurogo. html (1996).

◎ 2002

Van Der Werf, Erik, et al. "Local move prediction in Go." International Conference on Computers and Games. Springer, Berlin, Heidelberg, 2002.

◎ 2004

Enzenberger, Markus. "Evaluation in Go by a neural network using soft segmentation." Advances in Computer Games. Springer US, 2004. 97-108.

◎ 2008

Sutskever, Ilya, and Vinod Nair. "Mimicking go experts with convolutional neural networks." Artificial Neural Networks-ICANN 2008 (2008): 101-110.

◎ 2014

Maddison, Chris J., et al. "Move evaluation in go using deep convolutional neural networks." arXiv preprint arXiv:1412.6564 (2014).

◎ 2015

Clark, Christopher, and Amos Storkey. "Training deep convolutional neural networks to play go." International Conference on Machine Learning. 2015.

◎ 2016 AlphaGo。

Silver, David, et al. "Mastering the game of Go with deep neural networks and tree search." Nature 529.7587 (2016): 484-489.

◎ 2017 AlphaGo Zero。

Silver, David, et al. "Mastering the game of Go without human knowledge." Nature 550.7676 (2017): 354-359.

-----

4.1 NTM

◎ NTM

Graves, Alex, Greg Wayne, and Ivo Danihelka. "Neural turing machines." arXiv preprint arXiv:1410.5401 (2014).

◎ DNC

Graves, Alex, et al. "Hybrid computing using a neural network with dynamic external memory." Nature 538.7626 (2016): 471-476.

◎

Olah, Chris, and Shan Carter. "Attention and augmented recurrent neural networks." Distill 1.9 (2016): e1.

-----

5.1 Regularization

◎ Weight decay。

Krogh, Anders, and John A. Hertz. "A simple weight decay can improve generalization." Advances in neural information processing systems. 1992.

◎ Dropout。

Srivastava, Nitish, et al. "Dropout: a simple way to prevent neural networks from overfitting." Journal of machine learning research 15.1 (2014): 1929-1958.

◎ Batch Normalization。

Ioffe, Sergey, and Christian Szegedy. "Batch normalization: Accelerating deep network training by reducing internal covariate shift." International Conference on Machine Learning. 2015.

-----

5.2 Optimization

◎ Overview of optimization

Ruder, Sebastian. "An overview of gradient descent optimization algorithms." arXiv preprint arXiv:1609.04747 (2016).

◎ 1 SGD

Bottou, Léon. "Stochastic gradient descent tricks." Neural networks: Tricks of the trade. Springer Berlin Heidelberg, 2012. 421-436.

◎ 2 Momentum

Polyak, Boris T. "Some methods of speeding up the convergence of iteration methods." USSR Computational Mathematics and Mathematical Physics 4.5 (1964): 1-17.

◎ 3 NAG

Sutskever, Ilya, et al. "On the importance of initialization and momentum in deep learning." International conference on machine learning. 2013.

◎ 4 AdaGrad

Duchi, John, Elad Hazan, and Yoram Singer. "Adaptive subgradient methods for online learning and stochastic optimization." Journal of Machine Learning Research 12.Jul (2011): 2121-2159.

◎ 5 AdaADelta

Zeiler, Matthew D. "ADADELTA: an adaptive learning rate method." arXiv preprint arXiv:1212.5701 (2012).

◎ 6 RMSProp

Hinton, G., N. Srivastava, and K. Swersky. "RMSProp: Divide the gradient by a running average of its recent magnitude." Neural networks for machine learning, Coursera lecture 6e (2012).

◎ 7 Adam

Kingma, Diederik, and Jimmy Ba. "Adam: A method for stochastic optimization." arXiv preprint arXiv:1412.6980 (2014).

-----

6.1 AE

◎

Hinton, Geoffrey E., and Richard S. Zemel. "Autoencoders, minimum description length and Helmholtz free energy." Advances in neural information processing systems. 1994.

◎

Bengio, Yoshua, et al. "Greedy layer-wise training of deep networks." Advances in neural information processing systems. 2007.

◎

Poultney, Christopher, Sumit Chopra, and Yann L. Cun. "Efficient learning of sparse representations with an energy-based model." Advances in neural information processing systems. 2007.

◎

Vincent, Pascal, et al. "Extracting and composing robust features with denoising autoencoders." Proceedings of the 25th international conference on Machine learning. ACM, 2008.

-----

6.2 RBM

◎ 這篇把 RBM 的物理意義講得很清楚浅析 Hinton 最近提出的 Capsule 计划 - 知乎专栏
https://zhuanlan.zhihu.com/p/29435406

◎

Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast learning algorithm for deep belief nets." Neural computation 18.7 (2006): 1527-1554.

◎

Hinton, Geoffrey E., and Ruslan R. Salakhutdinov. "Reducing the dimensionality of data with neural networks." science 313.5786 (2006): 504-507.

◎

Salakhutdinov, Ruslan, and Geoffrey Hinton. "Deep boltzmann machines." Artificial Intelligence and Statistics. 2009.

-----

7.1 Robotics

◎ 機器人論文，精選裡面的精選

Levine, Sergey, et al. "End-to-end training of deep visuomotor policies." Journal of Machine Learning Research 17.39 (2016): 1-40.

-----

7.2 Fintech

◎ Machine Learning

Cavalcante, Rodolfo C., et al. "Computational intelligence and financial markets: A survey and future directions." Expert Systems with Applications 55 (2016): 194-211.

◎ ANN

Tkáč, Michal, and Robert Verner. "Artificial neural networks in business: Two decades of research." Applied Soft Computing 38 (2016): 788-804.

◎ Deep Learning

Chong, Eunsuk, Chulwoo Han, and Frank C. Park. "Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies." Expert Systems with Applications 83 (2017): 187-205.

◎

Li, Yawen, et al. "On neural networks and learning systems for business computing." Neurocomputing (2017).

The Star Also Rises

Wednesday, November 22, 2017

AI 從頭學（三八）：Highlight