深度學習論文研討（一）：機器學習（一）
2020/12/08
-----
前言：
本篇文章建議了「一種」學習機器學習的方法。另外，參考文獻 [3]，handout slides 省略了放映過程。
-----
https://pixabay.com/zh/photos/tractor-tractors-drive-drives-243431/
圖一、Wood、Vehicle、Field。
-----
Summary：
機器學習 [1] - [3] 是許多演算法的總稱，最有名的可能是支援向量機 Support Vector Machine [4] 以及決策樹 Decision Tree，較基本的則是回歸分析 [1] - [3]。
2012 年興起的深度學習 [5] 原本叫做類神經網路，是機器學習裡面的一支。特點在於可以用高效的平行運算的 GPU 與深層的大型網路架構，解決龐大資料的分類問題等等。[6] 的 Fig. 1 說明了人工智慧、機器學習、類神經網路、深度學習之間的關係。[6] 的 Fig.2 則圖示為何深度學習會逐漸取代一般的機器學習。
對機器學習若很有興趣，可以找機器學習的論文仔細研讀 [7]。先學深度學習 [8]，再回頭看機器學習，也是一種方式。
-----
圖二、人工智慧、機器學習、深度學習 [6]。
-----
圖三、機器學習與深度學習 [6]。
-----
說明：
如果我們比較機器學習 [1] - [3] 與深度學習 [5], [8] ，可以觀察到一件事：基礎都是數學，特別是線性代數與機率統計，然後最簡單的演算法都是回歸分析。另外，Overfitting 與 Regularization 則不管是機器學習 [1] 與深度學習都是非常重要的進階主題，簡單說，Overfitting 就是考古題做太多，臨場表現反而不好。Regularization 則是避免 Overfitting 的方法總稱。
李宏毅老師的線上課程，主要內容是深度學習，但一開始都會先講回歸分析 [8]。這是因為複雜的神經網路，可以說是從簡單的回歸分析演化而來。精神上都是用一條直線或曲線（也許是多項式）來 fit 一堆資料。只是神經網路複雜很多，資料的維度也高很多。
要不要學機器學習？何時學機器學習，每個人目的不同，基礎不同，所以不會有一定的答案。如果時間允許，[1], [2] 慢慢看完，並且每段影片都寫心得，是很好的學習方式。時間不夠，先看看 SVM [4] 也就夠了。
就像 [5] 是很好的教科書，但如果要你先複習線性代數與機率，陣亡的機會很高，還不如先研習 CNN 與 RNN 那兩章，雖然直接研讀 CNN、RNN，如果沒人帶，自己讀的陣亡率也很高。
如果你需要一些建議，那我建議從深度學習開始 [8]，我把龐大的教材，選了一些我認為的，比較重要的主題，列舉出來。這些重要的主題你研習完後，可以繼續研習其他的深度學習 [8]，或者開始認真打好機器學習、深度學習的基礎 [1] - [3]。
-----
機器學習基石
重點一：Linear Regression（34 - 37）。
重點二：Logistic Regression（38 - 41）。
重點三：Overfitting（50 - 53）。
重點四：Regularization（54 - 57）。
-----
機器學習技法
重點五：Linear Support Vector Machine（01 - 05）。
重點六：Dual Support Vector Machine（06 - 09）。
重點七：Kernel Support Vector Machine（10 - 13）。
重點八：Soft-Margin Support Vector Machine（14 - 17）。
重點九：Kernel Logistic Regression（18 - 21）。
重點十：Support Vector Regression（22 - 25）。
重點十一：Blending and Bagging（26 - 29）。
重點十二：Adaptive Boosting（30 - 33）。
重點十三：Decision Tree（34 - 37）。
重點十四：Random Forest（38 - 41）。
重點十五：Gradient Boosted Decision Tree（42 - 45）。
重點十六：Neural Network（46 - 49）。
重點十七：Radial Basis Function Network（54 - 57）。
-----
(01) The Learning Problem :: Course Introduction @ Machine Learning Foundations (機器學習基石) - YouTube
(02) The Learning Problem :: What Is Machine Learning @ Machine Learning Foundations (機器學習基石) - YouTube
(03) The Learning Problem :: Applications of Machine Learning @ Machine Learning Foundations (機器學習基石) - YouTube
(04) The Learning Problem :: Components of Learning @ Machine Learning Foundations (機器學習基石) - YouTube
(05) The Learning Problem :: Machine Learning and Other Fields @ Machine Learning Foundations (機器學習基石) - YouTube
(06) Learning to Answer Yes/No :: Perceptron Hypothesis Set @ Machine Learning Foundations (機器學習基石) - YouTube
(07) Learning to Answer Yes/No :: Perceptron Learning Algorithm @ Machine Learning Foundations (機器學習基石) - YouTube
(08) Learning to Answer Yes/No :: Guarantee of PLA @ Machine Learning Foundations (機器學習基石) - YouTube
(09) Learning to Answer Yes/No :: Non-Separable Data @ Machine Learning Foundations (機器學習基石) - YouTube
(10) Types of Learning :: Learning with Different Output Space @ Machine Learning Foundations (機器學習基石) - YouTube
(11) Types of Learning :: Learning with Different Data Label @ Machine Learning Foundations (機器學習基石) - YouTube
(12) Types of Learning :: Learning with Different Protocol @ Machine Learning Foundations (機器學習基石) - YouTube
(13) Types of Learning :: Learning with Different Input Space @ Machine Learning Foundations (機器學習基石) - YouTube
(14) Feasibility of Learning :: Learning is Impossible? @ Machine Learning Foundations (機器學習基石) - YouTube
(15) Feasibility of Learning :: Probability to the Rescue @ Machine Learning Foundations (機器學習基石) - YouTube
(16) Feasibility of Learning :: Connection to Learning @ Machine Learning Foundations (機器學習基石) - YouTube
(17) Feasibility of Learning :: Connection to Real Learning @ Machine Learning Foundations (機器學習基石) - YouTube
(18) Training versus Testing :: Recap and Preview @ Machine Learning Foundations (機器學習基石) - YouTube
(19) Training versus Testing :: Effective Number of Lines @ Machine Learning Foundations (機器學習基石) - YouTube
(20) Training versus Testing :: Effective Number of Hypotheses @ Machine Learning Foundations (機器學習基石) - YouTube
(21) Training versus Testing :: Break Point @ Machine Learning Foundations (機器學習基石) - YouTube
(22) Theory of Generalization :: Restriction of Break Point @ Machine Learning Foundations (機器學習基石) - YouTube
(23) Theory of Generalization :: Bounding Function: Basic Cases @ Machine Learning Foundations (機器學習基石) - YouTube
(24) Theory of Generalization :: Bounding Function: Inductive @ Machine Learning Foundations (機器學習基石) - YouTube
(25) Theory of Generalization :: A Pictorial Proof @ Machine Learning Foundations (機器學習基石) - YouTube
(26) The VC Dimension :: Definition of VC Dimension @ Machine Learning Foundations (機器學習基石) - YouTube
(27) The VC Dimension :: VC Dimension of Perceptrons @ Machine Learning Foundations (機器學習基石) - YouTube
(28) The VC Dimension :: Physical Intuition of VC Dimension @ Machine Learning Foundations (機器學習基石) - YouTube
(29) The VC Dimension :: Interpreting VC Dimension @ Machine Learning Foundations (機器學習基石) - YouTube
(30) Noise and Error :: Noise and Probabilistic Target @ Machine Learning Foundations (機器學習基石) - YouTube
(31) Noise and Error :: Error Measure @ Machine Learning Foundations (機器學習基石) - YouTube
(32) Noise and Error :: Algorithmic Error Measure @ Machine Learning Foundations (機器學習基石) - YouTube
(33) Noise and Error :: Weighted Classification @ Machine Learning Foundations (機器學習基石) - YouTube
(34) Linear Regression :: Linear Regression Problem @ Machine Learning Foundations (機器學習基石) - YouTube
(35) Linear Regression :: Linear Regression Algorithm @ Machine Learning Foundations (機器學習基石) - YouTube
(36) Linear Regression :: Generalization Issue @ Machine Learning Foundations (機器學習基石) - YouTube
(37) Linear Regression :: for Binary Classification @ Machine Learning Foundations (機器學習基石) - YouTube
(38) Logistic Regression :: Logistic Regression Problem @ Machine Learning Foundations (機器學習基石) - YouTube
(39) Logistic Regression :: Logistic Regression Error @ Machine Learning Foundations (機器學習基石) - YouTube
(40) Logistic Regression :: Gradient of Logistic Regression Error @ Machine Learning Foundations (機器學習基石) - YouTube
(41) Logistic Regression :: Gradient Descent @ Machine Learning Foundations (機器學習基石) - YouTube
(42) Linear Models for Classification :: Binary Classification @ Machine Learning Foundations (機器學習基石) - YouTube
(43) Linear Models for Classification :: Stochastic Grad. Descent @ Machine Learning Foundations (機器學習基石) - YouTube
(44) Linear Models for Classification :: Multiclass via Logistic @ Machine Learning Foundations (機器學習基石) - YouTube
(45) Linear Models for Classification :: Multiclass via Binary @ Machine Learning Foundations (機器學習基石) - YouTube
(46) Nonlinear Transformation :: Quadratic Hypotheses @ Machine Learning Foundations (機器學習基石) - YouTube
(47) Nonlinear Transformation :: Nonlinear Transform @ Machine Learning Foundations (機器學習基石) - YouTube
(48) Nonlinear Transformation :: Price of Nonlinear Transform @ Machine Learning Foundations (機器學習基石) - YouTube
(49) Nonlinear Transformation :: Structured Hypothesis Sets @ Machine Learning Foundations (機器學習基石) - YouTube
(50) Hazard of Overfitting :: What is Overfitting? @ Machine Learning Foundations (機器學習基石) - YouTube
(51) Hazard of Overfitting :: The Role of Noise and Data Size @ Machine Learning Foundations (機器學習基石) - YouTube
(52) Hazard of Overfitting :: Deterministic Noise @ Machine Learning Foundations (機器學習基石) - YouTube
(53) Hazard of Overfitting :: Dealing with Overfitting @ Machine Learning Foundations (機器學習基石) - YouTube
(54) Regularization :: Regularized Hypothesis Set @ Machine Learning Foundations (機器學習基石) - YouTube
(55) Regularization :: Weight Decay Regularization @ Machine Learning Foundations (機器學習基石) - YouTube
(56) Regularization :: Regularization and VC Theory @ Machine Learning Foundations (機器學習基石) - YouTube
(57) Regularization :: General Regularizers @ Machine Learning Foundations (機器學習基石) - YouTube
(58) Validation :: Model Selection Problem @ Machine Learning Foundations (機器學習基石) - YouTube
(59) Validation :: Validation @ Machine Learning Foundations (機器學習基石) - YouTube
(60) Validation :: Leave-One-Out Cross Validation @ Machine Learning Foundations (機器學習基石) - YouTube
(61) Validation :: V-Fold Cross Validation @ Machine Learning Foundations (機器學習基石) - YouTube
(62) Three Learning Principles :: Occam's Razor @ Machine Learning Foundations (機器學習基石) - YouTube
(63) Three Learning Principles :: Sampling Bias @ Machine Learning Foundations (機器學習基石) - YouTube
(64) Three Learning Principles :: Data Snooping @ Machine Learning Foundations (機器學習基石) - YouTube
(65) Three Learning Principles :: Power of Three @ Machine Learning Foundations (機器學習基石) - YouTube
-----
(01) Linear Support Vector Machine (SVM) :: Course Introduction @ Machine Learning Techniques (機器學習技法) - YouTube
(02) Linear SVM :: Large-Margin Separating Hyperplane @ Machine Learning Techniques (機器學習技法) - YouTube
(03) Linear SVM :: Standard Large-Margin Problem @ Machine Learning Techniques (機器學習技法) - YouTube
(04) Linear SVM :: Support Vector Machine @ Machine Learning Techniques (機器學習技法) - YouTube
(05) Linear SVM :: Reasons behind Large-Margin Hyperplane @ Machine Learning Techniques (機器學習技法) - YouTube
(06) Dual Support Vector Machine :: Motivation of Dual SVM @ Machine Learning Techniques (機器學習技法) - YouTube
(07) Dual Support Vector Machine :: Largange Dual SVM @ Machine Learning Techniques (機器學習技法) - YouTube
(08) Dual Support Vector Machine :: Solving Dual SVM @ Machine Learning Techniques (機器學習技法) - YouTube
(09) Dual Support Vector Machine :: Messages behind Dual SVM @ Machine Learning Techniques (機器學習技法) - YouTube
(10) Kernel Support Vector Machine :: Kernel Trick @ Machine Learning Techniques (機器學習技法) - YouTube
(11) Kernel Support Vector Machine :: Polynomial Kernel @ Machine Learning Techniques (機器學習技法) - YouTube
(12) Kernel Support Vector Machine :: Gaussian Kernel @ Machine Learning Techniques (機器學習技法) - YouTube
(13) Kernel Support Vector Machine :: Comparison of Kernels @ Machine Learning Techniques (機器學習技法) - YouTube
(14) Soft-Margin Support Vector Machine :: Motivation and Primal @ Machine Learning Techniques (機器學習技法) - YouTube
(15) Soft-Margin Support Vector Machine :: Dual Problem @ Machine Learning Techniques (機器學習技法) - YouTube
(16) Soft-Margin Support Vector Machine :: Messages @ Machine Learning Techniques (機器學習技法) - YouTube
(17) Soft-Margin Support Vector Machine :: Model Selection @ Machine Learning Techniques (機器學習技法) - YouTube
(18) Kernel Logistic Regression :: Soft-Margin SVM as Regularized @ Machine Learning Techniques (機器學習技法) - YouTube
(19) Kernel Logistic Regression :: SVM versus Logistic Regression @ Machine Learning Techniques (機器學習技法) - YouTube
(20) Kernel Logistic Regression :: SVM for Soft Binary @ Machine Learning Techniques (機器學習技法) - YouTube
(21) Kernel Logistic Regression :: Kernel Logistic Regression @ Machine Learning Techniques (機器學習技法) - YouTube
(22) Support Vector Regression :: Kernel Ridge Regression @ Machine Learning Techniques (機器學習技法) - YouTube
(23) Support Vector Regression :: Support Vector Regression Primal @ Machine Learning Techniques (機器學習技法) - YouTube
(24) Support Vector Regression :: Support Vector Regression Dual @ Machine Learning Techniques (機器學習技法) - YouTube
(25) Support Vector Regression :: Summary of Kernel Models @ Machine Learning Techniques (機器學習技法) - YouTube
(26) Blending and Bagging :: Motivation of Aggregation @ Machine Learning Techniques (機器學習技法) - YouTube
(27) Blending and Bagging :: Uniform Blending @ Machine Learning Techniques (機器學習技法) - YouTube
(28) Blending and Bagging :: Linear and Any Blending @ Machine Learning Techniques (機器學習技法) - YouTube
(29) Blending and Bagging :: Bagging (Bootstrap Aggregation) @ Machine Learning Techniques (機器學習技法) - YouTube
(30) Adaptive Boosting :: Motivation of Boosting @ Machine Learning Techniques (機器學習技法) - YouTube
(31) Adaptive Boosting :: Diversity by Re-weighting @ Machine Learning Techniques (機器學習技法) - YouTube
(32) Adaptive Boosting :: Adaptive Boosting Algorithm @ Machine Learning Techniques (機器學習技法) - YouTube
(33) Adaptive Boosting :: Adaptive Boosting in Action @ Machine Learning Techniques (機器學習技法) - YouTube
(34) Decision Tree :: Decision Tree Hypothesis @ Machine Learning Techniques (機器學習技法) - YouTube
(35) Decision Tree :: Decision Tree Algorithm @ Machine Learning Techniques (機器學習技法) - YouTube
(36) Decision Tree :: Decision Tree Heuristics in C&RT @ Machine Learning Techniques (機器學習技法) - YouTube
(37) Decision Tree :: Decision Tree in Action @ Machine Learning Techniques (機器學習技法) - YouTube
(38) Random Forest :: Random Forest Algorithm @ Machine Learning Techniques (機器學習技法) - YouTube
(39) Random Forest :: Out-of-bag Estimate @ Machine Learning Techniques (機器學習技法) - YouTube
(40) Random Forest :: Feature Selection @ Machine Learning Techniques (機器學習技法) - YouTube
(41) Random Forest :: Random Forest in Action @ Machine Learning Techniques (機器學習技法) - YouTube
(42) Gradient Boosted Decision Tree :: AdaBoost Decision Tree @ Machine Learning Techniques (機器學習技法) - YouTube
(43) Gradient Boosted Decision Tree :: Optimization of AdaBoost @ Machine Learning Techniques (機器學習技法) - YouTube
(44) Gradient Boosted Decision Tree :: Gradient Boosting @ Machine Learning Techniques (機器學習技法) - YouTube
(45) Gradient Boosted Decision Tree :: Summary of Aggregation @ Machine Learning Techniques (機器學習技法) - YouTube
(46) Neural Network :: Motivation @ Machine Learning Techniques (機器學習技法) - YouTube
(47) Neural Network :: Neural Network Hypothesis @ Machine Learning Techniques (機器學習技法) - YouTube
(48) Neural Network :: Neural Network Learning @ Machine Learning Techniques (機器學習技法) - YouTube
(49) Neural Network :: Optimization and Regularization @ Machine Learning Techniques (機器學習技法) - YouTube
(50) Deep Learning :: Deep Neural Network @ Machine Learning Techniques (機器學習技法) - YouTube
(51) Deep Learning ::Autoencoder @ Machine Learning Techniques (機器學習技法) - YouTube
(52) Deep Learning ::Denoising Autoencoder @ Machine Learning Techniques (機器學習技法) - YouTube
(53) Deep Learning :: Principal Component Analysis @ Machine Learning Techniques (機器學習技法) - YouTube
(54) Radial Basis Function Network :: RBF Network Hypothesis @ Machine Learning Techniques (機器學習技法) - YouTube
(55) Radial Basis Function Network :: RBF Network Learning @ Machine Learning Techniques (機器學習技法) - YouTube
(56) Radial Basis Function Network :: k-Means Algorithm @ Machine Learning Techniques (機器學習技法) - YouTube
(57) Radial Basis Function Network :: k-Means and RBFNet in Action @ Machine Learning Techniques (機器學習技法) - YouTube
(58) Matrix Factorization :: Linear Network Hypothesis @ Machine Learning Techniques (機器學習技法) - YouTube
(59) Matrix Factorization :: Basic Matrix Factorization @ Machine Learning Techniques (機器學習技法) - YouTube
(60) Matrix Factorization :: Stochastic Gradient Descent @ Machine Learning Techniques (機器學習技法) - YouTube
(61) Matrix Factorization :: Summary of Extraction Models @ Machine Learning Techniques (機器學習技法) - YouTube
(62) Finale :: Feature Exploitation Techniques @ Machine Learning Techniques (機器學習技法) - YouTube
(63) Finale :: Error Optimization Techniques @ Machine Learning Techniques (機器學習技法) - YouTube
(64) Finale :: Overfitting Elimination Techniques @ Machine Learning Techniques (機器學習技法) - YouTube
(65) Finale :: Machine Learning in Practice @ Machine Learning Techniques (機器學習技法) - YouTube
-----
References
[1] (79) Machine Learning Foundations (機器學習基石) - YouTube
https://www.youtube.com/playlist?list=PLXVfgk9fNX2I7tB6oIINGBmW50rrmFTqf
[2] (79) Machine Learning Techniques (機器學習技法) - YouTube
https://www.youtube.com/playlist?list=PLXVfgk9fNX2IQOYPmqjqWsNUFl2kpk1U2
[3] Hsuan-Tien Lin > MOOCs
https://www.csie.ntu.edu.tw/~htlin/mooc/
-----
[4] (14) 16. Learning: Support Vector Machines - YouTube
https://www.youtube.com/watch?v=_PwhiWxHK8o&t=6s
[5] Deep Learning Book
https://www.deeplearningbook.org/
[6] History DL。
Alom, Md Zahangir, et al. "The history began from alexnet: A comprehensive survey on deep learning approaches." arXiv preprint arXiv:1803.01164 (2018).
https://arxiv.org/ftp/arxiv/papers/1803/1803.01164.pdf
-----
延伸閱讀：
[7] The Star Also Rises: 深度學習論文研討（二）：機器學習（二）
https://hemingwang.blogspot.com/2020/12/problem.html
[8] The Star Also Rises: 深度學習論文研討（三）：深度學習（一）
https://hemingwang.blogspot.com/2020/11/hung-yi-lee.html
-----
No comments:
Post a Comment