AI 從頭學（一四）：Recommender

AI 從頭學（一四）：Recommender

2017/03/16

前言：

如何在 Azure 上建立 AI 的個人化新聞推薦系統？本文列出部分參考資料。

底下純屬紙上談兵，若有不足之處，還請不吝指教！

-----

Fig. Recommender（圖片來源：Pixabay）。

-----

Summary：

推薦系統 [1] 可用於個人化新聞推薦 [2]-[6]，或是個人化財經新聞推薦 [7], [8]。概念上是在 Hadoop [9]-[30] 上跑 Mahout [31]-[34]。也可以使用記憶體作為暫存的 Spark [35]-[45]，在速度上有驚人的提升。實作上則是先用 Sqoop [46] 把 SQL [47] 上的資料搬到 HDFS [48] 上。

如果要在 Azure [49]-[62] 上做，一樣要先把 Azure SQL [63], [64] 的資料先移到 Azure 的 Hadoop，也就是 HDInsight [65]-[67] 上，再跑 Azure 的 Machine Learning Service [68]-[70]。

如要自行將演算法 [2]-[8] 開發成系統，微軟推薦 F# [71]。F# [72]-[84] 屬於 Functional Programming 的程式語言，有簡潔的程式碼、較高的產能等種種好處。

Deep Learning [85]-[91] 也可用來開發推薦系統 [92]-[97]，今日頭條是此中翹楚；目前總員工 2,500 名，工程師有 1,500 個，其中 800 名工程師聚焦在演算法、資料分析 [98]。

-----

出版說明：

Collaborative Filtering - RDD-based API - Spark 2.4.4 Documentation
https://spark.apache.org/docs/latest/mllib-collaborative-filtering.html

之前不熟 Big Data 時，幫公司規劃推薦系統，洋洋灑灑找了近百篇資料。其實現在熟了，只要裝好 Spark 再跑一下預設的演算法就可以了：「Collaborative filtering is commonly used for recommender systems.」。較新的作法是使用深度學習，資料也不難找。

-----

References

◎ 01_Recommender

[1] 2015_Recommender Systems Handbook

-----

◎ 02_News

[2] 2013_Personalized News Recommendation Using Ontologies Harvested from the Web

[3] 2013_Mobile Recommender Systems and Their Applications

[4] 2011_News personalization using the CF-IDF semantic recommender

[5] 2010_Personalized news recommendation based on click behavior

[6] 2010_A contextual-bandit approach to personalized news article recommendation

-----

◎ 03_Financial

[7] 2015_Personalized Financial News Recommendation Algorithm Based on Ontology

[8] 2000_Language models for financial news recommendation

-----

◎ 04_Hadoop

[9] 2015_Learning Hadoop 2

[10] 2015_Hadoop,The definitive guide

[11] 2015_Hadoop Essentials

[12] 2015_Hadoop Backup and Recovery Solutions

[13] 2015_Guide to high performance distributed computing, case studies with Hadoop, Scalding and Spark

[14] 2015_Field Guide to Hadoop

[15] 2015_Big data made easy, a working guide to the complete Hadoop toolset

[16] 2015_Big Data Governance, Modern Data Management Principles for Hadoop, NoSQL & Big Data Analytics

[17] 2015_Big Data Forensics, Learning Hadoop Investigations

[18] 2014_Pro Apache Hadoop

[19] 2014_Practical Hadoop security

[20] 2014_Hadoop For Dummies

[21] 2013_Securing Hadoop

[22] 2013_Professional Hadoop Solutions

[23] 2013_Hadoop Real-World Solutions Cookbook

[24] 2013_Hadoop Operations and Cluster Management Cookbook

[25] 2013_Hadoop Cluster Deployment

[26] 2013_Hadoop Beginner's Guide

[27] 2012_Hadoop Operations

[28] 2012_Hadoop in Practice

[29] 2010_Hadoop in Action

[30] 2009_Pro Hadoop

-----

◎ 05_Mahout

[31] 2015_Learning Apache Mahout

[32] 2015_Learning Apache Mahout Classification

[33] 2013_Apache Mahout Cookbook

[34] 2011_Mahout in Action

-----

◎ 06_Spark

[35] 2016_Spark Guide

[36] 2016_Pro Spark Streaming

[37] 2016_High Performance Spark

[38] 2015_Spark for Python Developers

[39] 2015_Spark Core Programming

[40] 2015_Mastering Apache Spark

[41] 2015_Learning Spark

[42] 2015_Getting Started with Apache Spark

[43] 2015_Fast Data Processing with Spark

[44] 2015_Big Data Analytics with Spark

[45] 2015_Advanced Analytics with Spark

-----

◎ 07_Sqoop

[46] 2013_Apache Sqoop Cookbook

-----

◎ 08_SQL

[47] 2013_Microsoft SQL Server 2012 with Hadoop

-----

◎ 09_HDFS

[48] 2010_The Hadoop Distributed File System

-----

◎ 10_Azure

[49] 2015_Microsoft Azure, planning, deploying, and managing your data center in the cloud

[50] 2015_Microsoft Azure Essentials, Fundamentals of Azure

[51] 2015_Microservices, IoT, and Azure, leveraging DevOps and microservice architecture to deliver SaaS solutions

[52] 2015_Hardening Azure Applications

[53] 2015_15 minute Azure Installation, Set up the Microsoft Cloud Server by the Numbers

[54] 2014_Zen of Cloud, Learning Cloud Computing by Examples on Microsoft Azure

[55] 2014_Mastering Hyper-V 2012 R2 With System Center and Windows Azure

[56] 2014_Learning Windows Azure Mobile Services for Windows 8 and Windows Phone 8

[57] 2012_Programming Microsoft's Clouds, Windows Azure and Office 365

[58] 2012_Cloud Architecture Patterns Using Microsoft Azure

[59] 2011_Windows Azure platform

[60] 2011_Azure in Action

[61] 2009_Windows Azure platform

[62] 2009_Introduction to Windows Azure, an introduction to cloud computing using Microsoft Windows Azure

-----

◎ 11_SQL

[63] 2012_Pro SQL Database for Windows Azure, SQL Server in the Cloud

[64] 2010_Pro SQL Azure

-----

◎ 12_HDInsight

[65] 2014_Pro Microsoft HDInsight, Hadoop on Windows

[66] 2014_Microsoft Big Data Solutions

[67] 2013_HDInsight Essentials

-----

◎ 13_ML

[68] 2015_Predictive analytics with Microsoft azure machine learning, build and deploy actionable solutions in minutes

[69] 2015_Data Science in the Cloud with Microsoft Azure Machine Learning and R

[70] 2014_Predictive analytics with Microsoft azure machine learning, build and deploy actionable solutions in minutes

-----

◎ 14_ML_F#

[71] 2015_Machine Learning Projects for _NET Developers

-----

◎ 15_F#

[72] 2016_Beginning F# 4_0

[73] 2015_Learning F# Functional Data Structures and Algorithms

[74] 2015_Expert F# 4_0

[75] 2014_The Book of F#, Breaking Free with Managed Functional Programming

[76] 2013_Windows Phone 7_5 Application Development with F#

[77] 2013_Functional Programming Using F#

[78] 2013_F# for Quantitative Finance

[79] 2013_F# for C# Developers

[80] 2012_Programming F# 3_0

[81] 2012_Expert F# 3_0

[82] 2010_Visual Studio 2010 and _NET 4 Six-in-One

[83] 2010_Expert F# 2_0

[84] 2007_Expert F#

-----

◎ 16_DL_Overview

[85] 2016_Towards Bayesian Deep Learning, A Survey

[86] 2016_Deep Learning on FPGAs, Past, Present, and Future

[87] 2015_Deep learning

[88] 2015_Deep learning in neural networks, An overview

[89] 2014_Deep Learning, Methods and Applications

[90] 2012_Unsupervised feature learning and deep learning, A review and new perspectives

[91] 2009_Learning deep architectures for AI

-----

◎ 17_DL_RS

[92] 2016_Deep neural networks for youtube recommendations

[93] 2016_Collaborative denoising auto-encoders for top-n recommender systems

[94] 2015_Deep collaborative filtering via marginalized denoising auto-encoder

[95] 2015_Collaborative deep learning for recommender systems

[96] 2014_Improving content-based and hybrid music recommendation using deep learning

[97] 2013_Deep content-based music recommendation

[98] 4 年拿下 7,400 萬日活躍用戶，今日頭條已經準備跨入全球市場
http://technews.tw/2016/12/08/toutiao-china-global-market/