2019/01/18
-----
https://zhuanlan.zhihu.com/p/37601161
# Seq2seq
-----
https://zhuanlan.zhihu.com/p/37601161
# Attention
-----
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
# Global Attention and Local Attention。
-----
https://zhuanlan.zhihu.com/p/37601161
# Attention
-----
Fig. 2. An illustration of the attention mechanism (RNNSearch) proposed by [Bahdanau, 2014]. Instead of converting the entire input sequence into a single context vector, we create a separate context vector for each output (target) word. These vectors consist of the weighted sums of encoder’s hidden states.
Fig. 2. An illustration of the attention mechanism (RNNSearch) proposed by [Bahdanau, 2014]. Instead of converting the entire input sequence into a single context vector, we create a separate context vector for each output (target) word. These vectors consist of the weighted sums of encoder’s hidden states.
# Global Attention。[1]。
-----
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
# Global Attention and Local Attention。
-----
-----
-----
References
◎ 論文
[1] Attention - using GRU
[2] Global Attention - using LSTM
[3] Visual Attention
-----
-----
◎ 英文參考資料
# 綜述
[1] Attention Attention!
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
# 綜述
# 680 claps
[2] Attention in NLP – Kate Loginova – Medium
https://medium.com/@joealato/attention-in-nlp-734c6fa9d983
# 1.4K claps
Attn Illustrated Attention - Towards Data Science
https://towardsdatascience.com/attn-illustrated-attention-5ec4ad276ee3
# 1.3K claps
A Brief Overview of Attention Mechanism - SyncedReview - Medium
https://medium.com/syncedreview/a-brief-overview-of-attention-mechanism-13c578ba9129
# 799 claps
Intuitive Understanding of Attention Mechanism in Deep Learning
https://towardsdatascience.com/intuitive-understanding-of-attention-mechanism-in-deep-learning-6c9482aecf4f
# 126 claps
Understanding Attention Mechanism - Shashank Yadav - Medium
https://medium.com/@shashank7.iitd/understanding-attention-mechanism-35ff53fc328e
Attention and Memory in Deep Learning and NLP – WildML
http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/
-----
◎ 日文參考資料
深層学習による自然言語処理 - RNN, LSTM, ニューラル機械翻訳の理論 - ディープラーニングブログ
http://deeplearning.hatenablog.com/entry/neural_machine_translation_theory
-----
◎ 簡體中文參考資料
# 綜述
Attention-Mechanisms-paper_Attention-mechanisms-paper.md at master · yuquanle_Attention-Mechanisms-paper · GitHub
https://github.com/yuquanle/Attention-Mechanisms-paper/blob/master/Attention-mechanisms-paper.md
深度学习中的注意力模型(2017版) - 知乎
https://zhuanlan.zhihu.com/p/37601161
自然语言处理中的Attention Model:是什么及为什么 - 张俊林的博客 - CSDN博客
https://blog.csdn.net/malefactor/article/details/50550211
目前主流的attention方法都有哪些? - 知乎
https://www.zhihu.com/question/68482809/answer/264632289
# 110 claps
自然语言处理中注意力机制综述 - 知乎
https://zhuanlan.zhihu.com/p/54491016
# 14 claps
NLP硬核入门-Seq2Seq和Attention机制 - 知乎
https://zhuanlan.zhihu.com/p/73589030
注意力机制(Attention Mechanism)在自然语言处理中的应用 - Soul Joy Hub - CSDN博客
https://blog.csdn.net/u011239443/article/details/80418489
【NLP】Attention Model(注意力模型)学习总结 - 郭耀华 - 博客园
https://www.cnblogs.com/guoyaohua/p/9429924.html
-----
◎ 繁體中文參考資料
# 486 claps
[1] Seq2seq pay Attention to Self Attention Part 1(中文版)
https://medium.com/@bgg/seq2seq-pay-attention-to-self-attention-part-1-%E4%B8%AD%E6%96%87%E7%89%88-2714bbd92727
-----
◎ 代碼實作
Translation with a Sequence to Sequence Network and Attention — PyTorch Tutorials 1.0.0.dev20181228 documentation
https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html
PyTorch 1.1 Tutorials テキスト Sequence to Sequence ネットワークと Attention で翻訳 – PyTorch
http://torch.classcat.com/2019/07/20/pytorch-1-1-tutorials-text-seq2seq-translation/
# 832 claps
Attention in Deep Networks with Keras - Towards Data Science
https://towardsdatascience.com/light-on-math-ml-attention-with-keras-dc8dbc1fad39
References
◎ 論文
[1] Attention - using GRU
Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473 (2014).
https://arxiv.org/pdf/1409.0473.pdf[2] Global Attention - using LSTM
Luong, Minh-Thang, Hieu Pham, and Christopher D. Manning. "Effective approaches to attention-based neural machine translation." arXiv preprint arXiv:1508.04025 (2015).
https://arxiv.org/pdf/1508.04025.pdf [3] Visual Attention
Xu, Kelvin, et al. "Show, attend and tell: Neural image caption generation with visual attention." International conference on machine learning. 2015.
http://proceedings.mlr.press/v37/xuc15.pdf -----
-----
◎ 英文參考資料
# 綜述
[1] Attention Attention!
https://lilianweng.github.io/lil-log/2018/06/24/attention-attention.html
# 綜述
# 680 claps
[2] Attention in NLP – Kate Loginova – Medium
https://medium.com/@joealato/attention-in-nlp-734c6fa9d983
# 1.4K claps
Attn Illustrated Attention - Towards Data Science
https://towardsdatascience.com/attn-illustrated-attention-5ec4ad276ee3
# 1.3K claps
A Brief Overview of Attention Mechanism - SyncedReview - Medium
https://medium.com/syncedreview/a-brief-overview-of-attention-mechanism-13c578ba9129
# 799 claps
Intuitive Understanding of Attention Mechanism in Deep Learning
https://towardsdatascience.com/intuitive-understanding-of-attention-mechanism-in-deep-learning-6c9482aecf4f
# 126 claps
Understanding Attention Mechanism - Shashank Yadav - Medium
https://medium.com/@shashank7.iitd/understanding-attention-mechanism-35ff53fc328e
Attention and Memory in Deep Learning and NLP – WildML
http://www.wildml.com/2016/01/attention-and-memory-in-deep-learning-and-nlp/
-----
◎ 日文參考資料
深層学習による自然言語処理 - RNN, LSTM, ニューラル機械翻訳の理論 - ディープラーニングブログ
http://deeplearning.hatenablog.com/entry/neural_machine_translation_theory
-----
◎ 簡體中文參考資料
# 綜述
Attention-Mechanisms-paper_Attention-mechanisms-paper.md at master · yuquanle_Attention-Mechanisms-paper · GitHub
https://github.com/yuquanle/Attention-Mechanisms-paper/blob/master/Attention-mechanisms-paper.md
深度学习中的注意力模型(2017版) - 知乎
https://zhuanlan.zhihu.com/p/37601161
自然语言处理中的Attention Model:是什么及为什么 - 张俊林的博客 - CSDN博客
https://blog.csdn.net/malefactor/article/details/50550211
目前主流的attention方法都有哪些? - 知乎
https://www.zhihu.com/question/68482809/answer/264632289
# 110 claps
自然语言处理中注意力机制综述 - 知乎
https://zhuanlan.zhihu.com/p/54491016
# 14 claps
NLP硬核入门-Seq2Seq和Attention机制 - 知乎
https://zhuanlan.zhihu.com/p/73589030
注意力机制(Attention Mechanism)在自然语言处理中的应用 - Soul Joy Hub - CSDN博客
https://blog.csdn.net/u011239443/article/details/80418489
【NLP】Attention Model(注意力模型)学习总结 - 郭耀华 - 博客园
https://www.cnblogs.com/guoyaohua/p/9429924.html
-----
◎ 繁體中文參考資料
# 486 claps
[1] Seq2seq pay Attention to Self Attention Part 1(中文版)
https://medium.com/@bgg/seq2seq-pay-attention-to-self-attention-part-1-%E4%B8%AD%E6%96%87%E7%89%88-2714bbd92727
-----
◎ 代碼實作
Translation with a Sequence to Sequence Network and Attention — PyTorch Tutorials 1.0.0.dev20181228 documentation
https://pytorch.org/tutorials/intermediate/seq2seq_translation_tutorial.html
PyTorch 1.1 Tutorials テキスト Sequence to Sequence ネットワークと Attention で翻訳 – PyTorch
http://torch.classcat.com/2019/07/20/pytorch-1-1-tutorials-text-seq2seq-translation/
# 832 claps
Attention in Deep Networks with Keras - Towards Data Science
https://towardsdatascience.com/light-on-math-ml-attention-with-keras-dc8dbc1fad39
No comments:
Post a Comment
Note: Only a member of this blog may post a comment.