Transformer（二）：Overview

Transformer（二）：Overview

2020/12/27

-----

https://pixabay.com/zh/photos/caterpillar-branch-larva-1209834/

-----

◎ Abstract

-----

◎ Introduction

-----

本論文要解決（它之前研究）的（哪些）問題（弱點）？ 

-----

# GNMT。

-----

# ConvS2S。

-----

◎ Method

-----

解決方法？ 

-----

# Transformer。

-----

具體細節？

-----

◎ Result

-----

本論文成果。 

-----

◎ Discussion

-----

本論文與其他論文（成果或方法）的比較。 

-----

成果比較。 

-----

方法比較。 

-----

◎ Conclusion 

-----

◎ Future Work

-----

後續相關領域的研究。 

-----

後續延伸領域的研究。

-----

◎ References

-----

# ULMFiT。被引用 1339 次。

Howard, Jeremy, and Sebastian Ruder. "Universal language model fine-tuning for text classification." arXiv preprint arXiv:1801.06146 (2018).

https://arxiv.org/pdf/1801.06146.pdf

# Transformer。被引用 13554 次。

Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems. 2017.

https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

# BERT。被引用 12556 次。

Devlin, Jacob, et al. "Bert: Pre-training of deep bidirectional transformers for language understanding." arXiv preprint arXiv:1810.04805 (2018).

https://arxiv.org/pdf/1810.04805.pdf

-----

# GNMT。被引用 3391 次。

Wu, Yonghui, et al. "Google's neural machine translation system: Bridging the gap between human and machine translation." arXiv preprint arXiv:1609.08144 (2016).

https://arxiv.org/pdf/1609.08144.pdf

# ConvS2S。被引用 1772 次。

Gehring, Jonas, et al. "Convolutional sequence to sequence learning." arXiv preprint arXiv:1705.03122 (2017).

https://arxiv.org/pdf/1705.03122.pdf

-----

◎ 參考文章

The Star Also Rises: NLP（五）：Transformer

http://hemingwang.blogspot.com/2019/01/transformer.html

-----