Monday, December 28, 2020

Template

Template

2020/12/04

-----


https://pixabay.com/zh/photos/silk-screen-silk-screening-art-1246169/

-----

◎ Abstract

-----

◎ Introduction

-----

本論文要解決(它之前研究)的(哪些)問題(弱點)? 

-----

◎ Method

-----

解決方法? 

-----

具體細節?

-----

◎ Result

-----

本論文成果。 

-----

◎ Discussion

-----

本論文與其他論文(成果或方法)的比較。 

-----

成果比較。 

-----

方法比較。 

-----

◎ Conclusion 

-----

◎ Future Work

-----

後續相關領域的研究。 

-----

後續延伸領域的研究。

-----

◎ References

-----

# HDR。被引用 3589 次。針對數字的手寫辨識,較早的神經網路架構,無全連接層。

LeCun, Yann, et al. "Handwritten digit recognition with a back-propagation network." Advances in neural information processing systems 2 (1989): 396-404.

https://papers.nips.cc/paper/1989/file/53c3bce66e43be4f209556518c2fcb54-Paper.pdf


# LeNet。被引用 31707 次。經典的卷積神經網路,主要比 HDR 多了全連接層。

LeCun, Yann, et al. "Gradient-based learning applied to document recognition." Proceedings of the IEEE 86.11 (1998): 2278-2324.

http://yann.lecun.com/exdb/publis/pdf/lecun-01a.pdf


# AlexNet。被引用 74398 次。較早使用 GPU 的大型卷積神經網路之一,效能比之前有飛躍的提升,成功使用 dropout 避免 overfitting。

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Communications of the ACM 60.6 (2017): 84-90.

https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdf

-----

Thursday, December 17, 2020

What's the main points of Transformer?

What's the main points of Transformer?

2020/10/27

-----


-----

-----

一、基礎學習與論文理解(70%)。


◎ 1. 可以從這篇論文學到什麼(解決什麼問題)? 

◎ A. 問題原因。 

◎ 1.a.1:過往這個領域已經做到甚麼程度?

-----


# ConvS2S 論文。

-----

◎ 1.a.2:最近的研究遇到了甚麼瓶頸?

◎ 1.a.3:議題發生的根本原因是甚麼?


◎ B. 解決方法。 

◎ 1.b.1:作者採用甚麼方式解決?

◎ 1.b.2:細節內容是如何解決的?

◎ 1.b.3:(optional)- 作者是否有說明思路? - 或是後續研究者的討論?


◎ C. 效能評估。 

◎ 1.c.1:成果效能的比較。

◎ 1.c.2:目前這個方法是否還有限制,是甚麼?

◎ 1.c.3:(optional)- 作者對後續發展的期許? - 其他研究者後續的發展?


二、後續發展與延伸應用(30%)


◎ 2. 可以應用在那些垂直領域(應用領域)? 

◎ 3. 這篇論文的價值在哪(如何跨領域延伸應用)? 

◎ 4. 如果要改進可以怎麼做(後續的研究)?

-----

-----

References

◎ 相關論文

◎ 延伸論文

十、Transformer - 英宗 Transformer Transformer Lab GPT-1、(GPT-2、GPT-3) BERT

◎ 參考文章

The Star Also Rises: NLP(五):Transformer

http://hemingwang.blogspot.com/2019/01/transformer.html

-----

[翻譯] The Illustrated Transformer

[翻譯] The Illustrated Transformer

2019/10/02

----- 

-----

References

-----

The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time
https://jalammar.github.io/illustrated-transformer/

Monday, December 07, 2020

What's the main points of ConvS2S?

What's the main points of ConvS2S?

2020/10/27

-----



-----

一、Introduction

本論文要解決(它之前研究)的哪些問題(弱點)?

-----


# GNMT

-----


# GNMT

-----


# PreConvS2S

-----

二、Method

-----

三、Result

-----

四、Discussion

-----

五、Conclusion and Future Work

-----

Conclusion

-----

Future Work

-----

References

◎ 主要論文

[1] LSTM。被引用 39743 次。

Hochreiter, Sepp, and Jürgen Schmidhuber. "Long short-term memory." Neural computation 9.8 (1997): 1735-1780.

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.676.4320&rep=rep1&type=pdf


[2] Seq2seq。被引用 12676 次。

Sutskever, Ilya, Oriol Vinyals, and Quoc V. Le. "Sequence to sequence learning with neural networks." Advances in neural information processing systems. 2014.

http://papers.nips.cc/paper/5346-sequence-to-sequence-learning-with-neural-networks.pdf


[3] Attention 1。被引用 14895 次。

Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. "Neural machine translation by jointly learning to align and translate." arXiv preprint arXiv:1409.0473 (2014).

https://arxiv.org/pdf/1409.0473.pdf


[4] ConvS2S。被引用 1772 次。

Gehring, Jonas, et al. "Convolutional sequence to sequence learning." arXiv preprint arXiv:1705.03122 (2017).

https://arxiv.org/pdf/1705.03122.pdf


[5] Transformer。被引用 13554 次。

Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems. 2017.

https://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

-----

◎ 相關論文

-----

[6] GNMT。被引用 3391 次。

Wu, Yonghui, et al. "Google's neural machine translation system: Bridging the gap between human and machine translation." arXiv preprint arXiv:1609.08144 (2016).

https://arxiv.org/pdf/1609.08144.pdf


[7] PreConvS2S。被引用 273 次。

Gehring, Jonas, et al. "A convolutional encoder model for neural machine translation." arXiv preprint arXiv:1611.02344 (2016).

https://arxiv.org/pdf/1611.02344.pdf

-----

◎ 參考文章

The Star Also Rises: NLP(四):ConvS2S

https://hemingwang.blogspot.com/2019/04/convs2s.html

-----

[翻譯] Understanding incremental decoding in fairseq

[翻譯] Understanding incremental decoding in fairseq

2019/10/02


-----




-----

References

# 綜述
Understanding incremental decoding in fairseq – Telesens
http://www.telesens.co/2019/04/21/understanding-incremental-decoding-in-fairseq/

-----