ResNet-S

ResNet-S

2019/11/11

前言：

ResNet-S 是論文《Skip connections eliminate singularities》的簡稱，取其 singularity 的涵義。論文作者發現，ResNet 雖然能部分減緩梯度消失的問題，但真正解決的是網路的退化問題，強制加入的恆等映射，可以讓網路避免落入退化的困境。

-----

# ResNet-S

-----

References

# ResNet-S

Orhan, A. Emin, and Xaq Pitkow. "Skip connections eliminate singularities." arXiv preprint arXiv:1701.09175 (2017).

https://arxiv.org/pdf/1701.09175.pdf 

// Degeneracy

Shang, Wenling, et al. "Understanding and improving convolutional neural networks via concatenated rectified linear units." international conference on machine learning. 2016.

http://proceedings.mlr.press/v48/shang16.pdf

// Degeneracy

Greff, Klaus, Rupesh K. Srivastava, and Jürgen Schmidhuber. "Highway and residual networks learn unrolled iterative estimation." arXiv preprint arXiv:1612.07771 (2016).

https://arxiv.org/pdf/1612.07771.pdf

// Degeneracy

Jastrzębski, Stanisław, et al. "Residual connections encourage iterative inference." arXiv preprint arXiv:1710.04773 (2017).

https://arxiv.org/pdf/1710.04773.pdf

[2017] Skip Connections Eliminate Singularities · Issue #12 · yoheikikuta_paper-reading · GitHub
https://github.com/yoheikikuta/paper-reading/issues/12

【模型解读】resnet中的残差连接，你确定真的看懂了？ - 知乎
https://zhuanlan.zhihu.com/p/42833949