아티히어로

https://arxiv.org/abs/2003.07845v1 Rethinking Batch Normalization in Transformers The standard normalization method for neural network (NN) models used in Natural Language Processing (NLP) is layer normalization (LN). This is different than batch normalization (BN), which is widely-adopted in Computer Vision. The preferred use of LN in arxiv.org abstract NLP에서 사용되는 Neural network 모델의 표준 정규화 방법은 ..

https://arxiv.org/abs/2002.12327v1 A Primer in BERTology: What we know about how BERT works Transformer-based models are now widely used in NLP, but we still do not understand a lot about their inner workings. This paper describes what is known to date about the famous BERT model (Devlin et al. 2019), synthesizing over 40 analysis studies. We als arxiv.org abstract Transformer-based models 모델은 N..