Back to Media

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Devlin, Chang, Lee, Toutanova
paper completed ai-ml
Year 2018
External Link https://arxiv.org/pdf/1810.04805
pre-training masked language model NLU from:language-models

Notes

Bidirectional pre-training via masked language modeling. Defined the pre-train/fine-tune paradigm.

View Resource All Media More in Ai-Ml

More in Ai-Ml

An Introduction to Statistical Learning (ISLR2)
James, Witten, Hastie, Tibshirani · book
Artificial Intelligence: A Modern Approach
Russell & Norvig · book
Attention Is All You Need
Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser, Polosukhin · paper
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models
Wei, Wang, Schuurmans, Bosma, Xia, Chi, Le, Zhou · paper