Masked Language Model

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

Notes

Bidirectional pre-training via masked language modeling. Defined the pre-train/fine-tune paradigm.