In-Context-Learning

Attention Weight Is Not Information Flow

The trained pointer model reads exactly the right memory cell, provably. Its attention barely shows where. The gap, and the causal probe that closes it.

Language Models are Few-Shot Learners (GPT-3)

Notes

175B parameters. In-context learning emerges at scale. Changed the field.