Training Compute-Optimal Large Language Models (Chinchilla)

Hoffmann, Borgeaud, Mensch, Buchatskaya, Cai, Rutherford, et al.

paper completed ai-ml

Year 2022

External Link https://arxiv.org/pdf/2203.15556

scaling laws compute-optimal data efficiency from:language-models

Notes

Showed most LLMs were undertrained. Optimal ratio of data to parameters.

View Resource All Media More in Ai-Ml