Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer

Shazeer, Mirhoseini, Maziarz, Davis, Le, Hinton, Dean

paper completed ai-ml

Year 2017

External Link https://arxiv.org/pdf/1701.06538

mixture of experts sparse conditional computation from:language-models

Notes

Mixture of Experts with learned gating. Conditional computation at scale.

View Resource All Media More in Ai-Ml