This paper explores inductive biases and projection functions to enhance out-of-distribution generalization in autoregressive models. With the theory in place, we aim to implement these ideas in Python and develop tests and evaluations to validate our approach.
