n-gram models

This paper explores inductive biases and projection functions to enhance out-of-distribution generalization in autoregressive models. With the theory in place, we aim to implement these ideas in Python and develop tests and evaluations to validate our approach.

I experiment with simple predictive / generative models to approximate Solomonoff induction for a relatiely simple synthetic data-generating process.