Browse posts by tag
The sequential-prediction series is now an eight-episode animated playlist: what-comes-next as the foundation of the language-model era, traced from Solomonoff induction to the transformer.
Validating Context Tree Weighting through experiments, including a bug that changed everything.
The evolution of neural sequence prediction, and how it connects to classical methods
The bias-data trade-off in sequential prediction: when to use CTW, n-grams, or neural language models.
The classical approach to sequence prediction: counting and smoothing
Markov processes and tree sources: understanding where sequences come from
Model averaging over hypotheses, the principled way to handle uncertainty in prediction
The optimal predictor is incomputable. What we can learn from it anyway.
The problem of predicting what comes next, from compression to language models