Alex Towell

Alex Towell has a masters in computer science and a masters in mathematics (statistics) from SIUe.

Infini-gram: LLM-scale *-gram models

Recently, I watched a presentation on infini-grams, which utilize a suffix array to avoid precomputing

n

-grams and allow for arbitrary context lengths, up to a suffix that is found in the training data.

May 20, 2024 1 min read

Gave a SLUUG talk "Demystifying Large Language Models (LLMs) on Linux: From Theory to Application"

I gave a presentation for the St. Louis Unix Users Group (SLUUG) about Large Language Models (LLMs) on Linux. The talk was titled “Demystifying Large Language Models (LLMs) on Linux: From Theory to Application”.

Last updated on Feb 23, 2024 1 min read

Approximations of Solomonoff Induction

I experiment with simple predictive / generative models to approximate Solomonoff induction for a relatiely simple synthetic data-generating process.

Feb 19, 2024 Statistics, Machine Learning, Inference

Fine-Tuning Tiny LLMs for ElasticSearch DSL

I am creating a tiny LLM for ElasticSearch DSL as a proof of concept.

Feb 19, 2024 Statistics, Machine Learning, Inference

Master's project: Reliability Estimation in Series Systems

I presented my master’s project in October 2023. It was titled ‘Reliability Estimation in Series Systems: Maximum Likelihood Techniques for Right-Censored and Masked Failure Data’.

Feb 19, 2024 1 min read Series-Systems, Masked-Failure-Data, Censoring, Reliability-Analysis, Bootstrap, Mle, Likelihood-Model, Weibull

Master's project: Reliability Estimation in Series Systems

Various LLM Research Projects

Various research projects for LLMs and foundation models.

Feb 19, 2024 Statistics, Machine Learning, Inference

Entropy Maps

The basic theory behind an entropy map is to map values in the domain to values in the codomain by hashing to a prefix-free code in the …

Feb 18, 2024 11 min read Entropy Map, Rate-Distortion, Bernoulli-Map, Probabilistic Data Structure

Entropy Maps

The Bernoulli Model: A Closer Look at the Boolean Bernoulli Model

This blog post introduces the Bernoulli Model, a framework for understanding probabilistic data structures and incorporating uncertainty into data types, particularly Boolean values. It highlights the model’s utility in optimizing space and accuracy in data representation.

Jun 17, 2023 19 min read Boolean, Bernoulli, Probabilistic Data Structure, Bloom-Filter

The Bernoulli Model: A Closer Look at the Boolean Bernoulli Model

Uses and limits of abstractions

I’m been thinking about the power and limitations of abstractions in our understanding of the world. This blog post is from a chat I had with a ChatGPT, which can be found here and here.

Jun 17, 2023 9 min read Philosophy, Machine Learning

Uses and limits of abstractions

Working memory as an inductive bias

This blog post is from a chat I had with a ChatGPT, which can be found here and here. I’m not sure if this is a good blog post, but I’m posting it anyway.

Jun 17, 2023 4 min read Machine Learning

Working memory as an inductive bias