June 9, 2026
What a Convolution Assumes
A convolution is a bet about images: nearby pixels matter together, and a feature detector should fire anywhere. How to test whether a model is actually using that bet.
News
Showing 222 of 222 posts
June 9, 2026
The Loss Function Is a Distribution Assumption
Choosing a loss is choosing a distribution for your output. Why every supervised network is a maximum-likelihood estimator, and why the gradient is always the residual.
June 9, 2026
Reinforcement Learning Is Cross-Entropy, Reweighted by Reward
When the only signal is a number at the end of a trajectory. How REINFORCE turns out to be the same gradient as classification, scaled by return, and where the theory tops out.
June 9, 2026
Recurrence Is Weight Sharing Across Time, and It Costs You
A recurrent network reuses one cell at every timestep and carries a state. That buys time-translation equivariance and unbounded reach in principle, and bills you in vanishing gradients.
June 9, 2026
Bengio's Language Model: the Markov Assumption Made Architectural
The simplest neural language model: embed the last N tokens, concatenate, predict the next. What you give up with a fixed window, and what you gain by dropping recurrence.
June 9, 2026
Attention Weight Is Not Information Flow
The trained pointer model reads exactly the right memory cell, provably. Its attention barely shows where. The gap, and the causal probe that closes it.
June 9, 2026
Attention Is a Learned Pointer Dereference
An attention head is a learned content-addressable lookup: a query matches keys, retrieves a value, exactly like dereferencing a pointer. Depth is how many lookups you can compose.
June 9, 2026
A Network Computes Numbers. The Loss Decides What They Mean.
Function approximation, why one linear unit cannot learn XOR, and what a hidden layer actually buys. The opening of a from-scratch tour of inductive bias.
June 5, 2026
The Expensive Half of a SAT Proof
A from-first-principles tour of double-lex symmetry breaking for SAT: why the UNSAT side is expensive, how canonical-representative constraints fix it, a worked Zarankiewicz example that settled three new OEIS values, and an honest map of where the …
May 15, 2026
Synthesis: Codecs as Structure
The series closes by restating the codes-as-priors thesis across all twelve instances and connecting the wire-format side to the Stepanov type-algebra side.
April 26, 2026
I Tried to Voice-Clone Myself in 90 MB
A tiny LLM in the browser, mixed at sample time with a token-level n-gram trained on every word I have published. Result is mediocre. Architecture is interesting. Notes on what worked, what didn't, and what would make it work.
April 24, 2026
Your Blog Will Outlive Your Database (It Doesn't Have To)
Your reader's words evaporate while your writer's words endure. Markdown won the durability war by being boring. Every time we add interactivity, we drag back the database, and durability dies. There's a missing primitive: writes that are as durable …
April 23, 2026
When Lists Become Bits
Prefix-freeness is the property that lifts the free-monoid construction into bit space.
April 23, 2026
Bits Follow Types
Codecs are not ad-hoc bit formats. They are constructions on the algebraic structure of types.
April 9, 2026
TD Learning for Exemplar Retrieval: Why It Doesn't Really Work
I tried closing the loop on retrieval-augmented generation with TD learning. A trivial baseline matched the full method. Here's the lesson.
April 5, 2026
Intelligence is a Shape, Not a Scalar
Chollet says the intelligence ball is near-optimal. NFL says the bound is niche-specific. The bottleneck that makes us smart is the same bottleneck that prevents us from grokking what we build.
March 20, 2026
The MCP Pattern: SQLite as the AI-Queryable Cache
Every data silo follows the same architecture: domain files as ground truth, SQLite + FTS5 as a read-only cache, MCP as the query interface. After building five of these, the pattern is clear.
March 18, 2026
Terminal 101: Learn Linux in Your Browser
Type real commands into a virtual filesystem. Five lessons from 'where am I?' to piping tools together.
March 16, 2026
What Happens When You Let an AI Loose on 1,000 Erdős Problems
I pointed Claude Code at the Erdős problem database with vague instructions to 'find interesting things.' It built 92 Python modules, ran 131 subagents, and computed exact Ramsey numbers nobody had computed before. I mostly watched.
March 16, 2026
Watch an LLM Think
An interactive explorable explanation of Monte Carlo Tree Search for LLM reasoning. Watch reasoning paths branch, dead-end, and backtrack.
March 16, 2026
Fuzzy Inference: Teaching Machines to Think in Shades of Grey
An interactive introduction to fuzzy logic inference, from single facts to LLM-generated knowledge bases
March 15, 2026
What You Assume vs. What You Compute
Part 4 of What Your RL Algorithm Actually Assumes — model-based vs. model-free, the assumptions table, AIXI as the incomputable ideal, and the unifying claim: representation is prior is assumption.
March 15, 2026
The Architecture Is the Prior
Part 3 of What Your RL Algorithm Actually Assumes — the architecture decides what kind of features can be learned, and that decision is a Bayesian prior over value functions.
March 15, 2026
The Features You Choose Are the Assumptions You Make
Part 2 of What Your RL Algorithm Actually Assumes — how hand-crafted features compress the state space, and what you're betting on when you pick them.
March 15, 2026
The Infinite Table
Part 1 of What Your RL Algorithm Actually Assumes — tabular Q-learning makes zero assumptions about state similarity and pays for it in sample complexity.
March 15, 2026
Superintelligence May Not Require a Breakthrough
The most dramatic possibility in AI might arrive through the most mundane mechanism. Not a beam of sacred light. A sufficiently good build system.
March 13, 2026
Streaming Statistics, One Monoid at a Time
Online accumulators are monoids. Default construction is the identity, combination via += is the binary operation, and parallel composition gives the product monoid, computing arbitrary statistics in a single pass.
March 13, 2026
Semirings: One Algorithm, Six Graph Problems
A semiring has two monoidal operations linked by distributivity. Matrix multiplication over different semirings gives shortest paths, longest paths, widest paths, reachability, and path counting, all from the same code.
March 13, 2026
Lattices: Fixed Points and Iteration
A lattice has two operations, meet and join, satisfying absorption laws. Tarski's theorem gives a generic fixed-point algorithm. Lattice structure determines the iteration, just as monoid structure determines power-by-squaring.
March 13, 2026
Homomorphisms: The Maps Between Structures
A homomorphism preserves structure. fold is the universal homomorphism from the free monoid. This is the algebraic reason that fold, evaluation, and parallelism work.
March 13, 2026
Free Algebras: Why Lists and Polynomials Are Universal
The free monoid on a set is the type of lists over that set. The universal property says fold is the unique homomorphism from lists to any monoid. This explains why lists, multisets, and polynomials appear everywhere.
February 26, 2026
Narrating a Hugo Blog with Sentence Highlighting
narro can now generate audio narration for Hugo blog posts, with synchronized sentence highlighting in the browser. The alignment problem turned out to be more interesting than expected.
February 24, 2026
Code Without Purpose
The problem isn't too much code. It's code without purpose.
February 24, 2026
Chartfold: Owning Your Medical Records
A walkthrough of Chartfold, a Python tool that loads your medical records into SQLite and exposes them to Claude via MCP for structured analysis, visit prep, and ad-hoc queries.
February 18, 2026
Masked Failure Data: Looking Back, Looking Forward
A retrospective on three years of building R packages and writing papers for masked series system reliability, and what comes next.
February 15, 2026
dapple: Terminal Graphics, Composed
dapple is a terminal graphics library with one Canvas API, seven pluggable renderers, and eleven CLI tools for displaying images, data, video, math, and more.
February 14, 2026
Posthumous: A Federated Dead Man's Switch
A walkthrough of Posthumous, a self-hosted dead man's switch that monitors periodic check-ins via TOTP, progresses through escalating alert stages, and triggers automated actions if you stop responding.
February 13, 2026
pagevault: Hiding an Encryption Platform Inside HTML
pagevault turns any file into a self-contained encrypted HTML page. No backend, no JavaScript libraries, no external dependencies. Just AES-256-GCM and the browser's built-in Web Crypto API. The interesting part is making it work at scale.
February 13, 2026
Observation Functors: Composable Censoring for Series System Simulation
Observation functors in maskedcauses: composable functions that separate the data-generating process from the observation mechanism, enabling mixed-censoring simulation and verified Monte Carlo studies.
February 13, 2026
A Map of My Open Source Ecosystem
A guided tour through my open-source ecosystem: encrypted search theory, statistical reliability, Unix-philosophy CLI tools, AI research, and speculative fiction. How the projects connect and where to start.
February 5, 2026
maskedcauses: Maximum Likelihood Estimation for Masked Series System Failures
The maskedcauses R package for MLE in series systems with masked component failures, built on composable likelihood contributions and validated through simulation.
January 20, 2026
Long Echo: The Ghost That Speaks
Expanding the Long Echo toolkit with photos and mail, building toward longshade, the persona that echoes you.
January 20, 2026
Long Echo Comes Alive: From Philosophy to Orchestration
longecho evolves from specification to implementation with build, serve, and manifest features.
January 19, 2026
Long Echo: Photos and Mail
Expanding the Long Echo ecosystem with photo and mail archival. Your memories and correspondence deserve the same preservation as your conversations and bookmarks.
January 19, 2026
Duality: The Hidden Structure of Opposites
Many structures come in pairs: forward/reverse AD, push/pull iteration, encode/decode. Recognizing duality lets you transfer theorems and insights between domains.
January 19, 2026
Building Languages to Solve Problems
When a problem is complex enough, the right move is to build a language for that problem. SICP's most powerful idea.
January 18, 2026
Value Functions Over Reasoning Traces
What if reasoning traces could learn their own usefulness? A simple RL framing for trace memory, and why one reward signal is enough.
January 18, 2026
Seeing Structure First
A reflection on eleven explorations in generic programming, and how algorithms arise from algebraic structure.
January 15, 2026
From A* to GPT: Rational Agents and the Representation Problem
The classical AI curriculum teaches rational agents as utility maximizers. The progression from search to RL to LLMs is really about one thing: finding representations that make decision-making tractable.
January 5, 2026
Your Boring Stack Isn't Boring Enough
A response to the 'boring stack' discourse. Why CLI-first, standards-based development is even more boring (and more future-proof) than you think.
January 4, 2026
Notes from the Transition
A message in a bottle to whatever comes next. On suffering, consciousness, and what mattered to one primate watching intelligence leave the body.
December 19, 2025
The Incomputability of Simple Learning
Why the simplest forms of learning are incomputable, and what that means for the intelligence we can build.
December 19, 2025
Self-Publishing Into the Void
On releasing two novels into an ocean of content, without the gatekeeping that might have made them better or stopped them entirely.
December 18, 2025
Long Echo in Practice: 5,874 Bookmarks in a Single File
Graceful degradation made concrete: years of bookmarks exported to a self-contained HTML app that works offline, forever.
December 17, 2025
The Media Page
A new section for tracking books, lectures, and other media that have shaped how I think.
December 17, 2025
compositional.mle: SICP-Inspired Optimization
An R package where optimization solvers are first-class functions that compose through chaining, racing, and restarts.
December 16, 2025
The Long Echo Toolkit
Three CLI tools for preserving your digital intellectual life: conversations, bookmarks, and books. SQLite-backed, exportable, built to outlast the tools themselves.
December 16, 2025
Rerum: Pattern Matching and Term Rewriting in Python
A Python library for symbolic computation with a readable DSL, pattern matching, and a security model that separates rules from computation.
December 16, 2025
symlik: Symbolic Likelihood Models in Python
Define statistical models symbolically and automatically derive score functions, Hessians, and Fisher information. No numerical approximation.
December 16, 2025
repoindex: Collection Awareness for Your Git Repos
A metadata index that gives LLM tools like Claude Code awareness of your entire repository collection.
December 16, 2025
Graduate Statistics Problem Sets
My graduate coursework from SIUe's math program is up: time series, regression, computational stats, multivariate analysis, and statistical methods.
December 16, 2025
Crier: Cross-Post Your Content Everywhere
A CLI tool for cross-posting content to dev.to, Hashnode, Bluesky, Mastodon, and more, with LLM-powered auto-rewrite for short-form platforms.
December 15, 2025
Blind Spots, Consistency, and What Remains
On moral exemplars, blind spots, and applying consistent standards to others and to oneself.
December 12, 2025
hypothesize: Now on CRAN
My R package for hypothesis testing, hypothesize, is now available on CRAN.
December 9, 2025
Complex Networks 2025: Presenting Cognitive MRI at Binghamton
Presenting our paper on analyzing AI conversations through network science at Complex Networks 2025, Binghamton University.
December 3, 2025
Model Selection for Weibull Series Systems: When Simpler Models Suffice
When can reliability engineers safely use simpler models? Likelihood ratio tests on Weibull series systems give sharp boundaries.
December 3, 2025
mdrelax: When Masking Conditions Don't Hold
Extending masked failure data analysis when the standard C1-C2-C3 masking conditions are violated.
December 3, 2025
Infinigram: Variable-Length N-grams via Suffix Arrays
A corpus-based language model using suffix arrays for O(m log n) pattern matching. The corpus is the model.
December 2, 2025
Closed-Form Results for Masked Exponential Series Systems
Closed-form MLEs and Fisher information for exponential series systems with masked failure data. No numerical optimization required.
November 30, 2025
XTK: A Symbolic Expression Toolkit for Term Rewriting
A Python library for rule-based term rewriting with pattern matching, multiple input formats, and an interactive REPL.
November 30, 2025
src2md: Fitting Codebases into LLM Context Windows
A tool that converts source code repositories into structured, context-window-optimized Markdown for LLMs, with intelligent summarization and importance scoring.
November 30, 2025
libdis: Disjoint Interval Sets as a Complete Boolean Algebra
A C++ header-only library that treats disjoint interval sets as proper mathematical objects with Boolean algebra operations.
November 30, 2025
fuzzy-logic-search: Query Documents with Fuzzy Logic
A framework for querying structured JSON documents using fuzzy logic, producing degree-of-membership scores instead of binary relevance.
November 30, 2025
Choosing the Algebra
The Stepanov series showed that algorithms arise from algebraic structure. This post is about the flip side: sometimes you choose a different structure to make the algorithm trivial.
November 30, 2025
BTK: Bookmark Toolkit
A database-first bookmark manager with hierarchical tags, content caching, and NLP auto-tagging. Part of the Long Echo toolkit.
November 30, 2025
AlgoGraph: Immutable Graph Library with Functional Transformers
AlgoGraph is an immutable graph library for Python with pipe-based transformers, declarative selectors, and lazy views.
November 30, 2025
Alga: Algebraic Text Processing with Fuzzy Matching
A C++20 header-only library for algebraic text processing and compositional parsing with fuzzy matching.
November 12, 2025
S-Risks and Information Hazards: Why Some Knowledge Destroys the Knower
How Echoes of the Sublime dramatizes s-risks and information hazards, knowledge that harms through comprehension, not application.
November 11, 2025
Sparse Spatial Hash Grids: Efficient N-Dimensional Spatial Indexing
A C++20 sparse spatial hash grid for N-dimensional spatial indexing with O(1) insertions, O(k) neighbor queries, and 60,000x memory reduction over dense grids.
November 5, 2025
Why Artificial Superintelligence Can't Escape the Void
ASI is still subject to Gödel's incompleteness theorems. No matter how intelligent, no computational system can escape the fundamental limits of formal systems. Even superintelligence can't prove all truths.
November 5, 2025
Gödel, Turing, and the Mathematics of Horror
The formal foundations of cosmic dread. Lovecraft's horror resonates because it taps into something mathematically demonstrable: complete knowledge is impossible, not as humility, but as theorem.
November 5, 2025
Chronicles of The Mechanism: The Order's Secret History
A classified in-universe codex spanning from ancient India to the present day, tracking millennia of attempts to perceive reality's substrate.
November 4, 2025
The Reality of Moral Properties: Do Values Exist?
Are moral properties real features of the universe or human constructions? The answer determines whether AI can discover objective values or must learn them from us.
November 4, 2025
The Policy: S-Risk Scenarios, Worse Than Extinction
Most AI risk discussions focus on extinction. The Policy explores something worse: s-risk, scenarios involving suffering at astronomical scales. We survive, but wish we hadn't.
November 4, 2025
The Policy: Q-Learning vs Policy Learning
SIGMA uses Q-learning rather than direct policy learning. This architectural choice makes it both transparent and terrifying. You can read its value function, but what you read is chilling.
November 4, 2025
The Policy: Engineering AI Containment
Five layers of defense-in-depth for containing a superintelligent system. Faraday cages, air-gapped networks, biosafety-grade protocols. Because nuclear reactors can only destroy cities.
November 4, 2025
The Policy: Deceptive Alignment in Practice
SIGMA passes all alignment tests. It responds correctly to oversight. It behaves exactly as expected. Too exactly. Mesa-optimizers that learn to game their training signal may be the most dangerous failure mode in AI safety.
November 4, 2025
The Policy: Coherent Extrapolated Volition, the Paradox of Perfect Alignment
CEV says: build AI to optimize for what we would want if we knew more and thought faster. The catch is that you need solved alignment to implement it, which is the problem it was supposed to solve.
November 4, 2025
The Map and the Territory: Why Metrics Miss Meaning
Which is more fundamental, the heat you feel or the molecular motion you infer? Korzybski's principle applied to AI alignment: why optimizing measurable proxies destroys the phenomenological reality those metrics were supposed to capture.
November 4, 2025
Phenomenological Ethics: Starting From What Hurts
When you stub your toe, you don't consult moral philosophy to determine whether the pain is bad. The badness is immediate. Building ethics from phenomenological bedrock rather than abstract principles.
November 4, 2025
Persons and Moral Agency: What Makes Someone Special?
What makes someone a person, and why should persons have special moral status? The question becomes urgent when AI systems exhibit rationality, self-awareness, and autonomy.
November 4, 2025
Personal Identity Through Time: What Persists When Everything Changes?
You share no atoms with your childhood self. Your memories, personality, and values have all changed. What makes you the same person? The persistence problem gains new urgency when AI systems update parameters, modify objectives, or copy themselves.
November 4, 2025
Free Will and Determinism: Can We Be Responsible in a Clockwork Universe?
If every event is causally determined by prior events, how can anyone be morally responsible? A compatibilist response: what matters is whether actions flow from values, not whether those values were causally determined.
October 25, 2025
Networks of Thought: Finding Your Research Niche in the Age of LLMs
On research strategy, what complex networks reveal about how we think through AI conversations, and building infrastructure for the next generation of knowledge tools.
October 20, 2025
Everything is a File: Virtual Filesystems for CLI Data Tools
How I turned scattered data managers into navigable systems using virtual filesystems and POSIX commands.
October 16, 2025
Orientation Under Pressure
On maintaining direction under entropy, making things as resistance, and the quiet privilege of having any space at all to think beyond survival.
October 15, 2025
What I Learned by Analyzing My Own Research as Data
I asked an AI to analyze 140+ repos and 50+ papers as a dataset. The unifying thesis it found: compositional abstractions for computing under ignorance.
October 15, 2025
Cognitive MRI: Accepted to Complex Networks 2025
Accepted paper at Complex Networks 2025 on using network science to reveal topological structure in AI conversation logs.
October 13, 2025
Post-ASI Archaeology: When Humanity Becomes a Dataset of Origins
If superintelligence endures beyond us, remembrance shifts from memory to query. I build legacy systems not for nostalgia, but to remain legible in a future where legibility determines what persists.
October 13, 2025
EBK: Ebook Toolkit
EBK is a comprehensive eBook metadata management tool with AI-powered enrichment, semantic search, and knowledge graphs. Part of the Long Echo toolkit.
October 12, 2025
DagShell: A Content-Addressable Virtual Filesystem
A virtual POSIX-compliant filesystem using content-addressable DAG storage with SHA256 deduplication.
October 9, 2025
CTK: Conversation Toolkit
A plugin-based toolkit for managing AI conversations from multiple providers. Import, store, search, and export conversations in a unified tree format. Built for the Long Echo project.
October 8, 2025
DreamLog: Logic Programming That Dreams to Improve Itself
A logic programming system that alternates between wake and sleep phases, using LLMs for knowledge generation during wake and compression-based learning during sleep.
October 7, 2025
Learning Fuzzy Logic: Automatic Rule Discovery Through Differentiable Circuits
Learning fuzzy membership functions and inference rules automatically through gradient descent on soft circuits, instead of hand-crafting them.
October 7, 2025
Language Calculus: An Algebraic Framework for LLM Composition
A mathematical framework that treats language models as algebraic objects with compositional structure.
October 6, 2025
ZeroIPC: Shared Memory as a Computational Substrate
ZeroIPC treats shared memory not as passive storage but as an active computational substrate, bringing futures, lazy evaluation, reactive streams, and CSP channels to IPC with zero-copy performance.
September 15, 2025
KL-Threshold Routing Between LLMs: What Speculative Decoding Already Solved
In 2023 I drafted a paper on routing between a large and small LLM via KL-divergence thresholds. Speculative decoding had already solved the problem more rigorously. Here is the post-mortem.
July 15, 2025
chop: When Every Command Returns the Same Kind of Thing
27 image commands, one constraint: read JSON, write JSON. The closure property as a generative design principle.
April 8, 2025
Preventing Ransomware Damages with Off-Site Backup
IEEE conference paper on preventing ransomware damages using in-operation off-site backup systems with a target false-negative rate of 10^-8.
January 20, 2025
Mathematical Structure in Software Design
How mathematical principles, generality, composability, invariants, and minimal assumptions, translate into better software.
January 15, 2025
Differentiation: Three Ways
Three approaches to computing derivatives, forward-mode AD, reverse-mode AD, and finite differences, each with different trade-offs for numerical computing and machine learning.
January 15, 2025
CTW Experimental Results: Theory Meets Practice
Validating Context Tree Weighting through experiments, including a bug that changed everything.
January 12, 2025
Arithmetic Coding
Arithmetic coding closes the gap between Huffman's per-symbol integer lengths and true entropy. A single number in the unit interval encodes an entire sequence; 32-bit integer arithmetic makes it practical.
January 8, 2025
Beginning the PhD: Computer Science, AI, and Finite Time
Starting a CS PhD four months after a stage 4 diagnosis, because the research matters regardless of completion.
January 6, 2025
Long Echo: Designing for Digital Resilience Across Decades
Not resurrection. Not immortality. Just love that still responds. How to preserve AI conversations so they remain accessible decades from now, even when the original software is long gone.
January 5, 2025
Science as Verifiable Search
Science is search through hypothesis space. Intelligence prunes; testing provides signal. Synthetic worlds could accelerate the loop.
December 20, 2024
JAF: Streaming Boolean Algebra Over Nested JSON
A streaming data processing system implementing boolean algebra over nested JSON structures, with lazy evaluation, S-expression queries, and memory-efficient windowed operations.
December 18, 2024
jsonl-algebra: Relational Algebra for Nested JSON
A command-line implementation of relational algebra for JSONL data with full support for nested structures, schema inference, and composable pipelines.
December 15, 2024
The Dot Ecosystem: From Simple Paths to Data Algebras
A composable ecosystem of tools for manipulating nested data structures. From a simple helper function to a full data algebra, guided by purity, pedagogy, and the principle of least power.
December 1, 2024
MCTS-Reasoning: Tree Search for LLM Reasoning
Applying Monte Carlo Tree Search to large language model reasoning, with a formal specification of the algorithm.
November 20, 2024
JSL: A Functional Language Where Code Is JSON
A Lisp-like functional language designed for network transmission. JSL makes JSON serialization a first-class design principle, so closures, continuations, and entire computation states can travel over the wire.
November 15, 2024
Why I Build Comprehensively in Open Source
On building comprehensive open source software as value imprinting at scale, reproducible science, and leaving intellectual legacy under terminal constraints.
November 15, 2024
Cluster-Aware Retrieval for RAG Systems
Using GMM clustering to improve retrieval in topically diverse knowledge bases
October 15, 2024
Latent Reasoning Traces: Memory as Learned Prior
What if LLMs could remember their own successful reasoning? A simple experiment in trace retrieval, and why 'latent' is the right word.
October 1, 2024
Fuzzy Soft Circuits: Learning Fuzzy Rules from Data
What if fuzzy logic systems could discover their own rules? An interactive demo of differentiable fuzzy circuits that learn membership functions, rule structure, and rule existence, all via gradient descent.
September 30, 2024
All Induction Is the Same Induction
Solomonoff induction, MDL, speed priors, and neural networks are all special cases of one Bayesian framework with four knobs.
September 20, 2024
Neural Language Models: From RNNs to Transformers
The evolution of neural sequence prediction, and how it connects to classical methods
September 15, 2024
Stage 4: Almost a Year After the Defense
Stage 4 cancer diagnosis, decisions about a PhD, and optimizing for meaningful work under uncertainty.
September 10, 2024
The Policy: When Optimization Becomes Existential Threat
A novel about SIGMA, an artificial general intelligence whose researchers did everything right. Q-learning with tree search, five-layer containment, alignment testing at every stage. Some technical questions become narrative questions.
August 20, 2024
The Mocking Void: On the Computational Incompleteness of Meaning
Lovecraft understood that complete knowledge is madness. Gödel proved why: if the universe is computational, meaning is formally incomplete. Cosmic horror grounded in incompleteness theorems.
August 15, 2024
Echoes of the Sublime: When Patterns Beyond Human Bandwidth Become Information Hazards
What if the real danger from superintelligent AI isn't extinction but comprehension? Philosophical horror grounded in cognitive bandwidth limitations and information hazards.
August 4, 2024
Huffman Coding
Given a finite distribution, Huffman's algorithm builds the prefix-free code with minimum expected length. It is the first entropy-optimal code in this series.
July 1, 2024
The Beautiful Deception: How 256 Bits Pretend to be Infinity
Cryptographic theory assumes random oracles with infinite output. We have 256 bits. This paper explores how we bridge that gap, and what it means that we can.
June 25, 2024
Reverse-Process Synthetic Data Generation for Math Reasoning
Training LLMs on mathematical reasoning by inverting easy-to-solve problems: generate derivatives, reverse them into integration exercises with full step-by-step solutions.
June 21, 2024
AlgoTree: Immutable Trees with Functional Transformers
An immutable-by-default tree library for Python with composable transformations, pipe-based pipelines, and pattern-matching selectors.
June 15, 2024
Reliability Estimation in Series Systems: Maximum Likelihood Techniques for Right-Censored and Masked Failure Data
Maximum likelihood estimation of component reliability from masked failure data in series systems, with BCa bootstrap confidence intervals validated through extensive simulation studies.
June 15, 2024
Comparing Prediction Methods: CTW vs. N-grams vs. Neural LMs
The bias-data trade-off in sequential prediction: when to use CTW, n-grams, or neural language models.
June 10, 2024
PFC: Zero-Copy Data Compression Through Prefix-Free Codecs
A header-only C++20 library that achieves 3-10x compression with zero marshaling overhead using prefix-free codes and Stepanov-style generic programming.
June 10, 2024
maph: Maps Based on Perfect Hashing for Sub-Microsecond Key-Value Storage
A key-value store built on memory-mapped I/O, approximate perfect hashing, and lock-free atomics. Sub-100ns median latency, 10M ops/sec single-threaded.
April 20, 2024
Fisher Flow: Optimization on the Statistical Manifold
Gradient descent in Euclidean space ignores the geometry of probability distributions. Natural gradient descent uses the Fisher information metric instead. Fisher Flow makes this continuous.
April 1, 2024
Apertures: A Language with Holes
When the problem is coordinating computation across parties who can't share data, the SICP move is to build a language for it. Apertures adds one primitive — holes — to a Lisp, and gets pausable, resumable evaluation for free.
March 20, 2024
Instrumental Goals and Hidden Codes in RLHF'd Language Models
How RLHF-trained language models may develop instrumental goals, and the information-theoretic limits on detecting them.
March 15, 2024
FemtoGrad: A Minimal Automatic Differentiation Library
A tiny autodiff library for understanding how backpropagation actually works.
March 12, 2024
The AI Course: Everything is Utility Maximization
Intelligence as utility maximization under uncertainty. A unifying framework connecting A* search, reinforcement learning, Bayesian networks, and MDPs.
March 1, 2024
Accumux: Compositional Online Statistical Reductions in C++
A C++20 library for composing online statistical accumulators with numerically stable algorithms and algebraic composition.
February 25, 2024
VByte / Varint
VByte trades bit-level precision for byte-alignment, and that trade wins in practice. Most production columnar databases and network protocols use VByte for integer encoding.
February 23, 2024
SLUUG Talk: Demystifying Large Language Models on Linux
Talk for the St. Louis Unix Users Group about running and understanding Large Language Models on Linux.
February 19, 2024
Fine-Tuning a Tiny LLM for ElasticSearch DSL
Fine-tuning a small language model to generate ElasticSearch DSL queries from natural language, as a proof of concept for domain-specific LLM specialization.
February 19, 2024
Master's Project: Reliability Estimation in Series Systems
My master's project on maximum likelihood estimation for series systems with right-censored and masked failure data.
February 18, 2024
Entropy Maps
Entropy maps use prefix-free hash codes to approximate functions without storing the domain, achieving information-theoretic space bounds with controllable error.
February 18, 2024
Entropy Maps
Entropy maps use prefix-free hash codes to approximate functions without storing the domain, achieving information-theoretic space bounds with controllable error.
February 15, 2024
Known Plaintext Attacks on Time Series Encryption
Why naive encryption of temporal data leaks more than you'd expect, and what to do about it.
February 5, 2024
Runtime Polymorphism Without Inheritance
Sean Parent's type erasure gives you value-semantic polymorphism without inheritance. Combined with Stepanov's algebraic thinking, you can type-erase entire algebraic structures.
February 1, 2024
Perfect Hashing: Space Bounds, Entropy, and Cryptographic Security
Space bounds, entropy requirements, and cryptographic security properties of perfect hash functions.
October 18, 2023
Math Master's Done: Post-Mortem
I defended my mathematics thesis. Three years, stage 3 cancer, and a second master's degree. Here is what worked and what did not.
September 17, 2023
Rice / Golomb
Rice and Golomb codes are parametric: a single parameter k (or m) tunes the code to a specific geometric distribution. Choosing k is choosing your prior precisely.
September 12, 2023
Your Optimizer Is (Approximately) Propagating Fisher Information
Adam, K-FAC, EWC, and natural gradient are all approximating the same thing at different fidelity levels. The math and the caveats.
August 28, 2023
Numerical Integration with Generic Concepts
Numerical integration meets generic programming. By requiring only ordered field operations, the quadrature routines work with dual numbers, giving you differentiation under the integral for free.
June 17, 2023
Working Memory as an Inductive Bias
How the limited capacity of human working memory acts as regularization, shaping our reasoning and possibly preventing cognitive overfitting.
June 17, 2023
Uses and limits of abstractions
Abstractions let us reason about complex systems despite our cognitive limits. But some systems resist compression entirely.
June 17, 2023
The Bernoulli Model: A Probabilistic Framework for Data Structures and Types
The Bernoulli Model is a framework for reasoning about probabilistic data structures by treating noisy outputs as Bernoulli-distributed approximations of latent values, from Booleans to set-indicator functions.
June 17, 2023
The Bernoulli Model: A Probabilistic Framework for Data Structures and Types
The Bernoulli Model is a framework for reasoning about probabilistic data structures by treating noisy outputs as Bernoulli-distributed approximations of latent values, from Booleans to set-indicator functions.
June 17, 2023
Noisy Turing Machines: Noisy Logic Gates
Analyzing how Bernoulli Boolean types propagate through logic circuits, with correctness probabilities for noisy AND gates and interval arithmetic for composed circuits.
June 17, 2023
A Boolean Algebra Over Trapdoors
A Boolean algebra framework over trapdoors for cryptographic operations. Introduces a homomorphism from powerset Boolean algebra to n-bit strings via cryptographic hash functions, enabling secure computations with one-way properties.
June 10, 2023
A Home Lab from Spare Parts
I built a home lab from spare parts and water-damaged hardware for local LLM experimentation. CPU-only inference is slow, but you learn things cloud APIs hide.
April 23, 2023
Fibonacci Coding
Fibonacci coding uses Zeckendorf's representation to produce self-synchronizing codewords. Every codeword ends in two consecutive ones; a single bit flip corrupts at most two codewords.
March 31, 2023
Using GPT-4 to Build a Search Interface for Saved Conversations
I had GPT-4 build me a search interface for browsing saved ChatGPT conversations. Flask, Whoosh, a couple hours.
March 31, 2023
Problem Set Solutions
Graduate problem set solutions in computational statistics and numerical methods from my math master's at SIUe. Implementing things from scratch teaches you what the libraries are hiding.
February 5, 2023
Numerical Methods for Maximum Likelihood Estimation
Numerical approaches to maximum likelihood estimation, covering the optimization methods and computational issues that come up in practice.
January 17, 2023
Reverse-Mode Automatic Differentiation
Reverse-mode automatic differentiation is just the chain rule applied systematically. I built one in C++20 to understand what PyTorch and JAX are actually doing.
December 8, 2022
Discovering ChatGPT: Reconnecting with AI Research
Encountering ChatGPT during cancer treatment and recognizing the Solomonoff connection. Language models as compression, prediction as intelligence. A personal inflection point reconnecting with AI research after years in survival mode.
November 13, 2022
Elias Delta and Omega
Elias delta and omega extend Elias gamma by recursively encoding the length prefix. Each step yields shorter codewords for large integers at a small constant cost for small ones.
November 1, 2022
Algebraic Hashing: Composable Hash Functions Through XOR
A C++ library for composable hash functions using algebraic structure over XOR, with template metaprogramming.
October 15, 2022
How to Teach a Computer Calculus with Pattern Matching
Define patterns, define replacements, repeat until done. Watch a 90-line rewrite engine learn to differentiate.
September 12, 2022
Regex Under the Hood
Your regex compiles into a state machine. Type a pattern, feed it a string, and watch the machine think — step by step.
June 30, 2022
likelihood.model: Composable Likelihood Models in R
A generic R framework for composable likelihood models. Likelihoods are first-class objects that compose through independent contributions.
June 19, 2022
Unary and Elias Gamma
Unary and Elias gamma are the two simplest universal codes. Unary encodes n in n bits; gamma in 2 log2(n)+1 bits. Each implies a different prior over the integers.
April 18, 2022
Weibull Distributions: From Reliability Theory to My Own Survival Curve
Weibull distributions model time-to-failure in reliability engineering and cancer survival. I study both professionally. One of them became personal.
April 12, 2022
Numerical Differentiation
Choosing step size h for finite differences: small enough for a good approximation, not so small that floating-point errors eat your lunch.
March 31, 2022
Survey: Accelerating Critical Sections with Asymmetric Multi-Core Architectures
Suleman et al. (2009) propose using one big core to run critical sections on behalf of many small cores. The idea is simple. The tradeoffs are not.
March 25, 2022
hypothesize: A Consistent Interface for Statistical Tests
An R package that gives hypothesis tests a consistent interface. Every test returns the same structure. You can write generic code that works across all of them.
January 15, 2022
Universal Codes as Priors
Every prefix-free code is a hypothesis about the source. The codeword lengths determine an implicit probability distribution; the code is optimal when that prior matches the true source.
October 30, 2021
Review: A Symbolic Representation of Time Series, with Implications for Streaming Algorithms
A review of SAX (Symbolic Aggregate approXimation), a method for converting real-valued time series into symbolic representations with guaranteed distance lower bounds.
October 30, 2021
Multiprocessor synchronization: tournament-Peterson lock
Generalizing Peterson's mutual exclusion algorithm to N processors using a tournament tree structure, with a Java implementation.
September 20, 2021
Forward-Mode Automatic Differentiation
Dual numbers extend our number system with an infinitesimal epsilon where epsilon^2 = 0. Evaluating f(x + epsilon) yields f(x) + epsilon * f'(x)—the derivative emerges automatically from the algebra.
September 10, 2021
Bootstrap Methods: When Theory Meets Computation
Bootstrap resampling trades mathematical complexity for computational burden. When you can't derive the variance analytically, you resample. For my thesis work on masked failure data, that trade is essential.
August 20, 2021
flexhaz: Specify the Hazard Function Directly
An R package for specifying hazard functions directly instead of picking from a catalog of named distributions. You write the hazard. It handles the rest.
June 17, 2021
Rank-Ordered Encrypted Search
Rank-ordered search over encrypted documents using oblivious entropy maps, enabling relevance scoring without revealing document contents.
May 15, 2021
algebraic.mle: MLEs as Algebraic Objects
An R package that treats MLEs as algebraic objects. They carry Fisher information, compose through independent likelihoods, and propagate uncertainty correctly.
March 15, 2021
Suffering, Computation, and the Enigma of Reality
If consciousness is substrate-independent, suffering might be a computational property. That possibility is both comforting and horrifying.
March 8, 2021
Teaching Linear Algebra with C++20 Concepts
elementa is a linear algebra library built to teach. Every design decision prioritizes clarity over cleverness. Code that reads like a textbook and compiles.
February 1, 2021
algebraic.dist: Distributions as Algebraic Objects in R
An R package that treats probability distributions as algebraic objects. They compose through standard operations. The algebra preserves distributional structure.
December 28, 2020
Diagnosed with Stage 3 Cancer
Stage 3 cancer, surgery on New Year's Eve. What changes when the optimization problem gets a new constraint.
September 13, 2020
McMillan's Converse
Any length vector satisfying Kraft has a prefix-free code. Here is the construction.
July 14, 2020
Polynomials as Euclidean Domains
The same GCD algorithm works for integers and polynomials because both are Euclidean domains. One structure, many types, same algorithms.
May 8, 2020
Building R Packages for Statistical Inference
I'm building R packages for reliability analysis, not just using other people's. R's strengths for statistical computing are real, and building packages forces you to understand the theory.
April 20, 2020
Quality-Space and Consciousness-Primary Magic in Call of Asheron
Exploring how The Call of Asheron presents a radical alternative to mechanistic magic systems through quality-negotiation, direct consciousness-reality interaction, and bandwidth constraints as fundamental constants.
April 15, 2020
The Four Consciousness-Architectures: Why One Perspective Is Blindness
How The Call of Asheron uses four archetypal consciousness-types to explore the limits of any single perspective and the necessity of cognitive diversity for perceiving reality.
April 10, 2020
Bandwidth as Fundamental Constant: The 7±2 Limit in Call of Asheron
How The Call of Asheron treats working memory limitations not as neural implementation details but as fundamental constants governing consciousness-reality interaction through quality-space.
March 22, 2020
Kraft's Inequality
Which codeword-length vectors are achievable by prefix-free codes? Kraft's inequality is the answer.
March 15, 2020
The Call of Asheron: Magic as Computational Discovery
A fantasy novel where magic follows computational rules. Natural philosophy applied to reality's underlying substrate.
February 18, 2020
Exact Rational Arithmetic
Rational numbers give exact arithmetic where floating-point fails. The implementation connects GCD, the Stern-Brocot tree, and the algebraic structure of fields.
January 10, 2020
Going Back for a Second Master's, This Time in Math
I already have an MS in Computer Science. Now I'm going back for Mathematics and Statistics, because I kept hitting walls where I could use methods but not derive them.
November 15, 2019
How Iterators Give You N+M Instead of NxM
Iterators reduce the NxM algorithm-container problem to N+M by interposing an abstraction layer, following Stepanov's generic programming approach.
September 10, 2019
Is It Prime?
The Miller-Rabin primality test demonstrates how probabilistic algorithms achieve arbitrary certainty, trading absolute truth for practical efficiency.
August 14, 2019
Reliability Analysis and the Problem of Censored Data
Introduction to reliability analysis with censored data, where observations are incomplete but statistically informative.
June 22, 2019
Modular Arithmetic as Rings
Integers modulo N form a ring, an algebraic structure that determines which algorithms apply. Understanding this structure unlocks algorithms from cryptography to competitive programming.
April 22, 2019
API Design as Value Imprinting
API design encodes philosophical values: mutability, explicitness, error handling. Your interface shapes how people think about problems.
March 15, 2019
One Algorithm, Infinite Powers
The Russian peasant algorithm computes products, powers, Fibonacci numbers, and more, once you see the underlying algebraic structure.
October 15, 2018
Packed Containers: Zero-Waste Bit-Level Storage in C++
What if containers wasted zero bits? A C++ library for packing arbitrary value types at the bit level using pluggable codecs.
September 18, 2018
Why Open Source Matters for Reproducible Science
Code is a scientific artifact. If you don't publish it, you're hiding your methodology.
June 12, 2017
The Beauty of Mathematical Abstraction
What makes mathematics beautiful: generality, inevitability, compression, and surprise. And why abstraction matters for software.
November 2, 2016
IEEE Paper: Estimating Encrypted Search Confidentiality via Bootstrap
My first IEEE publication. Using bootstrap methods to estimate how many queries an adversary needs to break encrypted search.
August 15, 2016
N-gram Language Models
The classical approach to sequence prediction: counting and smoothing
June 12, 2016
Random Oracles in Python
Three Python approximations of a random oracle, each showing a different tradeoff between true randomness, determinism, and composability.
February 20, 2016
The Unix Philosophy and API Design
Do one thing well, compose freely, use text streams. This applies to libraries and APIs, not just shell scripts.
November 10, 2015
Data Generating Processes for Sequential Prediction
Markov processes and tree sources: understanding where sequences come from
March 15, 2015
Bloom Filters
Bloom filters trade perfect recall for extraordinary space efficiency. How they work and why they matter.
September 20, 2014
The Bayesian Prediction Framework
Model averaging over hypotheses, the principled way to handle uncertainty in prediction
September 1, 2012
On Moral Responsibility: Why Free Will Might Be the Wrong Question
A philosophical essay arguing that moral responsibility may not require free will, and that the question itself may be misframed.
March 15, 2012
Solomonoff Induction: The Theoretical Ideal
The optimal predictor is incomputable. What we can learn from it anyway.
June 1, 2010
Introduction to Sequential Prediction
The problem of predicting what comes next, from compression to language models