Everything is a File: Virtual Filesystems for CLI Data Tools

October 20, 2025

I had a bookmark manager. Then an ebook library manager. Then a chat history manager. Each started with the standard CRUD CLI:

btk add https://example.com --tags python,tutorial
btk list --tag python
btk search "async"
btk delete 1234

ebk import book.pdf --author "Knuth"
ebk list --author Knuth
ebk search "algorithms"

This works fine until you have 10,000+ bookmarks organized with hierarchical tags like programming/python/async, research/ml/transformers, work/clients/acme. Your ebook library has similar structure. Your exported chat conversations from Claude, ChatGPT, and Copilot are piling up.

Traditional CRUD commands become unwieldy:

btk list --tag programming/python/async/io --format json | jq '.[].title'
ebk list --category "Computer Science/Algorithms/Graph Theory" --limit 50
ctk search "machine learning" --source ChatGPT --date-from 2024-01-01

Each command requires precise arguments. Each tool has different flag conventions. You can’t navigate your data. You can only query it. And queries require knowing exactly what you’re looking for.

The insight: everything is a file

When I have thousands of source files organized in directories, I don’t run:

list-files --path /src/components/auth --extension .tsx

I run:

cd src/components/auth
ls *.tsx

The difference matters. With a filesystem, I can navigate incrementally (cd from general to specific), explore (ls to see what’s there), compose (cat file | grep pattern | wc -l), and use familiar tools (find, grep, xargs, pipes, redirection).

What if my bookmarks, ebooks, and chat histories were filesystems?

The pattern

Over the past year, I built six Python tools that all follow the same architecture:

ToolDomainVFS Root Structure
btkBookmarks/bookmarks/, /tags/, /recent/, /domains/, /unread/, /popular/
ebkEbook library/books/, /authors/, /series/, /subjects/, /recent/, /unread/
ctkChat conversations/conversations/, /sources/, /topics/, /starred/, /recent/
ghopsGit repositories/repos/, /languages/, /topics/, /stars/, /recent/
infinigramN-gram models/datasets/, /models/, /corpora/
AlgoTreeTree structures/nodes/, /paths/, /subtrees/

Each tool provides:

  1. A stateless CLI for scripting: btk bookmark add URL, ebk import book.pdf
  2. An interactive shell with a virtual filesystem: btk shell, ebk shell, ctk chat
  3. POSIX-like commands: cd, ls, pwd, cat, mv, cp, rm, find, grep
  4. Unix pipeline support: most commands output JSONL by default for piping

The interesting part is the shell.

Live recording captured with asciinema. You can pause, copy text, and replay. The entire recording is 78KB of text.

Read More

AlgoTree: Immutable Trees with Functional Transformers

June 21, 2024

AlgoTree is a tree manipulation library for Python. Version 2.0 is a complete redesign built on immutable-by-default principles with composable transformers and pattern-matching selectors.

Why immutable trees?

Mutable tree libraries have hidden costs. Modifying a tree can break other code holding references to it. Changes are hard to track during debugging. Concurrent modifications cause subtle bugs. The usual story.

AlgoTree takes a different approach: all operations return new tree objects. The original is never modified.

from AlgoTree import Node, node

# Build a tree
tree = node("root",
    node("child1", value=1),
    node("child2", value=2)
)

# All operations return new trees
tree2 = tree.with_name("new_root")  # tree unchanged
tree3 = tree.with_child(Node("child3"))  # tree unchanged

This is the same idea behind persistent data structures in Clojure or Haskell. Immutability eliminates a whole class of bugs at the cost of some allocation overhead. For tree manipulation tasks (as opposed to, say, hot inner loops), the tradeoff is worth it.

Building Trees

Multiple construction styles for different use cases:

from AlgoTree import Node, node, TreeBuilder

# Simple construction with Node
tree = Node("root",
    Node("child1", attrs={"value": 1}),
    Node("child2", attrs={"value": 2})
)

# Convenience function (auto-converts strings)
tree = node("root",
    node("child1", value=1),
    "child2",  # Strings auto-convert to nodes
    node("child3",
        "grandchild1",
        "grandchild2"
    )
)

# Fluent builder API
tree = (TreeBuilder("root", type="container")
    .child("src")
        .child("main.py", type="file", size=1024)
        .child("utils.py", type="file", size=512)
        .up()
    .child("docs")
        .child("README.md", type="file")
    .build())

Functional Transformations

The standard functional toolkit, applied to trees:

# Map: transform all nodes
doubled = tree.map(lambda n: n.with_attrs(
    value=n.get("value", 0) * 2
))

# Filter: keep nodes matching predicate
filtered = tree.filter(lambda n: n.get("value", 0) > 5)

# Find: locate specific nodes
nodes = tree.find_all(lambda n: n.is_leaf)

Composable Selectors

Pattern matching with wildcards and logical composition:

from AlgoTree import name, attrs, leaf, type_

# Pattern matching with wildcards
selector = name("*.txt")

# Attribute matching with predicates
selector = attrs(size=lambda s: s > 1000)

# Logical composition with operators
selector = type_("file") & ~leaf()  # Files that aren't leaves

# Structural selectors
selector = type_("file").child_of(name("src"))
selector = leaf().at_depth(2)

# Use selectors with trees
matching_nodes = list(selector.select(tree))

The selectors compose with &, |, and ~. This means you can build complex queries from simple parts without writing custom traversal code.

Pipe-Based Transformers

Build transformation pipelines with the >> operator:

from AlgoTree import map_, filter_, prune, normalize, extract

# Build transformation pipelines
pipeline = (
    map_(lambda n: {"processed": True}) >>
    filter_(lambda n: n.get("active")) >>
    normalize(sort_children=True) >>
    extract(lambda n: n.name)
)

# Apply pipeline to tree
result = pipeline(tree)

This is the same idea as Unix pipes. Each stage takes a tree and returns a tree (or extracted values). The >> operator chains them left to right.

Read More