Back to CTK - Conversation Toolkit

Documentation

API reference and architecture for CTK.

API Documentation

CTK (Conversation Toolkit) is a plugin-based system for managing AI conversations from multiple providers. This document covers the core architecture, database schema, and Python API.

Architecture Overview

CTK is built around three core concepts:

  1. ConversationTree: All conversations are stored as trees, preserving branching history
  2. Plugin System: Auto-discovered importers/exporters for different formats
  3. SQLite Backend: Local, searchable database with full-text search

Core Classes

ConversationTree

The primary data structure representing a conversation.

from ctk.core.models import ConversationTree, Message, MessageContent, MessageRole

# Create a new conversation
tree = ConversationTree(
    id="conv_123",
    title="Python Discussion"
)

# Add messages
user_msg = Message(
    role=MessageRole.USER,
    content=MessageContent(text="What is Python?")
)
tree.add_message(user_msg)

# Get all paths (for branching conversations)
paths = tree.get_all_paths()

# Get longest path
longest = tree.get_longest_path()

ConversationDB

Database interface for storing and querying conversations.

from ctk import ConversationDB

with ConversationDB("chats.db") as db:
    # Save conversation
    db.save_conversation(tree)

    # Load by ID
    conv = db.load_conversation("conv_123")

    # Search
    results = db.search_conversations("python async")

    # Get statistics
    stats = db.get_statistics()

Message Types

ClassDescription
MessageSingle message with role, content, and metadata
MessageContentText content with optional media attachments
MessageRoleEnum: USER, ASSISTANT, SYSTEM, TOOL
ConversationMetadataTitle, tags, timestamps, source info

Database Schema

CTK uses SQLite with the following structure:

TableDescription
conversationsMetadata, title, timestamps, source, model
messagesContent, role, parent/child relationships
tagsSearchable tags per conversation
pathsCached conversation paths for fast retrieval

Key Indexes

  • Full-text search on message content
  • Source and model filtering
  • Date range queries
  • Tag-based filtering

Plugin System

Registry

Plugins are auto-discovered at startup:

from ctk import registry

# List available plugins
print(registry.list_importers())
print(registry.list_exporters())

# Import file
conversations = registry.import_file("export.json", format="openai")

# Export conversations
registry.export_conversations(conversations, "output.jsonl", format="jsonl")

Available Importers

FormatDescription
openaiChatGPT exports (preserves tree structure)
anthropicClaude exports
geminiGoogle Gemini/Bard
copilotGitHub Copilot from VS Code
jsonlGeneric JSONL for local LLMs
filesystem_codingAuto-detect coding agent data

Available Exporters

FormatDescription
jsonlJSONL for fine-tuning
jsonNative CTK, OpenAI, or Anthropic format
markdownHuman-readable with tree visualization
htmlInteractive HTML5 app
hugoHugo page bundles for static sites

CLI Reference

Core Commands

CommandDescription
ctk import <file>Import conversations from file
ctk listList conversations with filtering
ctk search <query>Full-text search
ctk ask <query>Natural language query (LLM-powered)
ctk show <id>Display specific conversation
ctk tree <id>View tree structure
ctk export <output>Export conversations
ctk chatLaunch interactive TUI

Organization Commands

CommandDescription
ctk star <id>Star conversation
ctk pin <id>Pin conversation
ctk archive <id>Archive conversation
ctk title <id> <title>Rename conversation

Database Commands

CommandDescription
ctk statsShow database statistics
ctk merge <db1> <db2>Merge databases
ctk diff <db1> <db2>Compare databases
ctk filterCreate filtered database
ctk pluginsList available plugins

Sanitization

CTK can remove sensitive data before sharing:

from ctk.core.sanitizer import Sanitizer, SanitizationRule
import re

sanitizer = Sanitizer(enabled=True)

# Add custom pattern
sanitizer.add_rule(SanitizationRule(
    name="internal_urls",
    pattern=re.compile(r'https://internal\.company\.com/[^\s]+'),
    replacement="[INTERNAL_URL]"
))

Built-in Patterns

  • API keys (OpenAI, Anthropic, AWS, etc.)
  • Passwords and tokens
  • SSH keys
  • Database URLs
  • Credit card numbers

Full Reference

For complete documentation: