Introduction

chaotic_semantic_memory is a Rust crate for AI memory systems built on:

• 10240-bit hyperdimensional vectors - Dense binary representations for semantic similarity
• Chaotic echo-state reservoirs - Temporal sequence processing with emergent dynamics
• libSQL persistence - Durable storage with SQLite compatibility

Why Chaotic Memory?

Traditional vector databases store static embeddings. Chaotic semantic memory adds:

1. Temporal processing - Sequences are compressed into single hypervectors
2. Emergent dynamics - Reservoir chaos enables associative recall
3. Sub-symbolic reasoning - Hypervector operations are neuro-symbolic

Vector Options

This crate uses Hyperdimensional Computing (HDC) for fast lexical similarity. For semantic similarity (synonyms, paraphrases), you can:

• External embeddings: Use sentence-transformers or similar, inject via inject_concept()
• Turso native vectors: Add F32_BLOB tables to the same libSQL/Turso database this crate uses
• Hybrid approach: HDC for lexical matching + Turso vector_top_k() for semantic search

See Text Encoding for details.

Features

• Native Rust with WASM target support
• SIMD-accelerated hypervector operations
• Sparse reservoir matrices for memory efficiency
• Async I/O with Tokio
• Connection pooling for remote databases
• Export/import for data migration

Quick Example

use chaotic_semantic_memory::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    let framework = ChaoticSemanticFramework::builder()
        .without_persistence()
        .build()
        .await?;

    let concept = ConceptBuilder::new("cat".to_string()).build();
    framework.inject_concept("cat".to_string(), concept.vector.clone()).await?;

    let hits = framework.probe(concept.vector.clone(), 5).await?;
    println!("Found {} similar concepts", hits.len());
    Ok(())
}

Getting Started

Installation

Add to your Cargo.toml:

cargo add chaotic_semantic_memory

For WASM bindings:

[dependencies]
chaotic_semantic_memory = { version = "0.2", features = ["wasm"] }

Requirements

• Rust 1.85+ (MSRV)
• For WASM: wasm32-unknown-unknown target

Minimal Example

use chaotic_semantic_memory::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    // Create in-memory framework
    let framework = ChaoticSemanticFramework::builder()
        .without_persistence()
        .build()
        .await?;

    // Inject concepts
    let cat = ConceptBuilder::new("cat".to_string()).build();
    let dog = ConceptBuilder::new("dog".to_string()).build();
    
    framework.inject_concept("cat".to_string(), cat.vector.clone()).await?;
    framework.inject_concept("dog".to_string(), dog.vector.clone()).await?;

    // Create association
    framework.associate("cat".to_string(), "dog".to_string(), 0.8).await?;

    // Query similar concepts
    let hits = framework.probe(cat.vector.clone(), 5).await?;
    for (id, similarity) in hits {
        println!("{}: {:.3}", id, similarity);
    }

    Ok(())
}

With Persistence

use chaotic_semantic_memory::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    let framework = ChaoticSemanticFramework::builder()
        .with_local_db("memory.db")
        .build()
        .await?;

    // Data persists across restarts
    framework.load_replace().await?;
    
    Ok(())
}

CLI Installation

The csm binary is included with default features:

cargo install chaotic_semantic_memory --features cli
csm --help

Architecture

Overview

┌─────────────────────────────────────────────────────────┐
│                    ChaoticSemanticFramework              │
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────┐  │
│  │  Singularity │  │  Reservoir  │  │   Persistence   │  │
│  │  (Concepts)  │  │  (Temporal) │  │    (libSQL)     │  │
│  └──────┬──────┘  └──────┬──────┘  └────────┬────────┘  │
│         │                │                   │          │
│  ┌──────┴────────────────┴───────────────────┴──────┐   │
│  │                    Hyperdim                       │   │
│  │              (HVec10240 Operations)               │   │
│  └───────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────┘

Core Modules

Hyperdim (hyperdim.rs)

10240-bit binary hypervectors with SIMD-accelerated operations:

• HVec10240 - Dense binary vector type
• bundle() - Superposition (XOR + majority)
• bind() - Binding (XOR)
• permute() - Cyclic shift
• cosine_similarity() - Similarity measure

Reservoir (reservoir.rs)

Echo State Network with chaotic dynamics:

• Sparse CSR matrix (fixed degree k=64)
• Spectral radius constraint [0.9, 1.1]
• step() - Process single input
• to_hypervector() - Project state to HVec10240

Singularity (singularity.rs)

Concept storage and semantic search:

• HashMap-based concept storage
• Association graph (weighted edges)
• LRU cache for similarity queries
• Rayon-parallel similarity search

Framework (framework.rs)

High-level async orchestration:

• Builder pattern configuration
• Persistence integration
• Batch operations
• Export/import

Data Flow

Concept Injection

inject_concept(id, vector)
    → Singularity::inject(id, vector)
    → Persistence::save_concept(id, vector) [if enabled]

Semantic Query

probe(vector, top_k)
    → Singularity::find_similar(vector, top_k)
    → Returns Vec<(id, similarity)>

Temporal Processing

process_sequence(inputs)
    → Reservoir::reset()
    → Reservoir::step(input) for each input
    → Reservoir::to_hypervector()
    → Returns HVec10240

Constraints

API Reference

Prelude

Import common types:

#![allow(unused)]
fn main() {
use chaotic_semantic_memory::prelude::*;
}

This exports:

• HVec10240, ChaoticSemanticFramework, FrameworkBuilder
• Concept, ConceptBuilder, MemoryError, Result

Core Types

HVec10240

10240-bit binary hypervector.

#![allow(unused)]
fn main() {
let v1 = HVec10240::random();
let v2 = HVec10240::random();

// Operations
let bundled = HVec10240::bundle(&[v1, v2])?;
let bound = v1.bind(&v2);
let permuted = v1.permute(128);

// Similarity
let sim = v1.cosine_similarity(&v2);
}

ChaoticSemanticFramework

Main orchestration type.

#![allow(unused)]
fn main() {
let framework = ChaoticSemanticFramework::builder()
    .with_reservoir_size(50_000)
    .with_chaos_strength(0.1)
    .with_local_db("memory.db")
    .build()
    .await?;
}

Concept

Stored concept with vector and metadata.

#![allow(unused)]
fn main() {
let concept = ConceptBuilder::new("my-id".to_string())
    .with_vector(HVec10240::random())
    .with_metadata("key", "value")
    .build()?;
}

Async Operations

All framework operations are async:

#![allow(unused)]
fn main() {
// Single operations
framework.inject_concept(id, vector).await?;
framework.associate(from, to, strength).await?;
let hits = framework.probe(vector, 10).await?;

// Batch operations
framework.inject_concepts(&concepts).await?;
framework.associate_many(&edges).await?;
let results = framework.probe_batch(&vectors, 10).await?;
}

Error Handling

All fallible APIs return Result<T, MemoryError>:

#![allow(unused)]
fn main() {
match framework.probe(vector, 10).await {
    Ok(hits) => println!("Found {} hits", hits.len()),
    Err(MemoryError::NotFound { entity, id }) => eprintln!("{} not found: {}", entity, id),
    Err(e) => eprintln!("Error: {}", e),
}
}

Text Encoding

TextEncoder maps text to HVec10240 for direct use with concept injection and probing.

Why It Exists

• Deterministic text-to-vector conversion for repeatable indexing/querying
• No external model dependency
• Works in native Rust and WASM builds

Basic Usage

#![allow(unused)]
fn main() {
use chaotic_semantic_memory::encoder::TextEncoder;

let encoder = TextEncoder::new();
let vector = encoder.encode("rust async memory");
}

N-gram Encoding

N-grams improve local phrase sensitivity:

#![allow(unused)]
fn main() {
use chaotic_semantic_memory::encoder::TextEncoder;

let encoder = TextEncoder::new();
let vector = encoder.encode_with_ngrams("chaotic semantic memory", 3);
}

Framework Convenience APIs

use chaotic_semantic_memory::prelude::*;

#[tokio::main]
async fn main() -> Result<()> {
    let framework = ChaoticSemanticFramework::builder()
        .without_persistence()
        .build()
        .await?;

    framework.inject_text("doc-1", "Rust uses ownership for memory safety").await?;
    let hits = framework.probe_text("memory safety in rust", 5).await?;
    assert!(!hits.is_empty());
    Ok(())
}

Semantic Similarity Alternative

TextEncoder produces vectors for lexical similarity (same tokens, same order). For semantic similarity (synonyms, paraphrases), you have two options:

Option 1: External Embedding Model

Use sentence-transformers or similar, then inject the resulting vector:

#![allow(unused)]
fn main() {
let embedding: HVec10240 = my_model.encode("an overview of echo-state networks");
framework.inject_concept("doc-2", embedding).await?;
}

Option 2: Turso Native Vectors

This crate uses libSQL (local SQLite or remote Turso) for persistence. You can add Turso’s native F32_BLOB vector tables alongside the crate’s HDC storage:

#![allow(unused)]
fn main() {
use libsql::Builder;

// Connect to the same database this crate uses
let db = Builder::new_local("memory.db").build().await?;
let conn = db.connect()?;

conn.execute_batch("
    CREATE TABLE IF NOT EXISTS semantic_vectors (
        id TEXT PRIMARY KEY,
        embedding F32_BLOB(384)
    );
    CREATE INDEX IF NOT EXISTS semantic_idx ON semantic_vectors(
        libsql_vector_idx(embedding, 'metric=cosine')
    );
").await?;
}

Both HDC concepts and semantic vectors live in the same database. The crate manages concepts and associations tables, while you manage semantic_vectors for float-vector similarity search via vector_top_k().

Hashing Notes

• Default hashing is FNV-1a for stable cross-platform behavior.
• Switching hash algorithms changes produced vectors for the same text.
• If you persist encoder-generated vectors, re-encoding policy should be part of migration planning.

Graph Traversal

Singularity stores weighted directed associations and exposes graph traversal APIs.

Available Traversals

• neighbors(id, min_strength) for one-hop expansion
• bfs(start, config) for bounded breadth-first traversal
• shortest_path(from, to, config) for weighted Dijkstra traversal
• shortest_path_hops(from, to, config) for hop-count pathing

Example

use chaotic_semantic_memory::graph_traversal::TraversalConfig;
use chaotic_semantic_memory::prelude::{ConceptBuilder, Result};
use chaotic_semantic_memory::singularity::Singularity;

fn main() -> Result<()> {
    let mut sing = Singularity::new();
    sing.inject(ConceptBuilder::new("a").build()?)?;
    sing.inject(ConceptBuilder::new("b").build()?)?;
    sing.inject(ConceptBuilder::new("c").build()?)?;
    sing.associate("a", "b", 0.9)?;
    sing.associate("b", "c", 0.8)?;

    let cfg = TraversalConfig::default();
    let bfs = sing.bfs("a", &cfg)?;
    let path = sing.shortest_path("a", "c", &cfg)?;
    assert!(!bfs.is_empty());
    assert!(path.is_some());
    Ok(())
}

Weighted Cost Model

shortest_path converts edge strengths to costs using -ln(strength).

• Higher strength -> lower cost
• strength <= 0.0 is treated as non-traversable
• This preserves intuitive preference for stronger semantic links

WASM Support

WASM bindings expose:

• neighbors(id, min_strength)
• bfs(start)
• shortest_path(from, to)

CLI Usage

The csm binary provides command-line access to the memory system.

Installation

cargo install chaotic_semantic_memory --features cli

Commands

inject

Inject a new concept:

# Random vector
csm inject my-concept --database memory.db

# Vector from file
csm inject my-concept --from-file vector.bin --database memory.db

# Vector from stdin
echo "vector data" | csm inject my-concept --vector-source stdin --database memory.db

# With metadata
csm inject my-concept --metadata '{"key":"value"}' --database memory.db

probe

Find similar concepts:

# By concept ID
csm probe my-concept -k 10 --database memory.db

# Output as JSON
csm probe my-concept -k 10 --output-format json --database memory.db

associate

Create associations:

csm associate source target --strength 0.8 --database memory.db

export

Export memory state:

# JSON format
csm export --output backup.json --database memory.db

# Binary format (smaller)
csm export --output backup.bin --format binary --database memory.db

import

Import memory state:

# Replace existing
csm import backup.json --database memory.db

# Merge with existing
csm import backup.json --merge --database memory.db

completions

Generate shell completions:

# Bash
csm completions bash > ~/.local/share/bash-completion/completions/csm

# Zsh
csm completions zsh > ~/.zsh/completions/_csm

# Fish
csm completions fish > ~/.config/fish/completions/csm.fish

# PowerShell
csm completions powershell > $HOME\.config\powershell\completions\csm.ps1

Output Formats

Exit Codes

WASM Bindings

Building

rustup target add wasm32-unknown-unknown
cargo build --target wasm32-unknown-unknown --no-default-features --features wasm

Using wasm-pack:

wasm-pack build --target web --no-default-features --features wasm

JavaScript API

import init, { WasmFramework } from 'chaotic_semantic_memory';

await init();

// Create framework
const framework = new WasmFramework();

// Inject concept
const vector = WasmFramework.random_vector();
await framework.inject_concept('my-concept', vector);

// Query similar
const hits = await framework.probe(vector, 10);
console.log(hits); // [{id: 'my-concept', score: 1.0}]

// Create association
await framework.associate('cat', 'dog', 0.8);

// Process sequence
const sequence = [
    new Float32Array([0.1, 0.2, ...]),
    new Float32Array([0.3, 0.4, ...])
];
const resultVector = await framework.process_sequence(sequence);

// Export/Import
const exported = await framework.export_to_bytes();
await framework.import_from_bytes(exported);

TypeScript Types

interface Concept {
    id: string;
    vector: Uint8Array;
    metadata?: string;
}

interface SimilarityHit {
    id: string;
    score: number;
}

declare class WasmFramework {
    inject_concept(id: string, vector: Uint8Array): Promise<void>;
    get_concept(id: string): Promise<Concept | null>;
    delete_concept(id: string): Promise<void>;
    probe(vector: Uint8Array, topK: number): Promise<SimilarityHit[]>;
    associate(from: string, to: string, strength: number): Promise<void>;
    get_associations(id: string): Promise<SimilarityHit[]>;
    disassociate(from: string, to: string): Promise<void>;
    update_concept(id: string, vector: Uint8Array): Promise<void>;
    process_sequence(inputs: Float32Array[]): Promise<Uint8Array>;
    export_to_bytes(): Promise<Uint8Array>;
    import_from_bytes(data: Uint8Array): Promise<number>;
    neighbors(id: string, minStrength: number): Promise<string[]>;
    bfs(start: string): Promise<string[]>;
    shortest_path(from: string, to: string): Promise<string[] | null>;
}

declare class WasmFramework {
    static random_vector(): Uint8Array;
    static bundle(vectors: Uint8Array[]): Uint8Array;
    static bind(a: Uint8Array, b: Uint8Array): Uint8Array;
    static cosine_similarity(a: Uint8Array, b: Uint8Array): number;
}

Limitations

• No persistence - WASM build excludes filesystem access
• No threading - Rayon parallelization is gated
• Memory only - All data is in-memory for browser context

NPM Package

npm install @d-o-hub/chaotic_semantic_memory

Or use directly with npx:

npx @d-o-hub/chaotic_semantic_memory

Configuration

Framework Builder

#![allow(unused)]
fn main() {
let framework = ChaoticSemanticFramework::builder()
    // Reservoir settings
    .with_reservoir_size(50_000)
    .with_reservoir_input_size(10_240)
    .with_chaos_strength(0.1)
    
    // Persistence settings
    .with_local_db("memory.db")
    .with_connection_pool_size(10)
    
    // Memory limits
    .with_max_concepts(1_000_000)
    .with_max_associations_per_concept(100)
    .with_max_metadata_bytes(4096)
    
    // Cache settings
    .with_concept_cache_size(1_000)
    .with_max_cached_top_k(100)
    
    // Query limits
    .with_max_probe_top_k(10_000)
    
    .build()
    .await?;
}

Parameters

Persistence Options

No Persistence

#![allow(unused)]
fn main() {
.without_persistence()
}

In-memory only, no database.

Local SQLite

#![allow(unused)]
fn main() {
.with_local_db("memory.db")
}

Local SQLite file.

Remote Turso

#![allow(unused)]
fn main() {
.with_turso("libsql://...", auth_token)
}

Remote Turso/LibSQL database with authentication.

Performance

Benchmarks

Run benchmarks:

cargo bench --bench benchmark -- --save-baseline main
cargo bench --bench benchmark -- --baseline main

Targets

Tuning Guide

Small Workloads (<10k concepts)

#![allow(unused)]
fn main() {
let framework = ChaoticSemanticFramework::builder()
    .without_persistence()
    .with_reservoir_size(10_240)
    .with_concept_cache_size(128)
    .build()
    .await?;
}

Medium Workloads (10k-1M concepts)

#![allow(unused)]
fn main() {
let framework = ChaoticSemanticFramework::builder()
    .with_local_db("memory.db")
    .with_max_concepts(1_000_000)
    .with_concept_cache_size(1_000)
    .build()
    .await?;
}

Large Workloads (>1M concepts)

#![allow(unused)]
fn main() {
let framework = ChaoticSemanticFramework::builder()
    .with_remote_db(turso_url, auth_token)
    .with_connection_pool_size(20)
    .with_max_concepts(10_000_000)
    .with_max_probe_top_k(1_000)
    .with_concept_cache_size(10_000)
    .build()
    .await?;
}

Memory Footprint

Tips:

• Set max_concepts to enforce memory ceiling
• Use max_associations_per_concept to limit edges
• Cache hit rate improves with concept_cache_size

Optimization Techniques

SIMD Hypervector Operations

Automatic via std::simd on x86-64:

#![allow(unused)]
fn main() {
// These use SIMD when available
HVec10240::bundle(&vectors);
HVec10240::cosine_similarity(&a, &b);
}

Parallel Similarity Search

Automatic via Rayon:

#![allow(unused)]
fn main() {
// Uses all cores for similarity computation
framework.probe(vector, 100).await?;
}

Batch Operations

Prefer batch over individual:

#![allow(unused)]
fn main() {
// Slower: N round trips
for (id, vec) in concepts {
    framework.inject_concept(id, vec).await?;
}

// Faster: 1 round trip
framework.inject_concepts(&concepts).await?;
}

Connection Pooling

For remote databases:

#![allow(unused)]
fn main() {
.with_connection_pool_size(20) // More connections for concurrent access
}

Release Engineering

Overview

Releases are automated via scripts/release-manager.sh with GitHub Actions for publishing. The system uses:

• Structured logging with JSON output for CI debugging
• Exit-code validation (no fragile grep patterns)
• Rollback automation for failed releases
• Version sync across Cargo.toml and wasm/package.json

Release Manager

# Validate all gates (tests, clippy, fmt, LOC, dry-run)
scripts/release-manager.sh validate

# Prepare release (version bump, changelog, sync)
scripts/release-manager.sh prepare 0.2.0

# Publish (creates git tag, pushes - triggers GitHub release via CI)
scripts/release-manager.sh publish 0.2.0

# Full pipeline (validate + prepare + publish)
scripts/release-manager.sh full 0.2.0

# CI mode (non-interactive)
scripts/release-manager.sh full 0.2.0 --yes --log release.log

# Dry run (simulate without side effects)
scripts/release-manager.sh full 0.2.0 --dry-run

Validation Gates

The validate command checks:

Documentation Sync

The prepare command automatically updates version references across all documentation:

CI Workflows

Release (release.yml)

Triggered by git tag push (v*):

1. validate — Extract version from git tag, match Cargo.toml, dry-run publish
2. build-artifacts — Build release binary + WASM, create tarballs
3. publish-crates — cargo publish to crates.io
4. create-github-release — Upload artifacts, extract changelog notes
5. notify — Report success or failure with per-job status table
6. update-rolling-tags — Update major/minor tags (v1, v1.2)

npm Publish (npm-publish.yml)

Triggered by tag push (v*):

• Builds WASM package via wasm-pack
• Publishes @d-o-hub/chaotic_semantic_memory to npm
• Includes npm provenance (--provenance)

GitHub Pages (pages.yml)

Triggered by push to main (book/ changes):

• Builds mdBook documentation
• Generates API docs via cargo doc
• Deploys to GitHub Pages

Commit Conventions

Rollback

If a release has issues:

# Automated rollback (deletes tag + GitHub release)
scripts/release-manager.sh rollback 0.2.0

# If already published to crates.io, yank manually:
cargo yank --version 0.2.0 chaotic_semantic_memory

Security

• No long-lived API tokens — Uses CARGO_REGISTRY_TOKEN secret scoped to environment
• Concurrency control — Release workflow uses cancel-in-progress: false
• Minimal permissions — Only contents: write + id-token: write
• Branch protection — main branch requires PR with passing CI
• Provenance — npm packages include build provenance attestation
• Audit — cargo audit runs as part of validation (when installed)

Architecture Decision Records

• ADR-0039 — Release engineering strategy
• ADR-0042 — Release automation and v0.1.0 readiness