Uncategorized

Hazina Document Store: Your Application’s Knowledge Base

Behind every RAG engine is a document store. Hazina’s Document Store is designed to be both a simple key-value store for documents and a sophisticated vector database for similarity search. Let’s explore how it works.

Architecture

The Document Store consists of three layers:

  1. Document Storage – Raw documents with metadata
  2. Chunk Storage – Processed text chunks
  3. Vector Index – Embeddings for similarity search

Basic Document Operations

var store = serviceProvider.GetRequiredService<IDocumentStore>();

// Store a document
var docId = await store.StoreAsync(new Document {
    Title = "Product Manual",
    Content = "...",
    ContentType = "text/plain",
    Tags = new[] { "documentation", "product" },
    Metadata = new Dictionary<string, object> {
        ["version"] = "2.0",
        ["department"] = "Engineering"
    }
});

// Retrieve
var doc = await store.GetAsync(docId);

// Search by tags
var docs = await store.SearchByTagsAsync(new[] { "documentation" });

// Full-text search
var results = await store.SearchAsync("product features");

Vector Similarity Search

// Find documents similar to a query
var similar = await store.FindSimilarAsync(
    query: "How do I configure the API?",
    topK: 5,
    threshold: 0.7f  // Minimum similarity score
);

foreach (var result in similar)
{
    Console.WriteLine($"{result.Score:P0} - {result.Document.Title}");
    Console.WriteLine($"  Chunk: {result.ChunkText[..100]}...");
}

SQLite Configuration

builder.Services.AddHazina(config => {
    config.Storage.UseSqlite(new SqliteSettings {
        DatabasePath = "./data/hazina.db",
        EnableWAL = true,         // Better concurrent access
        VacuumOnStartup = false   // Skip for faster startup
    });
});

PostgreSQL for Production

builder.Services.AddHazina(config => {
    config.Storage.UsePostgreSQL(new PostgresSettings {
        ConnectionString = connectionString,
        EnablePgVector = true,    // Required for vector search
        VectorDimensions = 1536  // OpenAI embedding size
    });
});

Document Lifecycle

Documents go through a clear lifecycle:

  1. Stored – Raw content saved
  2. Chunked – Split into searchable pieces
  3. Embedded – Vector embeddings generated
  4. Indexed – Available for similarity search

All steps happen automatically when you call IndexDocumentAsync.

In the next post, we’ll cover one of Hazina’s most powerful features: the Dynamic Plugin System.

Frequently Asked Questions

How does Hazina’s Document Store handle document storage and retrieval?

Hazina’s Document Store uses a three-layer architecture consisting of Document Storage for raw documents, Chunk Storage for processed text chunks, and a Vector Index for similarity search. You can store documents with metadata, retrieve them by their ID, and perform searches using tags or full-text queries.

What is vector similarity search in Hazina’s Document Store?

Vector similarity search allows users to find documents that are similar to a given query based on their vector embeddings. By specifying a query and a threshold for the minimum similarity score, you can retrieve the top K documents that match your criteria.

What configurations are available for using Hazina’s Document Store?

Hazina’s Document Store can be configured to use SQLite for development and PostgreSQL for production. The PostgreSQL setup requires enabling PgVector for vector searches and specifying the connection string and vector dimensions.

What is the document lifecycle in Hazina’s Document Store?

Documents in Hazina’s Document Store undergo a lifecycle that includes four main stages: Stored (raw content saved), Chunked (split into searchable pieces), Embedded (vector embeddings generated), and Indexed (available for similarity search). This lifecycle is automated when you call the IndexDocumentAsync method.

Terug naar overzicht
ENNL