Semantic Search Roadmap

Technical roadmap for implementing production-grade semantic search and RAG capabilities in doc-agent.

Architecture

Vector Store: Provider Pattern

Support multiple vector store backends via a common interface. The vector store is decoupled from chunk storage—it only knows about IDs and embeddings.

interface VectorStoreItem {
  id: string;                           // maps to Chunk.id
  embedding: number[];
  metadata?: Record<string, unknown>;   // for filtering
}

interface VectorStoreResult {
  id: string;
  score: number;
}

interface VectorStore {
  name: string;
  insert(items: VectorStoreItem[]): Promise<void>;
  search(
    queryEmbedding: number[],
    topK: number,
    filters?: Record<string, unknown>
  ): Promise<VectorStoreResult[]>;
  delete(ids: string[]): Promise<void>;
}

Implementations:

CustomVectorStore — brute-force → HNSW
LanceDBVectorStore — baseline comparison

The search orchestrator hydrates results by joining VectorStoreResult.id against the chunks table.

Chunking Strategies

Strategy	Flag	Implementation	Best For
Line	`--chunk line`	Split on `\n`, group empty lines	Receipts, invoices
Sentence	`--chunk sentence`	NLP tokenizer	Natural text
Semantic	`--chunk semantic`	LLM-assisted boundary detection	Contracts, reports

Auto-routing by document type:

Receipts/invoices → line
Bank statements → line or sentence
Contracts/reports → semantic

Embedding Providers

interface EmbeddingProvider {
  name: string;
  dims: number;
  embed(texts: string[]): Promise<number[][]>;
}

Provider	Models	Notes
Ollama (default)	`nomic-embed-text`, `mxbai-embed-large`	Local, no API key
OpenAI	`text-embedding-3-small`	High quality
Gemini	`text-embedding-005`, `text-multilingual-embedding-002`	Multilingual support
Transformers.js	Local ONNX	Zero external deps

LLM Providers

interface LLMProvider {
  name: string;
  generate(prompt: string, options?: { system?: string }): Promise<string>;
}

Provider	Models	Notes
Ollama (default)	`llama3.2`, `mistral`	Local
OpenAI	`gpt-4o-mini`	High quality
Gemini	`gemini-1.5-flash`	Fast

Storage Model

CREATE TABLE chunks (
  id TEXT PRIMARY KEY,
  document_id INTEGER REFERENCES documents(id),
  content TEXT NOT NULL,
  metadata JSON,
  chunk_index INTEGER NOT NULL
);

Embedding storage:

Phase 1: SQLite BLOB (brute-force search)
Phase 2+: Vector store's native format (HNSW memory-mapped files)

Hybrid Search

FTS5 contentless index alongside vector search:

CREATE VIRTUAL TABLE chunks_fts USING fts5(
  content,
  content='chunks',
  content_rowid='id'
);

Both FTS5 and vector search return chunk.id, enabling fusion:

interface HybridSearchResult {
  chunk: Chunk;
  vectorScore?: number;
  keywordScore?: number;
  combinedScore: number;
  ranks: {
    vectorRank?: number;
    keywordRank?: number;
  };
}

Search modes:

--mode vector — Cosine similarity only
--mode keyword — BM25 only
--mode hybrid — RRF fusion

Reranking

interface Reranker {
  rerank(query: string, candidates: ScoredChunk[]): Promise<ScoredChunk[]>;
}

Reranker receives scored results to preserve retrieval context for debugging and score blending.

RAG Pipeline

interface RAGResponse {
  answer: string;
  chunks: RAGChunk[];
  debug?: {
    vectorResults: ScoredChunk[];
    keywordResults: ScoredChunk[];
    rerankedResults: ScoredChunk[];
    stats: {
      vectorLatencyMs: number;
      keywordLatencyMs?: number;
      rerankLatencyMs?: number;
      totalLatencyMs: number;
    };
  };
}

Exposed via:

CLI: doc search "query" --rag
MCP: search_documents tool
HTTP: POST /rag (optional)

Evaluation

interface EvalQuery {
  id: string;
  query: string;
  relevantChunkIds: string[];
  category?: string;
}

interface EvalDataset {
  name: string;
  chunks: Chunk[];
  queries: EvalQuery[];
}

interface EvalResult {
  recallAtK: Record<number, number>;
  precisionAtK: Record<number, number>;
  mrr: number;
  byCategory?: Record<string, EvalResult>;
}

Phase 1: Vector Search Core

Scope

Chunking module (line, sentence)
Embedding provider abstraction + Ollama implementation
Custom vector store with brute-force cosine similarity
chunks table in SQLite
CLI: doc ingest and doc search
Evaluation harness

File Structure

packages/vector-store/src/
├── chunking/
│   ├── types.ts
│   ├── line.ts
│   └── sentence.ts
├── embeddings/
│   ├── types.ts
│   └── ollama.ts
├── stores/
│   ├── types.ts
│   └── custom.ts
├── eval/
│   ├── types.ts
│   ├── dataset.ts
│   └── metrics.ts
├── search.ts
└── index.ts

Deliverables

Benchmarks

Chunk size vs recall@k
Embedding latency by provider

Phase 2: Hybrid Search

Scope

FTS5 integration for keyword search
BM25 scoring
Reciprocal Rank Fusion (RRF)
HNSW index
Metadata filtering

File Structure Additions

packages/vector-store/src/
├── ranking/
│   ├── bm25.ts
│   ├── rrf.ts
│   └── hybrid.ts
├── stores/
│   └── hnsw.ts

Deliverables

Benchmarks

Vector vs keyword vs hybrid recall
HNSW accuracy vs brute-force
HNSW latency vs ef parameter
Custom vs LanceDB comparison

Phase 3: RAG & Evaluation

Scope

LLM provider abstraction
Reranking
RAG engine with citations
MCP tool integration
Provider comparison

File Structure Additions

packages/vector-store/src/
├── llm/
│   ├── types.ts
│   └── ollama.ts
├── rerank/
│   ├── types.ts
│   └── ollama.ts
├── rag/
│   ├── types.ts
│   ├── engine.ts
│   └── prompts.ts

Deliverables

Benchmarks

Reranking impact on precision
Context window size vs answer quality
Embedding provider comparison (recall, latency)

Future

Types Reference

// ─────────────────────────────────────────────────────────────
// Chunking
// ─────────────────────────────────────────────────────────────

interface Chunk {
  id: string;
  documentId: string;
  content: string;
  index: number;
  metadata: {
    page?: number;
    section?: string;
    source: string;
    [key: string]: unknown;
  };
}

type ChunkingStrategy = 'line' | 'sentence' | 'semantic';

interface Chunker {
  strategy: ChunkingStrategy;
  chunk(text: string, documentId: string, metadata?: Record<string, unknown>): Chunk[];
}

// ─────────────────────────────────────────────────────────────
// Embeddings
// ─────────────────────────────────────────────────────────────

interface EmbeddingProvider {
  name: string;
  dims: number;
  embed(texts: string[]): Promise<number[][]>;
}

// ─────────────────────────────────────────────────────────────
// Vector Store
// ─────────────────────────────────────────────────────────────

interface VectorStoreItem {
  id: string;
  embedding: number[];
  metadata?: Record<string, unknown>;
}

interface VectorStoreResult {
  id: string;
  score: number;
}

interface VectorStore {
  name: string;
  insert(items: VectorStoreItem[]): Promise<void>;
  search(
    queryEmbedding: number[],
    topK: number,
    filters?: Record<string, unknown>
  ): Promise<VectorStoreResult[]>;
  delete(ids: string[]): Promise<void>;
}

// ─────────────────────────────────────────────────────────────
// LLM
// ─────────────────────────────────────────────────────────────

interface LLMProvider {
  name: string;
  generate(prompt: string, options?: { system?: string }): Promise<string>;
}

// ─────────────────────────────────────────────────────────────
// Ranking
// ─────────────────────────────────────────────────────────────

interface ScoredChunk {
  chunk: Chunk;
  vectorScore?: number;
  keywordScore?: number;
  combinedScore: number;
}

interface HybridSearchResult extends ScoredChunk {
  ranks: {
    vectorRank?: number;
    keywordRank?: number;
  };
}

interface Reranker {
  rerank(query: string, candidates: ScoredChunk[]): Promise<ScoredChunk[]>;
}

// ─────────────────────────────────────────────────────────────
// RAG
// ─────────────────────────────────────────────────────────────

interface RAGRequest {
  query: string;
  topK?: number;
  mode?: 'vector' | 'keyword' | 'hybrid';
  filters?: Record<string, unknown>;
  rerank?: boolean;
}

interface RAGChunk {
  id: string;
  content: string;
  score: number;
  source: {
    documentId: string;
    filename: string;
    page?: number;
  };
}

interface RAGResponse {
  answer: string;
  chunks: RAGChunk[];
  debug?: {
    vectorResults: ScoredChunk[];
    keywordResults: ScoredChunk[];
    rerankedResults: ScoredChunk[];
    stats: {
      vectorLatencyMs: number;
      keywordLatencyMs?: number;
      rerankLatencyMs?: number;
      totalLatencyMs: number;
    };
  };
}

// ─────────────────────────────────────────────────────────────
// Evaluation
// ─────────────────────────────────────────────────────────────

interface EvalQuery {
  id: string;
  query: string;
  relevantChunkIds: string[];
  category?: string;
}

interface EvalDataset {
  name: string;
  description?: string;
  chunks: Chunk[];
  queries: EvalQuery[];
}

interface EvalResult {
  recallAtK: Record<number, number>;
  precisionAtK: Record<number, number>;
  mrr: number;
  byCategory?: Record<string, EvalResult>;
}

CLI Reference

# Ingestion
doc ingest <file>
doc ingest <file> --chunk line|sentence|semantic
doc ingest <file> --embed-provider ollama|openai|gemini|transformers
doc ingest <file> --embed-model <model-name>

# Search
doc search <query>
doc search <query> --mode vector|keyword|hybrid
doc search <query> --vector-store custom|lancedb
doc search <query> --filter "key:value"
doc search <query> --rag
doc search <query> --rerank
doc search <query> --top-k 10
doc search <query> --json

# Evaluation
doc eval --dataset <path>
doc eval --compare ollama,openai,gemini

# Servers
doc mcp
doc serve --port 3000

References

HNSW Paper — Hierarchical Navigable Small World graphs
BM25 Explained
RRF Paper — Reciprocal Rank Fusion
RAG Paper — Retrieval-Augmented Generation

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Semantic Search Roadmap

Architecture

Vector Store: Provider Pattern

Chunking Strategies

Embedding Providers

LLM Providers

Storage Model

Hybrid Search

Reranking

RAG Pipeline

Evaluation

Phase 1: Vector Search Core

Scope

File Structure

Deliverables

Benchmarks

Phase 2: Hybrid Search

Scope

File Structure Additions

Deliverables

Benchmarks

Phase 3: RAG & Evaluation

Scope

File Structure Additions

Deliverables

Benchmarks

Future

Types Reference

CLI Reference

References

FilesExpand file tree

SEMANTIC-SEARCH-ROADMAP.md

Latest commit

History

SEMANTIC-SEARCH-ROADMAP.md

File metadata and controls

Semantic Search Roadmap

Architecture

Vector Store: Provider Pattern

Chunking Strategies

Embedding Providers

LLM Providers

Storage Model

Hybrid Search

Reranking

RAG Pipeline

Evaluation

Phase 1: Vector Search Core

Scope

File Structure

Deliverables

Benchmarks

Phase 2: Hybrid Search

Scope

File Structure Additions

Deliverables

Benchmarks

Phase 3: RAG & Evaluation

Scope

File Structure Additions

Deliverables

Benchmarks

Future

Types Reference

CLI Reference

References