🦛 CHONK docs with Chonkie ✨ — The lightweight ingestion library for fast, efficient and robust RAG pipelines
🦛 CHONK your texts with Chonkie ✨ Type-friendly, light-weight, fast and super-simple chunking library
FastCDC implementation in Python https://pypi.org/project/fastcdc/
Your RAG pipeline is broken and you don't know it. Chunky lets you validate your Markdown and choose the best chunking strategy before indexing.
SmartChunk is a lightweight, structure-aware semantic chunking toolkit designed to supercharge RAG (Retrieval-Augmented Generation) and LLM pipelines. Unlike naive splitters that break text arbitrarily, SmartChunk respects document structure (headings, lists, tables, code blocks) and semantic flow, ensuring cleaner, more coherent chunks.
Go implementation of the AE chunking algorithm.
Chunkpad is designed to prepare documents for Retrieval-Augmented Generation (RAG) pipelines and AI applications.
Android Resumable Uploads SDK from Fastpix
A Rust library for Content-Defined Chunking (CDC).
Implementation of an interactive chatbot for summarizing legal and policy documents. Includes data preprocessing (cleaning, tokenization, hierarchical chunking), extractive TF-IDF baselines, and fine-tuned abstractive models (DistilBART, LED). Integrates a retrieval layer for document relevance and uses ROUGE, BLEU, and cosine similarity metrics.
A nodejs chunking system
A high-performance FastCDC 2020 implementation written in Python + Cython
Explore and benchmark the world of data chunking algorithms in 'ChunkingChampions' - a competitive arena to determine the most efficient and effective chunking strategies for varied data sizes.
[2024-2] AI 기반 회의 지원 플랫폼 서비스 "Clerker"
A smol Go package for splitting text into chunks while preserving semantic meaning.
This is my curated repo regarding Retrieval Augmented Generation(RAG).
MS PowerPoint extension created using Yoman generator and React JS
Lightweight Semantic Chunking Library. Plug any embedding provider/API. Batch embeddings for efficiency and handling API rate limits.
Add a description, image, and links to the chunking-algorithm topic page so that developers can more easily learn about it.
To associate your repository with the chunking-algorithm topic, visit your repo's landing page and select "manage topics."