DocArray
Open source Python library for representing and moving multimodal documents and embeddings across services for search, RAG and generative apps.
Overview
DocArray provides typed data structures for text, image, audio and video plus their embeddings, so teams can pass rich, validated objects between components without ad hoc JSON. Documents carry content, metadata and vectors, enabling clean boundaries between ingestion, preprocessing, embedding, storage and retrieval. Efficient binary serialization reduces overhead when batching across workers or microservices.
Adapters integrate with deep learning frameworks and popular vector databases, while helper ops handle chunking, merging and indexing. Because schemas are explicit, pipelines stay debuggable and reproducible, which is crucial for RAG and evaluation. The project is maintained by Jina AI and an active community, ships on PyPI, and is permissively licensed for commercial use.
Typical setups pair DocArray with embedding models and a vector store to build hybrid search, question answering and generative workflows that move reliably from notebook to production.
Key features
- Typed Document and DocumentArray classes for multimodal data
- Fast binary serialization for inter process and network transport
- Field validation and schema versions for reproducibility
- Helpers for chunking splitting and hierarchical docs
- Vector friendly ops for indexing similarity and ranking
- Integrations with PyTorch TensorFlow and ONNX runtimes
- Adapters for common vector databases and cloud stores
- Active community docs examples and release cadence
Best for
- RAG pipelines passing chunks and embeddings between steps
- Multimodal search services combining text and images
- ETL jobs moving vectors between stores during migrations
- Evaluation harnesses that track inputs outputs and scores
- Realtime inference systems that batch requests across workers
- Dataset curation with typed metadata for training
- Prototyping in notebooks that later scales to services
- Education demos that teach embeddings and retrieval patterns
Capabilities
Typed Documents
Define explicit schemas for content, embeddings and metadata so every stage of the pipeline consumes and emits compatible objects.
Efficient IO
Use binary serialization and batching to reduce overhead when moving large DocumentArrays between workers and services.
Frameworks and Vectors
Leverage adapters for deep learning frameworks and popular vector databases in cloud or on-prem environments.
Data Quality
Apply type checks and constraints to catch schema drift early and keep experiments reproducible and debuggable.
Frequently Asked Questions
What license and cost apply?
DocArray is open source and free to use in commercial projects through its permissive license and PyPI distribution.
Does it require Jina the framework?
No, DocArray can be used standalone inside any Python project; it also plays well with Jina when chosen.
Which workloads benefit most?
RAG, multimodal search, evaluation pipelines and any service that shuttles embeddings and metadata between components.
Is there GPU dependency?
The library itself is CPU-only; it integrates with GPU-accelerated frameworks and vector DBs when models require them.
How does it help with reproducibility?
Typed schemas, validation and versioning reduce hidden coupling so runs can be repeated and compared safely.
Can it connect to vector databases?
Yes, community adapters exist for popular stores, enabling insert, query and delete from DocumentArrays.
Is there a GUI included?
DocArray is code-first; most users work in Python, notebooks and services rather than a dedicated UI.
Where can teams learn quickly?
The official docs provide tutorials, examples and migration guides, and the community offers support in discussions and issues.
