Embedding and Rerank retrieval API

Beginner: recall and reranking have different jobs

POST /v1/embeddings maps a query and documents into vectors so a nearest-neighbor index can recall candidates from a large corpus. POST /v1/rerank reads a query and candidate list, then returns each original index and a relevance_score for precise ordering. A common RAG path is authorization filter → vector recall → Rerank → context assembly. Never retrieve unauthorized data first and attempt to hide it later.

Minimal Embedding request

Copycurl "$BASE_URL/v1/embeddings" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "text-embedding-3-small",
    "input": ["What is the refund policy?", "Cancel a subscription from the billing page."],
    "encoding_format": "float"
  }'

Each data[] item uses index to identify its input and embedding for the vector. Freeze model, dimensions, normalization, chunker version, and distance metric before indexing. Never mix vectors from different dimensions or models in one index. Batching improves throughput but needs item, token, and response-memory limits.

Minimal Rerank request

Copycurl "$BASE_URL/v1/rerank" \
  -H "Authorization: Bearer $API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jina-reranker-v3.5",
    "query": "How do I cancel a subscription?",
    "documents": ["Export usage logs.", "Open the subscription on the billing page and select cancel.", "Rotate API keys."],
    "top_n": 2,
    "return_documents": true
  }'

A result index points into the original documents; do not infer identity from return order. Setting return_documents=false reduces the body, but the application must maintain a stable candidate-ID mapping.

Chunking, indexing, and authorization

Store document ID, tenant, ACL, source location, update time, and content hash on every chunk. Apply structured authorization before vector recall, and send only authorized candidates to Rerank. Inspect ingested content for parser exploits, prompt injection indicators, malicious links, and sensitive data. Source deletion must also remove vectors, caches, and derived summaries.

Evaluation, thresholds, and migration

Build a labeled set from real queries and measure Recall@K, MRR, nDCG, no-answer accuracy, latency, and cost. relevance_score is not a universal probability, so calibrate by model and domain. A model, dimension, chunker, or distance change requires a new index. Rebuild or dual-write, compare on a canary, switch gradually, and preserve rollback.

Expert: capacity, caching, and failure modes

Bound Embedding batches by tokens, items, concurrency, and response bytes, with streaming source reads and a bounded worker queue. Include model, dimensions, normalization version, and content hash in cache keys. Reranking too many candidates increases latency and spend, so limit recall K and select top_n deliberately. Alert on empty vectors, NaN, dimension mismatch, indexing lag, stale ACLs, candidate-ID drift, and model fallback. Trace query version, index version, model, Request-ID, and cited documents in usage logs.