Index Configuration

Self-Hosting Documentation Access

This section requires a password to access. Interested in self-hosting? Contact sales to learn more.

Password:

Access Documentation

Self-Hosting Documentation Access Granted Logout

LlamaCloud’s Index resource ties a source directory to a vector store via a sync pipeline. The destination vector store — the index target — is configured deployment-wide via environment variables on the backend, jobs, and Temporal worker pods.

Index target storage is configured separately from the platform’s metadata databases (Postgres and MongoDB under Databases and Queues). The two serve different roles: the platform DB stores users, projects, jobs; the index target stores embeddings and chunks. Keeping them separate lets you pick a vector-optimized destination without coupling it to the rest of the deployment.

Supported Vector Store Exports

Target	When to use
MongoDB	Already using MongoDB; want Search features alongside. Supported on Atlas (8.1+) and on self-managed deployments running MongoDB Search / mongot (8.2+).
Turbopuffer	High-scale vector workloads; per-namespace tenant isolation
PostgreSQL (pgvector)	No extra vector infrastructure — reuse your platform Postgres (or a dedicated one); the default for store-less (minimal profile) deployments
Azure AI Search	Already on Azure; want native hybrid search; need typed filterable metadata
Custom	Not seeing a native integration for your vector store? We encourage users to read the parsed output from a sync directly and write the export logic themselves. See our example repo for samples of multiple custom solutions.

The configured target is used for all indexes in the deployment. You can switch targets at any time by changing environment variables and re-syncing indexes. Index data is not portable across different targets, so a target switch triggers a full re-sync to the new destination.

The destination new indexes default to is set by config.defaultIndex.destination in Helm values (env DEFAULT_DIRECTORY_INDEX_DESTINATION; one of mongodb, turbopuffer, postgres, azureai_search). When unset, new indexes default to mongodb — or to postgres on deployments where MongoDB is disabled.

MongoDB

Runs against either deployment model:

• MongoDB Atlas — Search and Vector Search are built in.
• Self-managed MongoDB (Community or Enterprise) — requires MongoDB Search, the separate mongot process that runs alongside mongod.

Minimum Versions

Deployment	Minimum MongoDB version	Additional requirement
Atlas	8.1	Search / Vector Search enabled on the cluster
Self-managed	8.2	mongot deployed and attached to the replica set

Cluster Requirements

Requirement	Why it is needed
Network reachability	The backend, jobs, and Temporal worker pods all open connections. On Atlas, add your cluster egress addresses to the IP access list or front the cluster with PrivateLink / VPC peering; self-managed, make sure the pods can reach both mongod and the mongot nodes.
A dedicated database	LlamaCloud creates and writes a collection and its indexes here. Do not point this at the platform’s own MongoDB database.

On Atlas, check the Atlas Search documentation for per-tier search index limits when sizing the cluster.

Self-Managed Deployments

mongot has deployment constraints independent of LlamaCloud. It attaches to a running, authenticated replica set and cannot start against a standalone or empty cluster, and each mongot group is bound to one replica set rather than shared across several. A common production shape is three mongod members with two mongot nodes; a single co-located machine running both is fine for development. See MongoDB’s self-managed Search documentation for sizing and topology guidance.

Search Features Used

Both deployments must serve the following. MongoDB documents feature compatibility between Atlas and self-managed Search if you need to check a specific capability.

Feature	Where it is used
createSearchIndexes, listSearchIndexes	Creating the search index on first export and checking whether it already exists
$rankFusion	Retrieval — fuses the vector and full-text pipelines. Served by mongod, not mongot, so it depends on the server version rather than on Search
$search	Both retrieval pipelines: once with the vectorSearch operator for the vector leg, once with compound for the full-text leg
knnVector field type	The embedding field in the search index definition

Connection

Variable	Description
DEFAULT_INDEX_MONGO_URI	Full SRV or standard connection URI (e.g. mongodb+srv://...)
DEFAULT_INDEX_MONGO_DB	Database name
DEFAULT_INDEX_MONGO_COLLECTION	Collection name (a single collection holds all index data)

All three are required together. In Helm values they render from config.defaultIndex.mongo.*:

config:
  defaultIndex:
    destination: mongodb
    mongo:
      uri: "mongodb+srv://<user>:<password>@<cluster>.mongodb.net/?retryWrites=true&w=majority"
      db: "llamacloud_index"
      collection: "index_chunks"

Setting only some of the three fails the chart render with config.defaultIndex.mongo requires uri, db, and collection to all be set. To supply the values from a pre-existing Kubernetes secret instead of inline, set config.defaultIndex.secret to its name and omit the mongo block — the secret must then provide all three DEFAULT_INDEX_MONGO_* keys itself.

The URI is treated as a secret and stored in the rendered default-index-secret alongside the platform’s MONGODB_URL. Keep it distinct from the platform’s own MongoDB connection: use a separate cluster, or at minimum a separate database.

Database User Privileges

LlamaCloud creates the collection, regular indexes, and the search index programmatically on first use. The database user therefore needs two built-in roles scoped to the target database:

• readWrite — covers insert, delete, find, aggregate (including $search and $rankFusion), createCollection, listCollections, listIndexes, and creating regular indexes.
• dbAdmin — covers the search index commands createSearchIndexes and listSearchIndexes, which are not in readWrite.

readWrite alone is insufficient: the sync workflow fails on its first export with a privilege error when it tries to create the search index. The platform never drops indexes or collections, so dropSearchIndex and dropCollection are not required. If your security review demands tighter scoping, a custom role granting the readWrite actions plus only createSearchIndexes and listSearchIndexes is sufficient. On Atlas, see custom roles.

What LlamaCloud Creates

On the first export to a given database, LlamaCloud provisions the following if they do not already exist. Nothing needs to be pre-created, provided the user holds the roles above.

Object	Name	Purpose
Collection	your DEFAULT_INDEX_MONGO_COLLECTION	Holds chunks, metadata, and embeddings for every index in the deployment
Regular index	ix_export_config_file	Compound index on (export_config_id, parsed_directory_file_id); the incremental-sync diff uses it to list which files are already exported
Search index	search_index_v1	A single search-type index carrying both the full-text mapping and the vector mapping

The search index maps content for full-text search; _id, tenant_id, export_config_id, and parsed_directory_file_id as tokens for filtering; metadata_fields as a dynamic document so arbitrary chunk metadata is filterable; and embedding as a cosine-similarity vector field with scalar quantization.

Two consequences:

• Search index builds are asynchronous. LlamaCloud issues createSearchIndexes and continues; it does not block until the index is queryable. On a first-ever sync, retrieval can return no results for the minute or two the deployment takes to build the index, even though the export itself succeeded. Retry the query rather than re-running the sync.
• Changing the embedding model requires a new index. Vector dimensionality comes from the configured embedding model and is fixed in the index definition, so switching models means a new search index and a full re-sync. See Embedding Model below.

Data Layout and Tenancy

All indexes in the deployment share one collection. Isolation is filter-based rather than physical: every document carries tenant_id (the project) and export_config_id (the index), and every retrieval query pins both as must clauses before any user-supplied filter is applied. If you need physical separation per tenant, use Turbopuffer, which allocates one namespace per index.

Each chunk is stored as a single document holding its text, its embedding, page and character offsets, and flattened custom metadata under metadata_fields. Re-syncing a file deletes its existing chunks and re-inserts them, so a file whose chunk count changes between parses leaves no orphans. Deleting a file from the source directory removes its chunks when the index’s deletion mode is mirror; append_only leaves them in place.

Verifying and Troubleshooting

Check the target is wired up with the configz endpoint. It validates only that the environment variables are present and parseable; it never connects to the cluster, so a CONFIGURED status can still be followed by connection failures at sync time.

Symptom	Cause	Fix
503 on retrieval, with MongoDB index V2 is not configured for this environment	One or more of the three DEFAULT_INDEX_MONGO_* variables is missing on the pod	Set the missing variables — the error message names them
Sync fails on its first export with a privilege error	The database user cannot run createSearchIndexes	Grant dbAdmin alongside readWrite, or an equivalent custom role — see Database User Privileges above
Retrieval fails with an unrecognized-stage error naming $rankFusion	The server predates 8.1	Upgrade the deployment to meet Minimum Versions above
Retrieval fails with an unrecognized-stage error naming $search	The deployment has no Search — mongot is not running, or Search is disabled on the Atlas cluster	Deploy mongot, or enable Search on the cluster
Retrieval returns zero results immediately after a first sync, though the sync succeeded	The search index is still building — index creation returns before the index is queryable	Retry the query; do not re-run the sync

Turbopuffer

Requires a Turbopuffer account and API key. Each Index gets its own Turbopuffer namespace derived from the Index ID, so a single Turbopuffer account scales to arbitrarily many indexes without per-index infrastructure.

Variable	Description
DEFAULT_INDEX_TPUF_API_KEY	Turbopuffer API key
DEFAULT_INDEX_TPUF_REGION	Region for new namespaces (default aws-us-east-1)
DEFAULT_INDEX_TPUF_NAMESPACE_PREFIX	Prefix prepended to every namespace name (default llamacloud) — final namespace is {prefix}-{index_id}

PostgreSQL (pgvector)

Stores chunks and embeddings in a PostgreSQL database using the pgvector extension — no extra vector infrastructure. This is the default target for store-less (minimal profile) deployments, and the connection may point at the platform’s own Postgres or a dedicated database.

Variable	Description
DEFAULT_INDEX_PG_HOST	PostgreSQL host
DEFAULT_INDEX_PG_PORT	Port (default 5432)
DEFAULT_INDEX_PG_NAME	Database name
DEFAULT_INDEX_PG_USER	Username
DEFAULT_INDEX_PG_PASSWORD	Password

In Helm values these render from config.defaultIndex.postgres.{host,port,database,username,password}. Any field left empty inherits the matching inline postgresql.* value, so deployments that pass platform-Postgres credentials inline (including the minimal profile) need no extra configuration. If the platform Postgres is supplied via an existing secret (postgresql.secret), set the config.defaultIndex.postgres.* fields explicitly or provide config.defaultIndex.secret.

Requirements and behavior:

• The server must have the pgvector extension available. LlamaCloud runs CREATE EXTENSION IF NOT EXISTS vector on first export. pgvector is not a trusted extension, so creating it normally requires superuser-level rights — on managed services the standard mechanisms apply (Amazon RDS master user / rds_superuser, the azure.extensions allow-list on Azure, Cloud SQL’s supported-extension flow). If the LlamaCloud database user cannot create extensions, pre-create it once as an admin (CREATE EXTENSION vector;) — the app’s CREATE EXTENSION IF NOT EXISTS then no-ops.
• One table is created per embedding model, named index_v2_<embedding-model>, with an HNSW index on the embedding column. Chunk text and metadata live alongside the vector in the same row.
• Tenant isolation within the table is filter-based (per export config / tenant id), matching the other managed targets.

Azure AI Search

Requires an Azure AI Search service with vector search enabled. LlamaCloud creates one index per (export config, embedding model) on first sync.

Variable	Description
DEFAULT_INDEX_AZUREAI_SEARCH_ENDPOINT	Service endpoint, e.g. https://<service>.search.windows.net
DEFAULT_INDEX_AZUREAI_SEARCH_AUTHENTICATION_CONFIGURATION	JSON auth config; for API-key auth: {"type": "api_key", "api_key": "<admin-key>"}. The key must be allowed to create indexes.
DEFAULT_INDEX_AZUREAI_SEARCH_INDEX_NAME_PREFIX	Optional prefix for created index names (default llamacloud)

Arbitrary chunk metadata is written to a typed metadata complex collection on the index, with one entry per key: val_s (string, lowercase-normalized for case-insensitive equality), val_n (double), val_b (boolean), and val_d (datetime — populated when a string parses as ISO 8601). list[str] values expand to one entry per item. Bag fields are filter-only — Azure does not support $orderby or facet correlation across collection sub-fields — so query via OData any() lambdas:

$filter=metadata/any(m: m/key eq 'department' and m/val_s eq 'legal')
$filter=metadata/any(m: m/key eq 'created_at' and m/val_d ge 2026-01-01T00:00:00Z)

Azure caps complex-collection elements at 3000 per document; entries beyond that count are dropped silently at ingest. Adding a new top-level field to an existing index is seamless (existing documents see null), but changing the type or filterable/sortable/facetable attribute of an existing field requires dropping and recreating the index — a full re-sync.

Reranking

Index retrieval optionally runs a reranker over the top-K vector results. Reranking is enabled by default and uses Cohere rerank-v3.5 if a Cohere key is configured. To disable reranking entirely (e.g. for deployments without Cohere or Bifrost access):

Variable	Description
RERANK_ENABLED	Set to false to disable reranking globally. Defaults to true.

Cohere

Variable	Description
COHERE_PRIVATE_KEY	Cohere API key
COHERE_RERANK_LANGUAGE	One of multi, en, foreign. Defaults to multi.

Gateway (Optional)

If you’ve deployed an LLM gateway such as Bifrost or Portkey, you can route reranking through it instead of calling Cohere directly. Useful for centralized credential management or per-tenant routing.

Variable	Description
RERANK_GATEWAY_ENABLED	true to route reranking through the gateway
RERANK_GATEWAY	Gateway used for attribution headers: bifrost (default) or portkey
RERANK_GATEWAY_BASE_URL	Gateway base URL
RERANK_GATEWAY_DEFAULT_MODEL	Provider/model string (default cohere/rerank-v3.5)
RERANK_GATEWAY_DEFAULT_TOP_N	Default number of results to return (default 10)
RERANK_GATEWAY_TIMEOUT_SECONDS	Request timeout in seconds (default 30)
COHERE_PRIVATE_KEY	Credential sent as Authorization: Bearer by the Cohere SDK. Still required to build the client even when the gateway authenticates via a custom header (use a placeholder if unused).

When both a gateway and direct Cohere are configured, the gateway takes precedence.

The legacy BIFROST_RERANK_* variables remain supported as aliases. If both names are set for a field, the corresponding RERANK_GATEWAY* variable takes precedence.

Gateways that authenticate via a custom header (e.g. Portkey)

Some gateways do not accept Authorization: Bearer for gateway auth and instead require a custom header (Portkey uses x-portkey-api-key). Sending only the SDK’s bearer token makes the gateway forward that key upstream, producing a 401 from the provider (e.g. Cohere).

Reranking reuses the shared LLM_HEADERS setting (the same mechanism used for chat/embedding gateway auth). Set the gateway’s auth header there and it is attached to every rerank request alongside the observability headers:

LLM_HEADERS_ENABLED=true
LLM_HEADERS={"x-portkey-api-key": "<portkey-api-key>"}

For example, to rerank a Bedrock-hosted Cohere model through Portkey:

RERANK_GATEWAY_ENABLED=true
RERANK_GATEWAY=portkey
RERANK_GATEWAY_BASE_URL=https://<your-portkey-host>/v1
RERANK_GATEWAY_DEFAULT_MODEL=@bedrock-uswest2/cohere.rerank-v3-5
COHERE_PRIVATE_KEY=<placeholder-or-unused>
LLM_HEADERS_ENABLED=true
LLM_HEADERS={"x-portkey-api-key": "<portkey-api-key>"}

Embeddings and Chat LLM

Index sync and retrieval also need:

• An embedding model to convert chunks into vectors at sync time
• A chat LLM to answer questions against the index at query time

Both are resolved through LlamaCloud’s centralized LLM configuration. See Centralized Provider Configuration for the full setup. Models become available to Index operations as soon as the corresponding provider is configured under config.llms.* in your Helm values, or registered via config.llms.providerConfigs for fine-grained control.

Embedding Model

The embedding model is hardcoded to openai-text-embedding-3-small. It cannot be overridden per-index and there is no automatic fallback to another model.

• If openai-text-embedding-3-small is registered in centralized LLM config, sync works.
• If it is not, sync fails at workflow runtime with an EmbeddingConfigNotFoundError. There is no fallback to any other model.

This means an operator must either:

1. Configure the OpenAI provider directly (config.llms.openAi.apiKey), which auto-registers both embedding models, or
2. Register openai-text-embedding-3-small explicitly via providerConfigs — useful when routing through a gateway or a non-OpenAI-hosted endpoint (see Custom Endpoints below).

Chat LLM

The chat agent prefers OpenAI GPT-5-family models, in this order:

1. openai-gpt-5-4
2. openai-gpt-5-4-mini
3. openai-gpt-5-2
4. openai-gpt-5
5. openai-gpt-5-mini

The first available model in the list is used. If none of the preferred models is registered, the resolver falls back to any other LLM in the centralized config (matched by priority). If the centralized config holds no chat-capable LLM at all, the chat endpoint returns status=UNAVAILABLE with a message indicating that no chat LLM is registered.

Notes:

• The chat agent currently uses the OpenAI raw client, so non-OpenAI models registered through providerConfigs need to be served via an OpenAI-compatible endpoint (see Custom Endpoints).
• Index creation does not validate that a chat LLM is available — the failure surfaces only when a chat message is sent.

Custom Endpoints (OpenAI-Compatible Gateways)

Every provider in providerConfigs supports a credentials.base_url field. This makes it possible to point LlamaCloud at LiteLLM, Ollama, Azure AI Foundry proxies, or any other OpenAI-compatible endpoint while continuing to use the OpenAI model identifiers (openai-gpt-5-4, openai-text-embedding-3-small, etc.):

config:
  llms:
    providerConfigs:
      - id: "embedding-via-litellm"
        provider: "openai"
        model_id: "openai-text-embedding-3-small"
        enabled: true
        priority: 200
        credentials:
          api_key: "sk-litellm-..."
          base_url: "https://litellm.example.com/v1"
      - id: "chat-via-litellm"
        provider: "openai"
        model_id: "openai-gpt-5-4"
        enabled: true
        priority: 200
        credentials:
          api_key: "sk-litellm-..."
          base_url: "https://litellm.example.com/v1"

The same base-url override is honored for anthropic and gemini providers (set on their respective credentials blocks). Azure uses a separate credentials.endpoint field — see Centralized Provider Configuration for the full credentials schema by provider.

Verifying Configuration

After applying your Helm values, check that each target is correctly configured by hitting the configz endpoint:

curl -H "Authorization: Bearer $TOKEN" \
     -H "Project-Id: $PROJECT_ID" \
     https://<your-llamacloud-host>/api/v1/indexes/configz

The response lists which vector targets, embedding models, chat LLMs, and rerankers are available. Targets with missing or invalid env vars appear as unconfigured.