Skip to content
Developer Documentation
API Reference
Data Sources

Create Data Source

POST/api/v1/data-sources

Create a new data source.

Query ParametersExpand Collapse
organization_id: optional string
project_id: optional string
Cookie ParametersExpand Collapse
session: optional string
Body ParametersJSONExpand Collapse
component: map[unknown] or CloudS3DataSource { bucket, aws_access_id, aws_access_secret, 5 more } or CloudAzStorageBlobDataSource { account_url, container_name, account_key, 8 more } or 9 more

Component that implements the data source

One of the following:
map[unknown]
CloudS3DataSource = object { bucket, aws_access_id, aws_access_secret, 5 more }
bucket: string

The name of the S3 bucket to read from.

aws_access_id: optional string

The AWS access ID to use for authentication.

aws_access_secret: optional string

The AWS access secret to use for authentication.

formatpassword
class_name: optional string
prefix: optional string

The prefix of the S3 objects to read from.

regex_pattern: optional string

The regex pattern to filter S3 objects. Must be a valid regex pattern.

s3_endpoint_url: optional string

The S3 endpoint URL to use for authentication.

supports_access_control: optional boolean
CloudAzStorageBlobDataSource = object { account_url, container_name, account_key, 8 more }
account_url: string

The Azure Storage Blob account URL to use for authentication.

container_name: string

The name of the Azure Storage Blob container to read from.

account_key: optional string

The Azure Storage Blob account key to use for authentication.

formatpassword
account_name: optional string

The Azure Storage Blob account name to use for authentication.

blob: optional string

The blob name to read from.

class_name: optional string
client_id: optional string

The Azure AD client ID to use for authentication.

client_secret: optional string

The Azure AD client secret to use for authentication.

formatpassword
prefix: optional string

The prefix of the Azure Storage Blob objects to read from.

supports_access_control: optional boolean
tenant_id: optional string

The Azure AD tenant ID to use for authentication.

CloudGoogleDriveDataSource = object { folder_id, class_name, service_account_key, supports_access_control }
folder_id: string

The ID of the Google Drive folder to read from.

class_name: optional string
service_account_key: optional map[string]

A dictionary containing secret values

supports_access_control: optional boolean
CloudOneDriveDataSource = object { client_id, client_secret, tenant_id, 6 more }
client_id: string

The client ID to use for authentication.

client_secret: string

The client secret to use for authentication.

formatpassword
tenant_id: string

The tenant ID to use for authentication.

user_principal_name: string

The user principal name to use for authentication.

class_name: optional string
folder_id: optional string

The ID of the OneDrive folder to read from.

folder_path: optional string

The path of the OneDrive folder to read from.

required_exts: optional array of string

The list of required file extensions.

supports_access_control: optional true
CloudSharepointDataSource = object { client_id, client_secret, tenant_id, 11 more }
client_id: string

The client ID to use for authentication.

client_secret: string

The client secret to use for authentication.

formatpassword
tenant_id: string

The tenant ID to use for authentication.

class_name: optional string
drive_name: optional string

The name of the Sharepoint drive to read from.

exclude_path_patterns: optional array of string

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

folder_id: optional string

The ID of the Sharepoint folder to read from.

folder_path: optional string

The path of the Sharepoint folder to read from.

get_permissions: optional boolean

Whether to get permissions for the sharepoint site.

include_path_patterns: optional array of string

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

required_exts: optional array of string

The list of required file extensions.

site_id: optional string

The ID of the SharePoint site to download from.

site_name: optional string

The name of the SharePoint site to download from.

supports_access_control: optional true
CloudSlackDataSource = object { slack_token, channel_ids, channel_patterns, 6 more }
slack_token: string

Slack Bot Token.

formatpassword
channel_ids: optional string

Slack Channel.

channel_patterns: optional string

Slack Channel name pattern.

class_name: optional string
earliest_date: optional string

Earliest date.

earliest_date_timestamp: optional number

Earliest date timestamp.

latest_date: optional string

Latest date.

latest_date_timestamp: optional number

Latest date timestamp.

supports_access_control: optional boolean
CloudNotionPageDataSource = object { integration_token, class_name, database_ids, 2 more }
integration_token: string

The integration token to use for authentication.

formatpassword
class_name: optional string
database_ids: optional string

The Notion Database Id to read content from.

page_ids: optional string

The Page ID’s of the Notion to read from.

supports_access_control: optional boolean
CloudConfluenceDataSource = object { authentication_mechanism, server_url, api_token, 11 more }
authentication_mechanism: string

Type of Authentication for connecting to Confluence APIs.

server_url: string

The server URL of the Confluence instance.

api_token: optional string

The API token to use for authentication.

formatpassword
class_name: optional string
cql: optional string

The CQL query to use for fetching pages.

failure_handling: optional FailureHandlingConfig { skip_list_failures }

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
skip_list_failures: optional boolean

Whether to skip failed batches/lists and continue processing

index_restricted_pages: optional boolean

Whether to index restricted pages.

keep_markdown_format: optional boolean

Whether to keep the markdown format.

label: optional string

The label to use for fetching pages.

page_ids: optional string

The page IDs of the Confluence to read from.

space_key: optional string

The space key to read from.

supports_access_control: optional boolean
sync_permissions: optional boolean

Whether to fetch space-level permissions (allowed users/groups) and attach them to document metadata for access control. Disable for Confluence Server/Data Center versions whose permission APIs are unavailable (e.g. the JSON-RPC API removed in Data Center 9.2.6+), which otherwise surface as 401 errors during sync.

user_name: optional string

The username to use for authentication.

CloudJiraDataSource = object { authentication_mechanism, query, api_token, 5 more }

Cloud Jira Data Source integrating JiraReader.

authentication_mechanism: string

Type of Authentication for connecting to Jira APIs.

query: string

JQL (Jira Query Language) query to search.

api_token: optional string

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
class_name: optional string
cloud_id: optional string

The cloud ID, used in case of OAuth2.

email: optional string

The email address to use for authentication.

server_url: optional string

The server url for Jira Cloud.

supports_access_control: optional boolean
CloudJiraDataSourceV2 = object { authentication_mechanism, query, server_url, 10 more }

Cloud Jira Data Source integrating JiraReaderV2.

authentication_mechanism: string

Type of Authentication for connecting to Jira APIs.

query: string

JQL (Jira Query Language) query to search.

server_url: string

The server url for Jira Cloud.

api_token: optional string

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
api_version: optional "2" or "3"

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
"2"
"3"
class_name: optional string
cloud_id: optional string

The cloud ID, used in case of OAuth2.

email: optional string

The email address to use for authentication.

expand: optional string

Fields to expand in the response.

fields: optional array of string

List of fields to retrieve from Jira. If None, retrieves all fields.

get_permissions: optional boolean

Whether to fetch project role permissions and issue-level security

requests_per_minute: optional number

Rate limit for Jira API requests per minute.

supports_access_control: optional boolean
CloudBoxDataSource = object { authentication_mechanism, class_name, client_id, 6 more }
authentication_mechanism: "developer_token" or "ccg"

The type of authentication to use (Developer Token or CCG)

One of the following:
"developer_token"
"ccg"
class_name: optional string
client_id: optional string

Box API key used for identifying the application the user is authenticating with

client_secret: optional string

Box API secret used for making auth requests.

formatpassword
developer_token: optional string

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
enterprise_id: optional string

Box Enterprise ID, if provided authenticates as service.

folder_id: optional string

The ID of the Box folder to read from.

supports_access_control: optional boolean
user_id: optional string

Box User ID, if provided authenticates as user.

name: string

The name of the data source.

source_type: "S3" or "AZURE_STORAGE_BLOB" or "GOOGLE_DRIVE" or 8 more
One of the following:
"S3"
"AZURE_STORAGE_BLOB"
"GOOGLE_DRIVE"
"MICROSOFT_ONEDRIVE"
"MICROSOFT_SHAREPOINT"
"SLACK"
"NOTION_PAGE"
"CONFLUENCE"
"JIRA"
"JIRA_V2"
"BOX"
custom_metadata: optional map[map[unknown] or array of unknown or string or 2 more]

Custom metadata that will be present on all data loaded from the data source

One of the following:
map[unknown]
array of unknown
string
number
boolean
ReturnsExpand Collapse
DataSource = object { id, component, name, 6 more }

Schema for a data source.

id: string

Unique identifier

formatuuid
component: map[unknown] or CloudS3DataSource { bucket, aws_access_id, aws_access_secret, 5 more } or CloudAzStorageBlobDataSource { account_url, container_name, account_key, 8 more } or 9 more

Component that implements the data source

One of the following:
map[unknown]
CloudS3DataSource = object { bucket, aws_access_id, aws_access_secret, 5 more }
bucket: string

The name of the S3 bucket to read from.

aws_access_id: optional string

The AWS access ID to use for authentication.

aws_access_secret: optional string

The AWS access secret to use for authentication.

formatpassword
class_name: optional string
prefix: optional string

The prefix of the S3 objects to read from.

regex_pattern: optional string

The regex pattern to filter S3 objects. Must be a valid regex pattern.

s3_endpoint_url: optional string

The S3 endpoint URL to use for authentication.

supports_access_control: optional boolean
CloudAzStorageBlobDataSource = object { account_url, container_name, account_key, 8 more }
account_url: string

The Azure Storage Blob account URL to use for authentication.

container_name: string

The name of the Azure Storage Blob container to read from.

account_key: optional string

The Azure Storage Blob account key to use for authentication.

formatpassword
account_name: optional string

The Azure Storage Blob account name to use for authentication.

blob: optional string

The blob name to read from.

class_name: optional string
client_id: optional string

The Azure AD client ID to use for authentication.

client_secret: optional string

The Azure AD client secret to use for authentication.

formatpassword
prefix: optional string

The prefix of the Azure Storage Blob objects to read from.

supports_access_control: optional boolean
tenant_id: optional string

The Azure AD tenant ID to use for authentication.

CloudGoogleDriveDataSource = object { folder_id, class_name, service_account_key, supports_access_control }
folder_id: string

The ID of the Google Drive folder to read from.

class_name: optional string
service_account_key: optional map[string]

A dictionary containing secret values

supports_access_control: optional boolean
CloudOneDriveDataSource = object { client_id, client_secret, tenant_id, 6 more }
client_id: string

The client ID to use for authentication.

client_secret: string

The client secret to use for authentication.

formatpassword
tenant_id: string

The tenant ID to use for authentication.

user_principal_name: string

The user principal name to use for authentication.

class_name: optional string
folder_id: optional string

The ID of the OneDrive folder to read from.

folder_path: optional string

The path of the OneDrive folder to read from.

required_exts: optional array of string

The list of required file extensions.

supports_access_control: optional true
CloudSharepointDataSource = object { client_id, client_secret, tenant_id, 11 more }
client_id: string

The client ID to use for authentication.

client_secret: string

The client secret to use for authentication.

formatpassword
tenant_id: string

The tenant ID to use for authentication.

class_name: optional string
drive_name: optional string

The name of the Sharepoint drive to read from.

exclude_path_patterns: optional array of string

List of regex patterns for file paths to exclude. Files whose paths (including filename) match any pattern will be excluded. Example: [‘/temp/’, ‘/backup/’, ‘.git/’, ‘.tmp$’, ’^~’]

folder_id: optional string

The ID of the Sharepoint folder to read from.

folder_path: optional string

The path of the Sharepoint folder to read from.

get_permissions: optional boolean

Whether to get permissions for the sharepoint site.

include_path_patterns: optional array of string

List of regex patterns for file paths to include. Full paths (including filename) must match at least one pattern to be included. Example: [‘/reports/’, ‘/docs/..pdf$’, ‘^Report..pdf$’]

required_exts: optional array of string

The list of required file extensions.

site_id: optional string

The ID of the SharePoint site to download from.

site_name: optional string

The name of the SharePoint site to download from.

supports_access_control: optional true
CloudSlackDataSource = object { slack_token, channel_ids, channel_patterns, 6 more }
slack_token: string

Slack Bot Token.

formatpassword
channel_ids: optional string

Slack Channel.

channel_patterns: optional string

Slack Channel name pattern.

class_name: optional string
earliest_date: optional string

Earliest date.

earliest_date_timestamp: optional number

Earliest date timestamp.

latest_date: optional string

Latest date.

latest_date_timestamp: optional number

Latest date timestamp.

supports_access_control: optional boolean
CloudNotionPageDataSource = object { integration_token, class_name, database_ids, 2 more }
integration_token: string

The integration token to use for authentication.

formatpassword
class_name: optional string
database_ids: optional string

The Notion Database Id to read content from.

page_ids: optional string

The Page ID’s of the Notion to read from.

supports_access_control: optional boolean
CloudConfluenceDataSource = object { authentication_mechanism, server_url, api_token, 11 more }
authentication_mechanism: string

Type of Authentication for connecting to Confluence APIs.

server_url: string

The server URL of the Confluence instance.

api_token: optional string

The API token to use for authentication.

formatpassword
class_name: optional string
cql: optional string

The CQL query to use for fetching pages.

failure_handling: optional FailureHandlingConfig { skip_list_failures }

Configuration for handling failures during processing. Key-value object controlling failure handling behaviors.

Example: { “skip_list_failures”: true }

Currently supports:

  • skip_list_failures: Skip failed batches/lists and continue processing
skip_list_failures: optional boolean

Whether to skip failed batches/lists and continue processing

index_restricted_pages: optional boolean

Whether to index restricted pages.

keep_markdown_format: optional boolean

Whether to keep the markdown format.

label: optional string

The label to use for fetching pages.

page_ids: optional string

The page IDs of the Confluence to read from.

space_key: optional string

The space key to read from.

supports_access_control: optional boolean
sync_permissions: optional boolean

Whether to fetch space-level permissions (allowed users/groups) and attach them to document metadata for access control. Disable for Confluence Server/Data Center versions whose permission APIs are unavailable (e.g. the JSON-RPC API removed in Data Center 9.2.6+), which otherwise surface as 401 errors during sync.

user_name: optional string

The username to use for authentication.

CloudJiraDataSource = object { authentication_mechanism, query, api_token, 5 more }

Cloud Jira Data Source integrating JiraReader.

authentication_mechanism: string

Type of Authentication for connecting to Jira APIs.

query: string

JQL (Jira Query Language) query to search.

api_token: optional string

The API/ Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
class_name: optional string
cloud_id: optional string

The cloud ID, used in case of OAuth2.

email: optional string

The email address to use for authentication.

server_url: optional string

The server url for Jira Cloud.

supports_access_control: optional boolean
CloudJiraDataSourceV2 = object { authentication_mechanism, query, server_url, 10 more }

Cloud Jira Data Source integrating JiraReaderV2.

authentication_mechanism: string

Type of Authentication for connecting to Jira APIs.

query: string

JQL (Jira Query Language) query to search.

server_url: string

The server url for Jira Cloud.

api_token: optional string

The API Access Token used for Basic, PAT and OAuth2 authentication.

formatpassword
api_version: optional "2" or "3"

Jira REST API version to use (2 or 3). 3 supports Atlassian Document Format (ADF).

One of the following:
"2"
"3"
class_name: optional string
cloud_id: optional string

The cloud ID, used in case of OAuth2.

email: optional string

The email address to use for authentication.

expand: optional string

Fields to expand in the response.

fields: optional array of string

List of fields to retrieve from Jira. If None, retrieves all fields.

get_permissions: optional boolean

Whether to fetch project role permissions and issue-level security

requests_per_minute: optional number

Rate limit for Jira API requests per minute.

supports_access_control: optional boolean
CloudBoxDataSource = object { authentication_mechanism, class_name, client_id, 6 more }
authentication_mechanism: "developer_token" or "ccg"

The type of authentication to use (Developer Token or CCG)

One of the following:
"developer_token"
"ccg"
class_name: optional string
client_id: optional string

Box API key used for identifying the application the user is authenticating with

client_secret: optional string

Box API secret used for making auth requests.

formatpassword
developer_token: optional string

Developer token for authentication if authentication_mechanism is ‘developer_token’.

formatpassword
enterprise_id: optional string

Box Enterprise ID, if provided authenticates as service.

folder_id: optional string

The ID of the Box folder to read from.

supports_access_control: optional boolean
user_id: optional string

Box User ID, if provided authenticates as user.

name: string

The name of the data source.

project_id: string
source_type: "S3" or "AZURE_STORAGE_BLOB" or "GOOGLE_DRIVE" or 8 more
One of the following:
"S3"
"AZURE_STORAGE_BLOB"
"GOOGLE_DRIVE"
"MICROSOFT_ONEDRIVE"
"MICROSOFT_SHAREPOINT"
"SLACK"
"NOTION_PAGE"
"CONFLUENCE"
"JIRA"
"JIRA_V2"
"BOX"
created_at: optional string

Creation datetime

formatdate-time
custom_metadata: optional map[map[unknown] or array of unknown or string or 2 more]

Custom metadata that will be present on all data loaded from the data source

One of the following:
map[unknown]
array of unknown
string
number
boolean
updated_at: optional string

Update datetime

formatdate-time
version_metadata: optional DataSourceReaderVersionMetadata { reader_version }

Version metadata for the data source

reader_version: optional "1.0" or "2.0" or "2.1"

The version of the reader to use for this data source.

One of the following:
"1.0"
"2.0"
"2.1"

Create Data Source

curl https://api.cloud.llamaindex.ai/api/v1/data-sources \
    -H 'Content-Type: application/json' \
    -H "Authorization: Bearer $LLAMA_CLOUD_API_KEY" \
    -d '{
          "component": {
            "foo": "bar"
          },
          "name": "name",
          "source_type": "S3"
        }'
{
  "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
  "component": {
    "foo": "bar"
  },
  "name": "name",
  "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
  "source_type": "S3",
  "created_at": "2019-12-27T18:11:19.117Z",
  "custom_metadata": {
    "foo": {
      "foo": "bar"
    }
  },
  "updated_at": "2019-12-27T18:11:19.117Z",
  "version_metadata": {
    "reader_version": "1.0"
  }
}
Returns Examples
{
  "id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
  "component": {
    "foo": "bar"
  },
  "name": "name",
  "project_id": "182bd5e5-6e1a-4fe4-a799-aa6d9a6ab26e",
  "source_type": "S3",
  "created_at": "2019-12-27T18:11:19.117Z",
  "custom_metadata": {
    "foo": {
      "foo": "bar"
    }
  },
  "updated_at": "2019-12-27T18:11:19.117Z",
  "version_metadata": {
    "reader_version": "1.0"
  }
}
Note for AI agents: this documentation is built for programmatic access. - Overview of all docs: https://developers.llamaindex.ai/llms.txt - Any page is available as raw Markdown by appending index.md to its URL — e.g. https://developers.llamaindex.ai/llamaparse/parse/getting_started/index.md - Agent-friendly REST search APIs live under https://developers.llamaindex.ai/api/ — search (BM25 full-text), grep (regex), read (fetch a page), and list (browse the doc tree). See https://developers.llamaindex.ai/llms.txt for parameters. - A hosted documentation MCP server is available at https://developers.llamaindex.ai/mcp. If you support MCP, you can ask the user to install it for browsing these docs directly (an alternative to the REST API). Setup: https://developers.llamaindex.ai/python/shared/mcp/