API: How to Add Datasource Items to Swiftask

This documentation explains how to programmatically add datasource items to an existing datasource in Swiftask using the REST API.

Prerequisites

Before you can add datasource items, you need:

  1. An existing datasource created in your workspace

    • The datasource must be created from a category that supports item creation (canBeUpdatedByItemCreation = true)

    • You can view your datasources at: https://www.app.swiftask.ai/{workspaceId}/agents/data-sources/detail/{datasourceId}

    • The datasource ID is the last number in the URL (e.g., for URL https://www.app.swiftask.ai/186/agents/data-sources/detail/7216, the datasource ID is 7216)

  2. A valid API key

    • Create an API key from: https://www.app.swiftask.ai/{workspaceId}/profil/space-developper

    • Replace {workspaceId} with your actual workspace ID (e.g., 186)

    • The API key must belong to the same workspace as the datasource

    • API keys can have expiration dates - ensure your key is not expired

API Endpoint

URL: POST /api/datasource/items

Authentication: Bearer token (API Key) in the Authorization header

Content-Type: application/json

Request Format

Headers

Authorization: Bearer YOUR_API_KEY
Content-Type: application/json

Request Body Schema

{
  "parentDataSourceId": number,     // Required: ID of the parent datasource
  "data": [                        // Required: Array of items to create (minimum 1)
    {
      "categorySlug": "string",     // Optional: Category slug
      "uri": "string",              // Optional: URI for file/resource
      "name": "string",             // Optional: Item name
      "url": "string",              // Optional: URL for web resources, or files
      "chunkSize": number,          // Optional: Chunk size (minimum 64)
      "textContent": "string",      // Optional: Direct text content
      "metadata": {},               // Optional: Custom metadata object
      "textSplitter": "string",     // Optional: Text splitting method
      "headersToSplitOn": "string"  // Optional: Headers for splitting
    }
  ]
}

Field Reference

Required Fields

Field
Type
Description

parentDataSourceId

number

The ID of the existing datasource where items will be added

data

array

Array of datasource items to create (must contain at least 1 item)

Optional Item Fields

Field
Type
Constraints
Description

categorySlug

string

-

Category identifier for the item

uri

string

-

File URI or resource identifier (will be encrypted)

name

string

-

Display name for the item

url

string

-

Web URL for web-based resources, files

chunkSize

number

Minimum: 64

Size of text chunks for processing

textContent

string

-

Direct text content to be indexed

metadata

object

-

Custom metadata as JSON object

textSplitter

string

-

Method for splitting text content

headersToSplitOn

string

-

Specific headers to use for text splitting

Extended Fields (Available in Full Schema)

The system supports many additional fields for specific datasource types:

Field
Type
Description

description

string

Item description

webSiteScrapingMaxDepth

number

Maximum depth for website scraping

webSiteScrapingPreventOutsideDomain

boolean

Prevent scraping outside domain

webSiteScrapingIncludeFiles

boolean

Include files in website scraping

excludedUrls

string

URLs to exclude from scraping

jsonContentKey

string

Key for JSON content extraction

jsonJqSchema

string

JQ schema for JSON processing

restMethod

string

REST API method (GET, POST, etc.)

restHeaders

string

REST API headers (JSON string)

restParams

string

REST API parameters (JSON string)

restBody

string

REST API request body

sqlQuery

string

SQL query for database sources

sqlParams

string

SQL parameters (JSON string)

includeNumColumnsInMetadata

boolean

Include column numbers in metadata

timeout

number

Request timeout in seconds

delimiter

string

Delimiter for CSV/structured data

fetchSubUrl

boolean

Fetch sub-URLs

enablePagination

boolean

Enable pagination for API sources

pageKey

string

Pagination key field

maxPages

number

Maximum pages to fetch

startFromPage

number

Starting page number

extraConfig

object

Additional configuration as JSON

apifyDatasetId

string

Apify dataset identifier

Examples

Example 1: Adding Text Content Items

curl -X POST "https://graphql.swiftask.ai/api/datasource/items" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parentDataSourceId": 7216,
    "data": [
      {
        "name": "Company Policy Document",
        "textContent": "This is our company policy regarding remote work...",
        "metadata": {
          "department": "HR",
          "version": "1.0",
          "lastUpdated": "2024-01-15"
        }
      },
      {
        "name": "Product Documentation",
        "textContent": "Product features and specifications...",
        "chunkSize": 512,
        "metadata": {
          "product": "SwiftTask",
          "section": "features"
        }
      }
    ]
  }'

Example 2: Adding Web URL Items

curl -X POST "https://graphql.swiftask.ai/api/datasource/items" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parentDataSourceId": 7216,
    "data": [
      {
        "name": "Company Blog",
        "url": "https://company.com/blog",
        "webSiteScrapingMaxDepth": 2,
        "webSiteScrapingPreventOutsideDomain": true,
        "excludedUrls": "https://company.com/blog/private/*"
      }
    ]
  }'

Example 3: Adding Database Query Items

curl -X POST "https://graphql.swiftask.ai/api/datasource/items" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parentDataSourceId": 7216,
    "data": [
      {
        "name": "Customer Data Query",
        "sqlQuery": "SELECT name, email, created_at FROM customers WHERE active = ?",
        "sqlParams": "[true]",
        "includeNumColumnsInMetadata": true,
        "timeout": 30
      }
    ]
  }'

Example 4: Adding REST API Items

curl -X POST "https://graphql.swiftask.ai/api/datasource/items" \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "parentDataSourceId": 7216,
    "data": [
      {
        "name": "External API Data",
        "url": "https://api.external-service.com/data",
        "restMethod": "GET",
        "restHeaders": "{\"Authorization\": \"Bearer TOKEN\", \"Accept\": \"application/json\"}",
        "restParams": "{\"limit\": 100, \"page\": 1}",
        "enablePagination": true,
        "pageKey": "page",
        "maxPages": 10
      }
    ]
  }'

Response Format

Success Response (200 OK)

{
    "success": true,
    "message": "Data source items created successfully"
}

Error Responses

401 Unauthorized - Missing API Key

{
    "error": "API key is required"
}

401 Unauthorized - Invalid API Key

{
    "error": "Invalid API key"
}

401 Unauthorized - Expired API Key

{
    "error": "API Key Expired"
}

400 Bad Request - Validation Error

{
    "error": "Validation failed",
    "details": [
        {
            "message": "\"parentDataSourceId\" is required",
            "path": ["parentDataSourceId"],
            "type": "any.required"
        }
    ]
}

500 Internal Server Error

{
    "error": "You cannot create data source items for this data source"
}

Important Notes

  1. Datasource Category Compatibility: Only datasources with categories that have canBeUpdatedByItemCreation = true support item creation. If you try to add items to an incompatible datasource, you'll get an error.

  2. Workspace Access: The API key must belong to the same workspace as the target datasource. Cross-workspace access is not allowed.

  3. Indexing Process: After creating items, they are automatically indexed to the vector store for AI search capabilities. This process runs asynchronously.

  4. Credit Consumption: Creating and indexing datasource items consumes Swiftask tokens/credits from your workspace account.

  5. Encryption: Sensitive fields like uri are automatically encrypted when stored.

  6. Validation: All input data is validated according to the schema. Invalid data will result in a 400 error with detailed validation messages.

  7. Batch Limit: While there's no explicit limit on the number of items per request, very large batches may time out. Consider splitting large datasets into smaller chunks.

Troubleshooting

Common Issues

  1. "You cannot create data source items for this data source"

    • The datasource category doesn't support item creation

    • Check that your datasource was created from a compatible category

  2. "Invalid API key"

    • Verify your API key is correct

    • Ensure the API key belongs to the same workspace as the datasource

  3. "API Key Expired"

    • Check the expiration date of your API key

    • Create a new API key if needed

  4. Validation errors

    • Check that required fields are provided

    • Verify data types match the schema requirements

    • Ensure chunkSize is at least 64 if provided

Best Practices

  1. Test with Small Batches: Start with 1-2 items to verify your setup works

  2. Use Meaningful Names: Provide descriptive names for better organization

  3. Include Metadata: Add relevant metadata for better searchability

  4. Monitor Credits: Keep track of your Swiftask credit consumption

  5. Handle Errors Gracefully: Implement proper error handling in your applications

Security Considerations

  • Keep your API keys secure and never expose them in client-side code

  • Use environment variables to store API keys

  • Regularly rotate API keys

  • Set appropriate expiration dates for API keys

  • Only grant API access to trusted applications

Rate Limiting

While not explicitly documented, be mindful of rate limits. If you're making many requests, consider:

  • Adding delays between requests

  • Implementing exponential backoff for retries

  • Monitoring response times and adjusting accordingly

For questions or support, refer to the Swiftask documentation or contact support through your workspace.

PreviousShare knowledge baseNextSkills library

Last updated