Connecting via the API

To offer customers an alternative to crawling, we have made several endpoints available for sharing information with our platform from external sources, or reading from our platform. These endpoints are primarily for reading, writing, updating and deleting so-called Documents.

Documents

Within the platform, Documents are the sources from which we generate answers. All documents for a customer together are what we call the knowledge base. With the API, an external party can influence this knowledge base and therefore the quality of the generated answers.

Before you start

The base URL, authentication with a Bearer token and the required headers are described on Working with the API. For the actions on this page, your token needs the permissions documents:create, documents:read, documents:update and documents:delete.

API documentation

General information and principles

The API follows the JSON:API standard on top of REST; the base URL and the required headers are described on Working with the API.

In responses you will also find the property content_cleaned. This contains the stored content after it has passed through our filter and parser. This lets you validate whether the submitted information is processed correctly.

Document schema

When you add documents to the knowledge base, you can provide the following information:

Basic fields

`external_reference`

Type: string (required)

A reference to the document within your own system. Use this field to link documents in the knowledge base to records in your own database or CMS.

`url`

Type: string (required)

The unique source URL of the document. This is how we make each document unique and distinguishable within the knowledge base. Make sure every URL is unique per document.

`mime_type`

Type: string (required)

The file type of the source. We currently support the following types:

text/html for HTML documents
application/pdf for PDF files

`title`

Type: string (optional, but recommended)

The title of the document. It is used for display in search results and helps users identify the document.

Automatic extraction: If you do not provide a title, we automatically extract the title from the document itself (for example from an <h1> tag in HTML documents).

`content`

Type: string (optional)

The full text content of the document. This content is indexed and forms the basis for searching the knowledge base.

Automatic download: If you do not provide content, we automatically download the document based on the given URL and extract the content. This is convenient when you do not want to fetch and extract the content yourself.

Please note
In the case of "mime_type": "text/html" we expect HTML that is as complete as possible and valid.

Always wrap content in full HTML tags: <html><body> ... </body></html>

Additional fields

`meta_data`

Type: object (optional)

A flexible object for additional metadata that can enrich the behavior of the knowledge base and search results. You are completely free to add your own fields here. This metadata is used to make the systems smarter and can be used to enrich search results.

Commonly used `meta_data` fields (examples):

Although there is no prescribed structure, the following fields are frequently used and recognized by the system:

author (string) - The author or creator of the document
date (string, ISO 8601 format) - Publication or modification date of the document. This date can be used to determine how up to date sources are.
language (string, ISO 639-1 code) - The language of the document (e.g. "nl", "en")
category (string) - Categorization of the document (e.g. "product documentation", "blog post", "manual")

Using custom fields:

You are completely free to add your own metadata fields that are specific to your use case. These custom fields can be used for:

Custom functionality within systems
Making your Agent and Answer system smarter
Filtering and faceting of search results
Enriching your search results

{
  "data": {
    "external_reference": "scrapedPage:9001",
    "url": "https://www.domein-met-het-nieuwe-zoeken.nl/over-ons",
    "mime_type": "text/html",
    "title": "About us",
    "content": "<html><body><p>Hello world!</p></body></html>",
    "meta_data": {
        "date": "2024-12-12T12:00:00+00:00"
    }
  }
}

Endpoints

Creating a new document

POST https://{klant}.vragen.ai/api/v1/documents

Example request body:

{
    "data": {
        "type": "documents",
        "attributes": {
            "url": "https://domein-met-het-nieuwe-zoeken.nl",
            "title": "Homepage",
            "content": "<html><body><p>SWIS has everything in house to discover what your organization needs to become digitally successful. And stay that way. Discover what you really need. Starting from a successful strategy, we conceive and design a solution. More about discovering Making what is most valuable. Our agile teams know exactly how to build such a solution. In a smart, fast and smooth process. More about making. Optimizing to stay successful. That means you need to keep turning the dials of your product or service. We help you with that. More about optimizing</p></body></html>",
            "external_reference": "foo_1",
            "mime_type": "text/html",
            "meta_data": {
                "author": "SWIS"
            }
        }
    }
}

Example response

{
    "jsonapi": {
        "version": "1.0"
    },
    "links": {
        "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
    },
    "data": {
        "type": "documents",
        "id": "9001",
        "attributes": {
            "external_reference": "foo_1",
            "url": "https://domein-met-het-nieuwe-zoeken.nl",
            "mime_type": "text/html",
            "title": "Homepage",
            "meta_data": {
                "author": "SWIS"
            },
            "content_cleaned": "..." 
        },
        "links": {
            "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
        }
    }
}

Reading an existing document

GET https://{klant}.vragen.ai/api/v1/documents/{document_id}

Example response (similar to 'Creating a new document'):

{
    "jsonapi": {
        "version": "1.0"
    },
    "links": {
        "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
    },
    "data": {
        "type": "documents",
        "id": "9001",
        "attributes": {
            "external_reference": "foobar_1",
            "url": "https://domein-met-het-nieuwe-zoeken.nl",
            "mime_type": "text/html",
            "title": "Homepage",
            "meta_data": null,
            "content_cleaned": "This is all the content of the page."
        },
        "links": {
            "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
        }
    }
}

Updating an existing document

PATCH https://{klant}.vragen.ai/api/v1/documents/{document_id}

Example request body:

{
    "data": {
        "id": "9001",
        "type": "documents",
        "attributes": {
            "url": "https://domein-met-het-nieuwe-zoeken.nl",
            "external_reference": "foo_1",
            "mime_type": "text/html",
            "title": "Homepage",
            "content": "<html><body><p>SWIS has everything in house to discover what your organization needs to become digitally successful. And stay that way. Discover what you really need. Starting from a successful strategy, we conceive and design a solution. More about discovering Making what is most valuable. Our agile teams know exactly how to build such a solution. In a smart, fast and smooth process. More about making. Optimizing to stay successful. That means you need to keep turning the dials of your product or service. We help you with that. More about optimizing</p></body></html>",
            "meta_data": {
                "author": "SWIS - test"
            }
        }
    }
}

Example response:

{
    "jsonapi": {
        "version": "1.0"
    },
    "links": {
        "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
    },
    "data": {
        "type": "documents",
        "id": "9001",
        "attributes": {
            "external_reference": "foo_1",
            "url": "https://domein-met-het-nieuwe-zoeken.nl",
            "mime_type": "text/html",
            "title": "Homepage",
            "meta_data": {
                "author": "SWIS - test"
            },
            "content_cleaned": "This is all the content of the page."
        },
        "links": {
            "self": "https://{klant}.vragen.ai/api/v1/documents/9001"
        }
    }
}

Reading all available documents

GET https://{klant}.vragen.ai/api/v1/documents

Example response body

{
    "meta": {
        "page": {
            "currentPage": 1,
            "from": 1,
            "lastPage": 901,
            "perPage": 10,
            "to": 10,
            "total": 9001
        }
    },
    "jsonapi": {
        "version": "1.0"
    },
    "links": {
        "first": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=1&page%5Bsize%5D=10",
        "last": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=634&page%5Bsize%5D=10",
        "next": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=2&page%5Bsize%5D=10"
    },
    "data": [
        {
            "type": "documents",
            "id": "1",
            "attributes": {
                "external_reference": "scrapedPage:119145",
                "url": "https://www.domein-met-het-nieuwe-zoeken.nl/nieuws",
                "mime_type": "text/html",
                "title": "Actueel | Het Nieuwe Zoeken",
                "meta_data": {
                    "date": "2023-07-19T12:00:00+00:00"
                },
                "content_cleaned": "Today we start searching, anno 2024"
            },
            "links": {
                "self": "https://{klant}.vragen.ai/api/v1/documents/1"
            }
        },
        ...
      ]
  }

The response will contain a list of documents, if any are present. The structure of these objects is the same for each document and similar to when you request a single document.

Please note: with the following information returned in the response, you can send follow-up requests to retrieve further information

{
  ... 
  "meta": {
    "page": {
        "currentPage": 1,
        "from": 1,
        "lastPage": 901,
        "perPage": 10,
        "to": 10,
        "total": 9001
    }
  },
  "links": {
      "first": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=1&page%5Bsize%5D=10",
      "last": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=634&page%5Bsize%5D=10",
      "next": "https://{klant}.vragen.ai/api/v1/documents?page%5Bnumber%5D=2&page%5Bsize%5D=10"
  },
  ...
}

Deleting a document

DELETE https://{klant}.vragen.ai/api/v1/documents/{document_id}

Support

Stuck? Email our support team at service@vragen.ai.

Didn't find what you were looking for?

Ask your question directly to vragen.ai.

You are asking an AI assistant from vragen.ai. Answers come from our own content, with the source included. Why we mention this (Dutch)

Connecting via the API

Documents

Before you start

API documentation

General information and principles

Document schema

Basic fields

external_reference

url

mime_type

title

content

Additional fields

meta_data

Commonly used meta_data fields (examples):

Using custom fields:

Endpoints

Creating a new document

Reading an existing document

Updating an existing document

Reading all available documents

Deleting a document

Support

Didn't find what you were looking for?

`external_reference`

`url`

`mime_type`

`title`

`content`

`meta_data`

Commonly used `meta_data` fields (examples):