Docs API

Analyzing via the API

In addition to the CRUD endpoints for Documents, our JSON:API also offers read endpoints for threads and runs. These endpoints are primarily intended for analysis, reporting and quality control of conversations within your knowledge base.

With these endpoints you can retrieve complete conversations including the underlying questions, or retrieve only questions and refine them with filters, sorting, includes and pagination.

Within this structure:

  • a thread represents a full conversation

  • a run represents a single question-answer interaction within that conversation

  • a thread can therefore contain multiple runs when a user asks follow-up questions within the same conversation

These endpoints are particularly useful when you:

  • want to analyze conversations and questions over a certain period

  • want to break down runs by system, agent, user, feedback or moderation status

  • want to build trends or quality insights based on individual runs

  • want to retrieve a thread to assess the context of a specific run

Before you start

The base URL, authentication with a Bearer token and the required headers are described on Working with the API. The endpoints below assume this setup.

Thread and run endpoints

Two relevant endpoints are available:

1. Retrieving threads

GET https://{klant}.vragen.ai/api/v1/threads

This endpoint returns conversations at the thread level. A thread is the parent container of a conversation and holds the context of all runs executed within that conversation.

With include=runs you can include the associated runs directly in the same response.

If you want to retrieve a specific thread by its internal JSON:API resource ID, use:

GET https://{klant}.vragen.ai/api/v1/threads/{id}

2. Retrieving runs

GET https://{klant}.vragen.ai/api/v1/runs

This endpoint returns runs as individual records. A run typically represents a single question with its answer within a thread.

This is useful when you mainly want to filter on properties of individual questions, answers, systems, feedback or moderation.

If you want to retrieve a specific run by its internal JSON:API resource ID, use:

GET https://{klant}.vragen.ai/api/v1/runs/{id}

Retrieving documents used in an answer

When you want to analyze which sources an answer is based on, you can also request the documents or document chunks used for a run.

The following includes are particularly relevant here:

  • include=sources

  • include=searchables

  • include=citedSearchables

Where:

  • sources returns the underlying source relations of the run

  • searchables returns the documents or document chunks that were used for the run

  • citedSearchables returns only the documents or document chunks that were actually cited in the answer

Example:

GET https://{klant}.vragen.ai/api/v1/runs/{id}?include=searchables,citedSearchables

This is especially useful when you:

  • want to reconstruct which documents were used to arrive at an answer

  • only want to show the cited sources in an analysis or review interface

  • want to connect the document level and the run level

Example: retrieving the documents used for a run

The example below is based on a real API call, but the content has been anonymized.

Example request:

GET https://{klant}.vragen.ai/api/v1/runs/501?include=sources,searchables,citedSearchables&fields[runs]=question,answer,sources,searchables,citedSearchables&fields[run-sources]=citation_reference,relevance_score,distance_score,is_cited&fields[semantic-searchables]=title,summary,url,relevance_score,distance

Example response:

{
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{klant}.vragen.ai/api/v1/runs/501"
  },
  "data": {
    "type": "runs",
    "id": "501",
    "attributes": {
      "question": "What are the town hall's opening hours?",
      "answer": "The town hall is open Monday through Friday from 09:00 to 17:00 [1]."
    },
    "relationships": {
      "sources": {
        "data": [
          {
            "type": "run-sources",
            "id": "91"
          }
        ]
      },
      "searchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      },
      "citedSearchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      }
    },
    "links": {
      "self": "https://{klant}.vragen.ai/api/v1/runs/501"
    }
  },
  "included": [
    {
      "type": "run-sources",
      "id": "91",
      "attributes": {
        "citation_reference": "1",
        "relevance_score": 0.9994,
        "distance_score": null,
        "is_cited": true
      },
      "links": {
        "self": "https://{klant}.vragen.ai/api/v1/run-sources/91"
      }
    },
    {
      "type": "semantic-searchables",
      "id": "9001",
      "attributes": {
        "url": "https://www.voorbeeldgemeente.nl/gemeentehuis",
        "summary": "The town hall is open Monday through Friday from 09:00 to 17:00.",
        "title": "Town hall opening hours",
        "relevance_score": 0.9994,
        "distance": 0
      },
      "links": {
        "self": "https://{klant}.vragen.ai/api/v1/semantic-searchables/9001"
      }
    }
  ]
}

In this example:

  • sources shows which source relations are linked to the run

  • searchables contains the documents or document chunks that were used for the answer

  • citedSearchables contains the documents or document chunks that were actually cited in the answer

  • citation_reference links the source relation to the reference in the answer, such as [1]

Retrieving threads with runs included

Use include=runs if you want to receive threads with the associated runs in the included section of the JSON:API response.

Example:

GET https://{klant}.vragen.ai/api/v1/threads?include=runs

If you want to retrieve a specific thread by UUID including runs, you can filter on uuid:

GET https://{klant}.vragen.ai/api/v1/threads?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&include=runs

General request parameters

The following query parameters are available for both threads and runs:

include

Type: string
Optional

This lets you include related resources in the response.

Examples:

  • include=runs for threads

  • include=thread for runs

sort

Type: string
Optional

Sort fields are provided comma-separated. Use:

  • fieldname for ascending sorting

  • -fieldname for descending sorting

  • +fieldname explicitly for ascending sorting

Examples:

  • ?sort=-created_at

  • ?sort=created_at

page[offset]

Type: integer
Optional
Default: 0

The number of results to skip.

page[limit]

Type: integer
Optional
Default: 10

The maximum number of results to return.

Example:

?page[offset]=0&page[limit]=25

fields[...]

Type: sparse fieldset
Optional

This lets you limit the number of response fields.

For threads:

fields[threads]=uuid,initiator_type,initiator_origin,user_name,runs_count,completion_time_ms_total,owner_labels,created_at

For runs:

fields[runs]=uuid,run_index,status,system_type,question,answer,owner_type,owner_name,feedback,created_at

If you use include=runs, you can combine both:

fields[threads]=uuid,runs_count,created_at&fields[runs]=uuid,question,answer,status,created_at

Filters for threads

On the threads endpoint you can filter using individual filter fields.

Available filters:

  • filter[id]

  • filter[uuid]

  • filter[created_from]

  • filter[created_until]

  • filter[initiator_type]

  • filter[system_type]

Meaning of thread filters

filter[uuid]

Looks up a thread by UUID.

Example:

?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913

filter[created_from] and filter[created_until]

Limit threads by creation date. The date is applied to the thread's created_at.

Example:

?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31

filter[initiator_type]

Filters by the type of initiator of the thread, for example a user, visitor or system.

Example:

?filter[initiator_type]=user

filter[system_type]

Returns only threads that contain at least one run with the given system_type.

Example:

?filter[system_type]=answerSystem

Filters for runs

On the runs endpoint you can filter using individual filter fields.

Available filters:

  • filter[id]

  • filter[uuid]

  • filter[thread_id]

  • filter[thread_uuid]

  • filter[created_from]

  • filter[created_until]

  • filter[tag_ids]

  • filter[feedback]

  • filter[flagged]

  • filter[user_ids]

  • filter[agent_ids]

  • filter[system_ids]

  • filter[system_type]

Meaning of run filters

filter[thread_uuid]

Returns only runs that belong to a thread with the given UUID.

Example:

?filter[thread_uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913

filter[created_from] and filter[created_until]

Limit runs by creation date. The date is applied to the run's created_at.

Example:

?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31

filter[tag_ids]

Returns only runs that have an annotation with one or more of the given tag IDs.

Example:

?filter[tag_ids][]=12&filter[tag_ids][]=19

filter[feedback]

Filters runs by thumbs feedback.

Supported values:

  • positive

  • negative

Example:

?filter[feedback]=negative

filter[flagged]

Filters by moderation status.

Supported values:

  • true, or omitting the value, for moderated runs only

  • false for non-moderated runs only

  • all for both moderated and non-moderated runs

Examples:

  • ?filter[flagged]=true

  • ?filter[flagged]=false

  • ?filter[flagged]=all

filter[user_ids]

Returns only runs from threads that are linked to one or more users.

Example:

?filter[user_ids][]=3&filter[user_ids][]=8

filter[agent_ids]

Returns only runs that were executed by one or more specific agents.

Example:

?filter[agent_ids][]=5

filter[system_ids]

Returns only runs that belong to one or more specific systems.

Example:

?filter[system_ids][]=2

filter[system_type]

Returns only runs with a certain system type.

Example:

?filter[system_type]=answerSystem

Sortable fields

Not every field is sortable.

For threads

Available:

  • created_at

Example:

?sort=-created_at

For runs

Available:

  • created_at

  • time_to_first_token_ms

  • completion_time_ms

Example:

?sort=-completion_time_ms

Example requests

Example 1: retrieving threads with runs included

The example below retrieves the most recent threads, including runs, and limits the response fields.

GET https://{klant}.vragen.ai/api/v1/threads?include=runs&sort=-created_at&page[offset]=0&page[limit]=10&fields[threads]=uuid,initiator_type,user_name,runs_count,created_at&fields[runs]=uuid,run_index,status,question,answer,created_at

Example 2: retrieving a thread by UUID including runs

GET https://{klant}.vragen.ai/api/v1/threads?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&include=runs

Example 3: retrieving only the runs within a thread

GET https://{klant}.vragen.ai/api/v1/runs?filter[thread_uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&sort=created_at

Example 4: retrieving runs with multiple filters

The example below retrieves runs:

  • from May 2026

  • of system type answerSystem

  • with negative feedback

  • that have not been flagged by a moderator

GET https://{klant}.vragen.ai/api/v1/runs?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31&filter[system_type]=answerSystem&filter[feedback]=negative&filter[flagged]=false&sort=-created_at&page[offset]=0&page[limit]=25

Example response for threads with runs

When you use include=runs, you get a JSON:API collection with threads in data and the associated runs in included.

{
  "meta": {
    "total": 1,
    "limit": 10,
    "offset": 0
  },
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{klant}.vragen.ai/api/v1/threads?include=runs"
  },
  "data": [
    {
      "type": "threads",
      "id": "3",
      "attributes": {
        "uuid": "5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913",
        "initiator_type": "user",
        "initiator_origin": "web",
        "user_name": null,
        "runs_count": 2,
        "completion_time_ms_total": 1834,
        "owner_labels": [
          "Answer module production"
        ],
        "created_at": "2026-05-15T12:00:00.000000Z",
        "updated_at": "2026-05-15T12:01:10.000000Z"
      },
      "relationships": {
        "runs": {
          "data": [
            {
              "type": "runs",
              "id": "501"
            },
            {
              "type": "runs",
              "id": "502"
            }
          ]
        }
      },
      "links": {
        "self": "https://{klant}.vragen.ai/api/v1/threads/3"
      }
    }
  ],
  "included": [
    {
      "type": "runs",
      "id": "501",
      "attributes": {
        "uuid": "b8cbbf3b-907d-4f4d-b8f9-cc7b3a7a6771",
        "run_index": 1,
        "status": "finished",
        "failure_reason": null,
        "system_type": "answerSystem",
        "question": "What are the town hall's opening hours?",
        "answer": "The town hall is open Monday through Friday from 09:00 to 17:00 [1].",
        "flagged_by_moderator": false,
        "owner_type": "system",
        "owner_name": "Answer module",
        "owner_version_tag": "prod",
        "time_to_first_token_ms": 242,
        "completion_time_ms": 911,
        "metrics": {
          "ares_faithfulness": 98,
          "ares_answer_relevance": 96,
          "ares_context_relevance": 93
        },
        "feedback": null,
        "feedback_labels": [],
        "search_context": null,
        "created_at": "2026-05-15T12:00:05.000000Z",
        "updated_at": "2026-05-15T12:00:21.000000Z"
      },
      "links": {
        "self": "https://{klant}.vragen.ai/api/v1/runs/501"
      }
    }
  ]
}

Meaning of response fields

Thread fields

runs_count

The number of runs visible for this thread via the JSON:API.

completion_time_ms_total

The total processing time of the visible runs within this thread, expressed in milliseconds.

owner_labels

A list of unique labels of the systems or agents that have executed runs within the thread.

Run fields

run_index

The position of the run within the thread.

question

The user question of the run.

answer

The answer returned for this run.

owner_type

Indicates whether the run was executed by a system or agent.

owner_name

The name of the system or agent that owns the run.

feedback

The thumbs feedback on the run, if present.

feedback_labels

A normalized list of feedback labels derived from the feedback.

flagged_by_moderator

Indicates whether the run has been flagged by a moderator.

failure_reason

The failure reason of the run, if present. For successful runs this value is null.

searchables and citedSearchables

Through these relations you can retrieve the documents or document chunks that were used for the run, or that were actually cited in the answer.

Practical points of attention

  • Use threads?include=runs when you want to display complete conversations.

  • Use runs when you mainly want to analyze at the individual question level.

  • Use include=searchables or include=citedSearchables on runs when you also want to analyze the documents or document chunks that were used.

  • If you work with UUIDs, use filter[uuid] or filter[thread_uuid]. The {id} in the path is the internal JSON:API resource ID.

  • For filters with multiple values, such as tag_ids, user_ids, agent_ids and system_ids, use array syntax such as filter[tag_ids][]=12.

  • The endpoints only return runs that are visible within the inbox JSON:API.

  • Without include, a resource contains only attributes and links by default; relationships and included only appear when you explicitly request them.

  • Runs without timing data can still appear in the same response; when sorting by timing fields, these records automatically fall behind results with a valid timing value.

  • Not every visible response field is automatically sortable.

Support

Stuck? Email our support team at service@vragen.ai.

Didn't find what you were looking for?

Ask your question directly to vragen.ai.

What exactly is vragen.ai?

Example answer by vragen.ai

vragen.ai lets visitors ask their question on your website and gives them a reliable answer straight from your own content, with the source included. You decide which sources the AI uses.

Source: How it works

This is an example. The interactive widget could not load here, for instance because of a script blocker or a slow connection.

Request a demo Try it yourself

You are asking an AI assistant from vragen.ai. Answers come from our own content, with the source included. Why we mention this (Dutch)