Analyze via the API

In addition to the CRUD endpoints for Documents, read endpoints for threads and runs are also available within our JSON:API. These endpoints are primarily intended for analysis, reporting, and quality control of conversations within your knowledge base.

With these endpoints, you can retrieve complete conversations including underlying questions, or query only questions and refine them with filters, sorting, includes, and pagination.

Within this structure, the following applies:

• a thread represents a complete conversation or chat

• a run represents an individual question-and-answer interaction within that conversation

• a thread can therefore contain multiple runs when a user asks follow-up questions within the same conversation

These endpoints are particularly useful when you want to:

• analyze conversations and questions over a specific period

• break down runs by system, agent, user, feedback, or moderation status

• build trends or quality insights based on individual runs

• retrieve a thread to review the context of a specific run

Authentication

To authorize HTTP requests to our system, we use a so-called long-lived Bearer token.

Example of an Authorization header with a Bearer token:

Authorization: Bearer 31090db4198c2bf9f9a7d768bf6107a40f102d47534cb122b0

The endpoints are available per customer under their own sub-domain, for example:

https://{customer}.vragen.ai/api/v1/

The API expects the following headers with each request:

Accept: application/vnd.api+json
Content-Type: application/vnd.api+json

Thread and Run Endpoints

There are two relevant endpoints available:

1. Retrieve Threads

GET https://{customer}.vragen.ai/api/v1/threads

This endpoint returns conversations at the thread level. A thread is the parent container of a conversation and contains the context of all runs executed within that conversation.

Using include=runs, you can directly include the corresponding runs in the same response.

If you want to retrieve a specific thread by its internal JSON:API resource ID, use:

GET https://{customer}.vragen.ai/api/v1/threads/{id}

2. Retrieve Runs

GET https://{customer}.vragen.ai/api/v1/runs

This endpoint returns runs as separate records. A run typically represents a single question with its corresponding answer within a thread.

This is useful when you primarily want to filter on properties of individual questions, answers, systems, feedback, or moderation.

If you want to retrieve a specific run by its internal JSON:API resource ID, use:

GET https://{customer}.vragen.ai/api/v1/runs/{id}

Retrieving Documents Used in an Answer

When you want to analyze which sources an answer is based on, you can also request the documents or document fragments used in a run.

The following includes are particularly relevant for this:

• include=sources

• include=searchables

• include=citedSearchables

The following applies:

• sources returns the underlying source relationships of the run

• searchables returns the documents or document fragments used for the run

• citedSearchables returns only the documents or document fragments that were actually cited in the answer

Example:

GET https://{customer}.vragen.ai/api/v1/runs/{id}?include=searchables,citedSearchables

This is especially useful when you want to:

• reconstruct which documents were used to formulate an answer

• display only the cited sources in an analysis or review interface

• connect document level and run level with each other

Example: Retrieve Documents Used in a Run

The example below is based on an actual API call, but the content has been anonymized.

Example request:

GET https://{customer}.vragen.ai/api/v1/runs/501?include=sources,searchables,citedSearchables&fields[runs]=question,answer,sources,searchables,citedSearchables&fields[run-sources]=citation_reference,relevance_score,distance_score,is_cited&fields[semantic-searchables]=title,summary,url,relevance_score,distance

Example response:

{
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/runs/501"
  },
  "data": {
    "type": "runs",
    "id": "501",
    "attributes": {
      "question": "What are the opening hours of the town hall?",
      "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1]."
    },
    "relationships": {
      "sources": {
        "data": [
          {
            "type": "run-sources",
            "id": "91"
          }
        ]
      },
      "searchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      },
      "citedSearchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      }
    },
    "links": {
      "self": "https://{customer}.vragen.ai/api/v1/runs/501"
    }
  },
  "included": [
    {
      "type": "run-sources",
      "id": "91",
      "attributes": {
        "citation_reference": "1",
        "relevance_score": 0.9994,
        "distance_score": null,
        "is_cited": true
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/run-sources/91"
      }
    },
    {
      "type": "semantic-searchables",
      "id": "9001",
      "attributes": {
        "url": "https://www.examplemunicipality.nl/townhall",
        "summary": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM.",
        "title": "Town hall opening hours",
        "relevance_score": 0.9994,
        "distance": 0
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/semantic-searchables/9001"
      }
    }
  ]
}

{
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/runs/501"
  },
  "data": {
    "type": "runs",
    "id": "501",
    "attributes": {
      "question": "What are the opening hours of the town hall?",
      "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1]."
    },
    "relationships": {
      "sources": {
        "data": [
          {
            "type": "run-sources",
            "id": "91"
          }
        ]
      },
      "searchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      },
      "citedSearchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      }
    },
    "links": {
      "self": "https://{customer}.vragen.ai/api/v1/runs/501"
    }
  },
  "included": [
    {
      "type": "run-sources",
      "id": "91",
      "attributes": {
        "citation_reference": "1",
        "relevance_score": 0.9994,
        "distance_score": null,
        "is_cited": true
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/run-sources/91"
      }
    },
    {
      "type": "semantic-searchables",
      "id": "9001",
      "attributes": {
        "url": "https://www.examplemunicipality.nl/townhall",
        "summary": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM.",
        "title": "Town hall opening hours",
        "relevance_score": 0.9994,
        "distance": 0
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/semantic-searchables/9001"
      }
    }
  ]
}

{
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/runs/501"
  },
  "data": {
    "type": "runs",
    "id": "501",
    "attributes": {
      "question": "What are the opening hours of the town hall?",
      "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1]."
    },
    "relationships": {
      "sources": {
        "data": [
          {
            "type": "run-sources",
            "id": "91"
          }
        ]
      },
      "searchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      },
      "citedSearchables": {
        "data": [
          {
            "type": "semantic-searchables",
            "id": "9001"
          }
        ]
      }
    },
    "links": {
      "self": "https://{customer}.vragen.ai/api/v1/runs/501"
    }
  },
  "included": [
    {
      "type": "run-sources",
      "id": "91",
      "attributes": {
        "citation_reference": "1",
        "relevance_score": 0.9994,
        "distance_score": null,
        "is_cited": true
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/run-sources/91"
      }
    },
    {
      "type": "semantic-searchables",
      "id": "9001",
      "attributes": {
        "url": "https://www.examplemunicipality.nl/townhall",
        "summary": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM.",
        "title": "Town hall opening hours",
        "relevance_score": 0.9994,
        "distance": 0
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/semantic-searchables/9001"
      }
    }
  ]
}

In this example:

• sources shows which source relationships are linked to the run

• searchables contains the documents or document fragments used for the answer

• citedSearchables contains the documents or document fragments actually cited in the answer

• citation_reference links the source relationship to the reference in the answer, such as [1]

Retrieve Threads with Runs Included

Use include=runs if you want to receive threads with their corresponding runs in the included section of the JSON:API response.

Example:

GET https://{customer}.vragen.ai/api/v1/threads?include=runs

If you want to retrieve a specific thread by UUID including its runs, you can filter by uuid:

GET https://{customer}.vragen.ai/api/v1/threads?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&include=runs

General Request Parameters

The following query parameters are available for both threads and runs:

include

Type: string
Optional

This allows you to include related resources in the response.

Examples:

• include=runs with threads

• include=thread with runs

sort

Type: string
Optional

Sorting fields are specified as comma-separated values. Use:

• fieldname for ascending sort

• -fieldname for descending sort

• +fieldname explicitly for ascending sort

Examples:

• ?sort=-created_at

• ?sort=created_at

page[offset]

Type: integer
Optional
Default: 0

The number of results to skip.

page[limit]

Type: integer
Optional
Default: 10

The maximum number of results to be returned.

Example:

?page[offset]=0&page[limit]=25

fields[...]

Type: sparse fieldset
Optional

This allows you to limit the number of response fields.

For threads:

fields[threads]=uuid,initiator_type,initiator_origin,user_name,runs_count,completion_time_ms_total,owner_labels,created_at

For runs:

fields[runs]=uuid,run_index,status,system_type,question,answer,owner_type,owner_name,feedback,created_at

If you use include=runs, you can combine both:

fields[threads]=uuid,runs_count,created_at&fields[runs]=uuid,question,answer,status,created_at

Filters for Threads

On the threads endpoint, you can filter using individual filter fields.

Available filters:

• filter[id]

• filter[uuid]

• filter[created_from]

• filter[created_until]

• filter[initiator_type]

• filter[system_type]

Meaning of Thread Filters

filter[uuid]

Searches for a thread by UUID.

Example:

?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913

filter[created_from] and filter[created_until]

Restrict threads by creation date. The date applies to the thread's created_at.

Example:

?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31

filter[initiator_type]

Filters by the initiator type of the thread, such as user, visitor, or system.

Example:

?filter[initiator_type]=user

filter[system_type]

Returns only threads that contain at least one run with the specified system_type.

Example:

?filter[system_type]=answerSystem

Filters for Runs

On the runs endpoint, you can filter using individual filter fields.

Available filters:

• filter[id]

• filter[uuid]

• filter[thread_id]

• filter[thread_uuid]

• filter[created_from]

• filter[created_until]

• filter[tag_ids]

• filter[feedback]

• filter[flagged]

• filter[user_ids]

• filter[agent_ids]

• filter[system_ids]

• filter[system_type]

Meaning of Run Filters

filter[thread_uuid]

Returns only runs belonging to a thread with the specified UUID.

Example:

?filter[thread_uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913

filter[created_from] and filter[created_until]

Restrict runs by creation date. The date applies to the run's created_at.

Example:

?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31

filter[tag_ids]

Returns only runs that have an annotation matching one or more specified tag IDs.

Example:

?filter[tag_ids][]=12&filter[tag_ids][]=19

filter[feedback]

Filters runs based on thumbs feedback.

Supported values:

• positive

• negative

Example:

?filter[feedback]=negative

filter[flagged]

Filters based on moderation status.

Supported values:

• true (or leaving the value blank) for only moderated runs

• false for only unmoderated runs

• all for both moderated and unmoderated runs

Examples:

• ?filter[flagged]=true

• ?filter[flagged]=false

• ?filter[flagged]=all

filter[user_ids]

Returns only runs from threads linked to one or more users.

Example:

?filter[user_ids][]=3&filter[user_ids][]=8

filter[agent_ids]

Returns only runs executed by one or more specific agents.

Example:

?filter[agent_ids][]=5

filter[system_ids]

Returns only runs belonging to one or more specific systems.

Example:

?filter[system_ids][]=2

filter[system_type]

Returns only runs with a specific system type.

Example:

?filter[system_type]=answerSystem

Sortable Fields

Not every field is sortable.

For Threads

Available:

• created_at

Example:

?sort=-created_at

For Runs

Available:

• created_at

• time_to_first_token_ms

• completion_time_ms

Example:

?sort=-completion_time_ms

Example Requests

Example 1: Retrieve threads with runs included

The example below retrieves the most recent threads, including runs, and limits the response fields.

GET https://{customer}.vragen.ai/api/v1/threads?include=runs&sort=-created_at&page[offset]=0&page[limit]=10&fields[threads]=uuid,initiator_type,user_name,runs_count,created_at&fields[runs]=uuid,run_index,status,question,answer,created_at

Example 2: Retrieve a thread by UUID including runs

GET https://{customer}.vragen.ai/api/v1/threads?filter[uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&include=runs

Example 3: Retrieve only runs within a thread

GET https://{customer}.vragen.ai/api/v1/runs?filter[thread_uuid]=5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913&sort=created_at

Example 4: Retrieve runs with multiple filters

The example below retrieves runs:

• from May 2026

• of system type answerSystem

• with negative feedback

• that have not been flagged by a moderator

GET https://{customer}.vragen.ai/api/v1/runs?filter[created_from]=2026-05-01&filter[created_until]=2026-05-31&filter[system_type]=answerSystem&filter[feedback]=negative&filter[flagged]=false&sort=-created_at&page[offset]=0&page[limit]=25

Example Response for Threads with Runs

When you use include=runs, you receive a JSON:API collection with threads in data and the corresponding runs in included.

{
  "meta": {
    "total": 1,
    "limit": 10,
    "offset": 0
  },
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/threads?include=runs"
  },
  "data": [
    {
      "type": "threads",
      "id": "3",
      "attributes": {
        "uuid": "5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913",
        "initiator_type": "user",
        "initiator_origin": "web",
        "user_name": null,
        "runs_count": 2,
        "completion_time_ms_total": 1834,
        "owner_labels": [
          "Answer module production"
        ],
        "created_at": "2026-05-15T12:00:00.000000Z",
        "updated_at": "2026-05-15T12:01:10.000000Z"
      },
      "relationships": {
        "runs": {
          "data": [
            {
              "type": "runs",
              "id": "501"
            },
            {
              "type": "runs",
              "id": "502"
            }
          ]
        }
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/threads/3"
      }
    }
  ],
  "included": [
    {
      "type": "runs",
      "id": "501",
      "attributes": {
        "uuid": "b8cbbf3b-907d-4f4d-b8f9-cc7b3a7a6771",
        "run_index": 1,
        "status": "finished",
        "failure_reason": null,
        "system_type": "answerSystem",
        "question": "What are the opening hours of the town hall?",
        "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1].",
        "flagged_by_moderator": false,
        "owner_type": "system",
        "owner_name": "Answer module",
        "owner_version_tag": "prod",
        "time_to_first_token_ms": 242,
        "completion_time_ms": 911,
        "metrics": {
          "ares_faithfulness": 98,
          "ares_answer_relevance": 96,
          "ares_context_relevance": 93
        },
        "feedback": null,
        "feedback_labels": [],
        "search_context": null,
        "created_at": "2026-05-15T12:00:05.000000Z",
        "updated_at": "2026-05-15T12:00:21.000000Z"
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/runs/501"
      }
    }
  ]
}

{
  "meta": {
    "total": 1,
    "limit": 10,
    "offset": 0
  },
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/threads?include=runs"
  },
  "data": [
    {
      "type": "threads",
      "id": "3",
      "attributes": {
        "uuid": "5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913",
        "initiator_type": "user",
        "initiator_origin": "web",
        "user_name": null,
        "runs_count": 2,
        "completion_time_ms_total": 1834,
        "owner_labels": [
          "Answer module production"
        ],
        "created_at": "2026-05-15T12:00:00.000000Z",
        "updated_at": "2026-05-15T12:01:10.000000Z"
      },
      "relationships": {
        "runs": {
          "data": [
            {
              "type": "runs",
              "id": "501"
            },
            {
              "type": "runs",
              "id": "502"
            }
          ]
        }
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/threads/3"
      }
    }
  ],
  "included": [
    {
      "type": "runs",
      "id": "501",
      "attributes": {
        "uuid": "b8cbbf3b-907d-4f4d-b8f9-cc7b3a7a6771",
        "run_index": 1,
        "status": "finished",
        "failure_reason": null,
        "system_type": "answerSystem",
        "question": "What are the opening hours of the town hall?",
        "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1].",
        "flagged_by_moderator": false,
        "owner_type": "system",
        "owner_name": "Answer module",
        "owner_version_tag": "prod",
        "time_to_first_token_ms": 242,
        "completion_time_ms": 911,
        "metrics": {
          "ares_faithfulness": 98,
          "ares_answer_relevance": 96,
          "ares_context_relevance": 93
        },
        "feedback": null,
        "feedback_labels": [],
        "search_context": null,
        "created_at": "2026-05-15T12:00:05.000000Z",
        "updated_at": "2026-05-15T12:00:21.000000Z"
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/runs/501"
      }
    }
  ]
}

{
  "meta": {
    "total": 1,
    "limit": 10,
    "offset": 0
  },
  "jsonapi": {
    "version": "1.0"
  },
  "links": {
    "self": "https://{customer}.vragen.ai/api/v1/threads?include=runs"
  },
  "data": [
    {
      "type": "threads",
      "id": "3",
      "attributes": {
        "uuid": "5d8f2d34-6a7a-4e41-8c4b-b4f4c2d8d913",
        "initiator_type": "user",
        "initiator_origin": "web",
        "user_name": null,
        "runs_count": 2,
        "completion_time_ms_total": 1834,
        "owner_labels": [
          "Answer module production"
        ],
        "created_at": "2026-05-15T12:00:00.000000Z",
        "updated_at": "2026-05-15T12:01:10.000000Z"
      },
      "relationships": {
        "runs": {
          "data": [
            {
              "type": "runs",
              "id": "501"
            },
            {
              "type": "runs",
              "id": "502"
            }
          ]
        }
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/threads/3"
      }
    }
  ],
  "included": [
    {
      "type": "runs",
      "id": "501",
      "attributes": {
        "uuid": "b8cbbf3b-907d-4f4d-b8f9-cc7b3a7a6771",
        "run_index": 1,
        "status": "finished",
        "failure_reason": null,
        "system_type": "answerSystem",
        "question": "What are the opening hours of the town hall?",
        "answer": "The town hall is open Monday to Friday from 9:00 AM to 5:00 PM [1].",
        "flagged_by_moderator": false,
        "owner_type": "system",
        "owner_name": "Answer module",
        "owner_version_tag": "prod",
        "time_to_first_token_ms": 242,
        "completion_time_ms": 911,
        "metrics": {
          "ares_faithfulness": 98,
          "ares_answer_relevance": 96,
          "ares_context_relevance": 93
        },
        "feedback": null,
        "feedback_labels": [],
        "search_context": null,
        "created_at": "2026-05-15T12:00:05.000000Z",
        "updated_at": "2026-05-15T12:00:21.000000Z"
      },
      "links": {
        "self": "https://{customer}.vragen.ai/api/v1/runs/501"
      }
    }
  ]
}

Meaning of Response Fields

Thread Fields

runs_count

The number of runs visible for this thread via the JSON:API.

completion_time_ms_total

The total processing time of the visible runs within this thread, expressed in milliseconds.

owner_labels

A list of unique labels of the systems or agents that have executed runs within the thread.

Run Fields

run_index

The sequence of the run within the thread.

question

The user question of the run.

answer

The answer returned for this run.

owner_type

Indicates whether the run was executed by a system` or `agent`.

owner_name

The name of the system or agent that owns the run.

feedback

The thumbs feedback on the run, if any.

feedback_labels

A normalized list of feedback labels derived from the feedback.

flagged_by_moderator

Indicates whether the run has been flagged by a moderator.

failure_reason

The error reason of the run, if any. For successful runs, this value is null.

searchables` and `citedSearchables

Through these relationships, you can retrieve the documents or document fragments used for the run, or those actually cited in the answer.

Practical Considerations

• Use threads?include=runs when you want to display entire conversations.

• Use runs when you primarily want to perform analysis at an individual question level.

• Use include=searchables or include=citedSearchables on runs when you also want to analyze the documents or document fragments used.

• If working with UUIDs, use filter[uuid]` or `filter[thread_uuid]`. The `{id}` in the path is the internal JSON:API resource ID.

• For filters with multiple values, such as `tag_ids`, `user_ids`, `agent_ids`, and `system_ids`, use array syntax like `filter[tag_ids][]=12`.

• The endpoints only return runs that are visible within the inbox JSON:API.

• Without an include`, a resource contains only `attributes` and `links` by default; `relationships` and `included` only appear when explicitly requested.

• Runs without timing data can simply occur in the same response; when sorting by timing fields, these records automatically fall behind results with a valid timing value.

• Not every visible response field is automatically sortable.

Support

Based on the above information and examples, you should have no trouble retrieving threads and runs via our JSON:API. If you still have questions, please reach out to our support team at info@vragen.ai.