Docs On your website

Drupal module

With this Drupal module you synchronize the content of your Drupal site to your vragen.ai environment. That is what the module is primarily for: your content is automatically available in vragen.ai and gets updated whenever something changes on your site, without you having to supply it separately.

In addition, you can use vragen.ai as a search backend behind the Search API module. You then replace your site's default search with semantic search, with support for facets, filters and "similar content" (more like this).

What you need

Installation steps

  1. Install the module.

    # Install the official module from drupal.org
composer require drupal/search_api_vragen_ai

# Enable the module
drush pm:enable search_api_vragen_ai

  2. Create a new Search API server with the Vragen.ai backend (via Manage → Configuration → Search and metadata → Search API). You configure the connection to your vragen.ai environment directly on this server. There is no separate settings page anymore; everything is in the server form:

    1. API Endpoint and API Bearer Token: the endpoint and token from your vragen.ai environment. The endpoint has the form https://[account].vragen.ai/api/v1.

    2. Search System: once the endpoint and token are correct, the module loads the available search systems from your environment. Choose the search system vragen.ai should use when searching, or leave it empty to search without a specific system.

    3. Language fallback: leave this enabled if you want a result to be shown in the site's default language when it does not exist in the requested language (see Multilingual content).

  3. Create a new index for this server and adjust the following settings:

    1. Datasources → "Content" (and "Media" if you also want to index PDFs, see Indexing PDFs and documents).

    2. Bundles for Content → the content types you want indexed.

    3. Languages → all languages you want to search. Multiple languages are fine; the module handles them properly (see Multilingual content).

    4. Index items immediately: "off". (You can also leave this on, but then Drupal sends an HTTP request to vragen.ai on every change instead of handling it via cron.)

  4. Configure the fields.

    We recommend rendering items to HTML. That way vragen.ai takes the semantics of the HTML markup into account. You can configure a view mode in which you render only the fields you actually want indexed.

    The module sends your content to vragen.ai in two ways:

    • Semantic content: the content fields are combined into a single HTML document. This is the text that semantic search runs on and that the AI builds an answer from. You cannot filter on it or create facets on it.

    • Metadata: every indexed field (except attachments) is also sent along as a separate metadata field, under its own machine name. You use metadata to filter on, to create facets and to show extra data in the search results (see Filtering and facets).

    By default, every suitable field goes both ways: it is rendered as semantic content and sent along as metadata. If you want to control which fields make up the searchable text, enable the Vragen.ai semantic content processor (under the index's Processors) and check the fields that should become content. That way you keep, for example, author and publication date available as a filter or facet, without them polluting the search text.

    Besides your own fields, the module adds a few metadata fields it extracts from the content itself: the author, the creation date (date), the entity type and bundle, and the languages the content exists in (languages).

  5. Have the entire website indexed once to build the index.

Indexing PDFs and documents

If you want to index PDF documents in addition to content, there are two ways to do so:

  • Via the "Media" datasource. Enable "Media" as a datasource as well and, in that datasource's settings, select the bundle containing the PDFs. vragen.ai then ingests the documents as separate items.

  • As an attachment to content. Add a field of type Vragen.ai attachment to your index and enable the Vragen.ai attachment files processor. The module then walks through the file and reference fields of your content (including nested paragraphs and referenced entities) and attaches the files it finds to the document. That way a PDF stays linked to the page it belongs to, instead of ending up in the index as a standalone document.

Multilingual content

vragen.ai and the module are fully multilingual, so feel free to enable multiple languages on the index. One document is kept per piece of content; the translations are linked to that document, so no duplicate content ends up in the index.

  • When searching, the module filters on the visitor's language.

  • If Language fallback is enabled (the default), the visitor sees the result in the site's default language when the requested language is missing. If you disable it, the module only shows results that actually exist in the requested language.

On the old version of the module you could only index one language. That is no longer necessary: from now on, index all languages you want to search.

Tuning search results

The backend has a few optional settings to fine-tune the search results. You can find them in the server configuration; leave them empty for vragen.ai's defaults.

  • Alpha (0–1): determines the balance in hybrid search. 1 is fully semantic, 0 fully keyword-based.

  • Max distance and Distance (0–1): filter out results whose semantic distance is too large. Lower is stricter. Distance applies to "similar content" queries (more like this), max distance to regular searches.

If you work with Views, you can also set these three values per view (under the Vragen.ai → Query settings section in the view). That way, for example, a "related articles" view can be tuned differently from the main search page.

The module also supports "similar content" (more like this), so you can show related items via the Search API.

Filtering and facets

Filtering and facets work only on metadata. The semantic content (the rendered HTML from step 4) is exclusively for semantic search and the AI answer. You cannot filter on it or create facets on it.

If you want to filter on a field or turn it into a facet, that field needs to be sent along as metadata. This happens automatically for every indexed field (see step 4): in practice you simply add the field to the index and it is immediately available. You then set up facets with the Facets module; the OR operator is supported as well. For example, you get the languages field automatically, and on multilingual sites the module already pre-filters on it.

Note: every field you index is sent along as metadata, including the fields rendered as semantic content. A long text field such as the body then appears twice in the document: processed into the content, and as a raw value in the metadata. Filtering on such a long field rarely makes sense. For filters and facets, prefer short, well-defined fields such as category, type or date.

Caching of search results

Search results and "similar content" results are stored in Drupal's cache, so the same query is not sent to vragen.ai on every page view. The retention time defaults to one hour and can be configured via the search_cache_ttl key in the backend configuration (there is no UI for it; change it via exported config or a recipe). Note: the cache only expires based on time. Indexing or deleting items does not clear the cache, so a change only becomes visible in the search results after the retention time has passed. If your content changes often and freshness matters, lower the retention time.

Embedding Vragen.ai

The module handles indexing and semantic search. If you also want to let visitors ask questions directly, place the widget using the embed code from The widget on your website. In Drupal you can put that code in a custom block, for example (with a text format that allows scripts, such as "Full HTML"), or directly in a Twig template of your theme. There you will also find how to style the widget to match your branding and include it in your Content Security Policy.

For developers: modifying documents

If you want to make changes just before a document is sent to vragen.ai, or skip certain items, there is a PostCreateIndexDocumentEvent event. An event subscriber receives the Search API item, the built document and the index. With shouldIndex(FALSE) you skip an item (an existing document is then removed from the index), and you can extend or modify the document itself.

Frequently asked questions

I have changed a lot on my website, can I delete my index in one go?

We have disabled support for completely emptying the index. We saw this was often done by accident, and rebuilding all vectors is an intensive process. Contact us if you still need this.

Support

Can't figure it out? Email our support team at service@vragen.ai.

Didn't find what you were looking for?

Ask your question directly to vragen.ai.

What exactly is vragen.ai?

Example answer by vragen.ai

vragen.ai lets visitors ask their question on your website and gives them a reliable answer straight from your own content, with the source included. You decide which sources the AI uses.

Source: How it works

This is an example. The interactive widget could not load here, for instance because of a script blocker or a slow connection.

Request a demo Try it yourself

You are asking an AI assistant from vragen.ai. Answers come from our own content, with the source included. Why we mention this (Dutch)