Generative AI Chat

This document outlines the concepts and the API of the Generative AI Chat feature in LupaSearch.

Overview

Generative AI Chat is a feature that allows users to interact with LupaSearch using natural language queries. It leverages advanced AI models to understand user intent and provide relevant responses not only with text, but also providing direct results from your search index.

General Flow

Generative AI Chat flow contains of these main steps:

  1. User Input: The user enters a query in natural language.
  2. Text response generation: While the more complex query is being processed, a text response is generated using the AI model. This response is based on the user's input and the context of the conversation.
  3. Picking the candidate items from the search index using various methods, gen AI model will narrow down the search results to a manageable set of candidate items. This step may involve filtering, ranking, or other techniques to ensure that the most relevant items are selected.
  4. Best items response generation: The final response is generated by the AI model, which picks the best items from the candidate set.
  5. Continuing the conversation: The user can continue the conversation by asking follow-up questions or providing additional context, which should be processed in a similar manner to the initial query.

As steps 2-3 can take up to 10-15 seconds, it is recommended to show the text response chunks as soon as they are returned, an in parallel load the steps 2 and 3.

Step 1: Text Response Generation

Request

Issue a POST request to:

POST /v1/chat/<queryKey>/text

Request body example:

{
  "userPrompt": "what phones would you recommend for under 700 euros?"
}

Choose your main queryKey that has the most facets configured, as this will help the AI model to choose the best candidate items from the search index.

Response

Response is a stream of text chunks, which can be used to display the response in real-time as it is generated.

Response Example

Here are some great smartphone options under 700 euros:

1. **Features to Consider:**
   - **Display:** Look for a high-resolution display, ideally AMOLED for vibrant colors.
   - **Camera:** Check for multiple lenses, including wide-angle and telephoto, for versatile photography.
   - **Battery Life:** A battery capacity of at least 4000mAh for all-day use.
   - **Performance:** A fast processor and at least 6GB of RAM for smooth multitasking.
   - **Storage:** Ensure at least 128GB of internal storage, with expandable options if possible.

2. **Benefits:**
   - **Value for Money:** These phones offer premium features at a mid-range price.
   - **Durability:** Many models come with water and dust resistance.
   - **Software Updates:** Look for brands that provide regular software updates for security and new features.

3. **Similar Recommendations:**
   - Consider models from brands like Samsung, Xiaomi, and OnePlus, which often offer excellent value in this price range.
   - Look for last year's flagship models, which may have dropped in price but still offer high-end features.

Explore these options to find a phone that best suits your needs and preferences!

Example on how to handle streamed chunks in JavaScript:

fetch("https://api.lupasearch.com/v1/chat/<queryKey>/text", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
  },
  body: JSON.stringify({
    userPrompt: "what phones would you recommend for under 700 euros?",
  }),
}).then((response) => {
  const reader = response.body.getReader();
  const decoder = new TextDecoder();

  function read() {
    reader.read().then(({ done, value }) => {
      if (done) {
        console.log("Stream finished");
        return;
      }
      const chunk = decoder.decode(value, { stream: true });
      result += chunk; // Append the chunk to the result
      console.log(chunk); // Display the chunk in real-time
      read(); // Continue reading
    });
  }
  read(); // Start reading the stream
});

Step 2: Picking Candidate Items

Request

POST /v1/chat/<queryKey>/candidates

Request body example:

{
  "userPrompt": "what phones would you recommend for under 700 euros?"
}

Response Example

{
  "candidates": [
    {
      "searchQuery": "smartphones",
      "items": [
        {
          "id": "84",
          "name": "Huawei P40 Pro",
          "price": 649.99,
          "description": "Smartphone with excellent camera and screen.",
          "color": "Pink",
          "brand": "Huawei",
          "category": "Phones",
          "image": "https://storage.googleapis.com/lupa-example-images/feed_images/huawei-p40-white-1.jpg",
          "rating": 5
        }
      ],
      "filters": {
        "price": {
          "gte": 0,
          "lte": 700
        }
      },
      "total": 5,
      "type": "filter"
    },
    {
      "searchQuery": "iPhones",
      "items": [
        {
          "id": "80",
          "name": "iPhone X",
          "price": 999.99,
          "description": "A modern phone with excellent camera and screen quality.",
          "color": "Silver",
          "brand": "Apple",
          "category": "Phones",
          "image": "https://storage.googleapis.com/lupa-example-images/feed_images/iphonex_front_back_glass_big.jpg.large.jpg",
          "rating": 5
        }
      ],
      "filters": {},
      "total": 1,
      "type": "direct"
    },
    {
      "searchQuery": "Samsung Galaxy",
      "items": [
        {
          "id": "81",
          "name": "Samsung Galaxy S21",
          "price": 799.99,
          "description": "The latest Samsung smartphone with excellent camera and LED lights.",
          "color": "Black",
          "brand": "Samsung",
          "category": "Phones",
          "image": "https://storage.googleapis.com/lupa-example-images/feed_images/samsung-galaxy-s21-ultra-juoda-128-5.jpeg",
          "rating": 1
        }
      ],
      "filters": {},
      "total": 2,
      "type": "direct"
    },
    {
      "searchQuery": "OnePlus phones",
      "items": [
        {
          "id": "82",
          "name": "OnePlus 9 Pro",
          "price": 699.99,
          "description": "High-quality smartphone with good cameras and screen.",
          "color": "Green",
          "brand": "OnePlus",
          "category": "Phones",
          "image": "https://storage.googleapis.com/lupa-example-images/feed_images/oneplus-9-pro-5g-128-gb-dual-78182_xbig.webp",
          "rating": 2
        }
      ],
      "filters": {},
      "total": 1,
      "type": "direct"
    },
    {
      "searchQuery": "Android phones",
      "alternativeQuery": "Smartphones",
      "items": [
        {
          "id": "84",
          "name": "Huawei P40 Pro",
          "price": 649.99,
          "description": "Smartphone with excellent camera and screen.",
          "color": "Pink",
          "brand": "Huawei",
          "category": "Phones",
          "image": "https://storage.googleapis.com/lupa-example-images/feed_images/huawei-p40-white-1.jpg",
          "rating": 5
        }
      ],
      "total": 5,
      "type": "alternative"
    }
  ],
  "chatReferenceKey": "c-xmm2gu94yz7n"
}

Response will contain a list of candidate items, grouped into different search queries. Each candidate will have a searchQuery field, which indicates the query used to find the items, and an items array containing the actual items that match the query.

If query was filtered, it will also contain filters field with tells you what filters were applied to the search query to obtain the results. You can use this information if you want to redirect user to the search page with the applied filters and search query.

Additionally, the response may include an alternativeQuery field, which suggests a different query that could yield similar results. Which might not always be the most relevant, but can be helpful for having more candidates in case the original query did not return enough results.

Once canditates are returned, you can choose to display them to the user as "intermediate results" or continue directly to the step 3.

Note the chatReferenceKey in the response. You will receive this field with a first call to the /candidates endpoint, and you will need to use it in the next step to generate the best items response. This key is used to reference the chat session and ensure that the AI model can access the context of the conversation. Additionally, you can use this key to continue the conversation by processing user's follow-up questions.

Step 3: Best Items Response Generation

Request

POST /v1/chat/<queryKey>/best

Request body example:

{
  "chatReferenceKey": "c-xmm2gu94yz7n",
  "userPrompt": "what phones would you recommend for under 700 euros?"
}

Response Example

{
  "items": [
    {
      "id": "82",
      "name": "OnePlus 9 Pro",
      "price": 699.99,
      "description": "High-quality smartphone with good cameras and screen.",
      "color": "Green",
      "brand": "OnePlus",
      "category": "Phones",
      "image": "https://storage.googleapis.com/lupa-example-images/feed_images/oneplus-9-pro-5g-128-gb-dual-78182_xbig.webp",
      "rating": 2
    },
    {
      "id": "83",
      "name": "Xiaomi Mi 11",
      "price": 599.99,
      "description": "High-quality smartphone with excellent cameras and screen.",
      "color": "White",
      "brand": "Xiaomi",
      "category": "Phones",
      "image": "https://storage.googleapis.com/lupa-example-images/feed_images/xiaomi-mi-11-lite-128gb-dual-sim_original.jpg",
      "rating": 4
    },
    {
      "id": "84",
      "name": "Huawei P40 Pro",
      "price": 649.99,
      "description": "Smartphone with excellent camera and screen.",
      "color": "Pink",
      "brand": "Huawei",
      "category": "Phones",
      "image": "https://storage.googleapis.com/lupa-example-images/feed_images/huawei-p40-white-1.jpg",
      "rating": 5
    }
  ],
  "chatReferenceKey": "c-xmm2gu94yz7n",
  "total": 3
}

Response will contain a list of best items, which are the most relevant items from the candidate set to the original userPrompt. Each item will have fields returned as defined in your search query configuration's selectFields.

Continuing the Conversation

You can continue the conversation by processing user's follow-up questions using the same endpoints. Just make sure to include the chatReferenceKey in the request body to maintain the context of the conversation. chatReferenceKey Expires in 1 hour.

Quotas

Your organization might have a daily limit of request to each of the endpoints. If you exceed the limit, you will receive a 429 Too Many Requests response. You can contact the LupaSearch support team to increase your quota.