Enhancing search results with Vector Search

What is Vector Search?

Vector search functionality allows you to apply numeric values to content, enabling you to return search results based on semantic similarity rather than keyword matching. Vector search can capture the conceptual meaning of the content to identify relevant results that are linguistically distinct, for example, "dog" and "canine".

This approach elevates the quality of search outcomes compared to traditional keyword-based methods. In vector search, the system identifies the nearest match to the input query, enhancing the precision of search results. By representing the user search query, words, phrases, or documents as vectors in a high-dimensional space, SmartHub can calculate the distance between vectors to reflect their semantic similarity.

Prerequisites

Note the following prerequisites for enabling vector search:

AutoClassifier

  • In AutoClassifier, you must add a custom script component to a pipeline that applies vector embeddings to your indexed content.
  • Additionally, Upland BA Insight recommends that you also chunk your documents, as AI/vector search works best when vector embeddings are generated on smaller chunks of text. For more information, see Generate vector embeddings for chunked content in the AutoClassifier documentation for full configuration steps.

Connectivity Hub

  • In Connectivity Hub, you must configure the enrichment pipeline integration on the content source to connect Connectivity Hub and AutoClassifier, Create the vector metadata in Connectivity Hub, and validate your implementation. For more information, see Adding AI Search capabilities to your Azure AI Search content in the Connectivity Hub documentation.

Supported Search Engines

Limitations

Note the following:

  • When vector query is included, all SmartHub functionalities that are set up or consumed in the end user UI are not guaranteed to work as expected.

  • For a list of all limitation and unavailable functionalities, see Vector Search limitations.

Vector query object model in SmartHub

Vector queries are controlled in SmartHub using the SearchQuery object, which is available for processing in Query Scripting Tuning stages. For more information on processing the query in scripting stages, see Add Scripting to Tuning Stages.

The query object model member that represents the vector queries

Copy 
public IEnumerable<VectorQuery> VectorQueries

  • VectorQueries: this is the collection of the VectorQuery objects that represent all of the vector query components that will be used in the search.

The VectorQuery class

Copy 
public class VectorQuery
 {
     public IEnumerable<string> Fields;
     public IEnumerable<double> Embeddings;
 }

  • Fields: These are the property fields that the vector query should be applied to (Example: body, Title, etc.).

  • Embeddings: This is the actual vector representation of the vector query (vector embedding).

Example of a Query Tuning Stage Script that Generates Vector Embedding

To see a sample script for generating vector queries using OpenAI, refer to Generate Vectors in Query Scripting Pipeline.