How to Modify the Elastic Search Query by using a Scripting Stage

The Elastic Translator stage produces the JSON to be sent to Elastic.

That JSON is then consumed by the Elastic backend The search engine your SmartHub instance uses to perform queries. SmartHub can be configured to use more than one search engine..

If you wish to enhance the elastic query generated, you can do so by implementing a scripting stage right after the translation stage.

In the scripting stage, you can access the request generated using:

var elasticSearchRequest = Query.ExtendedProperties[ElasticSearchRequestBuilder.ElasticSearchRequestParameterName] as ElasticSearchRequest;

In order to update the JSON that will be consumed by Elastic Search, you will use MergeJsonObjects function by passing the request JSON and the new JSON that will be used in Elastic request.

elasticSearchRequest.RawRequestJson = CommonUtil.MergeJsonObjects(elasticSearchRequest.RawRequestJson, newJson);

After RawRequestJson is updated by function MergeJsonObjects, you must set UseRawJson on true.

elasticSearchRequest.UseRawJson = true;

In order to modify the Elastic search query using a Scripting stage follow these steps:

1. Navigate to the administration page of SmartHub.
   
   
2. Click on the name of the Elastic backend that you want to modify.
   
   
3. The "Backend Configuration" page appears. From the "Query Pipeline Stages" section click the button Add New Query Stage.
   
   
4. In the modal window that appears do the following:
   1. From the drop-down list select "Query Scripting Processor".
      
   2. In the Name field enter a name for this pipeline stage Pipeline stages offer uniformity to the end user. Various functions include mapping names and values to match local refinements..
      

   3. In the Referenced Assemblies enter the following code snippet:

      Copy

      BAInsight.ElasticSearch;
      BAInsight.ElasticSearch.FederatorExtension;

   4. In the Imported Namespaces enter the following code snippet:

      Copy

      BAInsight.ElasticSearch.FederatorExtension.ElasticBackend.Builders;
      BAInsight.Elasticsearch.Models;

   5. In the Script section enter your script.

   6. Click the Compile button to make sure that you don't have any errors.
   7. Click Save.
      
      

Below you can find 3 scripts example:

1. The size was changed from 10 to 100, the query text was changed from * to new query search and a custom aggregation was added.

   Example

   Copy

   var newJson = "{\"size\":100,\"from\":0,\"query\":{\"bool\":{\"must\":[{\"query_string\":{\"query\":\"new query search\",\"default_operator\":\"AND\",\"type\":\"best_fields\"}}],\"filter\":[{\"bool\":{\"must\":[]}}]}},\"aggs\":{\"escbase_custom\":{\"terms\":{\"field\":\"escbase_custom.keyword\",\"size\":6,\"min_doc_count\":1,\"order\":{\"_count\":\"desc\"}}}},\"_source\":{\"excludes\":[\"FileContentRaw\",\"FileContent.content\"]},\"sort\":[{\"_score\":{\"order\":\"desc\"}}],\"highlight\":{\"pre_tags\":[\"<c0>\"],\"post_tags\":[\"</c0>\"],\"number_of_fragments\":3,\"require_field_match\":false,\"highlight_query\":{\"query_string\":{\"query\":\"new query search\"}},\"fields\":{\"FileContent\":{},\"FileContent.content\":{\"type\":\"unified\"},\"FileContent.author\":{},\"FileContent.title\":{},\"escbase_title\":{}}}}";
    
   var elasticSearchRequest = Query.ExtendedProperties[ElasticSearchRequestBuilder.ElasticSearchRequestParameterName] as ElasticSearchRequest;
   elasticSearchRequest.RawRequestJson = CommonUtil.MergeJsonObjects(elasticSearchRequest.RawRequestJson, newJson);
   elasticSearchRequest.UseRawJson = true;

2. The fragment_size was changed to 150 (by default is 100)

   Example

   Copy

   var newJson = "{\"highlight\":{\"fragment_size\":150}}";  
   var elasticSearchRequest = Query.ExtendedProperties[ElasticSearchRequestBuilder.ElasticSearchRequestParameterName] as ElasticSearchRequest;
   elasticSearchRequest.RawRequestJson = CommonUtil.MergeJsonObjects(elasticSearchRequest.RawRequestJson, newJson);
   elasticSearchRequest.UseRawJson = true;

3.  The fragment_size was changed for all fields.

   Example

   Copy

   var newJson = "{\"highlight\":{\"fields\":{\"FileContent\":{\"fragment_size\":150},\"['FileContent.content']\":{\"type\":\"unified\",\"fragment_size\":150},\"['FileContent.author']\":{\"fragment_size\":150},\"['FileContent.title']\":{\"fragment_size\":200},\"escbase_title\":{\"fragment_size\":200}}}}";
     
   var elasticSearchRequest = Query.ExtendedProperties[ElasticSearchRequestBuilder.ElasticSearchRequestParameterName] as ElasticSearchRequest;
   elasticSearchRequest.RawRequestJson = CommonUtil.MergeJsonObjects(elasticSearchRequest.RawRequestJson, newJson);
   elasticSearchRequest.UseRawJson = true;