How to Add a Pipeline Stage

How to Add a New Query/Results Pipeline Stage

Add scripting to stages to make it easier to use libraries by importing namespaces and referencing assemblies.

The XQL tools library is available by default.

When you add a pipeline stage Pipeline stages offer uniformity to the end user. Various functions include mapping names and values to match local refinements., choose one of the following operations:

• For all the backends that are listed on this page, click Add New Query Stage/Add New Results Stage and specify your parameters using the settings below.
• For a selected backend The search engine your SmartHub instance uses to perform queries. SmartHub can be configured to use more than one search engine., click <Backends ><name of your backend> to see the Edit SmartHub Backend page.
  
  • Perform the same operations that you perform to add pipeline stage(s) to the selected backend.

For either operation, use the following steps and graphics:

1. Go to the <name of your backend> → Pipeline Stages
   

2. Click "Add New Query" or "Add New Results Stage"
   
   
   

3. The pipeline page appears. The "Query Scripting Processor" stage is shown below.
   The page layout changes depending on the Pipeline Stage selected from the drop-down menu.
   
   
   
   

4. Enter the following values for each field:

   1. Pipeline Stage:

      1. Click the down arrow and select your stage from menu that appears.
      2. Property Mapper:
         1. This pipeline stage translates managed properties names so that they map to an existing property on a specific backend.
         2. In order to get the expected results, this pipeline stage must be added to both the Query and Results stages.
            
            Copy

            OriginalProperty1,MappedProperty1; 
            OriginalProperty2,MappedProperty2;
            PropertyToRemove,;
            Url,Path;
            format,fileextension;fcoid,;

            
            

      3. Range Facets:

         1. The Range Facets query pipeline stage must be added both before and after the Property Mapper pipeline stage.

            Usage Example
            Copy

            <RangeFacetsConfig> <Facet name="Size" isDateInterval = "false"> <Intervals minLabel="Less than 200 Kb" maxLabel = "More than 500 Kb"> <Interval startPoint="100" endPoint="200">100-200 Kb</Interval> <Interval startPoint="200" endPoint="500">200-500 Kb</Interval> </Intervals> </Facet> <Facet name="LastModifiedTime" isDateInterval = "true"> <Intervals minLabel="+3 years" maxLabel = "Today"> <Interval startPoint="-1065" endPoint="-7">1 week - 3 years</Interval> <Interval startPoint="-7" endPoint="0">1 week - today</Interval> </Intervals> </Facet> <Facet name="Date" isDateInterval = "false"> <Intervals minLabel="Before 2015" maxLabel = "After 2018"> <Interval startPoint="2015-01-10" endPoint="2017-01-10">2015-2016</Interval> <Interval startPoint="2017-01-10" endPoint="2018-01-10">2016-2017</Interval> </Intervals> </Facet> </RangeFacetsConfig>

      4. Refinement Mapper: Maps a backend refinement names and values to match the local refinements.
         
         1. In order to get the expected results, this pipeline stage must be added to both the Query and Results stages.
            
            On separate lines:
            Property#OriginalName1|OriginalValue1|...#MappedName|MappedValue
Format#docx|docx|doc|docf#Word|Microsoft Word
            
            
      5. NLQ Enables users to find information by making requests as if they were speaking to a person. Processing
         1. This stage is already configured in the NLQ Service Settings.
      6. Query Playback
         1. This pipeline stage overwrites the query with a previously captured query in JSON format.
      7. Results Playback
         1. This pipeline stage overwrites the results with a previously captured results set in JSON format.
      8. Query Recorder
         1. This pipeline stage saves in JSON format the executed search queries.
         2. Use local paths to make sure you have write access.
         3. The file is going to be created relative to the root level: ".\Folder"
      9. Results Recorder:
         1. This pipeline stage saves in JSON format the received search results.
         2. Use local paths to make sure you have write access.
         3. The file is going to be created relative to root level: ".\Folder"

      10. Query Scripting Processor

          1. Processes search queries and translates managed property names so that these names map to an existing property on a specific backend.

          2. Use this stage to process the received search query requests by providing complete scripting access to the SearchQuery object.

      11. Results Scripting Processor

          1. Processes search results and maps the backend refinement names and values to match the local refinements.

          2. This stage post-processes the returned search results by providing complete access to the SearchResults object.

      12. Suggestion Query Processing

          1. Performs suggestion query processing and allows search queries to be processed.

          2. Use this stage to process the query suggestion requests by providing complete access to the SearchQuery object.

      13. Query Suggestions Results Processor:

          1. Runs query suggestion processing.

          2. This stage processes query suggestion results by providing complete access to the QuerySuggestions, ResultSuggestions, and PeopleSuggestions members of the SearchResults object.

          3. All results pipeline stages have access to ExecutedQuery, which is a component of the SearchResults class.
          4. For further assistance, contact your BA Insight Support representative.
      14. Similar Documents Query Processor
          1. This pipeline stage allows processing similar documents queries using C# or VB scripts. This pipeline stage allows processing similar documents queries using C# or VB scripts.
             
      15. Similar Documents Results Processor
          1. This pipeline stage allows processing similar documents results using C# or VB scripts.Thi
      16. Suggestion Query Processor
          1. This pipeline stage allows processing suggestion queries using C# or VB scripts.hi
      17. Admin Relevancy
          1. This pipeline stage is tuning relevancy for user.
          2. Only full page URLs are supported.
          3. Example: https://smarthub.bainsight.com/Results.html
      18. Synonym Processor
          1. This pipeline stage expands Queries to include Synonyms.
          2. Both One-Way and Two-Way Synonyms are supported.
          3. Usage Examples:

             One-Way
             1,William,Will,Bill
             1,Anthony,Tony

             Two-Way
             2,William,Will,Bill
             2,Anthony,Tony

             Mixed
             1,William,Will,Bill
             2,Anthony,Tony
             
             

      19. Graph Boosting Stage
          1. Boosts results that are more relevant, according to the Microsoft Graph API.
      20. Personalized Query
          • This pipeline stage performs personalized query adjustments.
            Example:
            
      Copy

      First match:True
      Keyword: *
      Include results sources: *
      Exclude results sources:
      Type: Keyword
      Partial match: False
      Condition property: favorite_personaDepartment
      Condition value: Sales
      Operator: OR
      Multivalue delimiter: |
      Multivalue prefix:
      Property format:Role:{0}
      Property format default value: [userprofile].favorite_jobs
      Property format default value: *

      1. SharePoint OnPremise Refiner Mapper
         1. This pipeline stage performs refiner adjustments.
            1. See the below example.
         2. Each parameter must be on a new line and must have a semicolon at the end of the line.
            

            Without impersonation:

            Copy

            impersonation:false;
            siteurl:http://example.sharepoint.com/site/searchcenter;
            refiners:x,y;
            separator::;
            username:user;
            password:password;

            With impersonation:

            Copy

            impersonation:true;
            siteurl:http://example.sharepoint.com/site/searchcenter;
            refiners:x,y;
            separator::;
            nid:primarysid;
            nii:urn:office:idp:activedirectory;
            tid:GUID;
            issuerid:GUID;
            certpath:\Certificates\SmartHub.pfx;
            certpass:pass;
            appid:GUID;
            audience:spsiteurl;

            Parameters description:

            nid - NameID Claim name
            nii - NameID Provider
            tid - Tenant Authentication Realm
            issuerid -Registered Issuer Name
            certpath - Certificate path
            certpass - Certificate pass
            appid - App Client IDaudience - Audience Url
            
            

      2. SharePoint Online Refiner Mapper
         1. This pipeline stage performs refiner adjustments.
         2. See the below example.
         3. Each parameter must be separated by a semicolon.
         4. Use either:
            "Impersonation:true;",
            or
            "Username:myusername;
            Password:mypassword;"
            
            Example
            SiteURL:http://example.sharepoint.com/site/searchcenter;
Refiners:x,y;
Separator:|;
Impersonation:true;
            
      3. Elastic Aggregator Stage:
         1. Alters the elastic search query so that Admins can append aggregations.
         2. How to configure the stage:
            1. Related record property:
               1. Used to figure out how to attach aggregated values to the search results.
               2. Example: “people_name”
            2. Result source ID:
               1. Restricts the stage to execute only if query result source ID matches this value.

            3. Aggregation JSON:

               1. A JSON representation of the Elastic Query that is merged with the original query.
                  

               2. Example:

                  {"aggs":{"hoursByPeople":{"terms":{"field":"_PEOPLENAME", "size":300},"aggs":{"workedhours":{"sum":{"field":"_WORKEDHOURS"}}}}}}



      4. Elastic Translator Stage
         1. Translates both ways the search query into an elastic search query.
            

         2. Enter a valid elastic translator stage configuration:

            Copy

            <configuration><settings><setting Name='Indices' Value='index1,index2' /><setting Name='RefinablePropertiesSuffix' Value='keyword' /><setting Name='HighlightTagContent' Value='c0' /><setting Name='timeout' Value='30' /><setting Name='EnableFuzziness' Value='false' /><setting Name='TextQueryType' Value='best_fields' /><setting Name='QueryOperator' Value='AND' /><setting Name='DateProperties' Value='ElasticLastUpdate' /><setting Name='NumericProperties' Value='FileSize' /><setting Name='ElasticMappingTimeoutInHours' Value='2' /><setting Name='SourceMappings' Value='00000000-0000-0000-0000-000000000000#index1,index2;' /></settings></configuration>

      5. Longitude Security Trimmer

         1. This pipeline stage handles the native security trimming feature for the BA Insight Connectors.
            Note: This pipeline stage should be placed before the Property Mapper stage.

            Example

            <config><pathToGroupService>http:...</pathToGroupService> <userPrincipalNameProperty>upn</userPrincipalNameProperty> <domainRemapping>domain=domain.local</domainRemapping> <groupCacheTimeout>10</groupCacheTimeout> <additionalSecurityLevels>0</additionalSecurityLevels> <enableSidSecurity>true</enableSidSecurity> <includeResultSources>ASM</includeResultSources> <excludeResultSources>XYZ</excludeResultSources> <impersonateUser>domain\uA=domain\uB</impersonateUser> <debug>false</debug> <showErrorWhenCHNotAvailable>true</showErrorWhenCHNotAvailable></config>

      6. Graph Claims Provider
         1. Enhance the current list of user claims using the properties from the Microsoft Graph
      7. Learn to Rank Results Boosting
         1. This pipeline stage is boosting results based on clustering algorithms
      8. Learn to Rank Results Suggestions
         1. This pipeline stage is suggesting results based on clustering algorithms

      9. Advanced Security Module

         1. This pipeline stage handles the native security trimming feature for the BA Insight Connectors.
            Note: This should be places before the Property Mapper stage.
            

            Example
            Copy

            <configuration><settings><setting Name='Indices' Value='index1,index2' /><setting Name='RefinablePropertiesSuffix' Value='keyword' /><setting Name='HighlightTagContent' Value='c0' /><setting Name='timeout' Value='30' /><setting Name='EnableFuzziness' Value='false' /><setting Name='TextQueryType' Value='best_fields' /><setting Name='QueryOperator' Value='AND' /><setting Name='DateProperties' Value='ElasticLastUpdate' /><setting Name='NumericProperties' Value='FileSize' /><setting Name='ElasticMappingTimeoutInHours' Value='2' /><setting Name='SourceMappings' Value='00000000-0000-0000-0000-000000000000#index1,index2;' /></settings></configuration>

   1. Compile script:
      1. Use this option within the large text area, or as a link on the SmartHub Search Service page.

   2. Referenced assemblies

      1. Optional.

      2. Specify which assemblies the script needs in order to run.

      3. The format is one line per assembly and you must specify the assembly using the Fully Qualified Assembly format.

      4. Example:
            BAInsight.ElasticSearch.FederatorExtension, Version=1.0.0.0, Culture=neutral,PublicKeyToken=bd5ed2e4b6962cd0


      5. Imported Namespaces
         1. Optional

         2. The format is one line per namespace.

         3. Example:
               BAInsight.ElasticSearch.FederatorExtension.ElasticBackend.Backends

         4. Specify which namespaces to import in your script.

      6. Script
         1. Double-click to access the large text area:
            
            
            
            
5. If your script throws an error see the red explanatory text below the Script box.
   
6. Submit script:
   1. Click the Submit script button after you finalize your script.