About Pipeline Stages

You can use SmartHub Pipeline stages to further refine your metadata.

Pipeline Stage Key Facts

Note the following basic information about Pipeline stages:

Pipeline stages can offer uniformity to the end user
- This is done using various "pipeline stages," each of which provide a different function
Pipeline stages can apply to all of the active backends in SmartHub (Global), or to specific backends (Backend-specific)
- Global Pipeline stages run before backend-specific pipeline stages run.
- See Pipeline Categories: Global and Backend, below
Pipeline stages are set as one or both of the following two types:
- Query
- Results
Most Pipeline stages require BOTH a Query and Results stage.
- Pipeline stages that require both stages describe this requirement in the pipeline stage description found in the Pipeline stage dialogue box.
  - See Pipeline Categories: Global and Backend, below.
The various Pipeline stages are accessible from the drop-down menu in the pipelines dialogue box that appears when adding a Query or Results Pipeline stage.
- Example: "Backend-specific Pipeline Stage," below.
Pipeline stages are used to do things like map names and values to match local refinements
- Different content sources use different names and values for a given term such as "date," or "title"
- Pipelines such as the Property Mapper pipeline map these names and values
See Example: Achieving Term Uniformity section below

For a walk-through of the Pipeline tool and further explanation, see the following video: Introduction to BA Insight SmartHub Pipelines

In the example below, the SmartHub Property Mapper pipeline stage maps two terms:

This results in SmartHub users discovering consistent results when searching content across different content sources.

Pipelines are one of two categories:

Note: : Global Pipeline stages run first. Backend-specific Pipeline stages run second.

Global
- These pipeline stages apply to all of the backends configured in SmartHub
- Global pipelines are found under "General Settings"
Backend
- These pipelines apply to a single, specific backend configured in SmartHub
- Backend-specific pipelines are found under each backend's configuration page.
- Example: "Backend-specific Pipeline Stage," below.

Backend-specific Pipeline Stage

You must use a Registered Pipeline stage.

Available (Registered) Pipeline stages are listed within your SmartHub administration site (http://<SmartHubURL>/_admin).
See the graphic below.
- Select Pipeline stages from the left-side navigation menu to see a list of registered pipeline stages.
To register a new Pipeline stage, click the link at the bottom of the page (not shown in the graphic below).

Pipeline stages are available (and typically, must be created) as two different types of stages

The various available pipeline stages (typically available as both Query and Results type), and their functions, are listed below.

See the individual pipeline stage descriptions in the user interface for additional information.

Property Mapper
- Translates named property names to that these names map to an existing property on a specific backend.
- In order to get results, this stage must be added to both the Query and Results stages.
Refinement Mapper
- Maps backend refinement names and values to match the local refinements.
- In order to get the expected results, this pipeline stage must be added to both the Query and Results stages.
NLQ Processing Stage
- Extracts information from the query using machine learning (ML) algorithms and append the extracted intent and entities to the query.
Range Facets
- Customize ranges for date, size, and so on.
  - This pipeline stage must be added to both the Query and Results stages.

Playback and Recorder stages:

Query Scripting Processor
Query Suggestions Results Processor
Results Scripting Processor
Suggestion Query Processor
Personalized Query: See the Personalize queries page.
- The stage requires the User Profile to be set.

SharePoint Online Refiner Mapper:
- Maps taxonomies so that the taxonomy labels are displayed and not the GUID.
- To get results, this stage must be added only as a Results pipeline stage.
SharePoint On-Premise Refiner Mapper:
- Maps a backend refinement names and values to match the local refinements.
- To get the expected results, this pipeline stage must be added only as a Results pipeline stage.

Parameters for each SmartHub Pipeline stage are set in SmartHub.

SmartHub offers both global and backend-specific pipelines.

The Property Mapper pipeline stage below is a global pipeline stage that maps terms (parameters) across all of the backends configured in SmartHub.

The mapping process described below is a common and popular use of Pipeline stages.

For example, you may have 5 different content sources, and within each content source are various documents.

Every document in each content source has a title.

You want, in this case, all document titles labeled as "Title."

When performing queries with SmartHub users will see "Title" as an available search refinement.
Using a Pipeline stage, you can map all document titles from all of your content sources to the same term - "Title".
When performing queries with SmartHub users will see "Title" as an available search refinement. and it will match with the corresponding titles in documents from all content sources.