Elastic Adapter

The Elastic Adapter integrates AutoClassifier with Elasticsearch.

Elastic Adapter Capabilities and Limitations

  • Capable of pushing additional metadata to Elasticsearch Index items. 
  • Capable of incremental crawls, but only when specific properties are available in the Elastic Index.
  • Elasticsearch Version Supported: v8.5.3 and older

How to Configure the Elastic Adapter

Use the following steps to configure the Elastic Adapter:

  1. Select the Elastic Adapter Source to configure.
  2. Complete the fields shown in the graphic below.

  3. Enter the Server Address.
    1. This is the Elastic Server Address.
  4. Enter the User Id
    1. If the Elastic Index is secured, enter the user ID which will be used to access content in the Elastic Index.
  5. Enter the User Password
    1. If the Elastic Index is secured, enter the user ID which will be used to access content in the Elastic Index.
  6. Enter the Timeout (seconds)
    1. Enter web service timeout for accessing the Elastic Index.
    2. It should be a value between 5 and 120 seconds. 
  7. Select the Index
    1. An adapter is capable of crawling a single Elastic Index at a time.
  8. Enter the Elastic Query.
    1. Enter a valid Elastic query which will serve as the bases for what data from the index is crawled. 
    2. For example: "match_all":{}
  9. Enter the Elastic Filter
    1. Add additional Query filters which will be applied to the index.
    2. Multiple filters can be added separated by ";"
  10. Enter the Enumeration Scroll Size
    1. The enumeration process takes advantage of the Elastic Scroll capabilities for enumeration.
  11. Enter the Crawl Date Field (if available)
    1. If the Elastic Index has a Date field which represents the modification date/time of each item within the index, then incremental crawls are possible.
    2. Without a date-time field, you can run only Full Crawls.
  12. Enter the Crawl Full Start Date (utc)
    1. If a Crawl Date field is available, then a filter will be applied when enumerating.

Available Elastic Queries

Copy 
MatchAll
 
"match_all":{}
 
Term
 
"term": {"escbase_fileextension": "doc"}
 
Terms
 
"terms": { "escbase_fileextension": ["doc","pdf"] }
 
Range
 
"range": { "escbase_lastupdateutc": { "gt": "1990-01-01T10:00:00.000Z"}}
"range": { "escbase_lastupdateutc": { "gte": "1990-01-01T10:00:00.000Z", "lte": "2021-01-01T10:00:00.000Z" }}

Example Configurations

Crawl all indexed data

Crawl all indexed data where lastmodifieddate property is greater than the specified data


Crawl all indexed data where createddate property is between to dates

Crawl all indexed data where fileextension property contains "doc" or "pdf"

Crawl all indexed data where fileextension property contains "doc" and the language property equals "en"

Scaling

The Elastic Adapter can be scaled to increase performance

  1. Add more Crawl Components per Server


  2. The Adapter Crawler can also be installed on multiple Servers