View detailed historical information on your previous crawls

To improve visibility into past crawls, Connectivity Hub has introduced functionality to allow you to view detailed information regarding the history of your previously run crawls, including crawl-level and error-level data.

Your Connectivity Hub configuration database now contains tables for crawl data and error details. When a crawl is completed, failed, or stopped, Connectivity Hub will write the details of the crawl to these tables. To visualize your past crawl data, you can use Grafana. For more information on how Grafana can connect to your Connectivity Hub database, see Grafana data sources in the Grafana documentation.

Prerequisites

To be able to view reports on previous crawls, the following is required

Enabling the crawl history feature

By default, the crawl history feature is disabled. To enable it, do the following:

  1. In the connectivity Hub installation folder, navigate to the C:\Program Files\Upland BA Insight\Connectivity Hub\Quartz Server\bin folder.

  2. Open the Quartz.Server.exe.config file in a text editor.

  3. In the <appSettings>, change the key="EnableCrawlReport" value to True.

  4. Save the file.

Visualizing your crawl data

Connectivity Hub uses Grafana to display your crawl reports. To connect Grafana to Connectivity Hub, do the following:

  1. Ensure that you have completed the prerequisites for Grafana integration.

  2. In Grafana, click Dashboards from the menu.

  3. Click New > Import. For more information, see Import dashboards in the Grafana documentation.

  4. Optionally, you can change the dashboard name, folder, or UID to your preference.

  5. Upload the following json file: Connectivity Hub crawl reports Json.

    • When you select the link above, right click the page, select Save as... and save the json file.

  6. Select the data source you created in the prerequisites.

  7. Click Import.

  8. Verify that your dashboard is receiving your Connectivity Hub data.