How to Configure the SharePoint Online Connector Using Web.config

The SharePoint Online Connector does not retrieve 'hidden' properties for the items crawled.

• If there are properties which you want to include from this category, they must be listed in connector's web.config file under "ListOfHiddenColumnsToInclude".
• A complete record of the 'hidden' properties which have been ignored is listed in the connector logs.
• Note, you must set your Log level to DEBUG.

AppSettings CSOM and General Settings

1. <add key="CSOMTimeout" value="600000"/>
   CSOM timeout configuration
   
   
2. <add key="UserAgent" value="ISV|BA Insight|Sharepoint Online Connector/1.0"/>
   User Agent presented with each CSOM request
   
   
3. <add key="MaxDegreesOfParallelism" value="16"/>
   Value when parallelism is used to process data.  Adjust this value down if encountering Throttling. If Setting not present MaxDegreesOfParallelism will be 16.
4. <add key="ThrottlingRetryMode" value="True"/>
   If SPO rejects a request due to Throttling the request will be re-tried if ThrottlingRetryMode="True"
   
   
5. <add key="ThrottlingInitialTimeSeconds" value="30"/>
   Amount of time to await retries when throttling encountered
   
   
6. <add key="ThrottlingRetries" value="5"/>
   Number of retries when throttling encountered before request will be logged as a Failure
   
   
7. <add key="MemoryCacheTimeLimitHours" value="24"/>
   Time before Caching Expires.  Caching will automatically be flushed between Security Sync and each crawl
   
   
8. <add key="HtmlCleanRegex" value="&lt;[a-zA-Z\/][^&gt;]*&gt;"/>
   Regex for cleansing html content
   
   

Metadata Retrieval Settings

1. <add key="PropertyNamesToExclude" value=""/>
   List of properties to be excluded from item metadata (comma separated values)
   
   
2. <add key="ListOfHiddenColumnsToInclude" value=""/>
   List of hidden properties to be included in item metadata (comma separated values)
   Example: "Created_x0020_By,Modified_x0020_By,SharedWithUsers,SharedWithDetails"
   
   Security and Azure Active Directory Settings
3. <add key="GroupLoadPaginated" value="True"/>
   Determines if Groups should be loaded by page vs single retrieval
   
   
4. <add key="GroupPageSize" value="5000"/>
   Page Size to retrieve if GroupLoadPaginated="True"
   
   
5. <add key="EnableFullSecurityCache" value="True"/>
   Caching is used upon Group Loading. If memory becomes excessive during Security Synchronization disable this setting.
   
   
6. <add key="GraphAPITimeout" value="5"/>
   MS Graph API timeout
   
   
7. <add key="GetUsersPageSize" value="999"/>
   Max Size: 999
   Graph User Retrieval Page Size
   
   
8. <add key="GetGroupsPageSize" value="999"/>
   Max Size: 999
   Graph Group Retrieval Page Size
   
   
9. <add key="GetGroupMembersPageSize" value="999"/>
   Graph Group Membership Retrieval Page Size
   
   
10. <add key="UserProperties" value=""/>
    Azure Active Directory User Properties to retrieve during Security Sync
    By Default Security will return "id", "userPrincipalName", "displayName", "accountEnabled","mail" 
    To Return Additional Fields add a comma separated list. Example "onPremisesSecurityIdentifier,employeeId,givenName,mail,onPremisesDistinguishedName,onPremisesDomainName,onPremisesUserPrincipalName,surname,otherMails,proxyAddresses"
    
    
11. <add key="GroupProperties" value=""/>
    Azure Active Directory Group Properties to retrieve during Security Sync
    By Default Security will return "id", "displayName", "mail","groupTypes"
    To Return Additional Fields add a comma separated list Example "onPremisesSecurityIdentifier,mail,onPremisesNetBiosName,onPremisesSamAccountName"
    
    
12. <add key="SPOAuthorityEndpoint" value="https://login.microsoftonline.com"/>
    SPO Authorization EndPoint
    
    
13. <add key="GraphEntrypoint" value="https://graph.microsoft.com"/>
    MS Graph Endpoint