Alter chunks with a script component

After you have generated document chunks, you can add the ability to create, edit, and delete chunks through a custom script component in your pipeline. This component is useful when you want to add additional metadata properties to a chunk, add a custom chunk to the generated chunks, or remove a specific chunk from a document (for example, a privacy notice that you may have appended to all documents). To add this component to your pipeline, do the following:

  1. In your pipeline, add a script component.

  2. Enter a name for your script component and click Add.

  3. Click on the new component.

  4. In the Configuration section, enter the custom script you want to apply to your chunks. Refer to the following example which uses a C# script to add, edit, and remove chunks:

    Copy 
    var childItems = item.ChildItems;

    //
    var newChildItem = new BAInsight.ContentProcessing.V1.ProcessedChildItem(){
        ItemProperties = new List<AbstractProperty>(){
             new Property<string>(){Name="ChunkBody", Value="This is Chunk Body property Value"}
        }
    };

    // Add Child
    item.AddChild(newChildItem);

    // Add Child item inheriting all the parent properties
    item.AddChild(childItem, true);

    // Add Child item inheriting all the parent properties but skipping few properties from parent
    item.AddChild(childItem, true, new List<string>(){"Property1" "Property2"});

    // Add Properties to ChildItem
    foreach(var childItem in childItems){
        var updatedChildItem = childItem;
        updatedChildItem.AddProperty<string>("PropertyName", "PropertyValue");
    }

    // Get child Item property
    var property = childItem.GetProperty<string>("PropertyName");

    // Remove child Item property
    childItem.RemoveProperty<string>("PropertyName");

    //RemoveChild
    item.RemoveChild(childItems[0].UniqueId);

  5. Click the sprocket icon ().

  6. In the Namespaces textbox, add BAInsight.ContentProcessing.V1. When complete, the Namespaces textbox will look like the following:

    Copy 
    System.Text.RegularExpressions
    System.Text
    System.Linq
    System.Collections.Generic
    BAInsight.ContentProcessing.V1

    If you implement extra code in your script that calls third-party APIs, you will also need to add the namespaces for the necessary libraries in this textbox. For more information, see the "How to Define Assemblies and Namespaces to Import" section in how to use custom logic script.

  7. Click OK

  8. Click Compile to verify that the script successfully compiles.

  9. Click Save.

  10. When complete, you must position your script component after your document chunker component and before the chunk processor component.