Batching and commingling

A Connect Print process in its simplest form merges data with a template and creates the print job(s) in one go, as shown in Print processes with OL Connect tasks.

In reality, however, a company might want to create letters and invoices during the week, for example, and then print them out only over the weekend. For such a scenario, it would be better to separate the data mapping and content creation process from the production of the actual print job(s), and work with print batches.

Batching means gathering and then printing previously created print content items.
Commingling refers to documents originating from different templates being combined into one print batch.

This topic explains how to batch, commingle, and - not the least importantly - sort and separate items into print batches with Connect.

The Workflow tasks mentioned here can all be found on the OL Connect tab of the Plug-In Bar in Workflow. For descriptions of the tasks, see OL Connect tasks.

Batching

Batching refers to creating print jobs from print content items that were created earlier. A batching print process is split up in 2 phases:

  • Phase 1: Content is created by merging a print template with data. Newly created print content items are always automatically saved in the Connect database1. Information about the set of items (the "content set"), including the Content Set ID, gets stored in the database as well.

  • Phase 2: Print content items are retrieved from the Connect database using the Retrieve Items task. The Create Job and Create Output tasks transform these items or sets into one or more print jobs.

In order to retrieve print content from the Connect database in the second phase, some preparatory work must be done in the first phase. What needs to be done exactly depends on what you want to retrieve, and how.
Sorting and separating the output also requires some preparation (see Sorting and grouping items in print batches).

1Database entries get cleaned up automatically as well. Make sure that the Cleanup Service is configured appropriately, so that it doesn't remove relevant record sets and content sets before batching starts (see Clean-up Service preferences).

What to retrieve: content sets or content items

The Retrieve Items task can retrieve only one type of entity from the Connect database at a time: records, record sets, content items and so forth. Since the Create Job task can only work with print content items or content sets, in a batching print process the choice is narrowed down to these two possibilities.

Here are a few things to consider:

  • If you want the Create Job task to use a Job Preset, you must retrieve content sets. A Job Preset can filter, sort, and group print content items, add meta data and make finishing settings (see Print Presets).

  • From a performance perspective, working with content sets is better. Retrieving content sets is faster than retrieving content items and thus can improve performance. Also, working with Job Presets is more efficient than working with individual content items in Workflow.

  • If you decide to retrieve content sets, you may still have to do preparatory work on the content item level to enable a Job Preset to sort the output (see Sorting and grouping items in print batches).

How to retrieve: selection methods

The Retrieve Items task can retrieve items/sets by ID or based upon conditions.

Retrieving items/sets by ID

New print content items are automatically saved in the Connect database when the Create Content task creates them. The task returns the IDs of those items, as well as the ID of the content set, to the Workflow process via Metadata (see About Metadata).
In order to retrieve (sets of) items by ID later on, you will have to get the IDs from the Metadata and store them somewhere else. Workflow's Data Repository would be a good place to store them in; you can use the Push to Repository task to store the IDs there.

Retrieving items or sets using conditions

When the Retrieve Items task retrieves items or sets based upon conditions, the respective database entities are matched against the values of data fields or by the values of properties set on entities in the Connect database.

  • Values: Print content items and sets don't contain data fields, but they do have a link to the data record with which they were created, so selecting and sorting them by value is still a possibility.
  • Properties are key/value pairs that can be set on entities in the Connect database. There are two ways to do that:

    • Using the Set Properties task.
      Ideally, the Set Properties task directly follows the Create Print Content task in a Workflow process. The Create Print Content task returns the IDs of the content items as well as the ID of the content set to the process via the Metadata. Using those IDs, the Set Properties task can either set properties on all new Content Items or on the Content Set that was just created.
      Use two consecutive Set Properties tasks to set properties on both levels.

    • Through a Post Pagination script in the template. (See Post Pagination Scripts and contentitem.)

In order to retrieve (sets of) items by data values, you may have to adjust the data mapping configuration (see Data mapping configurations).

Note that a property can only be used to retrieve entities on the level on which the property was explicitly set. For example, when you set a property on a content set, you can only use that property to retrieve the content set in its entirety (i.e. all content items that belong to that content set); it isn't possible to retrieve individual content items using that property. Properties added to a content set are not propagated to the individual items in that content set.
To label both a set and the items in it, use the Set Properties task twice.

Note: Data field values and properties of items in the Connect database are also used when sorting the output. Make sure to set properties that are necessary for sorting on the content item level.

Commingling

When documents in one print batch originate from different templates, that is called commingling.

Commingling isn't very different from batching in how it's done. The content creation phase will likely be a little different, as there will probably be a separate process or branch for each of the templates used to create the print content.

However, sorting and grouping can become more important when print content items are commingled. For instance, when invoices and letters are commingled, they probably need to be grouped per customer, and sorted in a certain way. Ensure that there is a unique ID in the different data sets that can be used to group items together that belong to the same mail piece.

Sorting and grouping items in print batches

Print content items can be grouped and sorted by the values of data fields or by the values of properties set on them in the Connect database.

  • Values: Print content items and sets don't contain data fields, but they do have a link to the data record with which they were created, so selecting and sorting them by value is still a possibility.
  • Properties are key/value pairs that can be set on entities in the Connect database. There are two ways to do that:

    • Using the Set Properties task.
      Ideally, the Set Properties task directly follows the Create Print Content task in a Workflow process. The Create Print Content task returns the IDs of the content items as well as the ID of the content set to the process via the Metadata. Using those IDs, the Set Properties task can either set properties on all new Content Items or on the Content Set that was just created.
      Use two consecutive Set Properties tasks to set properties on both levels.

    • Through a Post Pagination script in the template. (See Post Pagination Scripts and contentitem.)

Sorting by property is only possible if the property has been set explicitly on the respective content items (not content sets) in the Connect database.

It is also important to note that in order to use a data field or property in a sorting rule, it should be present in all content items.
You may have to set properties on content items (not content sets) or add data fields to a data model (see Data mapping configurations), for no other reason than to sort the items by them eventually.

In Connect there are two places where you could group and sort print content after it has been retrieved from the Connect database:

  • In a Job Preset.

  • On the Batching/Commingling tab of the Retrieve Items task.

Job Preset

Based on the settings in a Job Preset, the Connect Server can filter, sort, and group print content items, add meta data and make finishing settings (see Job Presets). Compared with the Batching/Commingling tab of the Retrieve Items task, a Job Preset offers a lot more options. It is also usually more efficient; that's why it's the recommended method, for larger jobs at least.

A Job Preset can only take content sets as input. So, if you want to use a Job Preset, the Retrieve Items task must retrieve content sets from the Connect database, not content items.
This also means that if you want to use properties in conditions to retrieve the correct items, those properties must be set on the content set level.
However, any properties that you want to be used for filtering, grouping or sorting must be set on the content items.

Batching/Commingling tab of Retrieve Items task

The Batching/Commingling tab of the Retrieve Items task allows you to group and sort print content on two levels. You can:

  • Bundle content items into "documents" (mail pieces) and sort the items within each document.

  • Put documents in "groups", and define how documents are sorted within a group.

Note that this tab is only available when the Retrieve Items task is configured to retrieve content items - not content sets.

Separating the output

Print output in batches can be separated just like all other print output, using an Output Preset in the Create Output task (see Output Presets).

This requires that the output be grouped beforehand, either via a Job Preset or via the Batching/Commingling tab of the Retrieve Items task.

Note that the terminology used on the Batching/Commingling tab and in an Output Preset is slightly different.

  • Select Document Set in the Separation Settings to separate the "documents" defined on the Batching/Commingling tab.

  • Select Job Segment in the Separation Settings to separate the "groups" defined on the Batching/Commingling tab.