Data mapping configurations
A data mapping configuration file contains the information necessary for data mapping: the settings to read the source file (Delimiter and Boundary settings), the data mapping workflow with its extraction instructions ('Steps'), the Data Model and any imported data samples.
Data mapping configurations are used in the Designer to help add variable data fields and personalization scripts to a template. In fact, only a Data Model would suffice (see Importing/exporting a Data Model). The advantage of a data mapping configuration is that it contains the extracted records to merge with the template, which lets you preview a template with data instead of field names.
It is also possible to generate output of a data mapping configuration directly from the Designer (see Generating output).
Ultimately data mapping configurations are meant to be used by the OL Connect Server to extract data from a particular type of data file. Typically the extracted data is then merged with a template to generate output in the form of print, email and/or a web page. To make this happen, the data mapping configuration as well as the template and any Print Presets have to be sent to the Server.
If Workflow is used as automation tool, the files need to be sent to Workflow; see Sending files to Workflow. Next, they have to be linked to OL Connect tasks (i.e. the Execute Data Mapping task or All in One task) in OL Connect Workflow processes.
Alternatively the files can be sent to the Connect Server directly; see Sending files to Connect Server or to another server.
Note: The option to use AFP input is included only in the Enterprise edition of OL Connect.
AFP input is dependent on a third party library (CDP), which allows OL Connect to run up to 4 AFP input processes on a given machine, at any given time. This license requires a stable MAC address, and therefore AFP input is not compatible with virtual servers or cloud environments that cannot provide a network interface with a stable MAC address.
Creating a new data mapping configuration
A new data mapping configuration can be made with or without a wizard. When you open a data file with a DataMapper wizard, the wizard automatically detects a number of settings. You can adjust these settings. Next, the wizard automatically extracts as many data fields (or metadata, in case of a PDF/VT
Without a wizard you have to make the settings yourself, and configure the extraction workflow manually.
Note: The DataMapper doesn’t use the data source directly, rather it uses a copy of that data: a data sample. Although the data sample is a copy, it is updated each time the data mapping configuration is opened or whenever the data sample is selected.
More samples can be added via the Settings pane; see Data samples.
From a file
To start creating a data mapping configuration without a wizard, first select the data file.
-
Click the File menu and select New.
-
Click the Data mapping Configuration drop-down and select Files and then the file type:
-
CSV, XLSX or XLS (Comma Separated Values or Excel),
-
JSON File
-
Microsoft Access
-
PDF, PS, PCL or AFP file
-
Text File
-
XML File
Note: The option to use AFP input is included in the Enterprise edition of OL Connect.
The option to use PCL input can be purchased with the Professional and Enterprise editions of OL Connect.
-
-
Click Next.
-
Click the Browse button and open the file you want to work with.
-
Click Finish.
After opening the file, you have to make settings for the input data (see Data source settings) to make sure that the source data is parsed correctly and divided into logical units of data the way you want. Then you can start building the data extraction workflow.
Note:
- Excel files saved in "Strict Open XML" format are not supported yet.
- PCL and PostScript (PS) files are automatically converted to PDF format by the Connect Server. To allow for this, the default Connect Server and (if it is secured) an authenticated user must be configured via the Preferences (see Connect Servers preferences).
Note that when used in a production environment (e.g. a Connect Workflow process) the conversion to PDF may influence the processing speed, depending on the available processing power. - Some advanced PCL to PDF options are available by calling LincPDF (OL Connect's PCL to PDF converter) command line module; see Advanced PCL to PDF options.
- Extracting data from a PDF that comes from a Windows printer queue (a PDF converted to PostScript, converted back to PDF by an Input task in Workflow) might not work. The problem lies in the conversion of the original PDF to PostScript. PostScript code is usually optimized to print and not to keep text searchable, by maintaining character identities for example.
The rule of thumb is: if copy-paste from Acrobat works, so will data mapping; if not, the DataMapper won't either. If the original is a PDF, it is recommended to find an alternative way to get it in the system instead of going through a print operation. - Rotated pages in a PDF are supported (if rotated 0/90/180/270 degrees). The Extract step will be able to extract data from horizontal and vertical lines of text on rotated pages. Motion steps (such as the Repeat step and the Goto step) however, can only work as expected if text on a page has the same orientation as the page, not when text has been rotated after the page was rotated.
The page number and rotation of a page are shown in the status bar at the bottom, next to the region selection information. - The XML option also allows to select a JSON file. The JSON file will be converted to and treated like XML. If there is no root element in the JSON it will be added and each object without a named parent element is called an 'item'.
Note that in addition to being valid, the JSON should follow naming rules for XML elements. For example, "adress_line_1:" is a valid key name in JSON, but it cannot be converted to a valid element name in XML because the colon is reserved for namespaces. For XML naming rules and best naming practices, see: XML elements on W3Schools.
With a wizard
Data mapping wizards are available for PDF/VT, AFP, XML, JSON, CSV/XLSX/XLS and database tabular files, because these files are structured in a way that can be used to automatically set record boundaries.
Note: The option to use AFP input is included only in the Enterprise edition of OL Connect.
AFP input is dependent on a third party library (CDP), which allows OL Connect to run up to 4 AFP input processes on a given machine, at any given time. This license requires a stable MAC address, and therefore AFP input is not compatible with virtual servers or cloud environments that cannot provide a network interface with a stable MAC address.
The wizard for PDF/VT and AFP files cannot extract data, only metadata. After opening such a file with the wizard, you can build the data extraction workflow for data and metadata in the PDF.
The other wizards use the Extract All step to extract data, but they cannot create detail tables, so they are less suitable for files from which you want to extract transactional data.
There are two ways to open a data file with a wizard: from the Welcome screen or from the File menu.
-
From the Welcome screen
-
Open the OL Connect Welcome page by clicking the
Home icon at the top right or select the Help menu and then Welcome. -
Click New at the left.
-
Click Data and select the appropriate file type.
-
-
From the File menu
-
In the menu, click File > New.
-
Click the Data mapping Wizards drop-down and select the appropriate file type.
-
The steps to take with the wizard depend on the file type. See:
Using the wizard for XML files. This wizard can also be used for JSON files.
Generating a counter
Instead of creating a data mapping configuration for a certain type of data file, you may create a data mapping configuration that only contains a series of sequential numbers. This is a solution if, for instance, you need to create sequential tickets or anything that has an ID that changes on each record.
Note: You can’t join this configuration to another data file. It is just a counter to be applied on a static template.
To generate a counter:
- From the Welcome screen
Open the OL Connect Welcome page by clicking the
Home icon at the top right or select the Help menu and then Welcome.Click New at the left.
Click Data at the right and select Counters.
- From the File menu
In the menu, click File > New.
Click the Data mapping Wizards drop-down and select Generate counters.
You can set the following parameters:
-
Starting Value: The starting number for the counter. Defaults to 1.
-
Increment Value: The value by which to increment the counter for each record. For example, an increment value of 3 and starting value of 1 would give the counter values of 1, 4, 7, 10, [...]
-
Number of records: The total number of counter records to generate. This is not the end value but rather the total number of actual records to generate.
-
Padding character: Which character to add if the counter's value is smaller than the width.
-
Width: The number of digits the counter will have. If the width is larger than the current counter value, the padding character will be used on the left of the counter value, until the width is equal to the set value. For example for a counter value of "15", a width of "4" and padding character of "0", the value will become "0015".
-
Prefix: String to add before the counter, for example, adding # to get #00001. The prefix length is not counted in the width.
-
Suffix: String to add after the counter. The suffix length is not counted in the width.
Opening a data mapping configuration
To open an existing data mapping configuration, in the Menus, select File > Open. Make sure that the file type is either DataMapper files or Connect files. Browse to the configuration file to open, select it and click Open.
Alternatively, click on File > Open Recent to select one of the recently opened configuration files.
Saving a data mapping configuration
A data mapping configuration file has the extension .OL-datamapper. The file contains the settings, the extraction workflow ('Steps'), the Data Model and the imported Data Samples (excluding database source files such as mySQL, oracle, etc).
To save a data mapping configuration:
- In the Menus, click on File > Save, or click on Save As to save a copy of a data mapping configuration under a different name.
- In the Toolbar, click the Save button.
If the data mapping configuration has not been saved before, you have to browse to the location where the data mapping configuration should be saved and type a name, then click Save.
Down-saving a data mapping configuration
The Connect software is backwards compatible: data mapping configurations that were made with an older version of Connect can always be opened with the newest version of the software.
But newer data mapping configurations cannot be opened with an older version of the software.
Connect, however, allows you to save a data mapping configuration in the format of a previous version of the software, so that you may restore a data mapping configuration that was accidentally opened and saved in a newer version, or share a data mapping configuration with users of a previous version of Connect.
Note that it may not always be possible to down-save a data mapping configuration to a an older version. For instance, a JSON-based data mapping configuration cannot be saved to a version earlier than 2021.1 because the JSON data type did not exist prior to that version.
To down-save a data mapping configuration, select File > Save a Copy, from the menu. Select the software version and click OK. Next you can select a location and give the data mapping configuration a name, as usual.
To save a copy, follow the same procedure without selecting a previous version of the software.
Note: Since it was not possible to create a data mapping configuration for a JSON file (without conversion to XML) until version 2021.1, these data mapping configurations cannot be saved in the format of an earlier version.