Settings pane

Settings for the data source and a list of Data Samples and JavaScript files used in the current data mapping configuration, can be found on the Settings tab at the left. The available options depend on the type of data sample that is loaded.
The Input Data settings (especially Delimiters) and Boundaries are essential to obtain the data and eventually, the output that you need. For more explanation, see Data source settings.

Input Data

The Input Data settings specify how the input data must be interpreted. These settings are different for each data type. For a CSV file, for example, it is important to specify the delimiter that separates data fields. PDF files are already delimited naturally by pages, so the input data settings for PDF files are interpretation settings for text in the file.

• CSV file Input Data settings

• Excel file Input Data settings

• PDF file Input Data settings

• Database Input Data settings

• Text file Input Data settings

• XML File Input Data settings

• JSON File Input Data settings

Boundaries

Boundaries are the division between records: they define where one record ends and the next record begins; for an explanation see Record boundaries.

• CSV, Excel or Database file boundaries

• PDF file boundaries

• Text file boundaries

• XML file boundaries

• JSON file boundaries

Data samples

The Data Sample area displays a list of all the imported Data Samples that are available in the current data mapping configuration. As many Data Samples as necessary can be imported to properly test the configuration.
Only one of the data samples - the active data sample - is shown in the Data Viewer.

A number of buttons let you manage the Data Samples.
In addition to using the buttons listed below, you can right-click a file to bring up the context menu, which offers the same options plus the Copy and Paste options.

To select or deselect multiple Data Samples, keep the Ctrl key pressed down while clicking on them, or keep the Shift key pressed down to select consecutive Data Samples.

Tip: Data samples can be copied and pasted to and from the Settings pane using Windows File Explorer.

• Add: Add a new Data Sample from an external data source. The new Data Sample will need to be of the same data type as the current one. For example, you can only add PDF files to a PDF data mapping configuration. Multiple files can be added simultaneously.

• Delete: Remove the current Data Sample from the data mapping configuration.

• Move up: Move the selected Data Sample(s) up the list.

• Move down: Move the selected Data Sample(s) down the list.

• Replace: Open a Data Sample and replace it with the contents of a different data source.

• Reload: Reload the currently selected Data Sample and any changes that have been made to it.

• Set as Active: Activates the selected Data Sample. The active data sample is shown in the Data Viewer after it has gone through the Preprocessor step as well as the Input Data and Boundary settings.

  Tip: You can also switch between sample data files by clicking the down arrow in the Data Model toolbar and selecting the desired sample data file.

Editor Data Format

The Editor Data Format setting is only available for Excel files.

• Date Display Format: This setting specifies how dates must be displayed in the Data Viewer. Note that extracting a Date value will only be successful if the expected date format matches the actual format of a date in the Data Viewer (see:Data format settings.)

  • Excel Default Format: Displays dates and times the way they would be displayed in Excel, using the specified locale. For date formats without a locale, the US English locale is used. (In the format selection dialog in Excel, these date formats are marked with an asterisk.) Values can show a date, or a time, or both.
    Connect always uses this setting when opening an Excel file.

  • Current Locale Settings: Shows dates and times as formatted by Windows using the current Locale of the system on which Connect runs. All values are shown as a date including a time.

  • ISO 8601 (UTC): Uses the ISO 8601 (UTC) format to display dates and times. All values are shown as a date including a time, taking the time zone into account. Note that when no time was specified with a date in the original file, the default time (12.00 AM) is used and converted; this may influence the displayed date.

  Note: Some Korean and Chinese date formats can't be parsed yet, and won't display correctly with any of these settings.

External JS Libraries

Right-clicking in the box brings up a control menu, with the same options as are available through the buttons on the right.

• Add: Add a new external library. Use the standard Open dialog to browse and open the .js file.

• Delete: Remove the currently selected library from the data mapping configuration.

• Replace: Open a library and replace it with the contents of a different js file.

• Reload: Reload the currently selected library and any changes that have been made to it.

Default Data Format

The Default Data Format settings defined here apply to any new extraction in made in the current data mapping configuration. Any format already defined for an existing field remains untouched.
It is also possible to set a default format for dates and currencies in the user preferences (DataMapper preferences).
Specific settings for a field that contains extracted data are made via the properties of the Extract step that the field belongs to (see Editing fields).

• Negative Sign Before: Any value in a numeric field that has a "-" sign is interpreted as a negative value.
• Decimal Separator: Set the decimal separator for a numerical value.
• Thousand Separator: Set the thousand separator for a numerical value.
• Currency Sign: Set the currency sign for a currency value.
• Treat empty as 0: A numerical empty value is treated as a 0 value.
• Date/Time Format: Set the date format for a date value.
  • Automatic: Select this option to parse dates automatically, without specifying a format. This is the default setting for new Date fields.
  • ISO8601: This setting allows for dates with different timestamp formats, or belonging to different time zones, to be parsed inside a single job. Dates that do not include a specific time are automatically considered to use the current locale's time zone.
    Select the ISO template to be used when parsing the timestamp. Other ISO8601 formats can be handled via the Custom option.
  • Custom: Set a custom date format. For the markers available in the DataMapper see Date.
• Date Language: Set the language for a date value (ex: If English is selected, the term May will be identified as the month of May).
• Use offset from UTC: Select the default time zone, which is to be used to extract any timestamp that does not already include time zone information with the time.

Note: Default data formats tell the DataMapper how certain types of data are formatted in the data source. They don't determine how these data are formatted in the Data Model or in a template. In the Data Model, data are converted to the native data type. Dates, for example, are converted to a DateTime object in the Data Model, and will always be shown as "year-month-day" plus the time stamp, for example: 2012-04-11 12.00 AM.