Using scripts in the DataMapper

In the DataMapper every part of the extraction process can be customized using scripts.

A script can be used to set boundaries for a data source (see Setting boundaries using JavaScript). The script determines where a new record starts.

Scripts can also be used in different steps in the extraction workflow. You can:

• Modify the incoming data prior to executing the rest of the extraction workflow, via a Preprocessor action (see Preprocessor step).

• Edit extracted data in a field of the Data Model using a Post function script (entered on the Step properties pane, under Field Definition; see Modifying extracted data and Extract step properties).

  Tip: Read more about the Post function in the following how-to: DataMapper tidbits.

• Enter a script in a JavaScript-based field (see Expression-based field). Note that the last value attribution to a variable is the one used as the result of the expression. It is possible to refer to previously extracted fields if they are extracted higher in this list or in previous Extract steps in the extraction workflow.

• Let an Action step run a JavaScript, for example to:

  • Extract data to fields at the root level or in a detail table. See Extracting data with an Action step script.

  • Add a value to a custom property defined in the Preprocessor step. (See Properties and runtime parameters.) Note that only the value of properties of which the scope is set to "Each record" can be changed in a script.

• Change the left and right operands in a Condition step to a JavaScript expression. (On the Step properties pane, under Condition, set Based on to JavaScript; see Condition step properties and Multiple Conditions step properties.)

• Further process the resulting record set after the entire extraction workflow has been executed, via a Postprocessor (see Postprocessor step).

The script can always be written directly in a small script area or in the Edit script dialog. To invoke this dialog click the Use JavaScript Editor button .

Tip: In the Edit script dialog, press Ctrl + Space to bring up the list of available JavaScript objects and functions (see DataMapper Scripts API). Use the arrow keys to select a function or object and press enter to insert it. Type a dot after the name of the function or object to see which features are subsequently available.

Keyboard shortcuts for the script editor are listed in the following topic: Keyboard shortcuts.

Note that there is a contextual menu as well; right-click the editor to open it.

Syntax rules

In the DataMapper, all scripts must be written in JavaScript, following JavaScript syntax rules. For example, each statement should end with ; and the keywords that can be used, such as var to declare a variable, are JavaScript keywords. There are countless tutorials available on the Internet to familiarize yourself with the JavaScript syntax. For a simple script all that you need to know can be found on the following web pages: W3Schools website - JavaScript Syntax and https://www.w3schools.com/js/js_if_else.asp. A complete JavaScript guide for beginners can be found here: https://developer.mozilla.org/en-US/docs/Web/JavaScript.

DataMapper API

Certain features that can be used in a DataMapper script do not exist in the native JavaScript library. These are additional JavaScript features, designed for use in Connect only. All features designed for use in the DataMapper are listed in the DataMapper API (see DataMapper Scripts API).

External JavaScript libraries

The External JS Libraries box on the Settings pane lets you add JavaScript libraries to your configuration and displays all the libraries that have been imported (see Settings pane).
You can use JavaScript libraries to add more JavaScript functionality to your data mapping configuration. Any functions included in a JavaScript library that is imported in a data mapping configuration will be available in Preprocessor scripts as well as Action tasks, Post functions and JavaScript-based extraction steps.

Take the following JavaScript function, for example:

function myAddFunction(p1, p2) { 
    return p1 + p2; 
}; 
        

If this is saved as myFunction.js and imported, then the following would work anywhere in the configuration:

var result = myAddFunction(25, 12); // returns 37!