Performance Considerations

This page is a comprehensive guide to getting the most performance out of PlanetPress Connect as well as a rough guideline to indicate when it's best to upgrade.

Performance Analysis Details

In order to get the most out of PlanetPress Connect, it is important to determine how best to maximize performance. The following guidelines will be helpful in extracting the best performance from PlanetPress Connect before looking into hardware upgrades or extra PlanetPress Connect performance packs.

  • Job Sizes and Speed: In terms of pure output speed, it's important to first determine what job size is expected, and adjust Scheduling Preferences accordingly. The basic rules are:
    • If processing a small number of very large records (when each individual record is composed of a large number of pages), more instances with an equal amount of speed units is better. For hardware, RAM and Hard Drive speeds are most important, since the smallest divisible part (the record) cannot be split on multiple machines or even cores.
    • If creating a very large number of small records (hundreds of thousands of 2-3 page individual records, for instance), a smaller number of instances with a large number of speed packs would be better. As for hardware, then the number of cores becomes critical, whereas RAM and hard drive are secondary. Performance Packs, as well as the MySQL instance being separate, would be helpful if your most powerful machine starts struggling.
    • Mix and match. For example, one instance prioritized for large jobs and the rest for smaller, quicker jobs. Or the contrary. Or, whatever you want, really.
  • RAM Configuration: By default, each instance of the Merge Engine and Weaver Engine is set to use 640MB of RAM. This means that regardless of speed units, if not enough memory is available, output speed might not be as expected. Assuming that the machine itself is not running any other software, the rule of thumb is the following: The total number of used memory in the machine should be pretty much the maximum available (around 95%).

    For each engine, it's necessary to modify the .ini file that controls its JAVA arguments. Edit as follows:

    • For the Merge Engine: see C:\Program Files\Objectif Lune\OL Connect\MergeEngine\Mergeengine.ini
    • For the Weaver Engine: see C:\Program Files\Objectif Lune\OL Connect\weaverengine\Weaverengine.ini
    • The parameters are -Xms640m for the minimum RAM size, -Xmx640m for the maximum RAM size. Explaining Java arguments is beyond the scope of this document. Please read references here, here and here for more details (fair warning: these can get pretty technical!).
  • Template and data mapping optimization: Some functionality offered by theDataMapper and Designer modules are very useful, and sometimes downright awesome, but can cause the generation of records and of contents items to slow down due to their nature. Here are some of them:
    • Preprocessor and Postprocessor scripts: manipulating data using a script may cause delays before and after the data mapping action has actually taken place, especially file conversion and data enrichment from other sources.
    • Loading external and network resources: In Designer, using images, javascript or css resources located on a slow network or on a slow internet connection will obviously lead to a loss of speed. While we do our best for caching, a document with 100,000 records which queries a page that takes 1 second to return a different image each time will, naturally, slow output generation down by up to 27 hours.
    • External JavaScript Libraries: While loading a single JavaScript library from the web is generally very fast (and only done once for the record set), actually running a script on each generated page can take some time. Because yes, JavaScript will run for each record, and often take the same time for each record.
    • Inefficient Selectors: Using very precise ID selectors in script wizards can be much faster than using a text selector, especially on very large documents. (more details on this in another upcoming page).
    • Complex Scripts: Custom scripts with large, complex or non-optimized loops can lead to slowing down content creation. While it is sometimes difficult to troubleshoot, there are many resources online to help learn about JavaScript performance and coding mistakes. Here, here, and here are a few. Note that most resources on the web are about JavaScript in the browser, but the greatest majority of the tips do, indeed, apply to scripts in general, wherever they are used.

Hardware configuration

When processing speed is important, the following is suggested before looking into Performance Packs to enhance performance.

  • Consider using a physical machine instead of a virtual machine. When running on a Virtual Machine, the machine may report that it has sufficient hardware (cores) available, but in a virtual environment you need to make sure that this hardware is not being shared with lots of other virtual machines.
  • For both virtual and non-virtual environments, make sure the machine is not busy with all kinds of other processes.
  • Antivirus exclusions. Sometimes, virus scanners, other security software or indexing services can interfere. It can help to disable those kinds of tools for the areas where Connect stores intermediate files. You could exclude the entire C:\Users\<connectuser>\Connect folder. See also: Antivirus Exclusions.
  • Use a high-performance, low-latency hard drive. Connect benefits from fast I/O. This is especially true for the DataMapper engine. Preferably use a Solid State Drive (SSD) or similar for storage.
  • Use at least 8+ GB High-Quality RAM. Check memory usage while the Print command is being executed to see if you need more than the minimum of 8GB. Assuming that the Designer and the Connect database need 1GB each, that each Merge engine needs 1GB and the Weaver engine needs 1GB as well, you can roughly estimate how much memory is needed.
  • Consider using hardware with more physical cores. PlanetPress Connect doesn't limit the number of Merge engines that is used for a print job, so if the number of physical cores is low, it makes sense to see if that can be increased. When running on a virtual machine, this is usually easy. When running on a physical machine, it means that you may have to switch hardware. Make sure to address all issues mentioned in this topic before deciding that you need to invest.
 
  • Last Topic Update: 23, May, 2018 02:04 AM
  • Last Published: 2019-05-23 : 3:18 PM