Optimize and improve the performance of Jitterbit private agents

Introduction

As projects become larger and more complex, private agents can take longer to run operations, and to synchronize operation data. To improve private agent performance, follow the guides on this page. (For significant performance issues, contact Jitterbit support.)

Private agent performance tuning

This section gives a recommended configuration set to improve the performance of private agents experiencing higher than average workloads.

To use this configuration set, configure your private agent components according to the following table:

JITTERBIT_HOME is the installation directory, by default C:\Program Files\Jitterbit Agent\ on Microsoft Windows, /opt/jitterbit/ on Docker and Linux.

Tomcat server configuration

Edit jitterbit-agent-config.properties and set the following values:

1
2

agent.heart.beat.interval = 3
max.concurrent.requests = 200

• Line 1: Recommended for Microsoft Azure hosts to avoid premature WebSocket connection termination.
• Line 2: Increase the maximum number of requests.

Java virtual machine memory allocation

Increase the values for the private agent Java VM initial and maximum memory allocation (Java parameters -Xms and -Xmx) to the same value according to the following formula:

\((Total\ Available\ Host\ Memory - 4) \times 0.75\)

Private agent configuration

Edit jitterbit.conf and set the following values in each section:

1. Enable internal pooling:

   [DbInfo]
   UseInternalPooling = true
   

2. Increase worker threads:

   [ProcessEngine]
   MaxNumberOfWorkerThreads = X
   

   where \(X = 4 \times\) (number of vCPUs or cores).

   Note

   A value of auto defaults to \(2 \times\) the number of vCPUs/cores.

3. Change values for the following:

   • MaxOperationStackDepth: Maximum number of synchronous operations that can be queued for execution (default value 50).
   • MaxAsyncOperationChainLength: Maximum number of child asynchronous operations that can be spawned (default value 50).

   [OperationEngine]
   MaxOperationStackDepth = 200
   MaxAsyncOperationChainLength = 200
   

4. Enable logging:

   [APIOperation]
   EnableLogging = true
   

5. Change the file cleanup interval:

   [FileCleanup]
   FrequencyInHours = 6.0
   

6. Increase maximum number of HTTP connections in the pool, and the maximum number of connections per route:

   [HttpConnectionPoolLocal]
   MaxConnections = 200
   MaxConnectionsPerRoute = 200
   

7. Add the following section and values. This prevents a large log sync record backlog, and improves the efficiency of the log shipper:

   [LogSync]
   MaxRecordsToSync = 2000
   MinRecordsToSync = 2000
   

Apache multi-processing module configuration

Set the following values in the <IfModule mpm_event_module> section of the httpd-mpm.conf file:

<IfModule mpm_event_module>
    ServerLimit                8
    StartServers               8
    MinSpareThreads            128
    MaxSpareThreads            128
    ThreadsPerChild            16
    ThreadLimit                64
    MaxRequestWorkers          128
    MaxConnectionsPerChild     1000
</IfModule>

Cleanup rules

Review and edit the file cleanup service rules.

Environment sync performance tuning

When you deploy an Integration Studio or Design Studio project, the project's environment is locked while data is synchronized between the Harmony cloud servers and all private agents in the agent group. While an environment is locked, no operations are processed. To avoid this behavior affecting the running of your projects, apply the following tips:

• Reduce the size of projects:

  Large projects take longer to deploy and synchronize. To reduce the size of projects, do the following:

  • Minimize the number of operations.

  • In transformations, use only the fields you need.

  • Delete unused components.

• Reduce the number of projects per environment:

  During environment sync, the entire environment is locked. To reduce the effect of this, do the following:

  • Use more environments, and spread your project portfolio across them evenly.

  • Keep high-availability (HA) projects and non-HA projects in separate environments.

• Deploy during low load times:

  To reduce the impact of synchronization times on production systems, schedule your project deployments to run during off-peak hours.

• Monitor synchronization times:

  To monitor environment sync times, scan the JITTERBIT_HOME/log/jitterbit-agent.log for the text environment-deploy:

  Example log entry

  2024-07-30 15:54:24 environment-deploy-sync-thread-thread-1  INFO org.jitterbit.integration.server.messaging.agent.deploysync.SynchronizeAgent:620 - Agent synchronization for environment 227680 and agent group ID 171230 completed at 18:54:24,667 duration in MS: 4388 Rows updated: 1 by thread: 33 name environment-deploy-sync-thread-thread-1
  

  In the above example above, the log information is as follows:

  Description	Example
  Environment ID	227680
  Agent group ID	171230
  Sync duration in milliseconds	4388
  Number of TranDb rows updated	1