Cleanup service for Jitterbit private agents

Introduction

This page describes the cleanup service and its main configuration file CleanupRules.xml.

The cleanup service automatically removes the following types of private agent files:

Debug files.
Log files.
Success and failure files.
Temporary files.
Transformation data.

Which of these files to remove and when is defined in CleanupRules.xml. (The service run status and cleanup intervals are set elsewhere, in the [FileCleanup] section of jitterbit.conf.)

The cleanup rules configuration file and its formal syntax specification .dtd file are found in the installation directory (JITTERBIT_HOME), by default C:\Program Files\Jitterbit Agent\ on Microsoft Windows, /opt/jitterbit/ on Linux and Docker.

Important

Changes made to the file are not applied until the agent is restarted, and are not overwritten when upgrading.

Changes to the cleanup service configuration affect the amount of disk space consumed by these files. You should have a good understanding of how much disk space your private agent consumes on daily, weekly, and monthly periods under normal and peak operating conditions before changing the configuration.

Example

Follow these steps to make a simple change to the CleanupRules.xml file:

Open the file in a plain text editor. If possible, use one that understands XML.

Find the <CleanupRule> entry containing the specific directory path you want to change. In the example below, line 14 sets the location of debug log files to DataInterchange/Temp/Debug, and on line 16, the <FileAge> element sets the maximum file age:

<?xml version = "1.0"?>
<!DOCTYPE CleanupRules SYSTEM "CleanupRules.dtd">
<CleanupRules>
  <FileCleanup>
    <CleanupRule>
      <!-- Note: Wildcard support for DirectoryPath is limited.  Only one wildcard per path is supported. -->
      <!-- And wildcard must substitute one whole directory name.  Invalid: folder1/abc*/folder3 -->
      <DirectoryPath SearchSubDirectory = "YES" >DataInterchange/file_store/*/result_files</DirectoryPath>
      <Pattern>*</Pattern>
      <FileAge NumDays = "14" Comparator = "GE"/>
      <FileSize Size = "0" Comparator = "GE"/>
    </CleanupRule>
    <CleanupRule>
      <DirectoryPath SearchSubDirectory = "YES" >DataInterchange/Temp/Debug</DirectoryPath>
      <Pattern>*</Pattern>
      <FileAge NumDays = "1" NumOfHours = "12" Comparator = "GE"/>
      <FileSize Size = "0" Comparator = "GE"/>
    </CleanupRule>
    <CleanupRule>
      <DirectoryPath SearchSubDirectory = "YES" >DataInterchange/Temp/LOG</DirectoryPath>
      <Pattern>*</Pattern>
      <FileAge NumDays = "7" Comparator = "GE"/>
      <FileSize Size = "0" Comparator = "GE"/>
    </CleanupRule>
    <!-- Remainder of file removed for clarity -->
  </FileCleanup>
</CleanupRules>

Change the values of the parameters for the <FileAge> element according to the following definitions:
- NumDays: The maximum number of days (from 1 to 14) to retain the files located in the directory before deleting them.
- NumOfHours: The maximum number of hours to retain the files located in the directory before deleting them. (Functional only with private agent version 10.83 / 11.21 or later.)
- Comparator: How to compare a file's age with the values set by NumDays and NumOfHours:
  - EQ: Equal to.
  - GE: Greater than or equal to.
  - GT: Greater than.
  - LE: Less than or equal to.
  - LT: Less than.
  - NE: Not equal to.
In the example above, debug files are saved for 1 day and 12 hours before they are deleted.

To save the debug files for 3 days, you can change that line to <FileAge NumDays = "3" Comparator = "GE"/>, and either remove the NumOfHours parameter or set it to 0.
When all changes are completed, save the edited CleanupRules.xml file and close the file.
Restart the agent.

Tip

Additional instructions on the rules for transferred files, transformation data, and success and failure files can be found at the end of the CleanupRules.xml file.