Operation debug logging for Jitterbit private agents

Introduction

Operation debug logs are additional log files and data that are generated for operations when operation debug logging is enabled.

On private agents, enabling operation debug logging can result in the generation of three types of logs:

• Operation debug log files: Files written to a private agent directory that are generated when operation debug logging is enabled either on an individual Integration Studio or Design Studio operation or for all operations on a private agent through an agent's configuration file. The files are accessible directly on private agents and are downloadable through the Management Console Agents and Runtime Operations pages. The files are subject to the retention policy configured in the agent's cleanup rules, which is 1 day by default.

• Component input and output data: Data written to an Integration Studio operation log when operation debug logging is enabled on an Integration Studio operation running on a 10.48 or later private agent. The data is retained for 30 days by Harmony.

  Warning

  When component input and output data are generated, all request and response data for that operation are logged to the Harmony cloud and remain there for 30 days. Be aware that personally identifiable information (PII) and sensitive data such as credentials provided in a request payload will be visible in clear text in the input and output data within the Harmony cloud logs.

• API Operation Logs: Operation logs for both successful and unsuccessful API operations (configured for custom APIs or OData services). When operation debug logging is not enabled, only unsuccessful API operations are logged in the operation logs.

  Note

  The steps to enable logging of all API operations for the entire agent differ from the steps outlined in Entire private agent later on this page. For information on enabling API operation logging for the entire agent, see API Operation.

Enable operation debug logging

Operation debug logging can be enabled on an individual operation or on an entire private agent. Component input and output data are generated by default on private agents 10.48 or later but can be disabled using additional settings in the private agent configuration file.

Individual operation

Operation-level debug logging can help if you are having issues with an individual operation and do not need to turn on debug logging at the agent level, which may generate files that are large in quantity and/or size.

You can enable debug logging for an individual operation using either Integration Studio operation options or Design Studio operation options:

• Integration Studio: Use an operation's actions menu to select Settings. In the Options tab, select Enable Debug Mode Until and set a date for debug logging to be turned off. The option to cascade the debug mode settings to any child operations is also available.

• Design Studio: Right-click the background of any operation and select Options. In the Operation Options window, select Enable Debug Mode Until and set a date for debug logging to be turned off.

This date is limited to 2 weeks from the current date. Operation debug logging will be turned off at the beginning of this date (that is, at 12:00 am) using the time zone of the agent.

Entire private agent

Agent-level operation debug logging can be enabled from the private agent configuration file for all operations running on a private agent.

To enable debug logging, edit the private agent configuration file as follows, and then restart Jitterbit services.

• Under the [TransformationEngine] section, set WriteLog=true.

• Under the [OperationEngine] section, set Debug=true.

The files created in this directory can become very large. Turn off debug logging as soon as you can, and delete the files when they are no longer needed.

Additional settings

Private agents that are version 10.48 and later have the section [VerboseLogging] present in their private agent configuration file with the setting verbose.logging.enable=true, which enables generation of component input and output data:

[VerboseLogging]
verbose.logging.enable=true

When upgrading an earlier private agent version to 10.48 or later, the generation of input and output data is automatically enabled by default, even if the [VerboseLogging] section and the verbose.logging.enable setting are not present in the configuration file.

To disable generation of component input and output data, you must add the [VerboseLogging] section and set verbose.logging.enable=false to prevent component input and output data from being logged to the Harmony cloud.

Additional settings in the [VerboseLogging] section should not be changed unless instructed by Jitterbit support.

Operation debug log files

Once operation debug logging is enabled on an Integration Studio operation, Design Studio operation, or entire private agent, operation debug log files for the applicable operations are generated when those operations are executed.

Debug log file access

Operation debug log files can be accessed via the Download agent logs icon on the Management Console Agents page, the Debug files tab of the Management Console Runtime Operations page, or directly on the agent host in the following locations:

To change the operation debug log directory, edit the private agent configuration file as follows, and then restart Jitterbit services:

• Under the [TransformationEngine] section, set WriteLogDir to the desired directory location.

Debug log file retention policy

By default, operation debug log files are deleted automatically after 1 day.

This time period is configurable in CleanupRules.xml, which is accessible on private agents in these default locations:

Within the CleanupRules.xml file, you can edit this cleanup rule to change the operation debug log retention policy:

  <CleanupRule>
      <DirectoryPath SearchSubDirectory = "YES" >DataInterchange/Temp/Debug</DirectoryPath>
      <Pattern>*</Pattern>
      <FileAge NumDays = "1" Comparator = "GE"/>
      <FileSize Size = "0" Comparator = "GE"/>
  </CleanupRule>

Debug log file contents

Operation debug log files contain debug messages for each operation activity and include information that is logged before and after each transformation is executed. Information about other actions such as reading from and writing to the Harmony cache is also included.

These files are typically requested by Jitterbit support if more information is required during troubleshooting.

Component input and output data

Component input and output data is generated when an Integration Studio operation that has operation debug logging enabled at the operation level is executed on a 10.48 or later private agent that also has the default configuration settings. Input and output data are available only for activities and transformations used as operation steps.

Input and output data access

Component input and output data is accessed from the Integration Studio operation log screen or the Management Console's Runtime Operations page in additional table rows shown for each operation step that has input and output data. In the Input/Output column, icons for activities and transformations used as operation steps can be clicked to show or hide the component input and output data for that individual operation step. Each input and output log entry is limited to 100MB. If the data for an individual input or output log entry exceeds 100MB, no data will be displayed.

• Search: Search the data for the entered text.
• Copy: Copies the log data to your clipboard.
• Download: Downloads the input or output data. The file is named with the operation step name followed by _input or _output as appropriate.

Input and output data retention policy

Component input and output data is retained for 30 days by Harmony.

Input and output data contents

Component input and output data is generated for certain steps of an operation. The data is appropriate to the function of a step. The input is the structure that the component consumes, while the output is the structure that the component produces.

Input and output data is generated for these components used as operation steps:

• Activities: Input and output data and any related information for the execution of the activity. Some activities may have multiple input and output files.

• Transformations: Input and output data for the execution of the transformation.

Input and output data is not applicable to scripts when they are used as operation steps. Script content, such as that produced by the WritetoOperationLog function, is displayed in the log messages of an operation log.

Input and output data troubleshooting

Agent logs related to the input and output data should not normally need to be accessed. If you think there may be an issue with generating the input and output data, you can check the Verbose Log Shipper service log files on a private agent:

<JITTERBIT_HOME>/VerboseLogShipper/verbose-log-shipper.out.log

The Verbose Log Shipper service can be restarted by restarting a private agent.

On Linux private agents, you can also disable and re-enable this service using these Jitterbit command line options:

jitterbit start_verboselogshipper

jitterbit stop_verboselogshipper

These options can also be viewed using jitterbit --help.