Temporary Storage Write activity
Introduction
A Temporary Storage Write activity writes data to a Temporary Storage endpoint and is intended to be used as a target to consume data in an operation or to be called in a script. After configuring a Temporary Storage connection, you can configure as many Temporary Storage activities as you like for each Temporary Storage connection.
Note
Cloud agents have a temporary storage file size limit of 50 GB per file. Those who need to create temporary files larger than 50 GB are encouraged to use private agents.
Create a Temporary Storage activity
An instance of an activity is created from a connection using an activity type.
To create an instance of an activity, drag the activity type to the design canvas or copy the activity type and paste it on the design canvas. For details, see Creating an activity instance in Component reuse.
An existing activity can be edited from these locations:
- The design canvas (see Component actions menu in Design canvas).
- The project pane's Components tab (see Component actions menu in Project pane Components tab).
Configure a Temporary Storage Write activity
Follow these steps to configure a Temporary Storage Write activity:
- Step 1: Enter a name, provide a request schema, and specify settings
- Step 2: Review the data schemas
Caution
After successful execution of a Temporary Storage Write activity, by default any files written to temporary storage are cleaned up by the Jitterbit cleanup service after 24 hours. On private agents, the file cleanup setting can be changed through the private agent configuration file under the [FileCleanup]
section by changing the FrequencyInHours
.
Step 1: Enter a name, provide a request schema, and specify settings
Tip
Fields with a variable icon support using global variables, project variables, and Jitterbit variables. Begin either by typing an open square bracket [
into the field or by clicking the variable icon to display a list of the existing variables to choose from.
-
Name: Enter a name to use to identify the Temporary Storage Write activity. The name must be unique for each Temporary Storage Write activity and must not contain forward slashes (
/
) or colons (:
). -
Provide Request Schema: The request schema defines the structure of data that is used by the Temporary Storage Write activity. Whether a request schema is required depends on if the activity is used as the target of a transformation (see Schema usage). For instructions on completing this section of activity configuration, refer to Schemas defined in an activity.
-
Path (Optional): Enter the folder path that you want to write to within the temporary storage directory, or enter multiple paths separated by a comma (for example,
folder1, folder2
). If no path is entered, the files are written to directly within the default temporary storage folder. To write to a new or existing temporary storage sub-directory, enter the existing folder path(s) contained within the temporary storage directory.Warning
These special characters cannot be used in file paths:
~
,%
,$
,"
,<
,>
,:
,?
.However, some special characters are allowed:
!
,@
,#
,^
,&
,*
,(
,)
,[
,]
,'
,;
. In addition, either a forward slash/
or a backslash\
can be used as a path separator.Note
The default temporary storage directory used by Harmony is the temporary directory of the agent's operating system. Typically:
- Windows:
C:\Windows\Temp\jitterbit\TemporaryStorage
- Linux:
/tmp/jitterbit/TemporaryStorage
On private agents, the default temporary storage directory can be changed through the private agent configuration file. To do so, under the
[OperationEngine]
section, setTemporaryStorageRootDirectory
to an absolute path to an existing directory.On cloud agents, the default temporary storage directory cannot be changed. However, since the path is relative, if a file has been written to temporary storage with a particular path, that same file can be retrieved from temporary storage using the same path if created within the same operation chain. For additional explanation, see Important notes under Temporary Storage.
- Windows:
-
Filename(s): Enter the name of the file(s) that you want to write to within the path(s) entered above. Any of the filename keywords are available to generate unique filenames (for example,
result[date][time].[ext]
). If specifying data in ZIP format, see the Compression options described below. -
Optional Settings: Click to expand additional optional settings:
-
Do Not Create Empty Files: Select this checkbox to skip creating the target file if there is no target data.
-
Append to File: Select this checkbox to append to existing target file instead of overwriting.
-
Write Headers: Select this checkbox if you are using a flat text target and want to write headers (column/field names) to the first line.
-
End-of-line Type: Specify how to write line breaks when writing to a target. Note that end-of-line characters in the data itself are not converted. The options are as follows:
-
Default: Standard end-of-line characters for the platform the agent is running on.
-
Windows: CRLF (ASCII 13 and ASCII 10).
-
Unix: LF (ASCII 10).
-
-
Character Encoding: Specify character encoding to define a specific character encoding for the target file. If you leave the field blank, Harmony tries to detect the encoding. If you want to overwrite the default, you can choose from any of the encoding types listed in Supported character encodings.
-
Compression: Select the checkbox for Compress Target Files if you want the target data to be compressed. These options become available:
-
Format: Use the dropdown to select the desired compression format. Currently, only ZIP format is supported.
-
File Names in Archive(s): Enter the name of the target file(s) to be compressed within the archive. Any of the filename keywords are available to generate unique filenames. File(s) are automatically written into the archive at the root level.
Tip
To keep the filename(s) in the archive the same as the filename(s) defined in the target, enter
[file].[ext]
. -
Mode: Use the dropdown to select from these modes:
- Overwrite: Overwrites the existing archive.
- Append: If the archive is not encrypted, adds compressed files to the existing archive. Appending to an encrypted ZIP file is not supported.
-
Password: To encrypt the archive, enter the desired password.
-
Encryption Mode: If a password is supplied, use the dropdown to select between the two available encryption modes: AES and Standard ZIP.
-
-
-
Save & Exit: If enabled, click to save the configuration for this step and close the activity configuration.
-
Next: Click to temporarily store the configuration for this step and continue to the next step. The configuration will not be saved until you click the Finished button on the last step.
-
Discard Changes: After making changes, click to close the configuration without saving changes made to any step. A message asks you to confirm that you want to discard changes.
Step 2: Review the data schemas
-
Data Schema: If provided during activity configuration, the request data schema is displayed. If the operation uses a transformation, the data schemas are displayed again later during the transformation mapping process, where you can map to target fields using source objects, scripts, variables, custom values, and more. You can also define schemas directly in a transformation.
-
Add Plugin(s): Plugins are Jitterbit- or user-provided applications that extend Harmony's native capabilities. To apply a plugin to the activity, click to expand this section and select the checkbox next to the plugin to be used. For additional instructions on using plugins, including details on setting any required variables used by the plugin, see Plugins added to an activity.
-
Back: Click to temporarily store the configuration for this step and return to the previous step.
-
Finished: Click to save the configuration for all steps and close the activity configuration.
-
Discard Changes: After making changes, click to close the configuration without saving changes made to any step. A message asks you to confirm that you want to discard changes.
Next steps
After configuring a Temporary Storage Write activity, you can use it within an operation or script as described below.
Complete the operation
After configuring a Temporary Storage Write activity, complete the configuration of the operation by adding and configuring other activities, transformations, or scripts as operation steps. You can also configure an operation's operation settings, which include the ability to chain operations together that are in the same or different workflows.
Once a Temporary Storage Write activity has been created, menu actions for that activity are accessible from the project pane in either the Workflows or the Components tabs, and from the design canvas. See Activity actions menu for details.
Temporary Storage Write activities can be used as a target with these operation patterns:
- Archive pattern
- Script pattern
- Transformation pattern
- Two-target archive pattern (as the second target only)
- Two-target HTTP archive pattern (as the second target only)
- Two-transformation pattern (as the second target only)
- Salesforce bulk source pattern
Other patterns are not valid using Temporary Storage Write activities. See the validation patterns on the Operation validity page.
A common design pattern for using temporary storage is provided in Persisting data for later processing using Temporary Storage (This pattern uses Design Studio as an example, but the same concepts can be applied in Integration Studio).
When ready, deploy and run the operation and validate behavior by checking the operation logs.
Using Temporary Storage activities in scripts
Temporary Storage Write activities can also be referenced in a script for use with script functions that use a targetId
as a parameter, including these:
Jitterbit Script Functions
ArchiveFile
FlushAllFiles
FlushFile
WriteFile
JavaScript Jitterbit Functions
Jitterbit.WriteFile
For more details on referencing activities in scripts, see Endpoints in Jitterbit Script or Endpoints in JavaScript.