Create a local file source in Jitterbit Design Studio

Introduction

A Local File on a private agent can be used as a source in Jitterbit. It must reside on the machine where the private agent is installed, and cannot be used with cloud agents. (See Temporary Storage source for an approach applicable for cloud agents.)

Any file type can be used. Commonly-used types for local files are text (.txt), CSV (.csv), XML (.xml), and JSON (.json).

Prerequisite

Before you can use a Local File source, you must have enabled Local File Location on your Jitterbit private agent. See Enabling local file location for details.

Create a local file source

You can choose to create a new Local File source on its own, or within an existing operation. For more information on how sources work within operations, see Creating an operation.

Create a new local file source as a standalone source

Within your project in Jitterbit Design Studio, you create a new Local File source by any of:

• Go to File > New > New Source; or

• In the tree on the left, right-click on Sources and select New Source; or

• In the top toolbar, click the green source icon .

In the popup, select Local File as the type:

Your new source appears in its own New Source tab in the right pane of the window.

Create a new local file source in an existing operation

A source is usually created by default when you create a new operation. (The exception is an operation that consists only of a script.) With an existing operation, you can specify the type of its source by:

• Double-clicking on the source icon; in the popup, select Create New Source; or

• Right-clicking on the source icon and selecting Create New Source.

In the configuration screen that appears, use the Type dropdown to select Local File, as shown above.

Use an existing local file source in an existing operation

To use an existing Local File source in an existing operation with a source, you can set it by any of:

• Within the operation, double-clicking on the source icon, and in the resulting popup, selecting the desired Local File source from the list; or

• Within the operation, right-clicking on the source icon, choosing Select Existing Source, and in the resulting popup, selecting the desired Local File source from the list; or

• Dragging the desired Local File source from the tree on the left and dropping it onto the existing source icon of the operation.

Configure a local file source

After you have created a Local File source, the configuration screen will open in the main view of Design Studio. You can return to the configuration screen at any time by double-clicking on the source icon in the operation, or by double-clicking on the source in the tree on the left.

Basic configuration

The configuration screen will appear similar to this example:

• Name: Enter an appropriate, unique name for the source.
• Type: Use the dropdown to select Local File, if it is not already specified.

• Connection Parameters: Specify the details for your Local File connection:

  • Browse: The Browse button, if enabled, can be used to quickly enter the folder path. It can only be used if the Jitterbit agent is installed locally on the same machine that Design Studio is running on. If this option is disabled, you may need to enable local files as described in Enabling local file location. If you select a folder or file using Browse, the Folder(s) (and possibly Get Files) fields will automatically be populated.

  • Folder(s): Enter a local path on the machine where the Jitterbit agent is installed that points to the folder where the desired file(s) reside.

    • The path may not contain any of these characters: \~ % $ ? " \< > :
    • The path separators ('/'or '\') used must be appropriate for the OS of the machine where the private agent is installed.
    • Only one path can be specified.

  • Get Files: Enter the name of the source file(s) that you want to read from within the directory specified in Folder(s). This field may also include an asterisk * to use as a wildcard (for example, *.txt or *.*) or a question mark ? to match exactly one character (for example, file?.txt). Variables may also be used in this field. If specifying a compressed file (ZIP), see the "Compression" section of the additional connection parameters under Options below. The compressed file can automatically be uncompressed and read. (Required.)

  • Filter Options: Allows specifying if filename case should be ignored or if the Get Files information should be treated as a regular expression.

• Test Connection: Clicking the button will test the Local File source. It will check that a connection to the specified folder can be made, and return a list of files that are found; it will fail if the path is not a valid path that finds files on the operating system where the Jitterbit agent is running. If you receive an error, make sure you have enabled local files as described in Enabling local file location.

• Click the Save button in either the main toolbar or the source's toolbar to save the configuration.

Options

Additional connection parameters can be specified by clicking the Options at the bottom of the Connection Parameters section:

• Character Encoding: Used to specify a character encoding for the source file(s). If left blank, Jitterbit will attempt to detect the encoding. To specify a different encoding than that default, insert one of the supported encodings described in Character encoding.

• Success/Error Folder: Use this section to specify if success and/or error folders will be used to archive a copy of all files processed.

  • If a success folder is specified, upon successfully processing the source, a copy of the file will be written to a file in that folder.
  • The file will be named the same as that specified in the Get Files field above, but with a trailing underscore and a timestamp added. If there is already a file with the same name in that folder, an additional underscore and a counter will be added.
  • For instance, if a file named data.csv is successfully processed, a copy named data.csv_2018-01-01_12-00-00-000 could be written to the folder specified as the Success Folder. If such a file already exists from a previous run, a new file data.csv_2018-01-01_12-00-00-000_1 would be added.
  • In a similar fashion, if an error folder is specified, any files that fail will be written to a file in the Error Folder.

• Trigger File: Use this to specify a trigger file to initiate processing of the source file(s). The trigger file is deleted when processing starts.

  • After selecting the Use Trigger checkbox, specify an Absolute file path to a file whose presence will trigger processing.
  • Variables (though not wildcards) may be used in this field.
  • As the trigger file is deleted as processing starts, the trigger file cannot be the same file as a source file being processed.
  • Once the operation starts, the operation will poll, looking for the trigger file. Polling time interval is fixed at one second.
  • The trigger file is deleted as processing starts.
  • If the trigger file is not found, the operation will keep attempting to find the trigger file until the operation times out.
  • Setting a trigger file blocks any operation or script that uses the source until either the trigger file condition is satisfied or the operation times out.

• After Processing:

  • If the Delete File checkbox is selected, the source file(s) will be deleted after they have been processed.
  • If a value is specified in the Rename File field, the file will be renamed using the pattern supplied. Variables may be used in this field.
  • Only one of deleting or renaming is accepted; if the Delete File checkbox is checked, Rename File will be disabled and ignored. If the Rename File is empty, it will be ignored. If left as the default, no post-processing of source files will take place.

• Ignore Lines: The indicated numbers of first and last lines are ignored for each file. If there are headers or footers to be ignored, enter the number of lines into the First and Last fields as appropriate.

  Note

  This setting does not take effect when a Local File source is referenced from a script using the ReadFile or WriteFile functions.

• Compression: If the checkbox for The source data is compressed is selected, source files will be uncompressed using the ZIP format. A Password (for encrypted archives) can be specified. Only reading from the root directory of the archive is supported.