Skip to Content

AWS S3 Get plugin (deprecated) for Jitterbit cloud and private agent groups

Introduction

The AWS S3 Get plugin reads, renames, or deletes a file from Amazon AWS S3. A file that is read can be used as a source in an operation.

This plugin is available by default on Jitterbit cloud agent groups and can also be associated with private agent groups.

This page details the steps to use the AWS S3 Get plugin:

  1. Download and add the plugin
  2. Associate the plugin with an agent group
  3. Assign the plugin to a source
  4. Set global variables for the plugin
  5. Complete operation setup

Important

Plugins provided by Jitterbit are deprecated.

Encryption

The AWS S3 Get plugin passes your provided AWS S3 access key ID and secret access key (described below in Set global variables for the plugin) to Amazon. Security is handled by Amazon as described in Amazon's documentation on AWS Security Credentials.

Usage

The AWS S3 Get plugin can be used whenever you need to retrieve, rename, or delete files from AWS.

Additional functionality

In addition, these options offer additional functionality beyond the AWS S3 Get plugin:

Download and add the plugin

The AWS S3 Get plugin is a standard Jitterbit plugin that is already available in the Management Console Plugins page and does not need to be downloaded or added to the organization.

If directed by support, this plugin can be downloaded at the link provided below and added to the organization.

Associate the plugin with an agent group

The AWS S3 Get plugin is associated with Jitterbit cloud agent groups by default.

If using a private agent group, before you can use the plugin you must associate it with the private agent group to make the plugin available on all agents in the group.

The plugin is listed in the Management Console Plugins page with a display name of Jitterbit AWS S3 Get.

Assign the plugin to a source

In a Design Studio project, you must assign the plugin to a source within an operation. An example operation setup is provided at the end of this page.

Within the project:

  1. Create an operation. Any operation using a source may be used.
  2. Create a dummy source that the plugin will be assigned to. This source is used as a placeholder and will not actually be read from. We recommend using a global variable source. If other file-type sources are used, such as Temporary Storage, local file, or FTP, during configuration of the source it is recommended to provide a Get Files value of [file].[ext].
  3. Assign the plugin to the source. To do this, right-click on the source and select Plugin. In the popup, under Available plugins, select "Jitterbit AWS S3 Get" and click Assign. Then click OK.

Tip

Detailed documentation on assigning plugins is available in Applying pipeline plug-ins.

Set global variables for the plugin

In order for a plugin assigned to a source to be functional, any global variables required by the plugin must be set. These variables are set in a script, which should be placed before the source in the operation so that the variables can be initialized.

To add a script before the source:

  1. Right-click on the source in the operation and select Insert Before This > Script.

  2. Create a new Jitterbit Script that sets variables used by the plugin to authenticate with your S3 bucket and read the file that you specify. A script template for required variables is provided below, followed by documentation on all possible variables that may be included within the script.

    AWS S3 Get Script
    <trans>
$AWS.S3.BUCKET_NAME="<YOUR_BUCKET_NAME>";
$AWS.S3.ACCESS_KEY_ID="<YOUR_ACCESS_KEY>";
$AWS.S3.SECRET_ACCESS_KEY="<YOUR_SECRET_KEY>";
$AWS.S3.OBJECT_KEY="<myfile.txt>";
</trans>
Name Type Required. Description
AWS.S3.ENDPOINT String Optional The URL of the Amazon AWS S3 endpoint. Defaults to "s3.amazonaws.com" if not set. Valid endpoint names by region are provided in Amazon's documentation AWS Regions and Endpoints.
AWS.S3.BUCKET_NAME String Required The name of the S3 source bucket. To read from within a folder inside the bucket, add the folder name after a forward slash (e.g. "my_bucket/folder").
AWS.S3.ACCESS_KEY_ID String Required The S3 access key (e.g. "ABCDEFGHI1JK2LM3NOPQ").
AWS.S3.SECRET_ACCESS_KEY String Required The S3 secret access key (e.g. "1aBCd2e/F34gH56IJ7Kl/Mno8pqRsTUvw9x0yzzz").
AWS.S3.REGION String Optional The S3 region. Defaults to "DEFAULT_REGION" if not set (not recommended). Current region names are provided in Amazon's documentation AWS Regions and Endpoints.
AWS.S3.OBJECT_KEY String Required The S3 object key to be read. This is the name of the specific file that you want to be returned, such as "myfile.txt". The object key is case-sensitive. Specifying multiple files or using wildcards is not supported.
AWS.S3.DELETE Boolean Optional If set to "true", deletes the object from the S3 bucket after reading it. The default is "false". Do not use with the AWS.S3.NEWNAME variable.

Warning

This variable deletes the specified object on the S3 bucket itself.
AWS.S3.NEWNAME String Optional The name used for renaming the object on the S3 bucket after reading it. Using this variable will change the S3 object key. Do not use with the AWS.S3.DELETE variable.

Warning

This variable deletes the specified object on the S3 bucket itself.

Warning

Although included in the plugin manifest, the variable AWS.S3.OBJECT_KEY_PREFIX is not functional. At this time, it is not possible to read multiple files at once; only one file may be read at a time.

Caution

Some of the variables above should not be used in combination with others. If they are used together, this will cause errors with your operation.

Complete operation setup

The plugin portion of the operation setup is complete after you have (1) the script defining the variables to be used with the plugin and (2) the plugin assigned to a source in the operation.

To complete your operation setup, define a target for archiving your S3 data, or add both a target and a transformation if you want to transform your data prior to reaching the target.

setup operation

Caution

If using a Temporary Storage, local file, or FTP site target, in order for the S3 filename and extension to be written to the target, it is recommended to enter [AWS.S3.OBJECT_KEY] in the Filename(s) field to use the same filename in the target as that being read by the plugin. Using a value of [file].[ext] will return the S3 ID of the file rather than the S3 filename.