High availability deployment on-premise for Jitterbit App Builder

Introduction

The following diagram illustrates the key components in a high-availability deployment of App Builder:

attachment

App Builder requires encryption key synchronization, Redis as a message bus and simple data synchronization, and application server configuration.

Data encryption keys

App Builder uses data encryption for two purposes:

Protecting security tokens such as session cookies.
Application-level, column encryption.

Both use the same underlying data encryption technology. Specifically, they use ASP.Net Data Protection API. Although App Builder will attempt to configure the Data Protection library automatically, additional configuration may be necessary.

Data encryption key storage

Data Encryption Keys (DEKs) are symmetric encryption keys used to protect data. Because DEKs are used to encrypt data stored in the database, the DEKs themselves cannot be stored in the database. App Builder supports multiple, configurable DEK storage policies. Administrators must choose the appropriate storage policy for their environment. App Builder supports the following storage policies:

File system
S3

The storage policy is configured via AppSettings. See Custom .NET configuration for additional information on AppSettings.

File system storage

By default, App Builder will store DEKs on the file system in plain text, beneath the App Builder web application root directory.

Setting	Example	Notes
`DataEncryption.KeyStorage`	`FileSystem`	Indicates that App Builder should store DEKs on the file system. This is the default value.
`DataEncryption.Directory`	`C:\inetpub\wwwroot\App Builder\App_Data\Keys`	Identifies the directory in which DEKs will be stored. Defaults to the App_Data\Keysd irectory, beneath the App Builder installation directory. The application pool user must have full control of this directory.
`DateEncryption.CertificateThumbprint`	`C123B3E899807189F11F0EC4AC320760F00ECE34`	Optional. X.509 thumbprint of the Key Encryption Key (KEK). The KEK certificate should be registered in the Personal certificate store for the Local Computer. The thumbprint can be found by looking at the certificate properties.

Example AppSettings JSON Configuration

{
    "DataEncryption": {
        "KeyStorage": "FileSystem",
        "Directory": "C:\Path\To\Keys",
        "CertificateThumbprint": "C123B3E899807189F11F0EC4AC320760F00ECE34"
    }
}

Amazon S3 storage

Because EC2 instance local storage cannot be used for long-term storage, DEKs must be stored elsewhere. App Builder supports storing DEKs in S3 buckets. The following settings configure S3 DEK storage.

Setting	Example	Notes
`DataEncryption.KeyStorage`	`S3`	Indicates that {{nm.ab}} should store DEKs on Amazon S3.
`DataEncryption.S3BucketEndpoint`	`https://s3.amazonaws.com/vinyl-data-encryption-keys` or `https://vinyl-data-encryption-keys.s3.amazonaws.com/`	Identifies the AWS region and S3 bucket in which S3 keys will be stored. The URL must take one of the following forms: `https://s3{-aws-region}.amazonaws.com/{bucket}` or `https://{bucket}.s3{-aws-region}.amazonaws.com`. For more information, see the following document: http://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html
`DataEncryption.S3KeyPrefix`	`dev`	Optional. Allows multiple environments to store DEKs in the same bucket, isolating the keys by prefix.

Example AppSettings JSON Configuration

{
    "DataEncryption": {
        "KeyStorage": "S3",
        "S3BucketEndpoint": "https://bucket.s3.us-east-2.amazonaws.com",
        "S3KeyPrefix": "dev"
    }
}

Redis setup

Use Redis to facilitate communication between App Builder servers. A small server, with no replicas or shards is all that is required.

App Builder has been tested to work with Redis version 3.2 as well as Redis 6.X

Application server configuration

You will need to make the follow changes to a default App Builder setup – these settings should be applied to every instance of App Builder. All settings should be identical for App Builder instances serving the same environment (DEV or QA or PROD)

The following should be setup in C:\inetpub\wwwroot\App Builder\AppSettings.json (Using S3 for data encryption key storage)

{
    "SharedState": {
        "Provider": "Redis",
        "RedisConnectionString": "REDISSERVER:6379,password=abc123"
    },
    "DataEncryption": {
        "KeyStorage": "S3",
        "S3BucketEndpoint": "https://BUCKET.s3.us-east-2.amazonaws.com",
        "S3KeyPrefix": "ENV"
    }
}

The following should be setup in C:\inetpub\wwwroot\App Builder\AppSettings.json (Using NFS for data encryption key storage)

{
    "SharedState": {
        "Provider": "Redis",
        "RedisConnectionString": "REDISSERVER:6379,password=abc123"
    },
    "DataEncryption": {
        "KeyStorage": "FileSystem",
        "Directory": "C:\Path\To\Keys"
    }
}

Every instance of App Builder should have the same connection.xml file.

After making changes, and restarting App Builder, check the logs to verify that settings are being picked up correctly.