High availability deployment on-premise for Jitterbit App Builder

Introduction

You can set up App Builder as a high-availability, on-premise environment by following this suggested architecture:

flowchart LR A[fa:fa-plug API client] B[fa:fa-balance-scale Load balancer] W[fa:fa-globe Web browser] AB1[fa:fa-window-restore App Builder] AB2[fa:fa-window-restore App Builder] AB3[fa:fa-window-restore App Builder] MB["fa:fa-random Message bus
(Redis)"] ABDB[(fa:fa-database App Builder
database)] ODS@{ shape: lin-cyl, label: "fa:fa-table Other
data sources" } A --> B W --> B B --> AB1 B --> AB2 B --> AB3 AB1 --> MB AB2 --> MB AB3 --> MB MB --> ABDB MB --> ODS

App Builder requires encryption key synchronization, Redis as a message bus and simple data synchronization, and application server configuration.

Data encryption keys

App Builder uses data encryption for two purposes:

Protecting security tokens such as session cookies.
Application-level, column encryption.

Both use the same underlying data encryption technology. Specifically, they use ASP.Net Data Protection API. (Although App Builder attempts to configure the data protection library automatically, additional configuration may be necessary.)

Data encryption key storage

Data Encryption Keys (DEKs) are symmetric encryption keys used to protect data. Because DEKs are used to encrypt data stored in the database, the DEKs themselves cannot be stored in the database. App Builder supports multiple, configurable DEK storage policies. Administrators must choose the appropriate storage policy for their environment. App Builder supports the following storage policies:

File system
S3

The storage policy is configured via the AppSettings.json file.

File system storage

By default, App Builder stores DEKs on the file system in plain text, in the App Builder web application root directory.

Setting	Example	Notes
`DataEncryption.KeyStorage`	`FileSystem`	Indicates that App Builder should store DEKs on the file system. This is the default value.
`DataEncryption.Directory`	`C:\inetpub\wwwroot\Vinyl\App_Data\Keys`	Identifies the directory in which DEKs will be stored. Defaults to the `App_Data\Keys` directory, within the App Builder installation directory. The application pool user must have full control of this directory.
`DateEncryption.CertificateThumbprint`	`C123B3E899807189F11F0EC4AC320760F00ECE34`	(Optional) X.509 thumbprint of the Key Encryption Key (KEK). The KEK certificate should be registered in the Personal certificate store for the Local Computer. The thumbprint can be found by looking at the certificate properties.

Example AppSettings.json

{
    "DataEncryption": {
        "KeyStorage": "FileSystem",
        "Directory": "C:\Path\To\Keys",
        "CertificateThumbprint": "C123B3E899807189F11F0EC4AC320760F00ECE34"
    }
}

Amazon S3 storage

Because EC2 instance local storage cannot be used for long-term storage, DEKs must be stored elsewhere. App Builder supports storing DEKs in S3 buckets. The following settings configure S3 DEK storage.

Setting	Example	Notes
`DataEncryption.KeyStorage`	`S3`	Indicates that App Builder should store DEKs on Amazon S3.
`DataEncryption.S3BucketEndpoint`	`https://s3.amazonaws.com/vinyl-data-encryption-keys` or `https://vinyl-data-encryption-keys.s3.amazonaws.com/`	Identifies the AWS region and S3 bucket in which S3 keys will be stored. The URL must take one of the following forms: `https://s3{-aws-region}.amazonaws.com/{bucket}` or `https://{bucket}.s3{-aws-region}.amazonaws.com`. For more information, see AWS: General purpose buckets overview.
`DataEncryption.S3KeyPrefix`	`dev`	(Optional) Allows multiple environments to store DEKs in the same bucket, isolating the keys by prefix.

Example AppSettings.json

{
    "DataEncryption": {
        "KeyStorage": "S3",
        "S3BucketEndpoint": "https://bucket.s3.us-east-2.amazonaws.com",
        "S3KeyPrefix": "dev"
    }
}

Redis setup

You can use Redis to communicate between App Builder servers. A small node, without replicas or shards, is sufficient. (App Builder has been tested to work with Redis versions 3.2 and 6.x.)

Application server configuration

To use S3 for data encryption key storage, configure your C:\inetpub\wwwroot\App Builder\AppSettings.json file as follows:

Example AppBuilder.json using S3 for data encryption key storage

{
    "SharedState": {
        "Provider": "Redis",
        "RedisConnectionString": "REDISSERVER:6379,password=abc123"
    },
    "DataEncryption": {
        "KeyStorage": "S3",
        "S3BucketEndpoint": "https://BUCKET.s3.us-east-2.amazonaws.com",
        "S3KeyPrefix": "ENV"
    }
}

To use NFS for data encryption key storage, configure your C:\inetpub\wwwroot\App Builder\AppSettings.json file as follows:

Example AppBuilder.json using NFS for data encryption key storage

{
    "SharedState": {
        "Provider": "Redis",
        "RedisConnectionString": "REDISSERVER:6379,password=abc123"
    },
    "DataEncryption": {
        "KeyStorage": "FileSystem",
        "Directory": "C:\Path\To\Keys"
    }
}

After making changes, and restarting App Builder, check the logs to verify that settings are being picked up correctly.

Important

Every instance of App Builder should have the same appsettings.json file.