Skip to Content

Apache Kafka connection

Introduction

An Apache Kafka connection, created using the Apache Kafka connector, establishes access to Apache Kafka. Once a connection is configured, you can create instances of Apache Kafka activities associated with that connection to be used either as sources (to provide data in an operation) or as targets (to consume data in an operation).

Note

This connector supports the Enable Re-authentication on Change organization policy. If enabled, a change to the Username in this connection requires users to re-enter the Password for the connection.

Create or edit an Apache Kafka connection

A new Apache Kafka connection is created using the Apache Kafka connector from one of these locations:

An existing Apache Kafka connection can be edited from these locations:

Configure an Apache Kafka connection

Each user interface element of the Apache Kafka connection configuration screen is described below.

Apache Kafka connection configuration

Tip

Fields with a variable icon support using global variables, project variables, and Jitterbit variables. Begin either by typing an open square bracket [ into the field or by clicking the variable icon to display a list of the existing variables to choose from.

  • Connection Name: Enter a name to use to identify the connection. The name must be unique for each Apache Kafka connection and must not contain forward slashes / or colons :. This name is also used to identify the Apache Kafka endpoint, which refers to both a specific connection and its activities.

  • Auth Mechanism: Select the SASL mechanism, either GSSAPI or PLAIN:

    • GSSAPI: Select if the Apache Kafka client uses the GSSAPI (Kerberos) mechanism.

    • PLAIN: Select if the Apache Kafka client uses the PLAIN mechanism (username/password).

  • Auth Protocol: Select the security protocol, one of SASL PLAINTEXT, SASL SSL, or SASL SSL (SCRAM-SHA-256):

    • SASL PLAINTEXT: Select if the Apache Kafka client uses the SASL_PLAINTEXT security protocol.

    • SASL SSL: Select if the Apache Kafka client uses the SASL_SSL security protocol.

    • SASL SSL (SCRAM-SHA-256): Select if the Apache Kafka client uses the SCRAM-SHA-256 security protocol (visible only when PLAIN is selected as the Auth Mechanism).

  • Bootstrap Servers: Enter a comma-separated list of bootstrap servers to use for establishing the connection to the Apache Kafka cluster.

  • Username: Enter the username for the Apache Kafka client. This field is visible only when PLAIN is selected as the Auth Mechanism.

  • Password: Enter the password for the Apache Kafka client. This field is visible only when PLAIN is selected as the Auth Mechanism.

  • Test: Click to verify the connection using the provided configuration. When the connection is tested, the latest version of the connector is downloaded by the agent(s) in the agent group associated with the current environment. This connector supports suspending the download of the latest connector version by using the Disable Auto Connector Update organization policy.

  • Save Changes: Click to save and close the connection configuration.

  • Discard Changes: After making changes to a new or existing configuration, click to close the configuration without saving. A message asks you to confirm that you want to discard changes.

  • Delete: After opening an existing connection configuration, click to permanently delete the connection from the project and close the configuration (see Component dependencies, deletion, and removal). A message asks you to confirm that you want to delete the connection.

Next steps

After an Apache Kafka connection has been created, you place an activity type on the design canvas to create activity instances to be used either as sources (to provide data in an operation) or as targets (to consume data in an operation).

Menu actions for a connection and its activity types are accessible from the project pane and design component palette. For details, see Actions menus in Connector basics.

These activity types are available: