Publish and receive Google Pub/Sub messages in Jitterbit Studio

Introduction

Google Cloud Pub/Sub is a managed messaging service that decouples systems by routing messages through topics and subscriptions. A publisher writes messages to a topic; one or more subscribers receive those messages through subscriptions attached to that topic.

The Google Pub/Sub connector provides activities for both sides of this pattern:

• Publishing: Use the Publish Message activity to write messages to a Pub/Sub topic from a Studio operation, for example when a record is created in a source system and downstream services need to be notified.
• Receiving: Use the Get Message activity to poll a subscription for pending messages, then use the Acknowledge Message activity to confirm that each message has been processed.

This guide covers both patterns. For event-driven message consumption (where Studio listens continuously rather than polling), see the Listen Message activity reference, which requires a private agent cluster with specific network configuration.

This guide assumes the following:

• A Google Cloud service account with the Pub/Sub API enabled and appropriate roles assigned, as described in Google Pub/Sub prerequisites.
• The topic and subscription already exist in Google Pub/Sub.

Part 1: Configure the Google Pub/Sub connection

1. In Studio, open your project and go to the Project endpoints and connectors tab in the design component palette.

2. Click the Google Pub/Sub connector to open the connection configuration.

3. Connection Name: Enter a unique name (for example, Google Pub/Sub). The name cannot contain forward slashes or colons.

4. Project ID: Enter the project_id value from the service account credentials JSON file.

5. Client Email: Enter the client_email value from the credentials JSON file.

6. Private Key String: Enter the private_key value from the credentials JSON file, formatted as:

   -----BEGIN PRIVATE KEY-----<private_key>-----END PRIVATE KEY-----
   

   Note

   The private_key value in the JSON file uses \n to represent line breaks. Replace each \n with an actual carriage return before pasting into this field. If the key is pasted with literal \n characters, the connection test returns an Invalid PKCS#8 data error.

7. Click Test to verify the connection.

8. Click Save Changes.

Part 2: Publish a message to a topic

The Publish Message activity is a target activity. Use it at the end of a transformation to write data as a message to a Pub/Sub topic.

Design pattern

The operation begins with a source activity that provides the data to publish. This can be any activity that returns data, such as a database or Salesforce Query activity, a file source, or another connector's response. The transformation in Step 2 maps that source data into the Publish Message request. The steps below assume a source activity is already in place on the operation; for source activity setup, see the relevant connector's documentation.

Step 1: Configure the Publish Message activity

1. On the design canvas, drag a Publish Message activity from the Google Pub/Sub connection onto the canvas.

2. Name: Enter a descriptive name (for example, Publish Order Created).

3. Choose Topic: Select how to specify the topic:

   • Select Existent Topic: Choose from a list of topics loaded from your Google Cloud project. Requires the pubsub.topics.list permission on the service account.
   • Inform Topic Manually: Enter the topic name directly. Use this if the topic name is dynamic or the service account does not have list permission.

4. Topic: Enter or select the topic name (for example, order-events).

5. Click Next, review the request schema, then click Finished.

Step 2: Map the transformation

1. Add a Transformation step between the source activity and the Publish Message activity.

2. Open the transformation. Map the data fields from the source to the Publish Message request schema.

   The request schema includes a body node for the message payload. If the topic has a configured schema in Google Pub/Sub, the body node reflects that schema. If no schema is configured, body accepts a string value; serialize your data to JSON before mapping it:

   JSONStringify($source_data)
   

3. Deploy and run the operation to publish a message.

Part 3: Receive and acknowledge messages from a subscription

Receiving messages requires two operations: one to retrieve pending messages from a subscription, and one to acknowledge them after processing. Unacknowledged messages are redelivered by Pub/Sub after the acknowledgement deadline expires.

Design pattern

Step 1: Configure the Get Message activity

1. On the design canvas, drag a Get Message activity from the Google Pub/Sub connection onto the canvas.

2. Name: Enter a descriptive name (for example, Get Order Events).

3. Choose Subscription: Select Select Existent Subscription or Inform Subscription Manually, then enter or select the subscription name.

4. Quantity of Messages: Enter the number of messages to retrieve per operation run (1–100, default 1). Set this to the batch size your downstream processing can handle.

5. Acknowledgement Deadline in Seconds: Optionally set how long Pub/Sub holds the message before redelivering it if not acknowledged (1–600 seconds). Leave blank to use the subscription's configured deadline.

6. Click Next, review the response schema, then click Finished.

Step 2: Process the messages in a transformation

1. Add a Transformation step after the Get Message activity.

2. Open the transformation and map the message fields from the response schema to the target.

   The response schema includes a messageId field and a body field for the message payload. If the message body is JSON, deserialize it in the transformation:

   JSONParser(Source.body)
   

   Store the messageId (or the ackId field) in a global variable so the acknowledge step can reference it:

   $ack_id = Source.ackId;
   

Step 3: Configure the Acknowledge Message activity

After the messages are processed, acknowledge them to prevent redelivery.

1. On the design canvas, drag an Acknowledge Message activity from the Google Pub/Sub connection onto the canvas.

2. Name: Enter a descriptive name (for example, Acknowledge Order Events).

3. Choose Subscription: Select the same subscription used in the Get Message activity.

4. Click Next, review the schema, then click Finished.

5. Add a Transformation step between the processing transformation and the Acknowledge Message activity. Map the ackId value stored in ack_id to the acknowledge request schema.

6. In the operation settings, configure the Acknowledge Message operation to run On Success of the processing operation.

Verify the integration

Publishing: Send a test record through the operation and verify in the Google Cloud Console (Pub/Sub > Topics > [topic name]) that the message count increased. Alternatively, use the Get Message pattern to pull the message back and confirm the payload.

Receiving: Publish a test message to the subscription's topic using the Google Cloud Console (Pub/Sub > Topics > [topic name] > Publish Message). Run the Get Message operation and check the operation logs to confirm the message was retrieved. Verify the target was written with the expected data, then confirm the message no longer appears in the subscription (indicating successful acknowledgement).

If the connection test fails, check that the service account has the correct roles assigned as described in Google Pub/Sub prerequisites.