Build a multi-turn LLM chat with conversation history in Jitterbit Studio

Introduction

A multi-turn conversation requires more than sending a single user message to an LLM: the model needs the full exchange history to respond coherently to follow-up questions and maintain context across turns. This guide covers the low-level pattern for building and persisting a conversation history array in Studio, using Cloud Datastore as the session store and SumCSV to accumulate messages between runs.

This guide is distinct from the How to build a contextual AI agent, which covers the overall agent architecture. The focus here is on the script and transformation patterns that construct the messages array: how to format each message as a CSV row, how to persist and retrieve the full history between operation runs, and how to handle optional fields like tool_call_id when combining conversation history with function calling.

This guide builds on:

Store and retrieve session state using Cloud Datastore for Cloud Datastore setup and the query-insert-update pattern.
Use OpenAI to process data in a Studio operation or Call a REST API using the HTTP v2 connector for LLM endpoint configuration.

Note

This guide manages the conversation history explicitly, which is what you need when you call the LLM over the HTTP v2 connector. If you send prompts through a native LLM connector instead, the connector can retain chat context for you: the OpenAI, Azure OpenAI, and Amazon Bedrock connectors have a Store chat context across operations setting that maintains history between operations sharing the same chatId on cloud agent groups, and private agents retain chat context in memory automatically.

Design pattern

The conversation history pattern adds two steps around a standard LLM call: a history retrieval step before the request and a history update step after the response.

flowchart LR A["Query Cloud Datastore
Retrieve history by session key"] --> B["Script
Build messages CSV"] --> C["Transformation
Convert CSV to JSON
call LLM"] --> D["Script
Append user + assistant
Update CDS"]

Each user message and assistant response is stored in Cloud Datastore as a row in a CSV string, keyed by a session identifier (typically a Slack channel ID or user ID). Before each LLM call, the full history is retrieved and assembled into the messages array. After the LLM responds, the current user message and the assistant's reply are both appended to the history, and the updated string is written back to Cloud Datastore.

Part 1: Set up the Cloud Datastore storage

Create a key storage with a ConversationHistory field to hold the CSV history string for each session. Follow Store and retrieve session state using Cloud Datastore for the full setup steps. To add fields, in the Harmony portal navigate to Harmony portal menu > Management Console > Cloud Datastore, open the key storage, and add a field named ConversationHistory with the type Big Text. The built-in Key, Alternative Key, and Value fields are always present and do not need to be added.

Note

Big Text fields support up to 25,000 bytes per item. For long-running conversations, implement a truncation strategy: retain only the most recent N exchanges before writing back to Cloud Datastore, or summarize earlier turns using the LLM itself before storing.

Part 2: Retrieve history and build the messages array

Retrieve the history

The first step in the operation queries Cloud Datastore for the session record, using the session identifier (for example, a Slack channel ID) as the key filter. After the Query Items activity, read the result in a script step:

<trans>
$historyMessages = "";
if(Source.json.pagination.totalItems > 0,
    $historyMessages = Source.json.items.item[0].ConversationHistory
);
</trans>

This sets historyMessages to the stored CSV string, or to an empty string if this is the first turn in the session.

Build the messages array

In the next script step, assemble the full messages array as a CSV string using SumCSV. Each call to SumCSV produces one CSV row from an array of field values:

<trans>
// System message (always first, not stored in history)
message = Array();
message[0] = "system";
message[1] = $systemPrompt;
$messages = SumCSV(message);

// Append stored conversation history (all prior user/assistant pairs)
if(length(trim($historyMessages)) > 0,
    $messages = $messages + "\n" + $historyMessages
);

// Append the current user message
message[0] = "user";
message[1] = $userInput;
$messages = $messages + "\n" + SumCSV(message);
</trans>

messages now holds a CSV string with one row per message. The system message is always first; stored history follows in the order it was accumulated; the current user message is last.

Tip

Store the session key in a global variable before this operation so the Cloud Datastore update in Part 4 uses the same key.

Part 3: Convert the messages array to JSON and call the LLM

Configure the transformation source schema

Add a transformation to the operation and set its source data to the messages variable, parsed as CSV. Define a two-column CSV source schema:

role (string)
content (string)

Map the messages array to the LLM request

Map the CSV columns to the OpenAI Chat Completions request body. The target path for each message entry is json/messages/item:

json/messages/item/role → source column role
json/messages/item/content → source column content

If the operation also appends tool call results to the history (for use with the tool-calling loop pattern), the CSV may include additional columns. Use Unmap in the target field script to omit the field when the column is empty, rather than sending a null or empty string value:

// In the target script for json/messages/item/tool_call_id
if(length(trim(tool_call_id)) == 0, Unmap(), tool_call_id)

Apply the same pattern to name if you include a tool_name column:

// In the target script for json/messages/item/name
if(length(trim(tool_name)) == 0, Unmap(), tool_name)

This ensures the field is absent from the serialized JSON object when empty. The OpenAI API requires tool_call_id and name to be absent (not null or empty) for standard user and assistant messages.

Connect to the LLM endpoint

Place the HTTP v2 POST activity targeting the OpenAI Chat Completions endpoint after the transformation. For connection and authentication setup, see Use OpenAI to process data in a Studio operation.

Part 4: Append the response and update Cloud Datastore

After the LLM call, a script step extracts the assistant's reply, appends both the user message and the assistant response to the stored history, and writes the updated string back to Cloud Datastore.

Build the updated history string

<trans>
// Extract the assistant response
$assistantReply = TrimChars(GetJSONString($jitterbit.response, "/choices/0/message/content"), "\"");

// Build the two new rows to append
userMessage = Array();
userMessage[0] = "user";
userMessage[1] = $userInput;

assistantMessage = Array();
assistantMessage[0] = "assistant";
assistantMessage[1] = $assistantReply;

newExchange = SumCSV(userMessage) + "\n" + SumCSV(assistantMessage);

// Combine prior history with the new exchange
$updatedHistory = trim($historyMessages);
if(length($updatedHistory) > 0,
    $updatedHistory = $updatedHistory + "\n"
);
$updatedHistory = $updatedHistory + newExchange;
</trans>

updatedHistory contains all prior exchanges plus the current user message and assistant response. The system message is not included: it is prepended dynamically at the start of each turn in Part 2 and does not need to be stored.

Write the updated history to Cloud Datastore

Use the query-insert-update pattern from Store and retrieve session state using Cloud Datastore to write updatedHistory to the ConversationHistory field, keyed by the same session identifier used in Part 2.

Note

The Update Items activity matches the record to update by its Key field. If the Query result from Part 2 returned totalItems = 0, the Insert operation must run instead. Structure the post-update chain to match the pattern in the Cloud Datastore guide.

Alternative approach: GetInstance()

The GetInstance function provides an alternative way to assign message roles when messages are already stored as a structured array (for example, retrieved from a JSON array or a database table with one row per message). In a transformation that iterates over message instances, use TargetInstanceCount to get the current row index and assign roles based on even/odd position:

<trans>
// Assign alternating user/assistant roles based on row position (1-based)
$role = if(TargetInstanceCount() % 2 == 1, "user", "assistant");
</trans>

Map role to json/messages/item/role. This approach works when all turns are stored with their content values in chronological order and role assignment can be derived from position alone. It does not support a system message as a distinct first entry or per-message role metadata.

The SumCSV approach in Part 2 is more flexible for the conversational agent pattern: it supports an explicit system message, allows any role per row, and stores the complete history as a single Cloud Datastore field without requiring a separate row-per-message schema.

Verify the integration

Deploy and run the operation with a session key that does not yet exist in the storage. Provide an initial user message.
In the operation logs, confirm that the Query result shows totalItems = 0 and that the Insert operation ran to create the session record.
In Management Console > Cloud Datastore, open the storage and confirm that a record with the expected key exists and that ConversationHistory contains one user row and one assistant row.
Run the operation again with the same session key and a follow-up question that refers to the first exchange.
Confirm that the LLM response reflects the prior context. In Cloud Datastore, confirm that ConversationHistory now contains two user/assistant pairs.
If the LLM response ignores previous context, use WriteToOperationLog to log messages before the transformation and confirm that prior turns appear in the CSV string.
If the transformation raises a schema mismatch error, confirm that the CSV source schema column count matches the number of values passed to each SumCSV call. All rows must have the same number of columns.