Skip to Content

Route LLM responses to Studio operations using function calling in Jitterbit Studio

Introduction

Function calling is an LLM capability where, instead of returning free text, the model returns a structured selection of which function to call and what arguments to pass. In Studio, this means the LLM can select which operation to run in response to a user request, routing a message about a missing paycheck to an HR notification operation, for example, while routing a message about a software access request to an IT provisioning operation.

This guide covers two approaches to implementing this pattern:

  • Part 1: Formal function calling: Define tool schemas in JSON, submit them with the LLM request via the HTTP v2 connector, and receive a structured tool_calls response. A script extracts the function name and arguments, then dispatches to the correct operation. This approach produces reliable, structured output and is recommended for production use.
  • Part 2: Prompt-based routing: Describe available operations as plain text in the system prompt, then parse the LLM's text response for the selected function name and dispatch using a Case statement. This approach requires no tool schema authoring but depends on the LLM following the prompt instruction precisely.

This guide assumes familiarity with the OpenAI Prompt activity and HTTP v2 connector. For setup details, see Use OpenAI to process data in a Studio operation and Call a REST API using the HTTP v2 connector.

Design pattern

Both approaches share the same conceptual flow and operation structure:

  1. The LLM receives the user's message along with a description of the available operations (the tool list).
  2. The LLM selects the appropriate operation and returns a function name (and in Part 1, structured arguments).
  3. A Studio script extracts the function name and routes execution to the corresponding operation.

The operation chain follows this structure:

flowchart LR A[LLM Prompt operation] -->|On Success| B[Dispatcher script operation] --> C[Target operation]

The LLM Prompt operation sends the user message to the model and stores the response. The Dispatcher script operation reads the response, extracts the selected function name, and calls the corresponding operation using RunOperation. Each target operation performs the actual work for that function.

Global variables carry the extracted function name and arguments from the dispatcher into each target operation.

Part 1: Formal function calling

The OpenAI Chat Completions API accepts a tools array in the request body. Each entry in the array defines one available function: its name, a description the model uses to decide when to call it, and a JSON Schema object describing its parameters. When the model selects a function, it returns a tool_calls array in its response rather than populating message.content.

Step 1: Define the tool schemas

Each tool schema is a JSON object with three fields: name, description, and parameters.

The name is the identifier that your dispatcher script will check. The description tells the model when to call this tool: write it as a clear statement of the function's purpose. The parameters object follows JSON Schema conventions: list each parameter under properties, set its type and description, and list required parameters under required.

Example schemas for two tools (one to notify HR, one to notify IT):

[
  {
    "type": "function",
    "function": {
      "name": "notify_hr",
      "description": "Send a message to the HR team, for example about onboarding, payroll, or leave requests.",
      "parameters": {
        "type": "object",
        "properties": {
          "email_body": {
            "type": "string",
            "description": "The full text of the message to send."
          },
          "recipient_name": {
            "type": "string",
            "description": "The name of the HR contact to address."
          }
        },
        "required": ["email_body", "recipient_name"]
      }
    }
  },
  {
    "type": "function",
    "function": {
      "name": "notify_it",
      "description": "Send a message to the IT team, for example about software access, hardware, or account issues.",
      "parameters": {
        "type": "object",
        "properties": {
          "email_body": {
            "type": "string",
            "description": "The full text of the message to send."
          },
          "issue_type": {
            "type": "string",
            "description": "A short label for the type of issue, for example 'access request' or 'hardware fault'."
          }
        },
        "required": ["email_body", "issue_type"]
      }
    }
  }
]

Tip

Write tool descriptions from the model's perspective: describe the situation in which the model should call this function, not what the underlying operation does. Vague descriptions cause the model to select the wrong tool.

Step 2: Include the tools in the LLM request

Use an HTTP v2 POST activity to call the OpenAI Chat Completions endpoint (https://api.openai.com/v1/chat/completions). In the request body transformation, construct the full JSON request as a string and map it to the body field.

A representative request body, using a project variable for the API key and global variables for the user message and conversation history:

{
  "model": "gpt-4",
  "messages": [
    {
      "role": "system",
      "content": "You are a helpful assistant. Use the available functions to handle the user's request."
    },
    {
      "role": "user",
      "content": "$user_message"
    }
  ],
  "tools": <tools array from Step 1>,
  "tool_choice": "auto"
}

Setting tool_choice to "auto" lets the model decide whether to call a function or return a plain text response. To force a function call, set tool_choice to "required".

For HTTP v2 connection and authentication setup, see Call a REST API using the HTTP v2 connector.

Step 3: Parse the tool_calls response

When the model selects a tool, the response body includes a tool_calls array in choices[0].message. Add a transformation script node to extract the function name and arguments and store them as global variables for use in the dispatcher and target operations.

In the transformation that follows the HTTP v2 activity, add a script node at the root level:

<trans>
// Check whether the model made a function call
toolCalls = Source.choices[0].message.tool_calls;

If(Length(toolCalls) > 0,
    // Extract the function name and parse the arguments JSON string
    $function_name = toolCalls[0].function.name;
    args = JSONParser(toolCalls[0].function.arguments);

    // Store individual arguments as global variables for downstream operations
    $arg_email_body     = args["email_body"];
    $arg_recipient_name = args["recipient_name"];
    $arg_issue_type     = args["issue_type"];

    WriteToOperationLog("Function selected: " + $function_name);
,
    // No function call: model returned plain text
    $function_name = "";
    WriteToOperationLog("No function call in response");
);
</trans>

JSONParser converts the arguments string (a JSON object) into a dictionary. The keys in args correspond to the parameter names defined in the tool schema.

Note

tool_calls is absent from the response when the model returns plain text instead of calling a function. The $function_name = "" branch handles this case so the dispatcher script in the next step can route it to a fallback operation.

Step 4: Dispatch to the target operation

Create a second operation, chained on the success of the LLM Prompt operation. This operation contains a single Script tool that reads function_name and calls the appropriate target operation:

Case($function_name == "notify_hr",
    RunOperation("<TAG>operation:Notify HR</TAG>");,

$function_name == "notify_it",
    RunOperation("<TAG>operation:Notify IT</TAG>");,

true,
    // Default case: unknown function name or plain text response
    WriteToOperationLog("No matching function for: " + $function_name)
);

RunOperation runs the target operation synchronously. The global variables set in Step 3 (arg_email_body, arg_recipient_name, etc.) are available inside each target operation.

Tip

To store conversation history between turns, use Cloud Datastore to persist the messages array across operation runs. If you send prompts through a native LLM connector rather than HTTP v2, the OpenAI, Azure OpenAI, and Amazon Bedrock connectors can instead retain chat context across operations for you on cloud agent groups (with private agents retaining it in memory).

Part 2: Prompt-based routing

This approach embeds the list of available operations directly in the system prompt as plain text, instructs the model to respond with exactly one function name, and uses a Case statement to dispatch. No tool schema JSON is required.

Step 1: Build the tool manifest

Use AddToDict to register each available operation. Store the description and optional parameter list together in a single string, separated by a | character:

AddToDict($tools_dict, "get_account_info",
    "Retrieve account details for a customer.|(account_name)");

AddToDict($tools_dict, "create_ticket",
    "Create a support ticket for a reported issue.|(subject,description)");

AddToDict($tools_dict, "send_notification",
    "Send a notification message to a user.|(user_id,message)");

AddToDict($tools_dict, "unknown_route",
    "Ask the user for clarification when the request is ambiguous.");

Include an unknown_route entry so the model has a safe fallback when the user's intent is unclear.

Step 2: Convert the manifest to text

Use GetKeys to iterate over the dictionary and build a plain-text list for embedding in the system prompt:

toolText = "";
keys = GetKeys($tools_dict);
i = 0;
while(i < Length(keys),
    key    = keys[i];
    entry  = $tools_dict[key];
    desc   = Split(entry, "|")[0];
    params = Split(entry, "|")[1];

    toolText = toolText + "- " + key + ": " + desc;
    If(Length(params) > 0,
        toolText = toolText + " Parameters: " + params
    );
    toolText = toolText + "\n";
    i++
);
$tools_text = toolText;

Step 3: Include the manifest in the system prompt

In the LLM request body, insert tools_text into the system message and instruct the model to respond with exactly one function name. Include the user's message as a separate user role message so the model has a request to route. The system instruction must be explicit: the model needs to know the expected output format.

{
  "model": "gpt-4",
  "messages": [
    {
      "role": "system",
      "content": "You are a routing assistant. Based on the user's message, respond with exactly one function name from the list below. Do not include any explanation or other text.\n\nAvailable functions:\n$tools_text"
    },
    {
      "role": "user",
      "content": "$user_message"
    }
  ]
}

The system message carries the routing instruction and the tool manifest (tools_text); the user message carries the actual request (user_message) the model routes.

Note

Prompt-based routing depends on the model following the output format instruction. If the model returns more than the function name, the Trim call in Step 4 removes surrounding whitespace, but it cannot correct multiword or formatted responses. Test the prompt with the target model before deploying.

Step 4: Extract the function name from the response

The model returns the function name in choices[0].message.content. Add a script node in the transformation following the HTTP v2 activity to store it as a global variable:

<trans>
$function_name = Trim(Source.choices[0].message.content);
WriteToOperationLog("Routing to: " + $function_name);
</trans>

Trim removes any leading or trailing whitespace from the model's response.

Step 5: Dispatch to the target operation

Chain a dispatcher operation on the success of the LLM Prompt operation, using the same Case pattern as Part 1:

Case($function_name == "get_account_info",
    RunOperation("<TAG>operation:Get Account Info</TAG>");,

$function_name == "create_ticket",
    RunOperation("<TAG>operation:Create Ticket</TAG>");,

$function_name == "send_notification",
    RunOperation("<TAG>operation:Send Notification</TAG>");,

$function_name == "unknown_route",
    RunOperation("<TAG>operation:Ask For Clarification</TAG>");,

true,
    WriteToOperationLog("Unrecognised function name: " + $function_name)
);

The final true branch catches any response that does not match a known function name (for example, if the model returns a sentence rather than a single keyword).

Verify the integration

Verifying Part 1 (formal function calling)

  1. Deploy and run the LLM Prompt operation with a user message that should trigger a specific tool (for example, a message about a payroll issue to trigger notify_hr).

  2. In the operation logs, confirm that the log entry shows Function selected: notify_hr (or the expected function name).

  3. Confirm that the dispatcher operation ran and that the corresponding target operation (Notify HR) completed successfully.

  4. Send a message that should not trigger any tool (for example, a general greeting). Confirm that the log shows No function call in response and that no target operation was called unexpectedly.

  5. If the model consistently selects the wrong tool, review the tool descriptions. Descriptions that overlap or lack specificity cause the model to guess incorrectly.

  6. If JSONParser raises an error, confirm that choices[0].message.tool_calls[0].function.arguments is a valid JSON string in the raw response. An empty or malformed arguments field typically indicates a model or prompt configuration issue.

Verifying Part 2 (prompt-based routing)

  1. Deploy and run the LLM Prompt operation with a message that maps clearly to one available function.

  2. In the operation logs, confirm that the log entry shows Routing to: followed by the expected function name as a single word.

  3. Confirm that the corresponding target operation ran and completed successfully.

  4. Send an ambiguous message. Confirm that the log shows Routing to: unknown_route and that the clarification operation ran.

  5. If the final true branch fires unexpectedly, inspect the raw choices[0].message.content value in the log. A response containing extra text (such as "I would call: get_account_info") indicates the system prompt instruction needs to be more directive. Add an explicit example to the prompt, such as: Example response: get_account_info.