Best practices for building AI agents in Jitterbit Harmony

Introduction

This document outlines the recommended architecture, conventions, and best practices for developing AI agents in Jitterbit Harmony using Jitterbit Integration Studio. These guidelines ensure scalability, maintainability, and consistency across all AI agents you build.

Core component architecture and naming conventions

A well-designed AI agent consists of modular, decoupled workflows with clear boundaries between logic, data processing, and external integrations.

Your architecture and naming conventions (critical for readability and debugging) should be based on these four core components, shown with examples in the diagram below:

① Main Entry

A single main entry workflow handles incoming API requests, authentication, and routing to the AI logic. It may be triggered by Slack, Microsoft Teams, a custom UI, a custom API endpoint, etc.

• Naming convention: Main Entry - [Handler Name]
• Example: Main Entry - Slack API Request Handler

② AI Logic

A single AI logic workflow serves as the central brain of the agent. It handles context retrieval, LLM calls (OpenAI, Amazon Bedrock, etc.), and routing to specific tools. This is the central orchestration layer responsible for reasoning, building prompts, routing to the correct tools, and returning formatted responses.

• Naming convention: Main - [Agent Name] Logic
• Example: Main - AI Agent Tools Logic

③ Tools (Optional)

You may have multiple independent modules that perform atomic data operations. These workflows encapsulate a specific business function or data source interaction. They should be stateless and reusable.

• Naming convention: Tool - [Action]
• Examples:
  • Tool - Customer Order Details
  • Tool - Get Account Details
  • Tool - Retrieve Case History

④ Utilities (Optional)

You may have multiple shared helper processes. These are supportive or cross-cutting operations. For example, blob uploads, error logging, or datastore writes.

• Naming convention: Utility - [Action]
• Examples:
  • Utility - Upload Customer Order Forms to Azure
  • Utility - Upload to Azure Blob
  • Utility - Format JSON

Workflow design and decoupling

To ensure your agent is maintainable and scalable, you must enforce a strict separation of concerns.

Decouple tools from the main logic

Each tool workflow (such as Tool - Get Accounts) must be standalone and callable independently. It should not assume any UI or AI orchestration context.

Ensure UI independence

The Main - Agent Logic workflow must not depend on any specific user interface such as Slack, Microsoft Teams, or a generic API. The main entry workflow (Slack handler, generic handler, etc.) should only translate incoming requests into a common internal payload format for LLM calls. The following diagram shows this layered architecture:

Centralize configuration values with project variables

Define environment-specific configurations (API keys, endpoints, etc.) as project variables, not hardcoded values.

LLM integration

Do not embed direct API calls to LLMs within your main orchestration logic. Instead, create an LLM abstraction layer and construct prompts dynamically.

Create an LLM abstraction layer

Create a dedicated utility workflow for invoking the LLM (such as Utility - Call Azure OpenAI). This makes switching to another LLM provider such as Amazon Bedrock or Anthropic Claude trivial.

Construct prompts dynamically

Construct prompts dynamically but in a controlled structure:

• Include context (retrieved data, query intent).
• Use system and user roles clearly when supported by the LLM API.
• Maintain temperature and token limits via project variable configuration.

Manage session and context data

Store transient session and user question-and-answer context in a datastore. Jitterbit provides an out-of-the-box cloud-native datastore for this purpose: Jitterbit Cloud Datastore.

Build generic utilities for reusability

Build generic utilities if common to multiple workflows so that they can be reused across workflows. For example:

• Utility - Upload to Azure Blob
• Utility - Write to Datastore
• Utility - Parse Slack Payload
• Utility - Call OpenAI

Avoid common pitfalls

Take care not to implement these common mistakes:

• Hardcoding configuration instead of using project variables.

• Embedding UI formatting logic (Slack, Teams) in core AI workflows.

• Creating monolithic workflows that handle multiple unrelated processes.

Document your AI agent project

Each AI agent project should include the following documentation to help new developers and for future maintenance:

• Architecture diagram (showing endpoints, main workflows, tools, datastore).

• List of source and consumer endpoints.

• List of Azure Blob containers and datastore tables created.

• List of API Manager custom APIs and their purpose.