Jitterbit Document Compliance Agent (Azure)

Overview

Jitterbit provides the Document Compliance Agent (Azure) to customers through Jitterbit Marketplace. This agent automates compliance-focused document processing by retrieving PDF files from Azure Blob Storage, extracting their text content using optical character recognition (OCR), and applying AI-powered analysis to detect and remove personally identifiable information (PII), helping organizations meet data privacy requirements without manual document review.

The agent retrieves PDF files from a configured source container, submits each file to an OCR service for analysis, and collects the full extracted text. An LLM then analyzes the extracted text in two passes: once to identify and store PII data, and again to produce a sanitized version with all PII removed. Both output files are written to an SFTP server, the original PDF is archived to a separate Azure Blob Storage archive container and deleted from the source container, and email notifications are sent to configured recipients when processing errors occur.

The agent performs the following tasks:

• Retrieves a list of PDF files from a configured Azure Blob Storage source container.
• Submits each PDF to Azure Document Intelligence for OCR text extraction and collects the extracted text.
• Sends extracted text to an LLM to detect and extract PII data, and writes the output to a file on an SFTP server.
• Sends extracted text to the LLM to produce a cleaned version with all PII removed, and writes the output to a file on an SFTP server.
• Archives processed PDF files to a configured destination Azure Blob Storage archive container and removes the originals from the source container.
• Sends email notifications to configured recipients when processing errors occur.

This document explains how to set up and operate this AI agent. It covers architecture, prerequisites, and steps to install, configure, and operate the AI agent.

AI agent architecture

This AI agent connects Azure Blob Storage, Azure Document Intelligence, an LLM, and an SFTP server to extract and sanitize text from PDF documents. A typical processing run follows these steps:

1. The Initial Controller operation initializes processing variables and triggers the blob listing step.
2. The agent connects to the configured Azure Blob Storage source container and retrieves a list of available PDF files.
3. For each file, the agent constructs the Document Intelligence source URL and submits the PDF to Azure Document Intelligence for OCR processing.
4. The agent retrieves the completed OCR analysis result and collects the full extracted text.
5. The agent sends the extracted text to the LLM with a PII-detection prompt. The identified PII data is written to a file on the SFTP server.
6. The agent sends the extracted text to the LLM with a data-cleaning prompt to produce a sanitized version. The cleaned text is written to a file on the SFTP server.
7. The original PDF is moved to the Azure Blob Storage archive container and deleted from the source container.
8. Email notifications are sent to configured recipients if any errors occur during processing.

Workflow diagram

The following diagram shows the main processing pipeline for the Document Compliance Agent (Azure).

Prerequisites

You need the following components to use this AI agent.

Harmony components

You must have a Jitterbit Harmony license with access to the following components:

• Jitterbit Studio
• Document Compliance Agent (Azure) purchased as a license add-on

Supported endpoints

The AI agent connects to the following endpoints. You can accommodate other systems by modifying the project's endpoint configurations and workflows.

Large language model (LLM)

The agent uses Azure AI Foundry to access large language models for PII detection and data sanitization. Azure AI Foundry is a managed platform that provides access to models from providers including OpenAI, Microsoft, and Meta. You must have an Azure subscription with an Azure AI Foundry hub and project configured and a model deployed and accessible via the Foundry API.

Azure Blob Storage

The agent uses Azure Blob Storage as both the PDF source and the archive destination. You must have an Azure subscription with two containers configured: one for incoming PDF files and one for archiving processed files. Both containers require shared access signature (SAS) tokens with read, write, and list permissions.

Azure Document Intelligence

The agent uses Azure Document Intelligence for OCR extraction from PDF files. You must have an Azure Document Intelligence resource with a valid endpoint URL and API key.

SFTP

The agent writes processed output files (PII data and cleaned text) to an SFTP server. You must have an SFTP server accessible from Jitterbit with valid connection credentials.

Email

The agent sends error notifications via SMTP email. The default configuration uses Gmail (smtp.gmail.com). You must have a sender email account with SMTP access enabled and, if using Gmail, an app password configured.

Installation, configuration, and operation

Follow these steps to install, configure, and operate this AI agent:

1. Download and install the project
2. Configure Azure resources
3. Configure project variables
4. Test connections
5. Deploy the project
6. Review project workflows
7. Trigger the project workflows

For troubleshooting guidance, see Troubleshooting.

Download and install the project

Follow these steps to install the Studio project for the AI agent:

1. Log in to the Harmony portal at https://login.jitterbit.com and open Marketplace.

2. Locate the AI agent named Jitterbit Document Compliance Agent (Azure). To locate the agent, use the search bar or, in the Filters pane under Type, select AI Agent to limit the display to AI agents.

3. Click the agent's Documentation link to open its documentation in a separate tab. Keep the tab open to refer back to after starting the project.

4. Click Start Project to open a configuration dialog.

   Note

   If you have not yet purchased the AI agent, Get agent is displayed instead. Click it to open an informational dialog, then click Submit to have a representative contact you about purchasing the AI agent.

   Tip

   The configuration dialog includes a warning not to import the template before applying endpoint customizations. That warning does not apply to this AI agent and can be ignored. Follow the recommended order of steps in this documentation instead.

5. In the Create a New Project dialog, select an environment where the Studio project will be created, then click Create Project.

6. After the progress dialog indicates the project is created, use the dialog link Go to Studio or open the project directly from the Studio Projects page.

Configure Azure resources

Before configuring project variables, set up the required Azure resources.

Create Azure Blob Storage containers

1. Sign in to the Azure portal and search for Storage accounts in the top search bar.

2. Click + Create, fill in the subscription, resource group, and storage account name, set Performance to Standard and Redundancy to LRS, then click Review + Create and Create.

3. After deployment completes, click Go to resource, then go to Data storage > Containers.

4. Click + Container, enter a name for the source container, set the access level to Private, and click Create.

5. Repeat step 4 to create an archive container.

6. For each container, generate a SAS token:

   1. In the storage account, go to Security + networking > Shared access signature.
   2. Enable Blob services, Container and Object resource types, and Read, Write, and List permissions.
   3. Set an expiry date and click Generate SAS and connection string.
   4. Copy the SAS token immediately — it is shown only once.

Create an Azure Document Intelligence resource

1. In the Azure portal, click + Create a resource, search for Document Intelligence, and click Create.

2. Configure the resource, selecting Standard S0 as the pricing tier for production workloads.

3. Click Review + Create, then Create. After deployment completes, click Go to resource.

4. Go to Resource Management > Keys and Endpoint and copy the endpoint URL and Key 1 for use in the project variables.

Create an Azure AI Foundry project

1. Go to ai.azure.com and sign in with your Azure account.

2. Click New hub, fill in the hub configuration (subscription, resource group, name, and region), and click Create.

3. Inside the hub, click New project, enter a project name, and click Create.

4. Go to Settings > Project details and copy the project endpoint URL and API key for use in the project variables.

Configure project variables

In the Studio project installed from Marketplace, set values for the following project variables.

To configure project variables, use the project's actions menu and select Project Variables to open the configuration drawer.

Azure Blob Storage

Azure Document Intelligence

Azure AI Foundry

SFTP

Email

Test connections

Test the endpoint configurations to verify connectivity using the defined project variable values.

To test connections, go to the design component palette's Project endpoints and connectors tab, hover over each endpoint, and click Test.

Deploy the project

Deploy the Studio project.

To deploy the project, use the project's actions menu and select Deploy.

Review project workflows

The Studio project contains one workflow that implements the Document Compliance Agent (Azure) processing pipeline.

PDF Parser

Initial Controller

The Initial Controller operation serves as the entry point for the workflow. It runs the Run Orchestrator#1 script to trigger the List Blobs From Azure Container operation, then executes logging scripts and the Logging and Notification Rollup script to record processing details and call the Send Email operation based on success or failure counts.

List Blobs From Azure Container

The List Blobs From Azure Container operation connects to the configured Azure Blob Storage source container and retrieves a list of available blobs. The response is transformed and stored in a variable, and the Extract Blob Names from Response script iterates through each blob name to call the Document Intelligence Source URL Formation operation.

Document Intelligence Source URL Formation

The Document Intelligence Source URL Formation operation runs the DI Source URL Formation script, which constructs the fully qualified source URL for the PDF file and calls the Post PDF Data to Azure DI operation to begin OCR processing.

Post PDF Data to Azure DI

The Post PDF Data to Azure DI operation transforms the Azure Document Intelligence request body and posts the PDF to Azure Document Intelligence via HTTP for analysis. The response is transformed, stored in a variable, and logged. The operation then delays execution to allow processing to complete before calling the Get Azure DI Analysis Result operation.

Get Azure DI Analysis Result

The Get Azure DI Analysis Result operation performs an HTTP GET to retrieve the completed OCR analysis result from Azure Document Intelligence. The response is transformed and stored in a variable for use in subsequent LLM processing steps.

Azure Foundry Request Formation

The Azure Foundry Request Formation operation reads the Document Intelligence result from the variable, transforms the content, and runs the PII Request Formation for Foundry script to prepare and dispatch requests for PII detection, clean data processing, and blob management.

Post DI Data to Foundry for PII Data

The Post DI Data to Foundry for PII Data operation transforms the Azure AI Foundry PII request body and posts the extracted text to Azure AI Foundry via HTTP with a PII-detection prompt. The LLM response is transformed, stored in a variable, and logged.

Write PII JSON Data to SFTP

The Write PII JSON Data to SFTP operation reads the PII data from the variable, transforms the content, and runs the Write PII Data in a file script to write the PII output to a file on the SFTP server and call the Post DI Data to Foundry for Clean Data operation.

Post DI Data to Foundry for Clean Data

The Post DI Data to Foundry for Clean Data operation runs the Clean Data Request Formation for Foundry script, transforms the request body, and posts the extracted text to Azure AI Foundry with a data-cleaning prompt to produce a sanitized version with all PII removed. The response is transformed, stored in a variable, and logged.

Write Clean JSON Data to SFTP

The Write Clean JSON Data to SFTP operation reads the clean data from the variable, transforms the content, and runs the Write Clean Data in a file script to write the sanitized output to a file on the SFTP server and call the Move Blob to Archive Container operation.

Move Blob to Archive Container

The Move Blob to Archive Container operation transforms the move request body and sends an HTTP request to copy the processed PDF from the source container to the archive container. The response is transformed and stored in a variable.

Delete Blob from Source Container

The Delete Blob from Source Container operation sends an HTTP DELETE request to remove the original PDF from the source container after it has been successfully archived, ensuring the file is not reprocessed on subsequent runs.

Error handling

On Failure Email

The On Failure Email operation runs the Send Error Email Script to log error details and call the Send Email operation.

Send Email

The Send Email operation transforms the email body, sends the notification via the configured Email adapter, and transforms any error response.

Trigger the project workflows

To run the Document Compliance Agent (Azure), deploy and run the Initial Controller operation. In Studio, hover over the operation and click the Deploy and Run icon in the top-right corner of the operation tile.

To automate the pipeline, configure operation schedules on the Initial Controller operation to run at your preferred frequency.

Troubleshooting

If you encounter issues, review the operation logs for detailed troubleshooting information.

For additional assistance, contact Jitterbit support.