AI Workflow Orchestrator — Build Multi-LLM Workflows with Spring Boot

A lightweight, Spring Boot–friendly plugin framework to orchestrate complex AI workflows — integrating multiple LLMs, local tools (@AiTool), and MCP services with zero boilerplate.

By Ravinderjeet Singh Nagpal

Clean, minimal cover – no external hosting needed (inline SVG).

Introduction

Artificial Intelligence is no longer limited to single-model interactions. Real-world apps often orchestrate multiple LLMs, integrate MCP services, and combine them with custom tools — all in one flow. The challenge is putting this together without heavy frameworks, glue code, or slow startup.

AI Workflow Orchestrator solves this with a lightweight, Spring Boot–friendly approach: multi-LLM support, @AiTool local tools, MCP clients, dynamic planning, schema transformations, and both CLI/Web execution — with zero boilerplate.

Table of Contents

• Features
• Project Structure
• Quick Start
• Local LLM Setup with Ollama
• API Usage
• Architecture
• Workflow Types & Step Types
• Plug & Play Workflow Creation
• Real-World Plug-and-Play Examples
• Web UI: Document Summarizer
• Advanced Configuration
• CLI Mode
• Logging
• Timeouts
• Environment Variables
• Troubleshooting
• License
• Conclusion & What’s Next

🚀 Features

• 🧠 Multi-LLM Support: OpenAI, Anthropic, and Ollama via custom HTTP clients
• ⚙️ Dynamic Workflow Planning: AI-driven step orchestration
• 🔌 MCP Protocol Support: HTTP, WebSocket & Async clients
• 🛠️ Tool Integration: Register & invoke @AiTool methods
• 🔄 Schema Transformation: JSON schema-based conversions
• 🖥️ CLI & Web Modes: Dual execution interfaces
• 🧩 Intelligent Transformations: LLM-powered data conversions
• 🪶 Lightweight: No external AI framework dependencies
• ⚡ Performance: Direct HTTP API calls with Java HttpClient

📁 Project Structure

SpringAIOrchestrator/
├── ai-orchestrator-bom/                 # Bill of Materials & Core Framework
│   └── ai-orchestrator-core/            # Core orchestration engine
├── examples/                            # Example workflow implementations
│   ├── ai-travel-workflow/              # Travel planning workflow example
│   ├── ai-resume_builder-workflow/      # Resume building workflow example
│   └── ai-document_summarizer-workflow/ # Summarizing + classification
├── LICENSE
├── README.md
└── USAGE.md

🛠️ Quick Start

✅ Prerequisites

• ☕ Java 21+
• 📦 Maven 3.8+
• 🔑 API Keys (OpenAI / Anthropic / Ollama)
• 🧠 Local Ollama Installation (optional but recommended)

⚙️ Code Installation

# 1) Clone
git clone <repository-url>
cd SpringAIOrchestrator/spring-ai-orchestrator

# 2) Build the framework
cd ai-orchestrator-bom
mvn clean install

# 3) Env Vars
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"
export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"
export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10

# 4) Run an example
cd ../examples/ai-travel-workflow
mvn spring-boot:run

# or
cd ../ai-resume_builder-workflow
mvn spring-boot:run

🧠 Local LLM Setup with Ollama

The framework can run fully offline using Ollama, letting you test workflows locally without API keys or cloud costs.

Install Ollama

# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

Windows: Download the installer from ollama.com/download.

Using Ollama for Local (Free) Models

• Ensure Ollama server is running at http://localhost:11434
• Verify by opening http://localhost:11434 in a browser
• Ollama models are quantized by default (e.g., q4); choose tags if needed

Common Pull Commands & Config

Goal	Ollama CLI	models.yml
Pull Default (q4)	ollama pull llama2	modelName: "llama2"
Pull Specific (e.g., 3-bit)	ollama pull llama2:7b-q3_K_L	modelName: "llama2:7b-q3_K_L"
Pull Latest	ollama pull llama3	modelName: "llama3"
Coder Model	ollama pull qwen2.5-coder:14b-instruct-q3_K_L	modelName: "qwen2.5-coder:14b-instruct-q3_K_L"
Lightweight	ollama pull mistral	modelName: "mistral"

Local Models Config Example

models:
  - alias: "local"
    provider: "ollama"
    modelName: "llama2"
    baseUrl: "http://localhost:11434"
    enabled: true

Using Ollama Cloud Models

# Sign in
ollama signin

# Pull a cloud-offloaded model
ollama pull qwen3-coder:480b-cloud

models:
- alias: "default"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"
  baseUrl: "http://localhost:11434"
  enabled: true

- alias: "transformer"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"
  baseUrl: "http://localhost:11434"
  enabled: true

🌐 API Usage

Run Travel Workflow

curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Plan a 5-day trip to Tokyo in March with budget of $3000"

curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Business trip to London for 3 days, need flights and hotel recommendations"

Run Resume Workflow

curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Create resume for Senior Java Developer with 5 years Spring Boot experience"

curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Build resume for Data Scientist role with Python and ML expertise"

List Workflows

curl -X GET "http://localhost:8282/api/workflows/list"

{
  "travel-planner": "travel-planner - AI-powered travel planning with weather, flights, and cost analysis",
  "resume-builder": "AI Resume Builder - Create a professional resume from user inputs and job descriptions"
}

🏗️ Architecture

• 🧠 Planner LLM – Generates and ranks intelligent execution plans
• 🔄 Transformer LLM – Handles format conversions between steps
• 🔗 Custom HTTP Clients – Direct API integration without external AI frameworks
• 🪶 Zero Heavy Dependencies – No LangChain4j or similar

┌─────────────────────────────────────────────────────────────────────────┐
│                        AI Orchestrator Framework                        │
└─────────────────────────────────────────────────────────────────────────┘

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ User Input   │  │ REST API     │  │ CLI Mode     │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       └─────────────────┼─────────────────┘
                         │
               ┌─────────▼─────────┐
               │ Workflow Controller│
               └─────────┬─────────┘
                         │
              ┌──────────▼──────────┐
              │ Dynamic Engine (AI) │
              └──────────┬──────────┘
                         │
           ┌─────────────▼─────────────┐
           │  🧠 Planner + 🔄 Transformer │
           └─────────────┬─────────────┘
                         │
           ┌─────────────▼─────────────┐
           │    Step Execution Engine   │
           └────────────────────────────┘

Everything connected: LLMs + @AiTool local tools + MCP services → Orchestrator → Outputs.

🔌 Workflow Types & Step Types

Workflow Types

• 🔁 Sequential – Predefined step order
• 🧠 Dynamic – AI-planned execution order

Step Types

• 🤖 LLM – Language model interactions
• 🔧 Tool – Custom @AiTool methods
• 🌐 MCP – External service calls (HTTP, WebSocket, Async)

⚡ Plug & Play Workflow Creation

1. Add Dependency

<dependency>
  <groupId>codetumblr.net.ai.orchestrator</groupId>
  <artifactId>ai-orchestrator-core</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>

2. Create Application

@SpringBootApplication
@ComponentScan({"codetumblr.net.ai.orchestrator.core", "your.package"})
public class MyWorkflowApplication {
  public static void main(String[] args) {
    SpringApplication.run(MyWorkflowApplication.class, args);
  }
}

3. Define Models (models.yml)

models:
  - alias: "default"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"

4. Create Workflow (workflows/my-workflow.yaml)

id: "document-analyzer"
executionMode: "dynamic"
steps:
  - id: "extract_content"
    type: "tool"
    parameters:
      tool: "document_extractor"
  - id: "analyze_sentiment"
    type: "llm"
    parameters:
      model: "default"
      prompt: "Analyze sentiment: {input}"

5. Add Tools

@Component
public class DocumentTools {
  @AiTool(name = "document_extractor")
  public String extractText(String filePath) {
    return "Extracted content...";
  }
}

6. Run

mvn spring-boot:run

📊 Real-World Plug-and-Play Workflow Examples

• 🛫 Travel Planner – Multi-day itineraries with weather, flights, and cost analysis
• 📄 Resume Builder – Tailored resumes from job descriptions/skills
• 📜 Document Summarizer – Extract, summarize, classify, and store metadata (PDF, DOCX, etc.) using a dynamic workflow

📂 Sample Workflow Implementations

Below are some practical examples showing how the AI Workflow Orchestrator can be extended with custom tools, MCP services, and dynamic orchestration — all without additional boilerplate.

---

📑 Example 1: Document Analyzer Workflow

This workflow demonstrates how to build an AI-powered document analyzer that extracts text, summarizes content, classifies it into categories, and stores metadata — all in a single orchestration pipeline.

id: "document-analyzer"
name: "AI Document Analyzer & Classifier"
description: "Extract, summarize, classify, and persist document insights."
version: "1.0"
executionMode: "dynamic"
outputMode: "all"

steps:
  - id: "extract_text"
    type: "mcp"
    description: "Extract text from documents with OCR fallback."
    parameters:
      client: "doc-processor"
      service: "extract_text"

  - id: "summarize_content"
    type: "llm"
    description: "Generate a human-readable summary."
    parameters:
      model: "default"
      promptTemplate: "prompts/document/summarize.txt"

  - id: "suggest_categories"
    type: "tool"
    description: "Suggest relevant categories based on content."
    parameters:
      tool: "suggest_categories"

  - id: "classify_document"
    type: "llm"
    description: "Classify into the most relevant category."
    parameters:
      model: "default"
      promptTemplate: "prompts/document/classify.txt"

  - id: "store_metadata"
    type: "tool"
    description: "Persist classification and summary results."
    parameters:
      tool: "save_document_metadata"

---

🛠️ Example 2: Local Tool - Category Suggestion

Local tools can be created with @AiTool to enrich workflows. Here’s a sample tool that suggests relevant categories for a given document text.

@Component
public class CategoryTool {

    @AiTool(
        name = "suggest_categories",
        description = "Suggest relevant document categories based on content."
    )
    public List suggestCategories(String text) {
        return List.of("Finance", "Healthcare", "Legal", "Technology");
    }
}

---

🌐 Example 3: Internal MCP Service - Document Text Extraction

MCP services allow external-like service calls to be part of the same workflow. The example below shows how to implement a document extraction service with OCR fallback.

@MCPService(
    name = "doc-processor",
    description = "Extract text from documents and scanned PDFs."
)
public class DocumentProcessor {

    @MCPMethod(
        name = "extract_text",
        description = "Extracts text with auto-detection and OCR fallback."
    )
    public Map extractText(String docName, String type, String base64Content) {
        // Example implementation: decode + process content
        Map result = new HashMap<>();
        result.put("document", docName);
        result.put("text", "Extracted content goes here...");
        result.put("status", "Success");
        return result;
    }
}

---

📌 Why These Matter

These examples demonstrate how easily you can:

• 🔁 Combine LLM, local tools, and MCP services into one workflow.
• ⚙️ Extend capabilities with zero boilerplate using @AiTool and @MCPService.
• 🌐 Enable both internal and external integrations within the same orchestration engine.

Upcoming Examples (Planned)

• 🧠 Intelligent Chat Agent
• 📈 Financial Report Analyzer
• 🧪 Code Review Assistant

🖥️ UI / Web Integration — Document Summarizer

The framework supports Spring Boot web controllers for file upload and dynamic prompt generation.

How it works: Upload → Base64 encode → Build planner prompt → Review → Submit to Orchestrator → Planner routes to initial MCP step (extract_text).

Run the Web Example

cd examples/ai-document_summarizer-workflow
mvn spring-boot:run

Open: http://localhost:8383/

Key Endpoints

Endpoint	Method	Description
/	GET	Upload form
/generate-prompt	POST	Base64 encodes file, builds planner prompt
/submit-prompt	POST	Submits to /api/workflows/run/document-summarizer

⚙️ Advanced Configuration Options

application.yml

server:
  port: ${SERVER_PORT:8282}
spring:
  application:
    name: ${APP_NAME:ai-orchestrator-document-summarizer}
 ...

ai:
  orchestrator:
    workflows:
      path: ${WORKFLOWS_PATH:classpath*:workflows/*-workflow.yaml}
      prompt:
        path: ${PLANNER_PROMPT_PATH:}
    transformer:
      prompt:
        path: ${TRANSFORMER_PROMPT_PATH:}
    mcp:
      clients:
        path: classpath:${MCP_CLIENTS_PATH:/mcp-clients.yml}

mcp-clients.yml

clients:
  # ============================================================
# MCP Clients Configuration for Document Summarizer Workflow
# ============================================================
# This configuration defines the MCP services used for text extraction.
# All other steps (summarization, classification, metadata storage)
# are handled by LLMs or tools within the orchestrator.
#
# Environment variables can override default values for flexibility.
# ============================================================

clients:
  # ----------------------------------------------------------
  # Document Processor Service (Primary MCP for Text Extraction)
  # ----------------------------------------------------------
  - name: "document-processor"
    endpoint: "${MCP_DOCUMENT_PROCESSOR_ENDPOINT:http://localhost:8383}"
    service: "extract_text"
    protocol: "${MCP_DOCUMENT_PROCESSOR_PROTOCOL:http}"
    timeout: "${MCP_DOCUMENT_PROCESSOR_TIMEOUT:15000}"   # Default: 15s timeout
    auth:
      type: "${MCP_DOCUMENT_PROCESSOR_AUTH_TYPE:none}"  # Supported: none, basic, oauth2, api-key, jwt

      # --- Basic Auth (Optional) ---
      username: "${MCP_DOCUMENT_PROCESSOR_USERNAME:}"
      password: "${MCP_DOCUMENT_PROCESSOR_PASSWORD:}"

      # --- API Key Auth (Optional) ---
      apiKey: "${MCP_DOCUMENT_PROCESSOR_API_KEY:}"
      apiKeyHeader: "${MCP_DOCUMENT_PROCESSOR_API_KEY_HEADER:X-API-Key}"

      # --- OAuth2 (Optional) ---
      tokenUri: "${MCP_DOCUMENT_PROCESSOR_TOKEN_URI:}"
      clientId: "${MCP_DOCUMENT_PROCESSOR_CLIENT_ID:}"
      clientSecret: "${MCP_DOCUMENT_PROCESSOR_CLIENT_SECRET:}"
      scope: "${MCP_DOCUMENT_PROCESSOR_SCOPE:}"

      # --- JWT Auth (Optional) ---
      jwtIssuer: "${MCP_DOCUMENT_PROCESSOR_JWT_ISSUER:}"
      jwtSubject: "${MCP_DOCUMENT_PROCESSOR_JWT_SUBJECT:}"
      jwtSecret: "${MCP_DOCUMENT_PROCESSOR_JWT_SECRET:}"

    enabled: "${MCP_DOCUMENT_PROCESSOR_ENABLED:true}"

models.yml

config:
  defaultMaxTokens: 1000
  promptTruncateLength: 50
  descriptionTruncateLength: 30
  autoRegisterForPlugins: true

models:
  - alias: "default"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.7
    maxTokens: 2000
    enabled: true

  - alias: "openai"
    provider: "openai"
    modelName: "gpt-4o-mini"
    apiKey: "${OPENAI_API_KEY}"
    temperature: 0.7
    maxTokens: 4000
    enabled: true

  - alias: "planner"
    provider: "ollama"
    modelName: "qwen2.5-coder:14b-instruct-q3_K_L"
    baseUrl: "http://localhost:11434"
    temperature: 0.3
    maxTokens: 1500
    enabled: true

  - alias: "transformer"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.1
    maxTokens: 1000
    enabled: true

🧪 CLI Mode

export CLI_MODE=true
mvn spring-boot:run
# or
CLI_MODE=true mvn spring-boot:run
# or
java -DCLI_MODE=true -jar ai-travel-workflow-1.0-SNAPSHOT.jar

Commands

# List workflows
lw

# Interactive
interactive
i
run-interactive

# Run specific
run --workflow-id travel-planner --prompt "Plan weekend trip to Paris"
run-workflow --workflow-id resume-builder --prompt "Create resume for DevOps engineer"

# Quick run
quick --number 1
q --number 2

# Help / Logs
help
h
?
log-info
logs
debug-help

# Exit
exit
quit
bye

🧰 Logging

logging:
  config: classpath:log4j2.xml

⏱️ Timeouts

# Planner (configurable)
ai:
  orchestrator:
    planner:
      max-iterations: 5
      target-rank: 10
      fallback-enabled: true

# MCP client timeouts (per client)
clients:
  - name: "weather-services-api"
    timeout: "5000"
  - name: "flight-services-api"
    timeout: "5000"

Hardcoded timeouts (not config-based yet):
LLM Planning Timeout: 120s · Planning Session Timeout: 60s · LLM Client Timeouts: 5m request / 30s connect · Schema Caching: ConcurrentHashMap · HTTP Client: Java 21 HttpClient

🔧 Environment Variables Reference

Core Application

export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"

LLM API Keys

export OPENAI_API_KEY="sk-your-openai-key"
export ANTHROPIC_API_KEY="sk-ant-your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"

Planner Configuration

export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10
export STEP_SEPARATOR=","
export PLANNER_PROMPT_PATH=""
export TRANSFORMER_PROMPT_PATH=""

File Paths

export WORKFLOWS_PATH="classpath*:workflows/*-workflow.yaml"
export MODELS_PATH="/models.yml"
export MCP_CLIENTS_PATH="/mcp-clients.yml"

Logging Levels

export PLANNER_LOG_LEVEL=INFO
export ENGINE_LOG_LEVEL=INFO
export WORKFLOW_LOG_LEVEL=INFO
export MCP_SERVICE_LOG_LEVEL=INFO
export SCHEMA_LOG_LEVEL=INFO

🆘 Troubleshooting

Issue	Solution
Schema not found	Verify schema files exist in resources/schemas/
LLM client not available	Check API keys and network connectivity
Workflow not found	Ensure workflow YAML is in resources/workflows/
Tool execution failed	Verify tool parameters and implementation
MCP service timeout	Check MCP client configuration and endpoints
Planning failed	Enable planner fallback: PLANNER_FALLBACK=true

Health Checks

curl -X GET "http://localhost:8282/api/workflows/list"
echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY
echo $OLLAMA_BASE_URL

Debug Steps

# Enable debug
export PLANNER_LOG_LEVEL=DEBUG
export ENGINE_LOG_LEVEL=DEBUG
export WORKFLOW_LOG_LEVEL=DEBUG

# Tail logs
tail -f logs/ai-orchestrator-**.log

# Validate YAML
cat src/main/resources/models.yml
cat src/main/resources/mcp-clients.yml

📄 License

Licensed under the MIT License. See LICENSE in the repository.

👨‍💻 Author

Ravinderjeet Singh Nagpal
Building next-generation AI orchestration systems.

📊 Performance Features

• No External AI Frameworks — custom HTTP clients instead of LangChain4j
• Smaller JAR Size — reduced by ~15–20MB
• Faster Startup — no auto-configuration overhead
• Direct API Control — optimized request/response handling

🔗 Related Projects

• Spring Boot – Java Backend Framework
• Model Context Protocol – MCP Specification

🏁 Conclusion — Build Smarter AI Pipelines

AI Workflow Orchestrator unifies LLMs, @AiTool local tools, and MCP services in a single, lightweight, Spring-friendly framework. With dynamic planning, schema transformations, and direct HTTP clients, it’s a practical foundation for modern AI apps — from document pipelines to enterprise AI agents.

⭐ Next Steps

• Try built-in workflows (Travel Planner, Resume Builder, Document Summarizer)
• Create your own workflow YAML in minutes
• Run locally with Ollama or integrate with cloud LLMs

Have ideas or feedback? Share them in the comments — I’d love to hear how you’re building AI workflows.

© 2025 • AI Workflow Orchestrator • Article by Ravinderjeet Singh Nagpal