Java Programmers and Coders: Automation

AI Workflow Orchestrator — Build Multi-LLM Workflows with Spring Boot

A lightweight, Spring Boot–friendly plugin framework to orchestrate complex AI workflows — integrating multiple LLMs, local tools (@AiTool), and MCP services with zero boilerplate.

By Ravinderjeet Singh Nagpal

Clean, minimal cover – no external hosting needed (inline SVG).

Introduction

Artificial Intelligence is no longer limited to single-model interactions. Real-world apps often orchestrate multiple LLMs, integrate MCP services, and combine them with custom tools — all in one flow. The challenge is putting this together without heavy frameworks, glue code, or slow startup.

AI Workflow Orchestrator solves this with a lightweight, Spring Boot–friendly approach: multi-LLM support, @AiTool local tools, MCP clients, dynamic planning, schema transformations, and both CLI/Web execution — with zero boilerplate.

Table of Contents

Features
Project Structure
Quick Start
Local LLM Setup with Ollama
API Usage
Architecture
Workflow Types & Step Types
Plug & Play Workflow Creation
Real-World Plug-and-Play Examples
Web UI: Document Summarizer
Advanced Configuration
CLI Mode
Logging
Timeouts
Environment Variables
Troubleshooting
License
Conclusion & What’s Next

🚀 Features

🧠 Multi-LLM Support: OpenAI, Anthropic, and Ollama via custom HTTP clients
⚙️ Dynamic Workflow Planning: AI-driven step orchestration
🔌 MCP Protocol Support: HTTP, WebSocket & Async clients
🛠️ Tool Integration: Register & invoke @AiTool methods
🔄 Schema Transformation: JSON schema-based conversions
🖥️ CLI & Web Modes: Dual execution interfaces
🧩 Intelligent Transformations: LLM-powered data conversions
🪶 Lightweight: No external AI framework dependencies
⚡ Performance: Direct HTTP API calls with Java HttpClient

📁 Project Structure

SpringAIOrchestrator/
├── ai-orchestrator-bom/                 # Bill of Materials & Core Framework
│   └── ai-orchestrator-core/            # Core orchestration engine
├── examples/                            # Example workflow implementations
│   ├── ai-travel-workflow/              # Travel planning workflow example
│   ├── ai-resume_builder-workflow/      # Resume building workflow example
│   └── ai-document_summarizer-workflow/ # Summarizing + classification
├── LICENSE
├── README.md
└── USAGE.md

🛠️ Quick Start

✅ Prerequisites

☕ Java 21+
📦 Maven 3.8+
🔑 API Keys (OpenAI / Anthropic / Ollama)
🧠 Local Ollama Installation (optional but recommended)

⚙️ Code Installation

# 1) Clone
git clone <repository-url>
cd SpringAIOrchestrator/spring-ai-orchestrator

# 2) Build the framework
cd ai-orchestrator-bom
mvn clean install

# 3) Env Vars
export OPENAI_API_KEY="your-openai-key"
export ANTHROPIC_API_KEY="your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"
export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"
export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10

# 4) Run an example
cd ../examples/ai-travel-workflow
mvn spring-boot:run

# or
cd ../ai-resume_builder-workflow
mvn spring-boot:run

🧠 Local LLM Setup with Ollama

The framework can run fully offline using Ollama, letting you test workflows locally without API keys or cloud costs.

Install Ollama

# macOS
brew install ollama

# Linux
curl -fsSL https://ollama.com/install.sh | sh

Windows: Download the installer from ollama.com/download.

Using Ollama for Local (Free) Models

Ensure Ollama server is running at http://localhost:11434
Verify by opening http://localhost:11434 in a browser
Ollama models are quantized by default (e.g., q4); choose tags if needed

Common Pull Commands & Config

Goal	Ollama CLI	`models.yml`
Pull Default (q4)	`ollama pull llama2`	`modelName: "llama2"`
Pull Specific (e.g., 3-bit)	`ollama pull llama2:7b-q3_K_L`	`modelName: "llama2:7b-q3_K_L"`
Pull Latest	`ollama pull llama3`	`modelName: "llama3"`
Coder Model	`ollama pull qwen2.5-coder:14b-instruct-q3_K_L`	`modelName: "qwen2.5-coder:14b-instruct-q3_K_L"`
Lightweight	`ollama pull mistral`	`modelName: "mistral"`

Local Models Config Example

models:
  - alias: "local"
    provider: "ollama"
    modelName: "llama2"
    baseUrl: "http://localhost:11434"
    enabled: true

Using Ollama Cloud Models

# Sign in
ollama signin

# Pull a cloud-offloaded model
ollama pull qwen3-coder:480b-cloud

models:
- alias: "default"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"
  baseUrl: "http://localhost:11434"
  enabled: true

- alias: "transformer"
  provider: "ollama"
  modelName: "qwen3-coder:480b-cloud"
  baseUrl: "http://localhost:11434"
  enabled: true

🌐 API Usage

Run Travel Workflow

curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Plan a 5-day trip to Tokyo in March with budget of $3000"

curl -X POST "http://localhost:8282/api/workflows/run/travel-planner" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Business trip to London for 3 days, need flights and hotel recommendations"

Run Resume Workflow

curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Create resume for Senior Java Developer with 5 years Spring Boot experience"

curl -X POST "http://localhost:8282/api/workflows/run/resume-builder" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "prompt=Build resume for Data Scientist role with Python and ML expertise"

List Workflows

curl -X GET "http://localhost:8282/api/workflows/list"

{
  "travel-planner": "travel-planner - AI-powered travel planning with weather, flights, and cost analysis",
  "resume-builder": "AI Resume Builder - Create a professional resume from user inputs and job descriptions"
}

🏗️ Architecture

🧠 Planner LLM – Generates and ranks intelligent execution plans
🔄 Transformer LLM – Handles format conversions between steps
🔗 Custom HTTP Clients – Direct API integration without external AI frameworks
🪶 Zero Heavy Dependencies – No LangChain4j or similar

┌─────────────────────────────────────────────────────────────────────────┐
│                        AI Orchestrator Framework                        │
└─────────────────────────────────────────────────────────────────────────┘

┌──────────────┐  ┌──────────────┐  ┌──────────────┐
│ User Input   │  │ REST API     │  │ CLI Mode     │
└──────┬───────┘  └──────┬───────┘  └──────┬───────┘
       │                 │                 │
       └─────────────────┼─────────────────┘
                         │
               ┌─────────▼─────────┐
               │ Workflow Controller│
               └─────────┬─────────┘
                         │
              ┌──────────▼──────────┐
              │ Dynamic Engine (AI) │
              └──────────┬──────────┘
                         │
           ┌─────────────▼─────────────┐
           │  🧠 Planner + 🔄 Transformer │
           └─────────────┬─────────────┘
                         │
           ┌─────────────▼─────────────┐
           │    Step Execution Engine   │
           └────────────────────────────┘

Everything connected: LLMs + @AiTool local tools + MCP services → Orchestrator → Outputs.

🔌 Workflow Types & Step Types

Workflow Types

🔁 Sequential – Predefined step order
🧠 Dynamic – AI-planned execution order

Step Types

🤖 LLM – Language model interactions
🔧 Tool – Custom @AiTool methods
🌐 MCP – External service calls (HTTP, WebSocket, Async)

⚡ Plug & Play Workflow Creation

Add Dependency

<dependency>
  <groupId>codetumblr.net.ai.orchestrator</groupId>
  <artifactId>ai-orchestrator-core</artifactId>
  <version>1.0-SNAPSHOT</version>
</dependency>

Create Application

@SpringBootApplication
@ComponentScan({"codetumblr.net.ai.orchestrator.core", "your.package"})
public class MyWorkflowApplication {
  public static void main(String[] args) {
    SpringApplication.run(MyWorkflowApplication.class, args);
  }
}

Define Models (models.yml)

models:
  - alias: "default"
    provider: "openai"
    model: "gpt-4"
    apiKey: "${OPENAI_API_KEY}"

Create Workflow (workflows/my-workflow.yaml)

id: "document-analyzer"
executionMode: "dynamic"
steps:
  - id: "extract_content"
    type: "tool"
    parameters:
      tool: "document_extractor"
  - id: "analyze_sentiment"
    type: "llm"
    parameters:
      model: "default"
      prompt: "Analyze sentiment: {input}"

Add Tools

@Component
public class DocumentTools {
  @AiTool(name = "document_extractor")
  public String extractText(String filePath) {
    return "Extracted content...";
  }
}

Run

mvn spring-boot:run

📊 Real-World Plug-and-Play Workflow Examples

🛫 Travel Planner – Multi-day itineraries with weather, flights, and cost analysis
📄 Resume Builder – Tailored resumes from job descriptions/skills
📜 Document Summarizer – Extract, summarize, classify, and store metadata (PDF, DOCX, etc.) using a dynamic workflow

📂 Sample Workflow Implementations

Below are some practical examples showing how the AI Workflow Orchestrator can be extended with custom tools, MCP services, and dynamic orchestration — all without additional boilerplate.

📑 Example 1: Document Analyzer Workflow

This workflow demonstrates how to build an AI-powered document analyzer that extracts text, summarizes content, classifies it into categories, and stores metadata — all in a single orchestration pipeline.


id: "document-analyzer"
name: "AI Document Analyzer & Classifier"
description: "Extract, summarize, classify, and persist document insights."
version: "1.0"
executionMode: "dynamic"
outputMode: "all"

steps:
  - id: "extract_text"
    type: "mcp"
    description: "Extract text from documents with OCR fallback."
    parameters:
      client: "doc-processor"
      service: "extract_text"

  - id: "summarize_content"
    type: "llm"
    description: "Generate a human-readable summary."
    parameters:
      model: "default"
      promptTemplate: "prompts/document/summarize.txt"

  - id: "suggest_categories"
    type: "tool"
    description: "Suggest relevant categories based on content."
    parameters:
      tool: "suggest_categories"

  - id: "classify_document"
    type: "llm"
    description: "Classify into the most relevant category."
    parameters:
      model: "default"
      promptTemplate: "prompts/document/classify.txt"

  - id: "store_metadata"
    type: "tool"
    description: "Persist classification and summary results."
    parameters:
      tool: "save_document_metadata"

---

🛠️ Example 2: Local Tool - Category Suggestion

Local tools can be created with @AiTool to enrich workflows. Here’s a sample tool that suggests relevant categories for a given document text.


@Component
public class CategoryTool {

    @AiTool(
        name = "suggest_categories",
        description = "Suggest relevant document categories based on content."
    )
    public List suggestCategories(String text) {
        return List.of("Finance", "Healthcare", "Legal", "Technology");
    }
}

---

🌐 Example 3: Internal MCP Service - Document Text Extraction

MCP services allow external-like service calls to be part of the same workflow. The example below shows how to implement a document extraction service with OCR fallback.


@MCPService(
    name = "doc-processor",
    description = "Extract text from documents and scanned PDFs."
)
public class DocumentProcessor {

    @MCPMethod(
        name = "extract_text",
        description = "Extracts text with auto-detection and OCR fallback."
    )
    public Map extractText(String docName, String type, String base64Content) {
        // Example implementation: decode + process content
        Map result = new HashMap<>();
        result.put("document", docName);
        result.put("text", "Extracted content goes here...");
        result.put("status", "Success");
        return result;
    }
}

---

📌 Why These Matter

These examples demonstrate how easily you can:

🔁 Combine LLM, local tools, and MCP services into one workflow.
⚙️ Extend capabilities with zero boilerplate using @AiTool and @MCPService.
🌐 Enable both internal and external integrations within the same orchestration engine.

Upcoming Examples (Planned)

🧠 Intelligent Chat Agent
📈 Financial Report Analyzer
🧪 Code Review Assistant

🖥️ UI / Web Integration — Document Summarizer

The framework supports Spring Boot web controllers for file upload and dynamic prompt generation.

How it works: Upload → Base64 encode → Build planner prompt → Review → Submit to Orchestrator → Planner routes to initial MCP step (extract_text).

Run the Web Example

cd examples/ai-document_summarizer-workflow
mvn spring-boot:run

Open: http://localhost:8383/

Key Endpoints

Endpoint	Method	Description
`/`	GET	Upload form
`/generate-prompt`	POST	Base64 encodes file, builds planner prompt
`/submit-prompt`	POST	Submits to `/api/workflows/run/document-summarizer`

⚙️ Advanced Configuration Options

`application.yml`

server:
  port: ${SERVER_PORT:8282}
spring:
  application:
    name: ${APP_NAME:ai-orchestrator-document-summarizer}
 ...

ai:
  orchestrator:
    workflows:
      path: ${WORKFLOWS_PATH:classpath*:workflows/*-workflow.yaml}
      prompt:
        path: ${PLANNER_PROMPT_PATH:}
    transformer:
      prompt:
        path: ${TRANSFORMER_PROMPT_PATH:}
    mcp:
      clients:
        path: classpath:${MCP_CLIENTS_PATH:/mcp-clients.yml}

`mcp-clients.yml`

clients:
  # ============================================================
# MCP Clients Configuration for Document Summarizer Workflow
# ============================================================
# This configuration defines the MCP services used for text extraction.
# All other steps (summarization, classification, metadata storage)
# are handled by LLMs or tools within the orchestrator.
#
# Environment variables can override default values for flexibility.
# ============================================================

clients:
  # ----------------------------------------------------------
  # Document Processor Service (Primary MCP for Text Extraction)
  # ----------------------------------------------------------
  - name: "document-processor"
    endpoint: "${MCP_DOCUMENT_PROCESSOR_ENDPOINT:http://localhost:8383}"
    service: "extract_text"
    protocol: "${MCP_DOCUMENT_PROCESSOR_PROTOCOL:http}"
    timeout: "${MCP_DOCUMENT_PROCESSOR_TIMEOUT:15000}"   # Default: 15s timeout
    auth:
      type: "${MCP_DOCUMENT_PROCESSOR_AUTH_TYPE:none}"  # Supported: none, basic, oauth2, api-key, jwt

      # --- Basic Auth (Optional) ---
      username: "${MCP_DOCUMENT_PROCESSOR_USERNAME:}"
      password: "${MCP_DOCUMENT_PROCESSOR_PASSWORD:}"

      # --- API Key Auth (Optional) ---
      apiKey: "${MCP_DOCUMENT_PROCESSOR_API_KEY:}"
      apiKeyHeader: "${MCP_DOCUMENT_PROCESSOR_API_KEY_HEADER:X-API-Key}"

      # --- OAuth2 (Optional) ---
      tokenUri: "${MCP_DOCUMENT_PROCESSOR_TOKEN_URI:}"
      clientId: "${MCP_DOCUMENT_PROCESSOR_CLIENT_ID:}"
      clientSecret: "${MCP_DOCUMENT_PROCESSOR_CLIENT_SECRET:}"
      scope: "${MCP_DOCUMENT_PROCESSOR_SCOPE:}"

      # --- JWT Auth (Optional) ---
      jwtIssuer: "${MCP_DOCUMENT_PROCESSOR_JWT_ISSUER:}"
      jwtSubject: "${MCP_DOCUMENT_PROCESSOR_JWT_SUBJECT:}"
      jwtSecret: "${MCP_DOCUMENT_PROCESSOR_JWT_SECRET:}"

    enabled: "${MCP_DOCUMENT_PROCESSOR_ENABLED:true}"

`models.yml`

config:
  defaultMaxTokens: 1000
  promptTruncateLength: 50
  descriptionTruncateLength: 30
  autoRegisterForPlugins: true

models:
  - alias: "default"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.7
    maxTokens: 2000
    enabled: true

  - alias: "openai"
    provider: "openai"
    modelName: "gpt-4o-mini"
    apiKey: "${OPENAI_API_KEY}"
    temperature: 0.7
    maxTokens: 4000
    enabled: true

  - alias: "planner"
    provider: "ollama"
    modelName: "qwen2.5-coder:14b-instruct-q3_K_L"
    baseUrl: "http://localhost:11434"
    temperature: 0.3
    maxTokens: 1500
    enabled: true

  - alias: "transformer"
    provider: "ollama"
    modelName: "qwen3-coder:480b-cloud"
    baseUrl: "http://localhost:11434"
    temperature: 0.1
    maxTokens: 1000
    enabled: true

🧪 CLI Mode

export CLI_MODE=true
mvn spring-boot:run
# or
CLI_MODE=true mvn spring-boot:run
# or
java -DCLI_MODE=true -jar ai-travel-workflow-1.0-SNAPSHOT.jar

Commands

# List workflows
lw

# Interactive
interactive
i
run-interactive

# Run specific
run --workflow-id travel-planner --prompt "Plan weekend trip to Paris"
run-workflow --workflow-id resume-builder --prompt "Create resume for DevOps engineer"

# Quick run
quick --number 1
q --number 2

# Help / Logs
help
h
?
log-info
logs
debug-help

# Exit
exit
quit
bye

🧰 Logging

logging:
  config: classpath:log4j2.xml

⏱️ Timeouts

# Planner (configurable)
ai:
  orchestrator:
    planner:
      max-iterations: 5
      target-rank: 10
      fallback-enabled: true

# MCP client timeouts (per client)
clients:
  - name: "weather-services-api"
    timeout: "5000"
  - name: "flight-services-api"
    timeout: "5000"

Hardcoded timeouts (not config-based yet):
LLM Planning Timeout: 120s · Planning Session Timeout: 60s · LLM Client Timeouts: 5m request / 30s connect · Schema Caching: ConcurrentHashMap · HTTP Client: Java 21 HttpClient

🔧 Environment Variables Reference

Core Application

export SERVER_PORT=8282
export CLI_MODE=false
export APP_NAME="ai-orchestrator"

LLM API Keys

export OPENAI_API_KEY="sk-your-openai-key"
export ANTHROPIC_API_KEY="sk-ant-your-anthropic-key"
export OLLAMA_BASE_URL="http://localhost:11434"

Planner Configuration

export PLANNER_FALLBACK=true
export PLANNER_MAX_ITERATIONS=5
export PLANNER_TARGET_RANK=10
export STEP_SEPARATOR=","
export PLANNER_PROMPT_PATH=""
export TRANSFORMER_PROMPT_PATH=""

File Paths

export WORKFLOWS_PATH="classpath*:workflows/*-workflow.yaml"
export MODELS_PATH="/models.yml"
export MCP_CLIENTS_PATH="/mcp-clients.yml"

Logging Levels

export PLANNER_LOG_LEVEL=INFO
export ENGINE_LOG_LEVEL=INFO
export WORKFLOW_LOG_LEVEL=INFO
export MCP_SERVICE_LOG_LEVEL=INFO
export SCHEMA_LOG_LEVEL=INFO

🆘 Troubleshooting

Issue	Solution
`Schema not found`	Verify schema files exist in `resources/schemas/`
`LLM client not available`	Check API keys and network connectivity
`Workflow not found`	Ensure workflow YAML is in `resources/workflows/`
`Tool execution failed`	Verify tool parameters and implementation
`MCP service timeout`	Check MCP client configuration and endpoints
`Planning failed`	Enable planner fallback: `PLANNER_FALLBACK=true`

Health Checks

curl -X GET "http://localhost:8282/api/workflows/list"
echo $OPENAI_API_KEY
echo $ANTHROPIC_API_KEY
echo $OLLAMA_BASE_URL

Debug Steps

# Enable debug
export PLANNER_LOG_LEVEL=DEBUG
export ENGINE_LOG_LEVEL=DEBUG
export WORKFLOW_LOG_LEVEL=DEBUG

# Tail logs
tail -f logs/ai-orchestrator-**.log

# Validate YAML
cat src/main/resources/models.yml
cat src/main/resources/mcp-clients.yml

📄 License

Licensed under the MIT License. See LICENSE in the repository.

👨‍💻 Author

Ravinderjeet Singh Nagpal
Building next-generation AI orchestration systems.

📊 Performance Features

No External AI Frameworks — custom HTTP clients instead of LangChain4j
Smaller JAR Size — reduced by ~15–20MB
Faster Startup — no auto-configuration overhead
Direct API Control — optimized request/response handling

🔗 Related Projects

Spring Boot – Java Backend Framework
Model Context Protocol – MCP Specification

🏁 Conclusion — Build Smarter AI Pipelines

AI Workflow Orchestrator unifies LLMs, @AiTool local tools, and MCP services in a single, lightweight, Spring-friendly framework. With dynamic planning, schema transformations, and direct HTTP clients, it’s a practical foundation for modern AI apps — from document pipelines to enterprise AI agents.

⭐ Next Steps

Try built-in workflows (Travel Planner, Resume Builder, Document Summarizer)
Create your own workflow YAML in minutes
Run locally with Ollama or integrate with cloud LLMs

Have ideas or feedback? Share them in the comments — I’d love to hear how you’re building AI workflows.