umbertorighetti.dev

Orchestrator Orchestrator

We’ve built the bridge (Part 2) and enabled the bot to find its own work (Part 3). Now, it’s time to look at the Orchestrator: the central hub that decides which agent is right for the job and manages the complex reasoning loops.

🧠 The Orchestrator: Smart Routing

The Orchestrator is an event-driven component that listens for GitHubTaskEvent messages published by the Fetcher. It makes a critical decision based on the task's metadata:

Backlog items are routed to the Analyst Agent to break down requirements.
Todo items are handed to the Coding Agent for implementation.

🧵 Leveraging Java 21 Virtual Threads

One of the most powerful features of Beto-Bot is its use of Virtual Threads. Traditional platform threads are expensive; if an agent gets stuck in a long reasoning loop or waits for a slow API response, it could hang the entire application.

By using Thread.ofVirtual().start(), we ensure that every agent runs in its own lightweight, non-blocking thread. As Mr.Burns would say:

"Excellent".

private void runAgent(GithubTask task, List<Tool> tools, Agent agent){
    logger.info(">>> Assigning {} agent for task: {} <<<", task.type(), task.number());
    Thread.ofVirtual().start(() -> {
        try {
            agent.start(task, tools);
        } catch (Exception e) {
            logger.error("Virtual Thread with agent failed: {}", e.getMessage());
        }
    });
}

This architecture allows Beto-Bot to handle multiple issues simultaneously, essentially giving you a parallel "AI dev team".

🧑‍🔬 Specialized Agents

Instead of one giant prompt, I refactored the Agent into an abstract class. This allows each sub-agent to have a dedicated prompt and keeps the code clean and organized.

For example, the Coding Agent is prompted as a "Senior Java Developer" with specific instructions to create feature branches and use push_files. This specialization significantly reduces hallucinations and ensures the agent follows the specific workflow.

🔁 Handling the Multi-Tool Challenge

A major lesson learned during development was that advanced models like Gemini 3.1 often want to call multiple tools at once to be efficient. My early code would crash because it only expected one FunctionCall at a time.

I then went to an updated approach where the agent fetched all requested tool calls, executed them iteratively, and fed the results back into the history before the agent proceeded:

List<FunctionCall> toolCalls = fetchAllToolCalls(modelResponse);
if (!toolCalls.isEmpty()) {
    toolCalls.forEach(call -> executeToolAndAddToHistory(call, history));
} else {
    // Check for final answer...
}

After upgrading my Google AI Client to Spring's more agnostic ChatClient, it became even simpler. As usual Spring does the heavy lifting for you. As long as you're using the Spring AI mcp client, Spring supplies you with something called a ToolCallbackProvider. This interface uses ToolCallbacks to register tools. More on the ChatClient and tools in the next post!

What’s Next?

Next time we will deep dive into the agent's code and prompts and how we use them to control the behavior of the bot.