CIAO👋 Core Components: Orchestration — Coordinating Conversations, Interfaces, and Agents
Orchestration requirements come from two places: the problem you're solving and the architecture you chose. Understanding both is essential to getting coordination right.
This article is part of a series exploring how would an architectural framework for AI-Application look like. We called it CIAO👋 architectural framework. Previous instalments have covered Conversations, Interfaces, and Agents.
With Conversations, Interfaces, and Agents each explored in depth, we turn to the component that connects them. Orchestration often gets treated as implementation wiring, the thing you figure out after the interesting design decisions have been made. That can lead to significant challenges later, when coordination gaps surface in production rather than in design.
Orchestration is a distinct architectural concern with its own requirements, its own design space, and its own failure modes. In the same way that Conversations merit deliberate design rather than defaulting to whatever message structure an API offers, Orchestration merits deliberate analysis rather than emerging as a byproduct of agent and interface decisions.
A working definition:
Orchestration is the runtime coordination that determines what happens next, which component handles it, and how information flows between them.
Two Sources of Orchestration Requirements
Following the same principle that guides the Agents article, namely that we need to understand the problem before reaching for solutions, we can identify two distinct sources of orchestration requirements.
Domain-derived requirements are constraints that exist in the problem space before any architecture decisions are made. They are discovered through domain analysis, not created through design. These include:
Sequencing constraints: dependencies inherent in the domain (fraud assessment must precede payout authorisation)
Consistency requirements: how aligned shared state must be across participants
Accountability chains: traceability of decisions to responsible parties
Human involvement requirements: mandated checkpoints and approvals
Temporal constraints: how fast coordination itself must happen
Recovery requirements: what must happen when something fails
Observability requirements: what must be visible, when, and to whom
Cross-component policy coherence: how tightly C, I, and A policies must align
These are the non-negotiable elements (unless one decides to reach for core organisational re-design that is!). They set the floor for any orchestration approach.
Architecture-induced requirements emerge from design decisions made across Conversations, Interfaces, and Agents. Decomposing work across multiple agents creates handoff and routing requirements. Supporting multiple interface modalities creates translation and synchronisation needs. Structuring conversations with explicit context introduces state management overhead. None of these coordination challenges exist in the problem domain. They are consequences of architectural choices.
This separation gives architects a useful design heuristic: if architecture-induced orchestration requirements grow disproportionate to domain-derived ones, the architecture may be more complex than the problem demands.
Making this concrete
Consider an insurance claims processing application. The domain-derived orchestration requirements are substantial and exist regardless of how you build the system. Fraud assessment must complete before payout authorisation. The regulator requires a decision audit trail. Human approval is mandated above a monetary threshold. Certain document types must be verified before assessment can begin.
Now layer in architectural decisions. You choose three specialised agents: one for document intake, one for assessment, one for payout. You support both voice and text interfaces for the claimant. You maintain separate conversation threads for the customer and the adjuster.
Each of these decisions creates additional orchestration requirements. The three agents need handoff protocols. The two interface modalities need synchronisation so that a claimant who starts on the phone and switches to a web form does not lose context. The separate conversation threads need state coordination so the adjuster’s view reflects what the customer has provided.
The domain-derived requirements are the cost of the problem. The architecture-induced requirements are the cost of the solution. Both are real, but only the second category is within your direct control to reduce.
A Taxonomy of Orchestration Approaches
With requirements understood, we can examine the design space for meeting them. The primary dimension is locus of control, what decides what happens next.
Zero orchestration applies when there is a single prompt, a single model call, and a single response. In essence, there is almost nothing to coordinate. This is appropriate when domain-derived requirements are minimal and the application involves one agent, one interface, and one conversation. A document Q&A tool that takes a question and returns an answer is a good example. There is no sequencing, no handoff, no state to manage across components. If your requirements genuinely fit here, adding orchestration adds complexity without value.
Model-intrinsic orchestration lets the model itself reason about task decomposition, sequencing, tool selection, and delegation. The model creates its own coordination logic at inference time. This is powerful and flexible, but the coordination reasoning is embedded in model behaviour. That has implications: observability is harder because the model’s routing decisions are not declared in advance. Accountability is harder because the path through the system is emergent rather than defined.
Framework-imposed orchestration defines coordination structures externally. State machines, directed graphs, explicit routing rules. Agents operate within defined nodes. This offers predictability and auditability. You can inspect the coordination logic without running the system. You can verify that compliance requirements are met structurally rather than hoping the model respects them. The cost is upfront design effort and reduced flexibility.
Hybrid orchestration combines the two. External structure defines phases, boundaries, and guardrails. Within those boundaries, model-intrinsic reasoning handles coordination. The insurance claims example might use framework-imposed orchestration for the overall process flow (intake → assessment → approval → payout) while allowing model-intrinsic reasoning within the assessment phase to determine which checks to run and in what order.
Two secondary dimensions further characterise the design space.
Static vs dynamic topology: is the coordination pattern fixed at design time or can it adapt at runtime?
Centralised vs distributed: is there a single orchestrating component or do multiple components negotiate coordination among themselves?
The right position on these dimensions is driven by requirements. High accountability, observability, and temporal constraints favour more explicit, centralised, and static approaches. Lower requirements on those dimensions create room for model-intrinsic, distributed, and dynamic coordination. Now, keep in mind that this is not a maturity ladder. Zero orchestration should not be considered primitive and hybrid is not necessarily advanced. Each fits different requirement profiles.
Orchestration, Governance, and Policies
Three concepts that are closely related but architecturally distinct.
Orchestration is operational. It is the runtime mechanism that routes work, manages state, sequences steps, and handles failures.
Governance is normative. It defines the boundaries within which orchestration operates: what is permissible, who is accountable, how compliance is verified. Governance should remain stable even when orchestration implementations change. You should be able to rework your coordination approach without renegotiating your governance model. If you cannot, the two are too tightly coupled.
Policies operate at multiple levels within a CIAO application. Conversation policies govern dialogue interaction, who can speak, what can be said, what actions are permitted. These are owned and enforced by the Conversation component. Agent constraints define autonomy boundaries and permitted actions, owned by each Agent. Orchestration policies govern coordination behaviour, routing rules, retry strategies, escalation triggers, owned by the Orchestration component. Domain policies capture business rules and regulatory requirements, defined through governance and enforced across components.
The architectural principle: each component enforces its own policies, but orchestration needs visibility into the policies of other components to coordinate coherently. It cannot route a task to an agent whose autonomy level does not permit that kind of decision. It cannot initiate a conversation that violates a conversation policy. Governance is the layer that ensures consistency across these policy boundaries.
Orchestration’s Relationship to C, I, and A
Orchestration has a bidirectional relationship with each of the other three components, and this is what distinguishes it within the framework.
Conversations generate events and state changes that inform orchestration decisions. A customer providing a key piece of information in conversation might trigger orchestration to route that information to an assessment agent. Orchestration also routes information into and across conversations, and may initiate or close conversations based on workflow needs.
Interfaces produce turns that orchestration must process and route. Orchestration determines what gets rendered back and through which interface, particularly when an application supports multiple modalities. A voice interface and a text interface presenting the same conversation require orchestration to manage the translation and synchronisation.
Agents receive work assignments through orchestration and return results. But the relationship is not purely top-down. Agents’ autonomy levels, capabilities, and constraints shape what orchestration can ask of them. An agent with limited autonomy cannot be assigned open-ended reasoning tasks. An agent with high autonomy may complete work without checking back, which orchestration needs to account for.
This bidirectional dependency across all three components is what makes orchestration the connective tissue of a CIAO application, and why its requirements emerge from two sources rather than one.
For Builders
When designing orchestration for an AI-powered application:
Start by mapping domain-derived orchestration requirements. These are your non-negotiable constraints. They exist before you write a line of code and they will outlast any specific implementation.
Track architecture-induced requirements as you design. As you make decisions across Conversations, Interfaces, and Agents, note the coordination overhead each decision creates. Ask whether it is proportionate to the problem you are solving.
Let your accountability and observability requirements guide locus of control. If regulators need to audit decision paths, framework-imposed or hybrid orchestration is likely necessary. If you are building an internal tool with low stakes and high tolerance for variability, model-intrinsic orchestration may be sufficient.
Be explicit about which policies live at which layer. Ambiguity about who enforces what leads to gaps, usually discovered in production. Document the policy boundaries between Conversations, Agents, and Orchestration.
Test governance independence. Can you change your orchestration implementation without changing your governance model? If you cannot, the two may be too tightly coupled. The goal is operational flexibility within normative stability.
Orchestration requirements come from two sources: the problem domain and the architecture you choose. Keeping that distinction clear is the most useful thing an architect can do early. If your coordination overhead is primarily architecture-induced, that is a signal to simplify. If it is primarily domain-derived, the problem genuinely requires sophisticated orchestration and you should invest accordingly.
The right amount of orchestration is the minimum that satisfies both sources of requirements. No less, because the domain constraints are non-negotiable. No more, because every additional coordination mechanism is complexity you must maintain, debug, and explain.