What if your AI agents could consult multiple models and synthesize the best answer? What if, instead of betting everything on a single LLM provider, you could assemble a panel of experts that reasons together? That is exactly what Ensemble Groups make possible.
Today we are announcing Ensemble Groups -- a new feature in the FireFoundry Broker Service that lets you route a single request to multiple LLM providers simultaneously and synthesize the results into a single, higher-quality response.
What Are Ensemble Groups?
An Ensemble Group is a broker configuration that defines two or more model pools as a collective. When a request arrives at an ensemble group, the broker does not pick one provider -- it queries all of them in parallel. A configurable synthesis step then combines those responses into a single result that is more accurate, more robust, and more trustworthy than any individual model could produce on its own.
Think of it as a panel discussion rather than a solo presentation. Each model brings its own strengths, biases, and reasoning patterns. By combining their perspectives, you get answers that are greater than the sum of their parts.
How It Works
Setting up an Ensemble Group takes just a few steps:
- Configure an ensemble group in the broker with two or more model pools -- for example, Claude, GPT, and Gemini.
- Send a request to the ensemble group endpoint, just like any other broker request.
- The broker routes in parallel to all configured providers simultaneously, respecting each pool's capacity limits.
- A synthesis step combines the results using one of several configurable strategies.
- Return the synthesized result to your agent as a single response.
The synthesis strategies are where the real power lives:
- Consensus: Pick the answer that multiple models agree on. When three models independently arrive at the same conclusion, confidence is high.
- Best-of-N: Generate multiple responses, score them against configurable quality criteria, and select the strongest.
- Specialized routing: Different models handle different aspects of the request. One model excels at reasoning, another at code generation, a third at creative writing. The synthesis step merges their specialized outputs into a cohesive result.
Use Cases
Ensemble Groups shine wherever accuracy matters more than minimizing latency:
- Critical decisions: When your agent is making high-stakes recommendations -- financial analysis, medical triage, compliance determinations -- get multiple opinions before committing to an answer.
- Code review: Have multiple models review the same code. Issues that one model misses, another catches. The union of their findings is far more comprehensive than any single review.
- Content generation: Generate multiple variations of marketing copy, email responses, or reports. Score and select the strongest, or blend the best elements from each.
- Fact checking: Cross-reference answers across models to reduce hallucination. When models disagree, flag the response for human review rather than serving a potentially incorrect answer.
Built on the Industrial Broker
Ensemble Groups are not a bolt-on feature. They are built directly into FireFoundry's industrial-grade Broker Service, which means they inherit all the production capabilities your agents already rely on:
- Capacity management ensures ensemble requests do not overwhelm any individual provider. The broker respects rate limits and token budgets across all pools in the ensemble.
- QoS tiering lets you prioritize ensemble requests appropriately. High-priority ensembles get capacity first; background ensembles yield gracefully.
- Full telemetry shows exactly which models contributed what. Trace each provider's response, compare quality scores, and understand how the synthesis step arrived at the final answer.
Because ensemble requests are just broker requests, they work seamlessly with your existing failover policies, cost controls, and audit logging. There is nothing new to operate.
Custom Synthesis with AgentML
The built-in synthesis strategies -- consensus, best-of-N, and specialized routing -- cover the most common use cases. But what happens when your domain requires something more specific? That is where AgentML comes in.
AgentML is FireFoundry's XML-based domain-specific language for defining executable agent programs and workflows. It is a declarative, procedural language that lets you orchestrate complex multi-step logic -- including custom ensemble synthesis patterns -- without writing TypeScript. AgentML programs execute as async generators, yielding progress updates and returning structured results.
When you configure an ensemble group, you can choose a predefined synthesis strategy or provide a custom AgentML program that defines exactly how responses should be combined. Here is an example of a custom synthesis pattern that scores each model's response, applies domain-specific validation, and merges the best elements:
<agent id="CustomSynthesis" display-name="Domain Synthesis"
description="Custom ensemble synthesis with validation">
<static-args>
<arg name="responses" type="array" required="true"/>
<arg name="originalQuery" type="string" required="true"/>
</static-args>
<run-impl>
<yield-status message="Scoring model responses"/>
<!-- Score each response for relevance and accuracy -->
<let name="scored" value="[]"/>
<loop items="args.responses" as="response" index="idx">
<call-bot name="ResponseScorerBot" result="score">
<arg name="query" value="args.originalQuery"/>
<arg name="response" value="response.content"/>
</call-bot>
<let name="scored" value="[...scored, { ...response, score }]"/>
</loop>
<!-- Select the highest-scoring response as the base -->
<let name="best" value="scored.sort((a,b) => b.score.total - a.score.total)[0]"/>
<!-- If top two scores are close, merge their insights -->
<if condition="scored.length > 1 && scored[0].score.total - scored[1].score.total < 0.1">
<yield-status message="Merging top responses"/>
<call-bot name="ResponseMergerBot" result="merged">
<arg name="primary" value="scored[0].content"/>
<arg name="secondary" value="scored[1].content"/>
<arg name="query" value="args.originalQuery"/>
</call-bot>
<return value="merged"/>
</if>
<return value="best.content"/>
</run-impl>
</agent>This AgentML program receives the array of model responses, runs each through a scoring bot, and then either selects the winner or merges the top two if they are close in quality. The entire process is observable through FireFoundry's telemetry -- you can trace exactly which models contributed what, how they were scored, and why the final answer was chosen.
AgentML is a core part of the FireFoundry Agent SDK, not just an ensemble feature. You can use it to define full agent workflows, multi-step pipelines, and conditional logic -- all declaratively in XML. Because it is XML, it is particularly well-suited for AI-assisted generation: LLMs produce valid AgentML more reliably than they produce TypeScript, making it an excellent choice for dynamically constructed workflows. For the full AgentML reference and getting-started guide, see the developer documentation.
Getting Started
Ensemble Groups are available now in the broker service configuration for all beta partners. Define an ensemble group in your broker config, specify the model pools and synthesis strategy -- or write a custom one in AgentML -- and start sending requests. It is that straightforward.
To learn more about how the broker handles multi-provider routing and capacity management, visit the Platform overview. For hands-on configuration details and the AgentML reference, check the developer documentation.
Ready to try Ensemble Groups? If you are already in the beta, update your broker configuration and start experimenting today. If you are not yet in the beta, request access and we will get you set up.