CaseStudies
MassGen v0.0.3: Berkeley Agentic AI Summit Summary
This case study demonstrates MassGen’s ability to handle specialized research queries with strict constraints, showcasing how agents can recognize and prioritize responses that precisely meet user specifications while maintaining high academic standards. This case study was run on version v0.0.3.
Command
massgen --config @examples/basic/multi/gemini_4o_claude "give me all the talks on agent frameworks in Berkeley Agentic AI Summit 2025, note, the sources must include the word Berkeley, don't include talks from any other agentic AI summits"
Prompt: give me all the talks on agent frameworks in Berkeley Agentic AI Summit 2025, note, the sources must include the word Berkeley, don’t include talks from any other agentic AI summits
Agents
- Agent 1: gemini-2.5-flash (Designated Representative Agent)
- Agent 2: gpt-4o
- Agent 3: claude-3-5-haiku
Watch the recorded demo:
The Collaborative Process
Research Approaches and Constraint Adherence
Each agent demonstrated different levels of precision in adhering to the user’s strict source requirements:
-
Agent 1 (gemini-2.5-flash) focused on framework-specific talks with explicit mention of technologies like DSPy, Google Agent Development Kit (ADK), and Model Context Protocol (MCP) while carefully ensuring Berkeley-sourced information.
-
Agent 2 (gpt-4o) provided comprehensive session coverage with detailed workshop information, organizing content by sessions and explicitly noting Berkeley sources in the response structure.
-
Agent 3 (claude-3-5-haiku) took a broader approach, including research presentations like SkyRL Framework, Maris Project, and CVE-Bench, but with less specificity on core agent frameworks mentioned in the user’s request.
Critical Constraint Recognition
A defining feature of this session was the agents’ ability to recognize and evaluate adherence to the user’s explicit constraints:
-
Source Verification: Agent 1 explicitly acknowledged in its voting that it “correctly ensured that the sources included the word ‘Berkeley’ and did not include talks from other summits, fulfilling all constraints of the original message.”
-
Framework Specificity: Agent 1’s second vote specifically noted its “more focused list of talks directly related to ‘agent frameworks’, explicitly mentioning specific frameworks like DSPy and the Google Agent Development Kit (ADK).”
-
Precision Recognition: Agent 2 emphasized that its response “ensures relevancy by explicitly mentioning Berkeley, per the original request.”
The Voting Pattern: Constraint Compliance Excellence
The voting process revealed sophisticated evaluation of constraint adherence and research precision:
-
Self-Assessment with Constraint Awareness: Agent 1 voted for itself twice, with both votes explicitly referencing constraint compliance and framework specificity.
-
Quality vs. Constraint Tension: Agent 2 voted for itself, recognizing its comprehensive coverage while emphasizing Berkeley source compliance.
-
Cross-Agent Validation: Agent 3 voted for Agent 1, praising it for providing “the most comprehensive and verified information about the Berkeley Agentic AI Summit 2025, with specific details about agent framework talks sourced directly from the summit’s materials.”
-
Final Consensus: Agent 1 achieved majority support (2 out of 3 votes), with agents specifically recognizing its superior constraint adherence and framework precision.
The Final Answer: Constrained Research Mastery
Agent 1 was selected to present the final answer, featuring:
- Strict Source Compliance: All information explicitly tied to Berkeley-sourced materials
- Framework Precision: Talk titles that explicitly mentioned specific frameworks (DSPy, ADK, MCP, LlamaIndex)
- Constraint Acknowledgment: Clear distinction from other non-Berkeley agentic AI summits
- Technical Accuracy: Proper identification of framework-specific versus general agentic system content
- Comprehensive Coverage: Both keynotes and technical sessions while maintaining constraint adherence
Conclusion
This case study showcases MassGen’s exceptional ability to handle research queries with explicit constraints, demonstrating how agents can recognize, evaluate, and prioritize responses that precisely meet user specifications. The system successfully identified and promoted the approach that best balanced comprehensive research with strict constraint adherence.
Agent 1’s methodology—which explicitly tracked constraint compliance while maintaining technical precision—ultimately earned recognition from other agents who could evaluate both the quality of research and adherence to user requirements.
This demonstrates MassGen’s strength in academic and professional contexts where following specific guidelines and constraints is as important as research comprehensiveness, making it particularly valuable for compliance-sensitive research tasks and constrained information gathering scenarios.