CPC Comparison AI Agent
Multi-Agent Benefits Comparison System
Google ADK · Vertex AI Agent Engine · Health Insurance Operations
THE CHALLENGE
Manual Benefits Administration is Broken
Benefits Administrators spend significant time manually cross-referencing health plan configurations — comparing deductibles, copays, and benefit categories across products to ensure accurate benefits coding.
Time-Intensive
Hours spent cross-referencing spreadsheets across multiple plan documents and benefit categories
Error-Prone
Manual data entry creates risks of miscoding benefits, leading to claims issues and compliance concerns
Spreadsheet-Driven
No structured tooling — comparisons exist only in ad hoc Excel files without version control or audit trails
THE SOLUTION
A Pipeline of Specialized Sub-Agents
Outcome: Natural language input: "Compare Plan A effective 2024-01-15 with Plan B" → Fully formatted Excel comparison report generated automatically — no manual steps.
ARCHITECTURE
System Architecture Diagram
Multi-Agent Benefits Comparison System
Google ADK · Vertex AI Agent Engine · Health Insurance Operations
Natural Language Input
Orchestration Layer
Interprets multi-turn conversations · Routes intent to sub-agents · Manages session state across turns · Coordinates pipeline execution
Specialized Sub-Agents
Extracts plan codes and effective dates from free-form natural language queries
Retrieves Plan Highlights, Benefit Details, and Plan Summaries from health insurance APIs
Structured diff across all benefit categories · Color-coded Excel report with 5 analytical sheets
Automated Output
AGENT CAPABILITIES
What Each Agent Does
Orchestrator Agent
Primary Agent- Interprets multi-turn conversation context
- Routes user intent to correct sub-agent
- Manages state across conversation turns
- Coordinates end-to-end pipeline execution
CPC Intake Agent
Sub-Agent- Extracts plan codes from free-form natural language
- Parses effective dates (e.g. 2024-01-15)
- Validates extracted parameters before downstream use
- Handles ambiguous or incomplete query inputs
Documents Agent
Sub-Agent- Retrieves Plan Highlights from insurance APIs
- Fetches Benefit Details and Plan Summaries
- Handles API authentication and error states
- Stores structured JSON for downstream comparison
Comparison Agent
Sub-Agent- Performs structured diff across all benefit categories
- Identifies deductible, copay, and OOP differences
- Exports color-coded Excel with 5 analytical sheets
- Flags MATCH vs DIFF per benefit line item
EVALUATION FRAMEWORK
Evaluation Was Harder Than Building
A single query can trigger: intent classification → tool selection → API calls → data transformation → file generation. Any break in that chain produces a wrong answer — traditional unit tests don't catch routing failures or hallucinated tool arguments.
- ✓ Intent → agent sequence mapping
- ✓ Correct agent routing per query
- ✓ Routing failure detection
- ✓ JSON schema integrity check
- ✓ Data structure validation
- ✓ Saved response verification
- ✓ Real difference detection
- ✓ False positive prevention
- ✓ Benefit category coverage
- ✓ Sheet structure validation
- ✓ Plan code header alignment
- ✓ Difference highlighting check
KEY LEARNINGS
What the AI Lab Taught Us
Agent Specialization Matters
Smaller, focused agents are easier to route, test, and iterate on independently. Avoid monolithic agents.
Build Evaluators Early
Multi-turn systems need custom evaluation logic before you can trust outputs. Retrofitting evals is painful.
Mock + Live Eval Modes
Essential separation: Mock mode for fast inner-loop dev; Live API mode for real-world validation signal.
Structured Output Contracts
Defining expected tool args, agent sequences, and output schemas upfront made debugging dramatically faster.
Google ADK + Vertex AI
A promising foundation for enterprise multi-agent deployment with built-in session management and tool routing.
NEXT STEPS
From Prototype to Production
Production Hardening
Error recovery, retry logic, input sanitization, and security review
Data Quality & Governance
Audit trails, output validation, and human-in-the-loop checkpoints
Workflow Integration
Connect to existing Benefits Admin tools and claims systems
Scale Evaluation
Expand eval coverage with more plan types and edge case queries
Note: This is an AI Lab prototype — meaningful work ahead before production deployment. Exploring multi-agent patterns in regulated industries requires careful evaluation design.