Challenge 3 - Build the Claims Intelligence Agent¶

Expected Duration: 30 minutes

Overview¶

In this challenge, you will build the Claims Intelligence Agent, the core decision-making component of the claims processing pipeline.

Like Challenge 2, this component is built in two steps: first you create a Foundry IQ knowledge base over the policy documents in Blob Storage, then you create one specialized Foundry agent. The agent receives the knowledge base as an MCP retrieval tool and handles policy extraction and coverage adjudication in one resource.

Together, the Claims Intake Agent and Claims Intelligence Agent form this flow:

flowchart LR A[Claim statement image] --> B[Mistral Document AI OCR] B --> C[Claims Intake Agent] D[Foundry IQ crash statements] --> C C --> E[Intake artifact] G[Policy documents in Blob Storage] --> H[Foundry IQ policies knowledge base] E --> P[Claims Intelligence Agent] H --> P P --> I{Coverage decision} I --> J[Approve] I --> K[Deny] I --> L[Escalate]

Challenge 2 creates claims-intake-agent. This challenge creates claims-intelligence-agent. After completing both challenges, the two Foundry agents required by Challenge 4 are available.

From mock policies to real policy documents¶

Enterprise claims adjudication starts with the model, but it only becomes useful when the model can read the same policy documents your organization uses. In this challenge, the Claims Intelligence Agent is the reasoning component, and the Foundry IQ knowledge base over the Storage account policies container is its enterprise grounding layer.

In short: policy documents → Foundry IQ knowledge base → Claims Intelligence Agent → coverage decision. The policy documents remain the source of truth. The agent retrieves the matching document, structures its policy fields, and adjudicates the claim in one Foundry resource.

The Claims Intelligence Agent combines Policy Matching and Coverage Validation into a single orchestrated flow:

Retrieve the policy document that matches the claim's policy number through the Foundry IQ knowledge base.
Extract structured policy fields (coverage types, limits, deductibles, exclusions) from the grounded document using a dedicated Foundry agent.
Run multi-factor coverage analysis using an LLM-backed adjudicator.
Apply the compliance rules engine to identify exclusions and limits.
Score how consistent the reported crash details are with the policy on a 1-100 scale, driving the approve/deny/escalate outcome.
Escalate edge cases with structured reasoning and severity levels.
Emit a detailed coverage decision with confidence scores, a consistency score, and a full audit trail.

Important

The supplied Claims Intelligence implementation is complete. Do not modify the Python code in this challenge. Run the commands below and validate each result.

Prerequisites¶

Complete Challenge 2.
Keep the .env configuration in the repository root.
Confirm the MicroHack platform or your event coach supplied access to the Foundry project and the Azure Storage account.

Tasks¶

Task 1: Create the policies knowledge base¶

Create a Foundry IQ Blob knowledge source over the Storage account's policies container and a policies-kb knowledge base over that source:

From the docs directory, run:

cd docs
python create_knowledge_base.py --policies

Foundry IQ creates and runs the ingestion resources asynchronously. The five Markdown documents already uploaded to the policies container become retrievable through the knowledge base without application code downloading the blobs.

The helper configures this knowledge base with Minimal retrieval reasoning and Extractive data output. This works without a separate model deployment on the knowledge base. The Claims Intelligence Agent still uses its Foundry model to interpret the extracted policy fields and reason about claim coverage; the knowledge base handles document retrieval, not adjudication.

If coverage validation reports that a Knowledge Base model must be specified for a reasoning effort other than Minimal, the policies-kb knowledge base was created before this setting was added. Rerun python create_knowledge_base.py --policies from the docs directory. The helper updates the existing knowledge base in place, then retry the coverage check.

Expected result:

Knowledge source 'policies-blob-ks' created or updated from Blob container 'policies'.
Knowledge base 'policies-kb' created or updated.

Open the Azure AI Search service in the Azure portal. Under Agentic retrieval, confirm that policies-blob-ks appears under Knowledge sources and policies-kb appears under Knowledge bases. Wait for the knowledge source synchronization to finish before continuing.

Task 2: Prepare the Claims Intelligence Agent¶

Create or update claims-intelligence-agent. It retrieves policy documents through the policies-kb MCP tool, converts them into structured policy information, and compares that policy with the structured claim to produce a coverage decision.

python claims-intelligence-agent.py --setup-agent

Expected result:

Foundry agent 'claims-intelligence-agent' is ready.

Open your Foundry project and confirm that claims-intelligence-agent appears under Agents with policies-knowledge-base and crash-statements-knowledge-base MCP tools. It calls knowledge_base_retrieve against the policies tool when extracting coverage types, limits, deductibles, and exclusions. The crash-statements tool lets the same agent inspect indexed accident evidence when requested.

Verify the crash-statements connection with a known indexed statement:

python claims-intelligence-agent.py --verify-crash-statement crash1_front

The Foundry IQ MCP endpoint uses the Search API version from FOUNDRY_IQ_MCP_API_VERSION; the default for this lab is 2026-08-01-preview. If setup or verification fails with an unsupported api-version error, confirm that the Search service supports that version. Only override the variable with a version supported by your Search service. The deployment's two Foundry project connection targets also use the default version; if you override it, update both connection targets to the same endpoint version before rerunning --setup-agent. This keeps the MCP tool URLs and their project connections aligned.

At this point, your project contains the two Foundry agents used in Challenge 4: claims-intake-agent from Challenge 2 and claims-intelligence-agent from this challenge.

Task 3: Verify the enterprise policy documents¶

Run the policy verification command from the docs directory:

python claims-intelligence-agent.py --verify-policies

The command asks claims-intelligence-agent to retrieve each lab policy through Foundry IQ. It does not download policy files from Blob Storage in application code.

Expected result:

Found 5 policy document(s) through Foundry IQ:
  - COMM-AUTO-001
  - COMP-AUTO-001
  - HV-AUTO-001
  - LIAB-AUTO-001
  - MOTO-001

This confirms that the Claims Intelligence Agent can retrieve the real policy documents uploaded from data/policies/ through the knowledge base.

If the command reports that no policy documents were found, ask your coach to verify the lab deployment and policy upload, then run the verification command again.

Task 4: Run the Claims Intelligence Agent¶

Feed the intake artifact produced by the Claims Intake Agent in Challenge 2 straight into the Intelligence Agent:

cd docs
python claims-intelligence-agent.py ../data/claims/crash1/derived/statements/crash1_front.intake.json

The command prints the coverage decision to the terminal. It does not save another local JSON file.

The intake artifact's policy_number is LIAB-AUTO-001 (Liability Only), so this run exercises the exclusion path: a liability-only policy denying a collision claim, with own_vehicle_damage or collision listed in exclusions_matched.

Optional: override the policy number to test the approval path:

python claims-intelligence-agent.py ../data/claims/crash1/derived/statements/crash1_front.intake.json --policy-number COMP-AUTO-001

Optional: try a high claim amount with the liability-only policy:

python claims-intelligence-agent.py ../data/claims/crash1/derived/statements/crash1_front.intake.json --policy-number LIAB-AUTO-001 --claim-amount 95000

This input does not deterministically trigger escalation. A high amount by itself does not cross a fixed escalation threshold; escalation depends on the adjudicator's response. Inspect status, coverage_decision.requires_escalation, and coverage_decision.escalation_reason in the printed JSON. When requires_escalation is true, the output also includes an ESCALATION_REQUIRED entry with severity WARN in errors. When it is false, a DENIED result with no escalation warning can be correct for an own-vehicle collision excluded by the liability-only policy.

Expected output¶

What the agent does¶

Retrieves the exact policy number from the policies-kb Foundry IQ knowledge base.
Parses the retrieved policy into a structured PolicyInfo object and caches it for the current process.
Builds a contextualized decision prompt combining claim details and policy fields.
Submits the prompt to the LLM adjudicator, which evaluates type match, limits, deductibles, exclusions, and crash-to-policy consistency simultaneously.
Parses the structured JSON response into a typed CoverageDecision object:
Approved amount after deductible
Matched exclusions
Risk flags and confidence score
consistency_score: a 1-100 rating of how well the crash details line up with what the policy should cover, and therefore whether the claim should be approved
requires_escalation and escalation_reason: the adjudicator's explicit manual-review signal and explanation
Escalates to ESCALATED status and records a WARN-severity error when the adjudicator signals manual review is needed.
Appends a complete audit trail entry for every significant action, including agent name, action type, outcome, and structured metadata (including the consistency score).

Validation checklist¶

Policy retrieval succeeds through the policies-kb Foundry IQ knowledge base and the cache is populated on the first call.
The claims-intelligence-agent appears under your Foundry project's agents with its MCP retrieval tool.
The LLM adjudicator returns a valid JSON decision with all required fields.
approved_amount reflects the claim amount minus the applicable deductible, capped at the policy limit.
The audit trail contains at least two entries: retrieve_policy and validate_coverage.
Escalation path sets status to ESCALATED and populates errors.
A liability-only policy correctly denies a collision claim and lists own_vehicle_damage or collision in exclusions_matched.
consistency_score is an integer between 1 and 100 that agrees with the decision: scores below 30 do not coincide with is_covered: true, and scores of 90+ do not require escalation unless a hard policy limit is exceeded.

Next step¶

Continue with Challenge 4: Agent Framework workflow (default path) to orchestrate the two Foundry agents from Challenges 2 and 3 as a graph with conditional human review and optional hosted-agent deployment. The manual sequential workflow remains available as an alternative.