Azure AI Foundry in the Field: Two Real Pilots, Zero Hype

There is a version of every Azure AI blog post where the author deploys a chat interface against a sample dataset and declares victory. This is not that post.

We are running two active pilots with Azure AI Foundry, both in production-adjacent environments, both with real compliance and data integrity requirements. Here is what we have learned so far.

Pilot One: PCI-DSS Evidence Automation

The Problem

PCI-DSS compliance generates a substantial evidence burden. Auditors want screenshots, logs, configuration exports, and control attestations — all organized, labeled, and traceable to specific requirements. In most environments this is manual: a junior engineer or compliance analyst collects artifacts across a dozen systems, renames files, drops them into folders, and hopes the auditor agrees with the taxonomy.

It is slow, error-prone, and adds zero business value. It is also exactly the kind of structured, repetitive extraction task that AI handles well.

What We Built

The pipeline looks like this:

Source Systems (Azure, M365, Entra ID)
        ↓
Azure AI Foundry — Document Intelligence
        ↓
Evidence Extraction + PCI Control Mapping
        ↓
Structured Output → Evidence Repository
        ↓
Audit-Ready Report Generation

Azure AI Foundry's Document Intelligence handles the heavy lifting on unstructured inputs — configuration exports, policy documents, audit logs that have been formatted as PDFs. We feed those in, the model extracts structured fields, and we map them to PCI-DSS control identifiers (Requirement 7, 8, 10, etc.) using a classification layer on top.

The output is a structured evidence package: each artifact tagged with the control it satisfies, the source system, the extraction timestamp, and a confidence score.

Architecture Decisions Worth Noting

Document Intelligence vs. raw GPT-4o for extraction: We tested both. Document Intelligence wins on structured forms and configuration exports — it understands layout, tables, and key-value pairs natively. Raw GPT-4o wins on narrative policy documents where context matters more than structure. We run both in parallel for certain document types and reconcile.

Control mapping as a separate step: The temptation is to ask the model to extract and classify in one pass. Don't. Extraction accuracy drops when you overload the prompt. Extract clean, classify separately. Two focused models beat one overloaded one.

Immutable audit trail: Every extraction event writes a record to an append-only log — what document came in, what model version ran, what was extracted, what confidence score was assigned, and who (or what) approved it. This is non-negotiable in a PCI environment. AI Foundry doesn't give you this out of the box; you build it around the pipeline.

def log_extraction_event(doc_id: str, model_version: str, 
                          extracted: dict, confidence: float, 
                          reviewer: str) -> None:
    event = {
        "timestamp": datetime.utcnow().isoformat(),
        "doc_id": doc_id,
        "model_version": model_version,
        "extracted_fields": extracted,
        "confidence_score": confidence,
        "reviewed_by": reviewer,
        "immutable": True
    }
    audit_log.append(event)  # append-only store

What the Docs Glossed Over

Rate limiting hits you faster than you expect. Document Intelligence has per-minute and per-hour limits that are easy to exceed when you batch a compliance sprint's worth of evidence in one run. Build retry logic with exponential backoff from day one, not after your first 429.

Model versioning matters for audits. If you run evidence through 2024-02-29 in March and 2024-07-31 in April, you now have a mixed-model evidence set that an auditor may flag. Pin your model version for a given audit cycle and document it explicitly.

Pilot Two: PDF Contract Ingestion

The Problem

A client manages a large volume of vendor and customer contracts — PDFs, mostly, with inconsistent formatting across different counterparties and different decades of signing. Key terms are buried in dense legal text: payment terms, SLA commitments, renewal clauses, liability caps, termination triggers.

Legal and operations teams needed to query across the contract corpus. "Show me every contract with a net-60 payment term." "Which contracts auto-renew in Q3?" Currently the answer to those questions requires a paralegal and several hours. The target is a natural language query interface over the full corpus.

The Architecture

This one has more moving parts:

PDF Contracts (source)
        ↓
Azure AI Foundry — Document Intelligence (extraction)
        ↓
Chunking Layer (semantic, not fixed-size)
        ↓
Azure Cosmos DB (chunk storage + metadata)
        ↓
Azure AI Foundry — AI Indexers (vector indexing)
        ↓
Query Interface (natural language → retrieval → synthesis)

Chunking is the most important decision you will make in a RAG pipeline. Fixed-size chunking (split every N tokens) is fast to implement and degrades retrieval quality in ways that are subtle and hard to debug. A clause that spans a chunk boundary disappears from retrieval. A chunk that contains half a payment term and half an indemnification clause returns false positives on both queries.

We use semantic chunking — splitting on logical boundaries (sections, clauses, defined terms) rather than token counts. Document Intelligence's layout analysis is useful here; it gives you paragraph and section structure to work with rather than raw text.

def semantic_chunk(doc_layout: dict, max_tokens: int = 512) -> list[dict]:
    chunks = []
    current_chunk = []
    current_tokens = 0
    
    for element in doc_layout["paragraphs"]:
        element_tokens = count_tokens(element["content"])
        
        # Respect section boundaries
        is_new_section = element.get("role") in ["sectionHeading", "title"]
        
        if is_new_section and current_chunk:
            chunks.append(merge_chunk(current_chunk))
            current_chunk = []
            current_tokens = 0
        
        if current_tokens + element_tokens > max_tokens and current_chunk:
            chunks.append(merge_chunk(current_chunk))
            current_chunk = []
            current_tokens = 0
        
        current_chunk.append(element)
        current_tokens += element_tokens
    
    if current_chunk:
        chunks.append(merge_chunk(current_chunk))
    
    return chunks

Cosmos DB for chunk storage gives us two things: flexible schema for metadata (contract ID, party names, effective date, jurisdiction, contract type) and the ability to filter before vector search. We do not want to retrieve chunks from a terminated contract when querying active obligations. Pre-filter on metadata, then vector search within the filtered set. This dramatically improves precision.

AI Foundry indexers handle the vector embedding and index update pipeline. When a new contract lands, the indexer picks it up, embeds the chunks, and updates the search index. The integration between Cosmos DB and AI Foundry indexers is reasonably clean — the indexer can watch a Cosmos DB container as a data source and trigger on change feed events.

Query Design

The query interface is a two-step process. Step one: retrieval — embed the user's query and run vector search against the index, filtered by active contracts. Step two: synthesis — pass the retrieved chunks to a GPT-4o completion with instructions to answer the query using only the retrieved text and cite the source contract and clause.

The citation instruction is critical. Without it, the model will occasionally synthesize an answer that sounds authoritative but blends details from multiple contracts inaccurately. Every answer surfaces the contract name, the clause, and the verbatim language it drew from.

Where AI Foundry Fits (and Where It Doesn't)

Azure AI Foundry is a solid platform when your use case is document-heavy, your data stays in Azure, and you need the compliance posture that comes with a managed Microsoft service. For regulated industries — financial services, healthcare, legal — that last point matters more than the benchmarks.

It is not a magic layer. The pipeline design, the chunking strategy, the confidence thresholds, the audit logging, the metadata schema — none of that is provided for you. The platform removes infrastructure friction. The hard thinking remains yours.

Both of these pilots are ongoing. The evidence automation pipeline is moving toward broader deployment across a second client environment. The contract ingestion system is in active user testing with the legal and operations teams. More to come as they mature.

If you are evaluating AI Foundry for a similar use case, start with a narrow, well-defined extraction problem where you can measure accuracy against a ground truth. Do not start with the query interface — start with the data quality.