Document Service

Enterprise document processing for AI agents. Extract text from PDFs, images, and Office documents. Advanced OCR and layout analysis with cloud-agnostic provider support. Generate, merge, split, and compress PDFs.

Why It Matters

AI agents work with documents. Contracts, reports, invoices, policies, spreadsheets. Getting clean text out of a PDF is surprisingly hard. Tables are worse. Scanned documents need OCR. And once you have the text, you often need to generate new documents, merge reports, or compress files for delivery.

The Document Service handles all of this with intelligent format detection and automatic fallback. Upload a PDF and the service determines whether it contains selectable text or is a scanned image. If text extraction yields poor quality — low character counts, bad alphanumeric ratios — it automatically falls back to OCR. Your agent gets clean text regardless of the document's origin.

The service follows a three-tier architecture: an orchestration layer that handles caching, logging, and storage routing; a processing layer with specialized providers for extraction, generation, and transformation; and a client layer with pluggable backends for each format. Content-based caching (SHA256 hashing) avoids re-processing identical documents, and all outputs can be stored in cloud blob storage for scalability.

Key Capabilities

• Multi-Format Extraction: PDF, Word (DOCX), Excel (XLSX), images, HTML — all through one API with intelligent format detection
• AI-Powered Layout Analysis: Advanced layout analysis, table extraction with cell structure, paragraph detection, and bounding boxes via cloud-agnostic AI document providers
• OCR: Extract text from scanned documents and images with high accuracy, including automatic fallback for PDFs with poor text quality
• Smart Detection: Automatically identifies scanned vs text PDFs using combined heuristics (chars/page, total chars, alphanumeric ratio) and triggers OCR when needed
• Table Extraction: Structured table data with cell coordinates, row/column spans, and confidence scores
• PDF Operations: Merge, split, extract pages, compress — full PDF manipulation toolkit
• HTML to PDF: Generate PDFs from HTML with configurable page format (A4, Letter, Legal), orientation, margins, and background printing
• Excel to CSV: Convert spreadsheets with sheet selection, custom separators, quote characters, and header options
• Content Caching: Content-based caching with TTL management — SHA256 hash of operation, content, and options avoids re-processing identical documents
• Blob Storage Integration: Input from and output to cloud storage (Azure Blob Storage, Google Cloud Storage) with working memory references

Extraction Capabilities

The Document Service provides multiple extraction endpoints, each optimized for different use cases. All endpoints support direct file upload via multipart form data or working memory references. Responses default to raw data (text or binary) but can return a full JSON envelope with metadata by setting Accept: application/json or ?format=json.

The extract-general endpoint is the recommended starting point. It identifies the extraction method automatically (pdf-basic, pdf-ocr-fallback, ocr, docx, excel, csv, plain-text) and includes quality metrics in the metadata when JSON output is requested.

AI-Powered Document Analysis

For advanced document analysis, the service integrates with cloud AI document intelligence providers. This provides AI-powered layout analysis that goes far beyond basic text extraction — detecting paragraphs, tables with cell structure, content blocks with bounding boxes, and confidence scores for every detected element. The provider layer is cloud-agnostic, with support for Azure Document Intelligence today and Google Document AI and AWS Textract coming soon.

Three specialized endpoints expose these capabilities:

Extract and Analyze Documents

Use the SDK to extract structured content with intelligent format detection. The service identifies the document type, selects the appropriate extraction method, and returns structured data including tables with cell-level detail.

Metadata in the response includes the backend used, processing time, and whether the result was served from cache — useful for monitoring extraction performance across your document pipeline.

import { Bot, docs } from '@firefoundry/agent-sdk';

@Bot({ name: 'doc-analyst' })
class DocAnalyst {
  async processDocument(fileUrl: string) {
    // Extract structured content with intelligent format detection
    const result = await docs.extractStructured({
      source: fileUrl,
      options: { includeMetadata: true, includeTables: true }
    });

    // Get tables with cell structure
    for (const table of result.tables) {
      console.log(`Table: ${table.rows}x${table.columns}`);
      for (const cell of table.cells) {
        console.log(`  [${cell.row},${cell.col}]: ${cell.text}`);
      }
    }

    return result;
  }
}

OCR and PDF Operations

The Document Service combines OCR for scanned content with a full PDF manipulation toolkit. OCR a scanned invoice, merge quarterly reports into a single document, or generate PDFs from HTML templates — all through the same API.

PDF operations include page extraction (specific pages by number or range), splitting into chunks, merging multiple documents, and compression with object stream optimization. All operations accept working memory references for seamless integration with the Context Service.

// OCR a scanned document with AI-powered layout analysis
const ocrResult = await docs.ocr({
  source: 'working-memory://scanned-invoice.pdf',
  model: 'prebuilt-layout'
});

// Merge multiple PDFs into one
const merged = await docs.mergePDFs({
  sources: ['report-q1.pdf', 'report-q2.pdf', 'report-q3.pdf'],
  output: 'annual-report.pdf'
});

// Generate PDF from HTML
const pdf = await docs.htmlToPdf({
  html: '<h1>Monthly Report</h1><p>Generated by AI agent</p>',
  options: { format: 'A4', orientation: 'portrait' }
});

Supported Formats

The Document Service handles a wide range of input formats. Each format routes to the appropriate processing method automatically. For the extract-general endpoint, format detection is automatic based on MIME type and file extension.

MCP Tools

The Document Service is available as MCP tools through the MCP Gateway, enabling any MCP-compatible client to process documents without the FireFoundry SDK. The docproc adapter exposes five tools covering extraction, OCR, format conversion, and PDF generation. Document content is passed as base64-encoded strings with the original filename for format detection.

Use Cases