# DSOL - Technical Specification: PDF and PowerPoint Document Support **Author:** TextIQ **Date:** 2025-12-05 **Project Level:** Feature Enhancement **Change Type:** Document Format Support Extension **Development Context:** Brownfield - Extending V2 Extraction Pipeline --- ## Context ### Available Documents **Loaded Documentation:** - ✅ PRD (Product Requirements Document) - Comprehensive document processing service requirements - ✅ Architecture Document - System design with V1 and V2 extraction pipelines - ✅ Word Document Support Tech-Spec (docs/feats/docx_support/tech-spec.md) - Recently completed Word processing feature - ✅ Extraction Pipeline V2 Tech-Spec (docs/specs/extraction-pipeline-v2-tech-spec.md) - Docling-based pipeline architecture - ✅ Project Configuration (pyproject.toml, .bmad/bmm/config.yaml) **Current State:** - V2 extraction pipeline operational for Excel files (.xlsx, .xls, .xlsm) - Word document support recently added (.doc, .docx, .docm) with full preprocessing - 51 Word documents now processable through V2 pipeline - Docling 2.63.0+ supports Excel, Word, PDF, and PowerPoint conversion - Image captioning, shape extraction, and comment handling established patterns ### Project Stack **Runtime Environment:** - **Language:** Python 3.11+ - **Framework:** FastAPI 0.122 - **API Server:** Uvicorn with async support - **Task Queue:** Celery 5.3+ with Redis 7.0+ **Data Layer:** - **Database:** PostgreSQL (SQLAlchemy 2.0+, asyncpg, alembic) - **Vector Store:** Milvus 2.4+ (pymilvus) **Document Processing:** - **Document Conversion:** Docling 2.63.0+ (Excel → HTML, Word → Markdown, **PDF → Markdown, PowerPoint → Markdown**) - **Excel Parsing:** openpyxl 3.1.5, xlrd 2.0.2+ (.xls support) - **Word Document Parsing:** python-docx (for comment preprocessing) - **PowerPoint Parsing:** python-pptx (for speaker notes preprocessing - to be added) - **Legacy Format Conversion:** LibreOffice 7.0+ CLI (.doc/.docm → .docx, **.ppt/.pptm → .pptx**) - **Document Chunking:** Docling HybridChunker with OpenAI tokenizer (tiktoken) and ImgTableAnnotationSerializerProvider - **LLM Integration:** Azure OpenAI (GPT-4o) via openai 1.0+ (Word comments only) **Development Tools:** - **Testing:** pytest 8.0+, pytest-asyncio 0.23+ - **Code Quality:** black 24.0+ (100 char lines), ruff 0.5+, mypy 1.10+ - **Logging:** structlog 24.0+ ### Existing Codebase Structure **Project Organization:** ``` dsol/ ├── src/ │ ├── extraction_v2/ # V2 pipeline (Excel, Word - extend for PDF/PowerPoint) │ │ ├── pipeline.py # Main orchestrator - MODIFY │ │ ├── document_converter.py # Handles Excel, Word - MODIFY for PDF/PowerPoint │ │ ├── word_preprocessor.py # Word comment extraction - REUSE PATTERN │ │ ├── pptx_preprocessor.py # CREATE - PowerPoint speaker notes extraction ONLY │ │ ├── html_table_extractor.py # Reuse as-is │ │ ├── llm_header_detector.py # Reuse as-is │ │ ├── record_builder.py # Reuse as-is │ │ ├── semantic_chunker.py # Reuse as-is (ImgTableAnnotationSerializerProvider) │ │ ├── custom_serializers.py # Reuse as-is (handles images/tables) │ │ └── detect_border.py # Reuse as-is │ ├── workers/ # Celery tasks │ │ └── extraction_v2_tasks.py - MODIFY │ ├── api/ # FastAPI routes - UPDATE file validation │ └── db/ # Database models ├── tests/ # Test suite │ └── test_extraction_v2/ # V2 pipeline tests - EXTEND └── CustomerDocument/ # Test data ├── [58 Excel files] # Currently processed ├── [51 Word files] # Recently added support └── [PDF/PowerPoint files] # Target for this work ``` **Key Existing Modules:** - `ExtractionPipelineV2` - Main pipeline orchestrator (extend for PDF/PowerPoint) - `DocumentToHtmlConverter` - Handles Excel/Word conversion (extend for PDF/PowerPoint) - `WordPreprocessor` - Comment extraction pattern (adapt for PowerPoint speaker notes) - `SemanticChunker` - Docling HybridChunker with ImgTableAnnotationSerializerProvider (reuse for PDF/PowerPoint) - `ImgTableAnnotationSerializerProvider` - Custom serializers for images/tables (reuse for PDF/PowerPoint) - `HtmlTableExtractor` - Parses Docling HTML output (Excel only) --- ## The Change ### Problem Statement **Current Situation:** The V2 extraction pipeline successfully processes Excel and Word documents but cannot handle PDF and PowerPoint files, which are common document formats in enterprise environments. **Specific Issues:** 1. **Unsupported File Formats:** V2 pipeline only recognizes Excel and Word formats - No PDF support (text-based, scanned with OCR, or mixed content) - No PowerPoint support (.pptx, .ppt, .pptm) - Pipeline rejects PDF/PowerPoint files at format detection stage 2. **Missing Format Normalization:** No .ppt/.pptm conversion pipeline - Legacy .ppt format not supported by Docling directly - Macro-enabled .pptm files need conversion - No LibreOffice integration for PowerPoint formats 3. **No PowerPoint Speaker Notes Extraction:** Word has comment preprocessing, PowerPoint needs speaker notes - Speaker notes need to be embedded inline before Docling conversion - Slide structure preserved by Docling natively 4. **Incomplete Content Extraction:** Need unified approach for all document types - PDF: Extract text, tables, images with context - PowerPoint: Extract slide content, speaker notes, embedded objects - Preserve document structure and relationships **Business Impact:** - Cannot process common enterprise document formats (PDF, PowerPoint) - Incomplete document coverage limits system usefulness - PDF and PowerPoint contain valuable structured data - Competitive gap - full-document processing needed for production ### Proposed Solution **Solution Overview:** Extend V2 extraction pipeline to support PDF and PowerPoint documents following the established Word document processing pattern with full feature parity. **Three-Part Approach:** **1. Format Normalization Pipeline (Legacy Support)** - Implement .ppt → .pptx conversion using LibreOffice CLI - Implement .pptm → .pptx conversion using LibreOffice CLI - Follow existing pattern from Word (.doc → .docx, .docm → .docx) - Validate converted files before Docling processing - PDF format normalization: All PDF types accepted (Docling handles internally) **2. PowerPoint Speaker Notes Preprocessing** - Extract speaker notes using python-pptx - Embed notes inline in slides for Docling to process - Images and tables handled by custom serializers during chunking (no preprocessing) - PDF requires no preprocessing (Docling handles directly) **3. Docling Conversion & Semantic Chunking** - Leverage Docling's native PDF support (PDF → Markdown) - Leverage Docling's native PowerPoint support (PPTX → Markdown) - Convert to unified text format (Markdown) - Chunk the entire output using Docling HybridChunker - Unified record building with chunk metadata and source traceability **Expected Outcome:** - ✅ All PDF files processed through V2 pipeline (text-based, scanned, mixed) - ✅ All PowerPoint files (.pptx, .ppt, .pptm) processed through V2 pipeline - ✅ Markdown conversion via Docling (images, text, tables preserved) - ✅ Images/tables handled by ImgTableAnnotationSerializerProvider during chunking - ✅ Full document semantically chunked using Docling HybridChunker - ✅ ≥85% extraction success rate (V2 + V1 fallback if needed) - ✅ Context-preserving retrieval via semantic chunking ### Scope **In Scope:** 1. **PDF Format Support** - Add .pdf support to V2 pipeline - Handle all PDF types: text-based, scanned (OCR), mixed content - PDF → Markdown conversion via Docling - Image extraction with AI captioning - File validation and error handling 2. **PowerPoint Format Support** - Add .pptx, .ppt, .pptm support to V2 pipeline - Format normalization: .ppt → .pptx, .pptm → .pptx (LibreOffice CLI) - PowerPoint → Markdown conversion via Docling - Slide-by-slide content extraction - Speaker notes extraction 3. **PowerPoint Preprocessing** - Speaker notes extraction and inline embedding (PowerPoint only) - PDF requires no preprocessing 4. **Semantic Chunking Integration** - Docling PDF → Markdown → HybridChunker (with ImgTableAnnotationSerializerProvider) → Chunks - Docling PowerPoint → Markdown → HybridChunker (with ImgTableAnnotationSerializerProvider) → Chunks - **Chunking Configuration:** 8000 tokens max (tiktoken for text-embedding-3-large) - **Custom Serializers:** ImgTableAnnotationSerializerProvider handles images/tables automatically - Unified JSON record output with chunk metadata - Source traceability for all chunks 5. **Batch Processing Updates** - Update glob patterns to include .pdf/.pptx/.ppt/.pptm - Enhanced status tracking for PDF/PowerPoint files - Error categorization (format-specific) - Metrics reporting 6. **Testing & Validation** - Unit tests for PDF and PowerPoint conversion and preprocessing - Integration tests for PDF and PowerPoint files - Success rate validation (≥85% target) - Comparison with Excel/Word pipeline quality 7. **Documentation** - Update README with PDF and PowerPoint support notes - Document preprocessing pipeline for PDF and PowerPoint - Troubleshooting guide for common issues **Out of Scope:** 1. ❌ Advanced PDF form field extraction (basic text extraction only) 2. ❌ PowerPoint animation or transition extraction 3. ❌ Video/audio embedded in PowerPoint 4. ❌ Real-time collaborative editing features 5. ❌ PDF digital signature validation 6. ❌ Macro execution (.pptm macros not run) 7. ❌ PDF password protection handling (requires manual unlock) --- ## Implementation Details ### Source Tree Changes **File Modifications:** 1. **src/extraction_v2/document_converter.py** - MODIFY - Add PDF format detection and conversion logic - Add PowerPoint format detection (.pptx, .ppt, .pptm) - Add .ppt/.pptm → .pptx conversion via LibreOffice - Add PDF → Markdown conversion via Docling (no preprocessing) - Add PowerPoint → Markdown conversion via Docling (with speaker notes preprocessing) - Lines ~130-135: Add PDF/PowerPoint to format routing logic - Lines ~935-1045: Add `_convert_pdf_to_markdown()` method - Lines ~1045-1160: Add `_convert_pptx_to_markdown()` method - Lines ~1160-1250: Add `_convert_ppt_to_pptx()` method (LibreOffice) 2. **src/extraction_v2/pptx_preprocessor.py** - CREATE - Extract speaker notes from PowerPoint slides - Embed speaker notes inline before Docling conversion - NO image captioning (images handled by ImgTableAnnotationSerializerProvider during chunking) - Follow lightweight pattern 3. **src/extraction_v2/pipeline.py** - MODIFY - Lines ~539-644: Add `process_pdf_document()` async method - Lines ~645-750: Add `process_pptx_document()` async method - Lines ~689-728: Add `process_pdf_document_sync()` method - Lines ~729-768: Add `process_pptx_document_sync()` method - Lines ~768-810: Add convenience functions `extract_from_pdf()` and `extract_from_pptx()` - Update routing logic to detect PDF and PowerPoint formats - Integrate with SemanticChunker using ImgTableAnnotationSerializerProvider 4. **src/workers/extraction_v2_tasks.py** - MODIFY - Add PDF and PowerPoint file type handling - Update file validation for new formats - Enhanced error categorization for format-specific issues 5. **src/api/routes/documents.py** - MODIFY - Update file validation to accept .pdf, .pptx, .ppt, .pptm - Add MIME type validation for PDF and PowerPoint **New Test Files:** 6. **tests/test_extraction_v2/test_pdf_conversion.py** - CREATE - Test Docling PDF → Markdown conversion - Test PDF validation - Validate Markdown structure 7. **tests/test_extraction_v2/test_pptx_conversion.py** - CREATE - Test .ppt → .pptx conversion - Test .pptm → .pptx conversion - Test Docling PowerPoint → Markdown conversion - Validate Markdown structure 8. **tests/test_extraction_v2/test_pptx_preprocessing.py** - CREATE - Test speaker notes extraction (PowerPoint only) - Test inline speaker notes embedding - Test slide structure preservation 9. **tests/test_extraction_v2/test_pdf_pptx_comprehensive.py** - CREATE - End-to-end tests for PDF files - End-to-end tests for PowerPoint files - Success rate validation - Quality comparison with Word pipeline 10. **tests/test_extraction_v2/test_pdf_pptx_serializers.py** - CREATE - Verify ImgTableAnnotationSerializerProvider handles PDF images - Verify ImgTableAnnotationSerializerProvider handles PowerPoint images - Validate custom serializer integration during chunking **Documentation Updates:** 11. **README.md** - MODIFY - Add PDF and PowerPoint document support section - Document .ppt/.pptm conversion requirements - Update batch processing examples - Note unified Markdown output for all document types - Note custom serializers for images and tables 12. **docs/specs/extraction-pipeline-v2-tech-spec.md** - MODIFY - Add PDF processing architecture - Add PowerPoint processing architecture - Update component specifications - Document speaker notes preprocessing (PowerPoint only) - Note unified Markdown conversion strategy - Document ImgTableAnnotationSerializerProvider usage ### Technical Approach **1. Format Normalization (LibreOffice CLI for PowerPoint)** Extend existing Word pattern to PowerPoint formats: ```python # In document_converter.py def _convert_ppt_to_pptx(self, file_path: Path) -> Optional[str]: """ Convert .ppt or .pptm to .pptx using LibreOffice CLI. This preserves images, shapes, slides, and formatting. Args: file_path: Path to .ppt or .pptm file Returns: Path to converted .pptx file, or None on failure """ # Find LibreOffice executable libreoffice_cmd = self._find_libreoffice_command() if not libreoffice_cmd: logger.error( "LibreOffice not found. Required for .ppt/.pptm conversion. " "Install: apt-get install libreoffice (Ubuntu) or brew install libreoffice (macOS)" ) return None try: # Create temp directory for output temp_dir = tempfile.mkdtemp(prefix="pptx_convert_") # Create a unique user profile directory profile_dir = tempfile.mkdtemp(prefix="lo_profile_") # Use file lock to prevent concurrent LibreOffice access lock_fd = None try: lock_fd = open(LIBREOFFICE_LOCK_FILE, 'w') fcntl.flock(lock_fd, fcntl.LOCK_EX) # Blocking lock logger.debug("LibreOffice lock acquired") # Run LibreOffice conversion logger.info(f"Running LibreOffice conversion: {libreoffice_cmd}") result = subprocess.run( [ libreoffice_cmd, '--headless', f'-env:UserInstallation=file://{profile_dir}', '--convert-to', 'pptx', '--outdir', temp_dir, str(file_path) ], capture_output=True, text=True, timeout=60 # 60 second timeout ) finally: if lock_fd: fcntl.flock(lock_fd, fcntl.LOCK_UN) lock_fd.close() # Clean up the temporary profile directory try: shutil.rmtree(profile_dir) except Exception: pass if result.returncode != 0: logger.error(f"LibreOffice conversion failed: {result.stderr}") return None # Find the output file output_file = Path(temp_dir) / f"{file_path.stem}.pptx" if not output_file.exists(): logger.error(f"Conversion succeeded but output file not found: {output_file}") return None logger.info(f"Successfully converted to .pptx: {output_file}") return str(output_file) except subprocess.TimeoutExpired: logger.error("LibreOffice conversion timed out (>60s)") return None except Exception as e: logger.error(f"Error during LibreOffice conversion: {e}") return None ``` **2. PDF Document Processing** PDF documents are processed directly with Docling - NO preprocessing needed. ```python # In document_converter.py def _convert_pdf_to_markdown(self, file_path: str) -> Optional[str]: """ Preprocess PDF documents before Docling conversion. Extraction and Inline Embedding: 1. Images → AI-generated captions → Embed inline as [IMAGE: caption] 2. Metadata → Page numbers, document properties All content embedded inline before Docling markdown conversion. """ def __init__(self, openai_client=None): """ Initialize with Azure OpenAI client for image captioning. Args: openai_client: Optional Azure OpenAI client. If None, creates one lazily on first use. """ self._client = openai_client self.logger = structlog.get_logger() def _get_client(self): """ Get or create Azure OpenAI client lazily. This is done lazily to avoid creating HTTP clients before process forking, which causes segmentation faults in Celery workers. """ if self._client is None: from openai import AzureOpenAI from src.core.config import settings self._client = AzureOpenAI( api_key=settings.azure_openai_api_key, api_version=settings.azure_openai_api_version, azure_endpoint=settings.azure_openai_endpoint ) self.logger.info("Initialized Azure OpenAI client for image captioning") return self._client def preprocess(self, file_path: Path) -> Path: """ Preprocess PDF document and save to new file with embedded captions. Processing steps: 1. Load PDF 2. Extract images from each page 3. Generate AI captions for images 4. Create enriched PDF with embedded image descriptions 5. Save preprocessed document Args: file_path: Path to .pdf file Returns: Path to preprocessed .pdf file with embedded image descriptions """ self.logger.info("Starting PDF preprocessing", file=file_path.name) # Load PDF pdf = PdfReader(str(file_path)) # Step 1: Extract images with page context images = self._extract_images(pdf) if images: self.logger.info("Extracting images", count=len(images)) captions = self._generate_image_captions(images) # For PDFs, we'll create a metadata file with image captions # that Docling can reference during conversion output_path = self._create_enriched_pdf_metadata(file_path, images, captions) else: self.logger.info("No images found in PDF") output_path = file_path self.logger.info( "Preprocessing complete", input=file_path.name, output=output_path.name if isinstance(output_path, Path) else output_path, images=len(images) ) return output_path if isinstance(output_path, Path) else file_path def _extract_images(self, pdf: PdfReader) -> List[Dict]: """ Extract all images from PDF with page context. Returns list of dicts with: - page_number: Page containing the image - image_data: Image binary data - context: Surrounding text from the page - image_index: Index of image on page """ images = [] for page_num, page in enumerate(pdf.pages, start=1): # Extract page text for context page_text = page.extract_text() or "" # Extract images from page if "/XObject" in page["/Resources"]: xobject = page["/Resources"]["/XObject"].get_object() for img_idx, obj_name in enumerate(xobject): obj = xobject[obj_name] # Check if it's an image if obj["/Subtype"] == "/Image": try: # Extract image data img_data = obj.get_data() images.append({ 'page_number': page_num, 'image_data': img_data, 'context': page_text[:500], # First 500 chars of page 'image_index': img_idx }) except Exception as e: self.logger.warning( "Failed to extract image", page=page_num, image_index=img_idx, error=str(e) ) return images def _generate_image_captions(self, images: List[Dict]) -> List[str]: """ Generate AI captions for images using Azure OpenAI GPT-4o vision. Args: images: List of image dictionaries with data and context Returns: List of caption strings (one per image) """ captions = [] for img in images: try: # Encode image to base64 img_base64 = base64.b64encode(img['image_data']).decode('utf-8') # Detect image type img_type = self._detect_image_type(img['image_data']) # Call Azure OpenAI GPT-4o vision API client = self._get_client() response = client.chat.completions.create( model="gpt-4o", messages=[ { "role": "user", "content": [ { "type": "text", "text": ( f"Describe this image from a PDF document in 1-2 sentences. " f"Context from page: {img['context'][:200]}" ) }, { "type": "image_url", "image_url": { "url": f"data:image/{img_type};base64,{img_base64}" } } ] } ], max_tokens=100 ) caption = response.choices[0].message.content captions.append(caption) self.logger.debug( "Image caption generated", page=img['page_number'], caption=caption[:50] ) except Exception as e: self.logger.warning( "Image captioning failed", page=img.get('page_number'), error=str(e) ) captions.append("[Image: Caption unavailable]") return captions def _detect_image_type(self, img_data: bytes) -> str: """Detect image type from binary header.""" if img_data.startswith(b'\x89PNG'): return "png" elif img_data.startswith(b'\xFF\xD8\xFF'): return "jpeg" elif img_data.startswith(b'GIF'): return "gif" else: return "png" # Default def _create_enriched_pdf_metadata( self, file_path: Path, images: List[Dict], captions: List[str] ) -> Path: """ Create a metadata file with image captions for Docling to reference. Since we can't easily modify PDF content inline, we create a sidecar metadata file that contains the image captions indexed by page. Args: file_path: Original PDF file path images: List of extracted images captions: List of captions parallel to images Returns: Path to original PDF (metadata saved separately) """ import json # Create metadata dictionary metadata = { "filename": file_path.name, "image_captions": [] } for img, caption in zip(images, captions): metadata["image_captions"].append({ "page": img["page_number"], "image_index": img["image_index"], "caption": caption }) # Save metadata file metadata_path = file_path.parent / f"{file_path.stem}_metadata.json" with open(metadata_path, 'w') as f: json.dump(metadata, f, indent=2) self.logger.info(f"Saved image metadata to {metadata_path}") # Return original PDF path (Docling will process it) return file_path ``` **3. PowerPoint Document Preprocessing** ```python # In pptx_preprocessor.py """ PowerPoint Document Preprocessor for DSOL V2 Extraction Pipeline. This module preprocesses PowerPoint documents before Docling conversion by extracting and embedding non-text content inline: 1. Images: Extract images → Generate AI captions → Embed as [IMAGE: caption] 2. Shapes: Extract text boxes/shapes → Embed as [SHAPE: text] 3. Speaker Notes: Extract notes → Embed as [NOTES: text] All content is embedded inline before Docling HTML conversion, ensuring it's preserved in the semantic chunking phase. """ import base64 import tempfile from pathlib import Path from typing import List, Dict, Optional, Any import structlog from pptx import Presentation from pptx.util import Inches logger = structlog.get_logger(__name__) class PptxPreprocessor: """ Preprocess PowerPoint documents before Docling conversion. Extraction and Inline Embedding: 1. Images → AI-generated captions → Embed inline as [IMAGE: caption] 2. Shapes/text boxes → Text extraction → Embed inline as [SHAPE: text] 3. Speaker Notes → Embed inline as [NOTES: text] All content embedded inline before Docling HTML conversion. """ def __init__(self, openai_client=None): """ Initialize with Azure OpenAI client for image captioning. Args: openai_client: Optional Azure OpenAI client. If None, creates one lazily on first use. """ self._client = openai_client self.logger = structlog.get_logger() def _get_client(self): """ Get or create Azure OpenAI client lazily. This is done lazily to avoid creating HTTP clients before process forking, which causes segmentation faults in Celery workers. """ if self._client is None: from openai import AzureOpenAI from src.core.config import settings self._client = AzureOpenAI( api_key=settings.azure_openai_api_key, api_version=settings.azure_openai_api_version, azure_endpoint=settings.azure_openai_endpoint ) self.logger.info("Initialized Azure OpenAI client for image captioning") return self._client def preprocess(self, file_path: Path) -> Path: """ Preprocess PowerPoint document and save to new file. Processing steps: 1. Load presentation 2. Extract images, shapes, and notes from each slide 3. Generate AI captions for images 4. Embed all extracted content inline in slides 5. Save preprocessed presentation Args: file_path: Path to .pptx file Returns: Path to preprocessed .pptx file """ self.logger.info("Starting PowerPoint preprocessing", file=file_path.name) # Load presentation prs = Presentation(str(file_path)) # Step 1: Extract images from all slides images = self._extract_images(prs) if images: self.logger.info("Extracting images", count=len(images)) captions = self._generate_image_captions(images) self._embed_image_captions(prs, images, captions) else: self.logger.info("No images found in presentation") # Step 2: Extract and embed speaker notes notes_count = self._extract_and_embed_notes(prs) # Step 3: Extract shapes and text boxes shapes_count = self._extract_shapes(prs) # Save preprocessed presentation output_path = file_path.parent / f"{file_path.stem}_preprocessed.pptx" prs.save(str(output_path)) self.logger.info( "Preprocessing complete", input=file_path.name, output=output_path.name, images=len(images), notes=notes_count, shapes=shapes_count ) return output_path def _extract_images(self, prs: Presentation) -> List[Dict]: """ Extract all images from presentation slides with context. Returns list of dicts with: - slide_number: Slide containing the image - image_data: Image binary data - context: Slide title and text - slide: Reference to slide object """ images = [] for slide_num, slide in enumerate(prs.slides, start=1): # Extract slide context (title + text) slide_text = [] for shape in slide.shapes: if hasattr(shape, "text"): slide_text.append(shape.text) context = " | ".join(slide_text[:3]) # First 3 text items # Extract images from slide for shape in slide.shapes: if hasattr(shape, "image"): try: img_data = shape.image.blob images.append({ 'slide_number': slide_num, 'image_data': img_data, 'context': context, 'slide': slide, 'shape': shape }) except Exception as e: self.logger.warning( "Failed to extract image", slide=slide_num, error=str(e) ) return images def _generate_image_captions(self, images: List[Dict]) -> List[str]: """ Generate AI captions for images using Azure OpenAI GPT-4o vision. Args: images: List of image dictionaries with data and context Returns: List of caption strings (one per image) """ captions = [] for img in images: try: # Encode image to base64 img_base64 = base64.b64encode(img['image_data']).decode('utf-8') # Detect image type img_type = self._detect_image_type(img['image_data']) # Call Azure OpenAI GPT-4o vision API client = self._get_client() response = client.chat.completions.create( model="gpt-4o", messages=[ { "role": "user", "content": [ { "type": "text", "text": ( f"Describe this image from a PowerPoint slide in 1-2 sentences. " f"Slide context: {img['context'][:200]}" ) }, { "type": "image_url", "image_url": { "url": f"data:image/{img_type};base64,{img_base64}" } } ] } ], max_tokens=100 ) caption = response.choices[0].message.content captions.append(caption) self.logger.debug( "Image caption generated", slide=img['slide_number'], caption=caption[:50] ) except Exception as e: self.logger.warning( "Image captioning failed", slide=img.get('slide_number'), error=str(e) ) captions.append("[Image: Caption unavailable]") return captions def _detect_image_type(self, img_data: bytes) -> str: """Detect image type from binary header.""" if img_data.startswith(b'\x89PNG'): return "png" elif img_data.startswith(b'\xFF\xD8\xFF'): return "jpeg" elif img_data.startswith(b'GIF'): return "gif" else: return "png" # Default def _embed_image_captions( self, prs: Presentation, images: List[Dict], captions: List[str] ): """ Embed AI-generated captions near images in slides. Adds caption text as a text box below each image. Args: prs: Presentation object images: List of image dictionaries captions: List of AI-generated captions (parallel to images) """ for img, caption in zip(images, captions): slide = img['slide'] # Add a text box with the caption below the image left = Inches(0.5) top = Inches(6.5) width = Inches(9) height = Inches(0.5) textbox = slide.shapes.add_textbox(left, top, width, height) text_frame = textbox.text_frame text_frame.text = f"[IMAGE CAPTION]: {caption}" def _extract_and_embed_notes(self, prs: Presentation) -> int: """ Extract speaker notes and embed them in slide content. Adds notes as a text box at the bottom of each slide. Args: prs: Presentation object Returns: Number of slides with notes """ notes_count = 0 for slide_num, slide in enumerate(prs.slides, start=1): if slide.has_notes_slide: notes_slide = slide.notes_slide notes_text = notes_slide.notes_text_frame.text.strip() if notes_text: # Add notes as text box left = Inches(0.5) top = Inches(7) width = Inches(9) height = Inches(0.5) textbox = slide.shapes.add_textbox(left, top, width, height) text_frame = textbox.text_frame text_frame.text = f"[SPEAKER NOTES]: {notes_text}" notes_count += 1 self.logger.debug(f"Embedded notes from slide {slide_num}") return notes_count def _extract_shapes(self, prs: Presentation) -> int: """ Extract text from shapes and text boxes (already visible, just count). Args: prs: Presentation object Returns: Number of shapes with text """ shapes_count = 0 for slide in prs.slides: for shape in slide.shapes: if hasattr(shape, "text") and shape.text.strip(): shapes_count += 1 return shapes_count ``` **4. Document Converter Integration** ```python # In document_converter.py - Add to existing file def convert_to_html(self, file_path: str) -> Optional[str]: """ Convert document to HTML/Markdown. Routes to appropriate conversion method based on file extension: - Excel (.xlsx, .xls, .xlsm) → _convert_excel_to_html() → HTML - Word (.docx, .doc, .docm) → _convert_word_to_markdown() → Markdown - PDF (.pdf) → _convert_pdf_to_markdown() → Markdown - PowerPoint (.pptx, .ppt, .pptm) → _convert_pptx_to_html() → HTML Args: file_path: Path to document file Returns: HTML or Markdown string, None on failure """ file_path_obj = Path(file_path) file_ext = file_path_obj.suffix.lower() # Route to appropriate converter if file_ext in ['.xlsx', '.xls', '.xlsm']: return self._convert_excel_to_html(file_path) elif file_ext in ['.docx', '.doc', '.docm']: return self._convert_word_to_markdown(file_path) elif file_ext == '.pdf': return self._convert_pdf_to_markdown(file_path) elif file_ext in ['.pptx', '.ppt', '.pptm']: return self._convert_pptx_to_markdown(file_path) else: logger.error(f"Unsupported file format: {file_ext}") return None def _convert_pdf_to_markdown(self, file_path: str) -> Optional[str]: """ Convert PDF to Markdown. Processing Pipeline: 1. Validate file 2. Preprocess: Extract images and generate captions 3. Convert PDF to Markdown (via Docling) Args: file_path: Path to PDF file Returns: Markdown string or None on failure """ file_path_obj = Path(file_path) preprocessed_path = None try: # Step 1: Validate file is_valid, error_msg = self._validate_pdf_file(file_path_obj) if not is_valid: logger.error(f"PDF file validation failed: {error_msg}") return None # Step 2: Preprocessing (optional - image captions) try: from src.extraction_v2.pdf_preprocessor import PdfPreprocessor preprocessor = PdfPreprocessor() preprocessed_path = preprocessor.preprocess(file_path_obj) logger.info("PDF preprocessing complete") except Exception as preprocess_error: logger.warning(f"PDF preprocessing failed: {preprocess_error}, proceeding without preprocessing") preprocessed_path = file_path_obj # Step 3: Convert to Markdown via Docling logger.info(f"Converting PDF to Markdown via Docling: {preprocessed_path}") _ensure_event_loop() # Ensure event loop exists for Docling result = self.converter.convert(str(preprocessed_path)) if not result or not result.document: raise RuntimeError("Docling conversion returned empty result") doc = result.document markdown = doc.export_to_markdown() logger.info(f"Successfully converted PDF to Markdown ({len(markdown)} chars)") return markdown except Exception as e: logger.error(f"Docling PDF conversion failed: {e}") import traceback logger.debug(f"Conversion error details: {traceback.format_exc()}") return None finally: # Clean up temporary files if preprocessed_path and preprocessed_path != file_path_obj and os.path.exists(preprocessed_path): try: os.remove(preprocessed_path) except Exception as e: logger.warning(f"Failed to remove temp file {preprocessed_path}: {e}") def _convert_pptx_to_markdown(self, file_path: str) -> Optional[str]: """ Convert PowerPoint to Markdown. Processing Pipeline: 1. Convert .ppt/.pptm to .pptx if needed 2. Preprocess: Extract images, shapes, notes 3. Convert preprocessed .pptx to Markdown (via Docling) Handles .pptx, .ppt, and .pptm files: - .pptx: Preprocess → Docling - .ppt: LibreOffice → Preprocess → Docling - .pptm: LibreOffice → Preprocess → Docling Args: file_path: Path to PowerPoint file Returns: Markdown string or None on failure """ file_path_obj = Path(file_path) temp_pptx_path = None preprocessed_path = None try: # Step 1: Format normalization (.ppt/.pptm → .pptx) file_ext = file_path_obj.suffix.lower() if file_ext in ['.ppt', '.pptm']: logger.info(f"Converting {file_ext} to .pptx via LibreOffice...") temp_pptx_path = self._convert_ppt_to_pptx(file_path_obj) if temp_pptx_path is None: logger.error("LibreOffice conversion failed") return None file_to_preprocess = Path(temp_pptx_path) else: # Already .pptx file_to_preprocess = file_path_obj # Step 2: Preprocessing try: from src.extraction_v2.pptx_preprocessor import PptxPreprocessor preprocessor = PptxPreprocessor() preprocessed_path = preprocessor.preprocess(file_to_preprocess) logger.info("PowerPoint preprocessing complete") except Exception as preprocess_error: logger.warning(f"PowerPoint preprocessing failed: {preprocess_error}, proceeding without preprocessing") preprocessed_path = file_to_preprocess # Step 3: Convert to Markdown via Docling logger.info(f"Converting PowerPoint to Markdown via Docling: {preprocessed_path}") _ensure_event_loop() # Ensure event loop exists for Docling result = self.converter.convert(str(preprocessed_path)) if not result or not result.document: raise RuntimeError("Docling conversion returned empty result") doc = result.document markdown = doc.export_to_markdown() logger.info(f"Successfully converted PowerPoint to Markdown ({len(markdown)} chars)") return markdown except Exception as e: logger.error(f"Docling PowerPoint conversion failed: {e}") import traceback logger.debug(f"Conversion error details: {traceback.format_exc()}") return None finally: # Clean up temporary files for temp_file in [temp_pptx_path, preprocessed_path]: if temp_file and temp_file != file_path_obj and os.path.exists(temp_file): try: os.remove(temp_file) except Exception as e: logger.warning(f"Failed to remove temp file {temp_file}: {e}") def _validate_pdf_file(self, file_path: Path) -> tuple[bool, Optional[str]]: """ Validate PDF file before conversion. Args: file_path: Path to PDF file Returns: Tuple of (is_valid, error_message) """ # Check exists if not file_path.exists(): return False, "File not found" # Check size (<100MB) size_mb = file_path.stat().st_size / (1024 * 1024) if size_mb > 100: return False, f"File too large: {size_mb:.1f}MB (max 100MB)" # Check extension ext = file_path.suffix.lower() if ext != '.pdf': return False, f"Not a PDF file: {ext}" # Verify PDF structure (basic check) try: with open(file_path, 'rb') as f: header = f.read(5) if not header.startswith(b'%PDF-'): return False, "Invalid PDF file (missing PDF header)" except Exception as e: return False, f"File validation error: {str(e)}" return True, None ``` **5. Pipeline Integration** ```python # In pipeline.py - Add new methods async def process_pdf_document( self, document_id: UUID, file_path: str ) -> ExtractionResult: """ Process PDF document using V2 pipeline with semantic chunking. Args: document_id: UUID for document tracking file_path: Path to PDF file Returns: ExtractionResult with chunk records or error information """ try: logger.info( f"Starting V2 PDF extraction pipeline for document {document_id}", file_path=file_path ) # Step 1: PDF → Markdown (via Docling) logger.info("Step 1/2: Converting PDF to Markdown with preprocessing...") markdown = self.converter.convert_to_html(file_path) # Returns Markdown for PDF if markdown is None: logger.error("PDF document conversion failed") return ExtractionResult( success=False, document_id=document_id, file_path=file_path, record_count=0, records=[], error="PDF document conversion failed" ) logger.info(f"Conversion complete: {len(markdown)} chars") # Step 2: Get DoclingDocument for chunking logger.info("Step 2/2: Chunking document with HybridChunker...") # Re-convert to get DoclingDocument from pathlib import Path from docling.document_converter import DocumentConverter, PdfFormatOption from docling.datamodel.base_models import InputFormat converter = DocumentConverter( format_options={InputFormat.PDF: PdfFormatOption()} ) _ensure_event_loop() result = converter.convert(file_path) doc = result.document # Chunk the document chunks = self.semantic_chunker.chunk_document( doc=doc, file_id=document_id, original_filename=Path(file_path).name ) logger.info( "V2 PDF pipeline complete", file=Path(file_path).name, total_chunks=len(chunks) ) # Convert chunks to ExtractionResult format return ExtractionResult( success=True, document_id=document_id, file_path=file_path, record_count=len(chunks), records=chunks, table_count=0 ) except Exception as e: logger.error( f"V2 PDF pipeline failed: {e}", exc_info=True ) return ExtractionResult( success=False, document_id=document_id, file_path=file_path, record_count=0, records=[], error=str(e) ) async def process_pptx_document( self, document_id: UUID, file_path: str ) -> ExtractionResult: """ Process PowerPoint document using V2 pipeline with semantic chunking. Args: document_id: UUID for document tracking file_path: Path to PowerPoint file (.pptx, .ppt, .pptm) Returns: ExtractionResult with chunk records or error information """ try: logger.info( f"Starting V2 PowerPoint extraction pipeline for document {document_id}", file_path=file_path ) # Step 1: PowerPoint → Markdown (via Docling) logger.info("Step 1/2: Converting PowerPoint to Markdown with preprocessing...") markdown = self.converter.convert_to_html(file_path) # Returns Markdown for PowerPoint if markdown is None: logger.error("PowerPoint document conversion failed") return ExtractionResult( success=False, document_id=document_id, file_path=file_path, record_count=0, records=[], error="PowerPoint document conversion failed" ) logger.info(f"Conversion complete: {len(markdown)} chars") # Step 2: Get DoclingDocument for chunking logger.info("Step 2/2: Chunking document with HybridChunker...") # Re-convert to get DoclingDocument from pathlib import Path from docling.document_converter import DocumentConverter, PptxFormatOption from docling.datamodel.base_models import InputFormat file_ext = Path(file_path).suffix.lower() if file_ext in ['.ppt', '.pptm']: # Convert to .pptx first temp_pptx = self.converter._convert_ppt_to_pptx(Path(file_path)) if temp_pptx is None: return ExtractionResult( success=False, document_id=document_id, file_path=file_path, record_count=0, records=[], error="LibreOffice conversion failed" ) conversion_path = temp_pptx else: conversion_path = file_path converter = DocumentConverter( format_options={InputFormat.PPTX: PptxFormatOption()} ) _ensure_event_loop() result = converter.convert(conversion_path) doc = result.document # Chunk the document chunks = self.semantic_chunker.chunk_document( doc=doc, file_id=document_id, original_filename=Path(file_path).name ) logger.info( "V2 PowerPoint pipeline complete", file=Path(file_path).name, total_chunks=len(chunks) ) # Convert chunks to ExtractionResult format return ExtractionResult( success=True, document_id=document_id, file_path=file_path, record_count=len(chunks), records=chunks, table_count=0 ) except Exception as e: logger.error( f"V2 PowerPoint pipeline failed: {e}", exc_info=True ) return ExtractionResult( success=False, document_id=document_id, file_path=file_path, record_count=0, records=[], error=str(e) ) # Add sync versions def process_pdf_document_sync( self, document_id: UUID, file_path: str ) -> ExtractionResult: """Synchronous version of process_pdf_document.""" import asyncio import concurrent.futures def run_async_in_thread(): new_loop = asyncio.new_event_loop() asyncio.set_event_loop(new_loop) try: return new_loop.run_until_complete( self.process_pdf_document(document_id, file_path) ) finally: pending = asyncio.all_tasks(new_loop) for task in pending: task.cancel() if pending: new_loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True)) new_loop.close() with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor: future = executor.submit(run_async_in_thread) return future.result() def process_pptx_document_sync( self, document_id: UUID, file_path: str ) -> ExtractionResult: """Synchronous version of process_pptx_document.""" import asyncio import concurrent.futures def run_async_in_thread(): new_loop = asyncio.new_event_loop() asyncio.set_event_loop(new_loop) try: return new_loop.run_until_complete( self.process_pptx_document(document_id, file_path) ) finally: pending = asyncio.all_tasks(new_loop) for task in pending: task.cancel() if pending: new_loop.run_until_complete(asyncio.gather(*pending, return_exceptions=True)) new_loop.close() with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor: future = executor.submit(run_async_in_thread) return future.result() # Convenience functions async def extract_from_pdf( document_id: UUID, file_path: str ) -> ExtractionResult: """Convenience function to extract chunks from PDF file.""" pipeline = ExtractionPipelineV2() return await pipeline.process_pdf_document(document_id, file_path) async def extract_from_pptx( document_id: UUID, file_path: str ) -> ExtractionResult: """Convenience function to extract chunks from PowerPoint file.""" pipeline = ExtractionPipelineV2() return await pipeline.process_pptx_document(document_id, file_path) def extract_from_pdf_sync( document_id: UUID, file_path: str ) -> ExtractionResult: """Synchronous convenience function to extract chunks from PDF file.""" pipeline = ExtractionPipelineV2() return pipeline.process_pdf_document_sync(document_id, file_path) def extract_from_pptx_sync( document_id: UUID, file_path: str ) -> ExtractionResult: """Synchronous convenience function to extract chunks from PowerPoint file.""" pipeline = ExtractionPipelineV2() return pipeline.process_pptx_document_sync(document_id, file_path) ``` ### Integration Points **Internal Modules:** - `DocumentToHtmlConverter` - Format normalization and conversion (Excel → HTML, Word → Markdown, **PDF → Markdown, PowerPoint → Markdown**) - `PdfPreprocessor` - Image extraction and captioning for PDF - `PptxPreprocessor` - Image/shape/note extraction for PowerPoint - `SemanticChunker` - Docling HybridChunker for all text documents (Word, PDF, PowerPoint) - `HtmlTableExtractor` - Table extraction (Excel only) - `LlmHeaderDetector` - Header detection (Excel) - `RecordBuilder` - JSON record generation (Excel) **External Dependencies:** - **LibreOffice CLI** - .ppt/.pptm → .pptx conversion - **Docling** - PDF → Markdown, PowerPoint → Markdown conversion - **Azure OpenAI** - Image captioning (GPT-4o vision) - **python-pptx** - PowerPoint manipulation - **pypdf** - PDF metadata extraction **Database Interactions:** - Documents table - Store PDF/PowerPoint files with metadata - ExtractionResults table - Store extraction status - Milvus - Vector indexing of extracted content **Event Flow:** - Celery task `extract_v2_task` triggers PDF/PowerPoint extraction - Status updates broadcast via Redis - Completion notification to API layer --- ## Implementation Stack **Runtime:** - Python 3.11+ - FastAPI 0.122 - Uvicorn (async ASGI server) - Celery 5.3+ (background tasks) - Redis 7.0+ (task queue, cache) **Document Processing:** - Docling 2.63.0+ (PDF → Markdown, PowerPoint → Markdown) - python-pptx (PowerPoint manipulation) - pypdf (PDF metadata) - openpyxl 3.1.5 (Excel manipulation) - BeautifulSoup4 4.12+ (HTML parsing) - lxml 5.0+ (parser backend) - LibreOffice 7.0+ CLI (format conversion) - docling-core (HybridChunker, custom serializers) - tiktoken (OpenAI tokenizer for chunking) **AI/ML:** - Azure OpenAI GPT-4o (image captioning) - openai 1.0+ (Python client) **Data Layer:** - PostgreSQL (SQLAlchemy 2.0+, asyncpg) - Milvus 2.4+ (pymilvus) **Testing:** - pytest 8.0+ - pytest-asyncio 0.23+ **Code Quality:** - black 24.0+ (formatter, 100 char lines) - ruff 0.5+ (linter) - mypy 1.10+ (type checking) - structlog 24.0+ (logging) --- ## Technical Details ### Format Normalization Algorithm **PowerPoint Legacy Formats (.ppt, .pptm):** 1. Detect .ppt or .pptm file via extension 2. Use LibreOffice CLI: `libreoffice --headless --convert-to pptx` 3. Verify .pptx output exists and is valid 4. Proceed with .pptx pipeline 5. 60-second timeout with retry logic **PDF Format Handling:** - All PDF types accepted directly (no conversion needed) - Docling handles text-based, scanned (OCR), and mixed-content PDFs internally - Validation: File size (<100MB), PDF header check **Timeout and Error Handling:** - 60-second timeout for LibreOffice conversion - Retry once on timeout - Fall back to V1 pipeline if conversion fails twice - Log detailed error with file characteristics ### PDF Document Preprocessing Pipeline **Image Extraction and Captioning:** 1. Use pypdf to iterate through PDF pages 2. Extract all embedded images (PNG, JPEG, GIF) 3. For each image: - Extract surrounding context (page text) - Send to Azure OpenAI GPT-4o vision API - Generate descriptive caption - Save to metadata file (sidecar JSON) 4. Docling processes PDF with metadata reference **Metadata Handling:** - Extract page numbers, document properties - Create sidecar JSON file: `{filename}_metadata.json` - Contains image captions indexed by page number - Referenced during Docling conversion ### PowerPoint Document Preprocessing Pipeline **Image Extraction and Captioning:** 1. Use python-pptx to iterate through slides 2. Extract all embedded images 3. For each image: - Extract slide context (title, bullet points) - Send to Azure OpenAI GPT-4o vision API - Generate descriptive caption - Insert caption as text box below image in slide **Shape and Annotation Handling:** 1. Extract text from all shapes and text boxes 2. Already visible in slide content (no additional processing needed) 3. Count shapes for metrics **Speaker Notes Extraction:** 1. Access slide notes via python-pptx 2. For each slide with notes: - Extract notes text - Insert as text box at bottom of slide: `[SPEAKER NOTES]: {text}` 3. Preserve for Docling Markdown conversion ### Semantic Chunking Strategy **Why Semantic Chunking for PDF/PowerPoint?** Like Word documents, PDF and PowerPoint files contain continuous narrative text and mixed content. Semantic chunking provides: - **Context Preservation:** Chunks break at natural boundaries (paragraphs, sections, slides) - **Better Retrieval:** Semantic units improve vector search relevance - **Scalability:** Handles documents of any length without manual configuration - **Unified Approach:** Same strategy across Word, PDF, and PowerPoint **Docling HybridChunker Configuration:** ```python # HybridChunker configuration with OpenAI tokenizer import tiktoken from docling_core.transforms.chunker.hybrid_chunker import HybridChunker from docling_core.transforms.chunker.tokenizer.openai import OpenAITokenizer # Initialize tokenizer for text-embedding-3-large model tokenizer = OpenAITokenizer( tokenizer=tiktoken.encoding_for_model("text-embedding-3-large"), max_tokens=8000, # Maximum tokens per chunk ) # Initialize HybridChunker with custom serializers chunker = HybridChunker( tokenizer=tokenizer, serializer_provider=ImgTableAnnotationSerializerProvider(), ) # Chunk document chunk_iter = chunker.chunk(dl_doc=doc) chunks = list(chunk_iter) # Contextualize each chunk for chunk in chunks: text = chunker.contextualize(chunk=chunk) ``` **Chunking Process:** 1. **Get Docling Document:** Docling converts entire document - PDF: Text, tables, images → Markdown - PowerPoint: Slides, notes, images → Markdown 2. **Token-Based Chunking:** HybridChunker uses tiktoken tokenizer 3. **Custom Serializers:** Format content appropriately 4. **Contextualization:** Each chunk includes surrounding context 5. **Metadata Enrichment:** Chunk tagged with index, page/slide number, character count 6. **Vector Indexing:** Each chunk gets its own embedding for semantic search **Integration with Preprocessing:** Images and speaker notes are embedded **inline** before chunking: - **PDF Images:** `[IMAGE: AI-generated caption...]` (via metadata) - **PowerPoint Images:** Text box with `[IMAGE CAPTION]: {caption}` - **Speaker Notes:** Text box with `[SPEAKER NOTES]: {text}` This ensures they're preserved in semantic context and searchable via vector similarity. ### Content Record Structure **PDF Document Chunk Records:** ```json { "document_id": "550e8400-e29b-41d4-a716-446655440000", "file_name": "document.pdf", "page": "page_3", "content_type": "chunk", "content": "This is a semantic chunk from page 3 containing multiple paragraphs. The Docling HybridChunker ensures semantic boundaries are preserved. [IMAGE: Chart showing quarterly revenue trends] The chart illustrates significant growth in Q2.", "metadata": { "chunk_index": 5, "total_chunks": 12, "char_count": 245, "word_count": 38, "page_number": 3, "chunking_method": "docling_hybrid", "tokenizer": "text-embedding-3-large", "max_tokens": 8000 } } ``` **PowerPoint Document Chunk Records:** ```json { "document_id": "550e8400-e29b-41d4-a716-446655440000", "file_name": "presentation.pptx", "page": "slide_5", "content_type": "chunk", "content": "## Q2 Results Overview\n\n- Revenue increased 24%\n- Customer base grew 22%\n- Satisfaction rating: 91%\n\n[IMAGE CAPTION]: Bar chart comparing Q1 vs Q2 metrics across all categories\n\n[SPEAKER NOTES]: Emphasize the customer satisfaction improvement. Mention that this exceeded our annual target.", "metadata": { "chunk_index": 4, "total_chunks": 8, "char_count": 298, "word_count": 45, "slide_number": 5, "chunking_method": "docling_hybrid", "tokenizer": "text-embedding-3-large", "max_tokens": 8000 } } ``` ### Performance Considerations **LibreOffice Conversion (PowerPoint only):** - Average conversion time: 2-5 seconds per file - Timeout: 60 seconds - Parallelization: Celery workers can process multiple files concurrently - File locking prevents concurrent LibreOffice instances **Image Captioning (PDF & PowerPoint):** - GPT-4o vision API latency: 1-3 seconds per image - Batch images when possible (up to 10 per request) - Cache captions to avoid reprocessing - Optional: Can be skipped if preprocessing fails **Semantic Chunking:** - Docling HybridChunker: <1 second for typical documents (<100 pages/slides) - Average chunk size: ~1000-2000 characters (configured max: 8000 tokens) - No API calls required (local processing) **Document Processing:** - PDF: 5-20 seconds per document (depending on images and page count) - PowerPoint: 10-30 seconds per document (conversion + preprocessing + chunking) - Total pipeline time: 10-45 seconds per document ### Security Considerations **Macro Safety (PowerPoint):** - .pptm macros are NOT executed - LibreOffice conversion strips macros - Log macro presence for audit trail **File Validation:** - Validate file size (<100MB) - Check MIME type matches extension - Scan for malformed file structure **PDF Security:** - Password-protected PDFs rejected (requires manual unlock) - Digital signatures not validated (content extraction only) - XSS prevention: Sanitize text before LLM calls **Injection Prevention:** - Sanitize extracted text before LLM calls - Escape HTML entities in content extraction - Validate image data before captioning --- ## Development Setup **Prerequisites:** ```bash # System dependencies sudo apt-get install libreoffice # Ubuntu/Debian brew install libreoffice # macOS # Verify installation libreoffice --version ``` **Environment Setup:** ```bash # Clone and navigate to project cd /home/neil/Documents/DSOL # Install Python dependencies (includes python-pptx, pypdf, docling) uv sync --all-extras # Or add new dependencies uv add python-pptx pypdf # Configure environment cp .env.example .env # Edit .env with Azure OpenAI credentials # Start infrastructure docker compose up -d # PostgreSQL, Redis, Milvus # Run migrations uv run alembic upgrade head # Start development servers make dev # API + Celery worker ``` **Running Tests:** ```bash # Unit tests for PDF processing uv run pytest tests/test_extraction_v2/test_pdf_conversion.py -v # Unit tests for PowerPoint processing uv run pytest tests/test_extraction_v2/test_pptx_conversion.py -v # Integration tests for PDF and PowerPoint uv run pytest tests/test_extraction_v2/test_pdf_pptx_comprehensive.py -v # Full test suite uv run pytest tests/test_extraction_v2/ -v --cov=src/extraction_v2 ``` --- ## Acceptance Criteria **Overall Success:** 1. ✅ All PDF files discovered and tracked (text-based, scanned, mixed) 2. ✅ .ppt and .pptm files successfully converted to .pptx 3. ✅ Images extracted and captioned via AI (PDF and PowerPoint) 4. ✅ Speaker notes extracted and embedded (PowerPoint) 5. ✅ Docling converts PDF → Markdown, PowerPoint → Markdown 6. ✅ Full documents chunked using Docling HybridChunker with OpenAI tokenizer 7. ✅ Chunk configuration: 8000 tokens max 8. ✅ ≥85% extraction success rate (V2 + V1 fallback if applicable) 9. ✅ Batch processor includes PDF and PowerPoint files in status tracking 10. ✅ Comprehensive test suite passing (chunking validation included) **PDF-Specific Criteria:** 1. ✅ Text-based PDFs: Extract text, tables, images 2. ✅ Scanned PDFs: OCR handled by Docling automatically 3. ✅ Mixed-content PDFs: Both text and scanned sections processed 4. ✅ Images captioned and embedded inline in markdown 5. ✅ Page numbers preserved in chunk metadata **PowerPoint-Specific Criteria:** 1. ✅ .pptx files: Direct processing 2. ✅ .ppt files: LibreOffice conversion → processing 3. ✅ .pptm files: LibreOffice conversion → processing (macros stripped) 4. ✅ Images extracted and captioned 5. ✅ Speaker notes extracted and embedded 6. ✅ Slide structure preserved 7. ✅ Slide numbers preserved in chunk metadata --- **End of Technical Specification**