# User Story: PowerPoint Format Normalization & Speaker Notes

**Story ID:** story-pdf-pptx-support-2
**Epic:** PDF and PowerPoint Document Processing Support
**Created:** 2025-12-05
**Status:** Done
**Story Points:** 3
**Priority:** High

---

## User Story

**As a** data engineer
**I want** PowerPoint files (.pptx, .ppt, .pptm) automatically converted and speaker notes embedded inline
**So that** the V2 pipeline can process all PowerPoint formats with speaker notes preserved (images/tables handled by serializers)

---

## Context

**Current Situation:**
- V2 pipeline handles Excel, Word, and now PDF documents
- Word documents use speaker notes-like preprocessing for comments
- No PowerPoint support currently exists
- PowerPoint is common for presentations, reports, training materials
- Docling 2.63.0+ has native PPTX support

**Problem:**
Cannot process PowerPoint presentations which contain valuable structured content in slides and speaker notes.

**Solution:**
Implement .ppt/.pptm → .pptx conversion (LibreOffice), create PptxPreprocessor for speaker notes extraction only (images/tables handled by custom serializers during chunking).

---

## Technical Details

**Files to Modify:**
- `src/extraction_v2/document_converter.py` - Add PowerPoint conversion methods

**File to Create:**
- `src/extraction_v2/pptx_preprocessor.py` - Speaker notes extraction only
- `src/extraction_v2/libreoffice_lock.py` - File locking (if not exists from Word support)

**PowerPoint Preprocessing (Speaker Notes Only):**
```python
# In pptx_preprocessor.py

"""
PowerPoint Preprocessor for DSOL V2 Extraction Pipeline.

This module ONLY handles speaker notes extraction.
Images and tables are handled by custom serializers during chunking.
"""

from pathlib import Path
from pptx import Presentation
from pptx.util import Inches
import structlog

logger = structlog.get_logger(__name__)


class PptxPreprocessor:
    """
    Preprocess PowerPoint documents before Docling conversion.
    
    ONLY extracts and embeds speaker notes inline.
    Images and tables are handled by ImgTableAnnotationSerializerProvider during chunking.
    """

    def __init__(self):
        self.logger = structlog.get_logger()

    def preprocess(self, file_path: Path) -> Path:
        """
        Preprocess PowerPoint: extract speaker notes only.
        
        Returns path to preprocessed .pptx file.
        """
        self.logger.info("Starting PowerPoint preprocessing (speaker notes)", file=file_path.name)
        
        prs = Presentation(str(file_path))

        # Extract and embed speaker notes
        notes_count = self._extract_and_embed_notes(prs)
        self.logger.info(f"Extracted {notes_count} speaker notes")

        # Save preprocessed presentation
        output_path = file_path.parent / f"{file_path.stem}_preprocessed.pptx"
        prs.save(str(output_path))

        self.logger.info(
            "Preprocessing complete",
            input=file_path.name,
            output=output_path.name,
            notes=notes_count
        )

        return output_path

    def _extract_and_embed_notes(self, prs: Presentation) -> int:
        """
        Extract speaker notes and embed inline in slides.
        
        Adds notes as text boxes at bottom of slides for Docling to process.
        """
        notes_count = 0
        
        for slide_num, slide in enumerate(prs.slides, start=1):
            if slide.has_notes_slide:
                notes_text = slide.notes_slide.notes_text_frame.text.strip()
                
                if notes_text:
                    # Add text box with speaker notes
                    textbox = slide.shapes.add_textbox(
                        Inches(0.5),  # left
                        Inches(7),    # top
                        Inches(9),    # width
                        Inches(0.5)   # height
                    )
                    textbox.text_frame.text = f"[SPEAKER NOTES]: {notes_text}"
                    notes_count += 1
                    
                    self.logger.debug(
                        f"Embedded notes from slide {slide_num}",
                        slide=slide_num,
                        notes_length=len(notes_text)
                    )
        
        return notes_count
```

**LibreOffice PowerPoint Conversion:**
```python
# In document_converter.py

def _convert_ppt_to_pptx(self, file_path: Path) -> Optional[str]:
    """
    Convert .ppt or .pptm to .pptx using LibreOffice CLI.
    
    Preserves:
    - Slides and content
    - Speaker notes
    - Images (handled by Docling + serializers)
    
    Strips:
    - Macros (.pptm)
    """
    libreoffice_cmd = self._find_libreoffice_command()
    
    if not libreoffice_cmd:
        logger.error(
            "LibreOffice not found. Required for .ppt/.pptm conversion. "
            "Install: apt-get install libreoffice (Ubuntu) or brew install libreoffice (macOS)"
        )
        return None

    try:
        temp_dir = tempfile.mkdtemp(prefix="pptx_convert_")
        profile_dir = tempfile.mkdtemp(prefix="lo_profile_")

        # File locking to prevent concurrent LibreOffice access
        from src.extraction_v2.libreoffice_lock import LIBREOFFICE_LOCK_FILE
        import fcntl
        
        lock_fd = open(LIBREOFFICE_LOCK_FILE, 'w')
        fcntl.flock(lock_fd, fcntl.LOCK_EX)  # Blocking lock
        logger.debug("LibreOffice lock acquired")

        try:
            result = subprocess.run(
                [
                    libreoffice_cmd,
                    '--headless',
                    f'-env:UserInstallation=file://{profile_dir}',
                    '--convert-to', 'pptx',
                    '--outdir', temp_dir,
                    str(file_path)
                ],
                capture_output=True,
                text=True,
                timeout=60  # 60 second timeout
            )
        finally:
            fcntl.flock(lock_fd, fcntl.LOCK_UN)
            lock_fd.close()
            # Clean up profile directory
            shutil.rmtree(profile_dir, ignore_errors=True)

        if result.returncode != 0:
            logger.error(f"LibreOffice conversion failed: {result.stderr}")
            return None

        output_file = Path(temp_dir) / f"{file_path.stem}.pptx"
        
        if not output_file.exists():
            logger.error(f"Conversion succeeded but output file not found: {output_file}")
            return None

        logger.info(f"Successfully converted to .pptx: {output_file}")
        return str(output_file)

    except subprocess.TimeoutExpired:
        logger.error("LibreOffice conversion timed out (>60s)")
        return None
    except Exception as e:
        logger.error(f"Error during LibreOffice conversion: {e}")
        return None
```

---

## Implementation Steps

1. **Create pptx_preprocessor.py**
   - Implement PptxPreprocessor class
   - Extract speaker notes using python-pptx
   - Embed notes inline as text boxes
   - NO image captioning (handled by serializers)

2. **Add LibreOffice conversion for PowerPoint**
   - Implement _convert_ppt_to_pptx() in document_converter.py
   - Add file locking mechanism
   - Timeout and retry logic
   - Validate converted files

3. **Update document_converter.py routing**
   - Add PowerPoint format detection
   - Route .pptx, .ppt, .pptm to appropriate methods

4. **Add new dependency**
   ```bash
   uv add python-pptx
   ```

5. **Test PowerPoint conversion**
   ```bash
   # Test .ppt conversion
   python -c "
   from src.extraction_v2.document_converter import DocumentConverter
   converter = DocumentConverter()
   pptx_path = converter._convert_ppt_to_pptx(Path('test.ppt'))
   assert pptx_path is not None
   "
   
   # Test speaker notes preprocessing
   python -c "
   from src.extraction_v2.pptx_preprocessor import PptxPreprocessor
   preprocessor = PptxPreprocessor()
   output = preprocessor.preprocess(Path('test_with_notes.pptx'))
   assert output.exists()
   "
   ```

---

## Acceptance Criteria

1. ✅ PptxPreprocessor class created
   - Speaker notes extraction working
   - Inline embedding functional
   - NO image captioning (not needed)

2. ✅ .ppt → .pptx conversion working
   - LibreOffice CLI integration functional
   - File locking prevents concurrent access
   - Timeout handling (60s)

3. ✅ .pptm → .pptx conversion working
   - Macro stripping logged
   - Security: macros not executed

4. ✅ Speaker notes preprocessing functional
   - Notes embedded inline as text boxes
   - Preserved for Docling markdown conversion
   - Slide structure maintained

5. ✅ Error handling robust
   - Missing LibreOffice detected
   - Conversion failures handled
   - Clear error messages

6. ✅ Logging comprehensive
   - Preprocessing metrics tracked
   - Conversion success/failure logged
   - Notes count reported

---

## Testing Strategy

**Unit Tests:**
```python
# tests/test_extraction_v2/test_pptx_conversion.py

def test_ppt_to_pptx_conversion():
    converter = DocumentConverter()
    pptx_path = converter._convert_ppt_to_pptx(Path("test.ppt"))
    assert pptx_path is not None
    assert Path(pptx_path).exists()

def test_pptm_macro_stripping():
    converter = DocumentConverter()
    pptx_path = converter._convert_ppt_to_pptx(Path("test.pptm"))
    assert pptx_path is not None
    # Macros should be stripped

def test_speaker_notes_extraction():
    preprocessor = PptxPreprocessor()
    output = preprocessor.preprocess(Path("test_with_notes.pptx"))
    
    # Load preprocessed file and verify notes embedded
    prs = Presentation(str(output))
    notes_found = False
    for slide in prs.slides:
        for shape in slide.shapes:
            if hasattr(shape, 'text') and '[SPEAKER NOTES]:' in shape.text:
                notes_found = True
                break
    
    assert notes_found is True

def test_no_speaker_notes():
    preprocessor = PptxPreprocessor()
    output = preprocessor.preprocess(Path("test_no_notes.pptx"))
    assert output.exists()
    # Should succeed even with no notes

def test_images_not_preprocessed():
    """Verify images are NOT preprocessed - handled by serializers."""
    preprocessor = PptxPreprocessor()
    output = preprocessor.preprocess(Path("test_with_images.pptx"))
    
    # Preprocessor should NOT extract or caption images
    # Images handled by ImgTableAnnotationSerializerProvider during chunking
    assert output.exists()
```

---

## Dependencies

**New Python Dependencies:**
- python-pptx (PowerPoint manipulation)

**External Dependencies:**
- LibreOffice 7.0+ CLI (.ppt/.pptm conversion)
- Docling 2.63.0+ (PowerPoint → Markdown)

**Internal Dependencies:**
- LibreOffice lock file mechanism (may exist from Word support)
- structlog logging

**Blockers:**
- None - can run in parallel with Story 1

---

## Definition of Done

- [x] pptx_preprocessor.py created
- [x] PptxPreprocessor class implemented (speaker notes only)
- [x] _convert_ppt_to_pptx() method implemented
- [x] File locking mechanism implemented
- [x] Speaker notes extraction working
- [x] python-pptx dependency added
- [x] All unit tests passing
- [x] .ppt files convert successfully
- [x] .pptm files convert successfully (macros stripped)
- [x] Speaker notes embedded inline
- [x] NO image preprocessing (confirmed)
- [x] Code reviewed
- [x] Ready for pipeline integration (Story 3)

---

## Reference

**Tech-Spec:** `docs/feats/pdf_pptx_support/tech-spec.md`
**Epic:** `docs/feats/pdf_pptx_support/epics.md`
**Pattern:** Word document preprocessing (WordPreprocessor for comments)
**Key Difference:** Only handles speaker notes, not images

---

**Story Status:** Done

## Implementation Notes

**Completed:** 2025-12-05

**Files Modified:**
- `src/extraction_v2/document_converter.py` (+125 lines)
  - Added `_convert_pptx_to_markdown()` method with conditional preprocessing
  - Added `_convert_ppt_to_pptx()` method for LibreOffice conversion (.ppt/.pptm → .pptx)
  - Updated `convert_to_html()` routing to handle .pptx, .ppt, .pptm files
  - Implemented file locking for LibreOffice concurrent access prevention
  - No format option specified for PPTX (Docling auto-detects)

- `src/services/file_validator.py` (+3 sections)
  - Updated `validate_file_extension()` to accept .pptx, .ppt, .pptm
  - Updated `validate_magic_bytes()` docstring to include PowerPoint formats
  - Updated `validate_uploaded_file()` docstring to include PowerPoint

- `src/extraction_v2/pptx_preprocessor.py` (97 lines, NEW)
  - Created `PptxPreprocessor` class for speaker notes extraction
  - Implements `preprocess()` method to extract and embed notes inline
  - Adds notes as text boxes at bottom of slides with `[SPEAKER NOTES]:` prefix

- `tests/test_extraction_v2/test_pptx_conversion.py` (335 lines, NEW)
  - 19 comprehensive unit tests covering all scenarios
  - All tests passing

**Key Implementation Decisions:**

1. **Conditional Preprocessing**: Only preprocess PowerPoint files if speaker notes exist
   - Check for notes before preprocessing
   - Skip preprocessing if no notes found (avoids compatibility issues)
   - Direct Docling conversion when no preprocessing needed

2. **No Format Option for PPTX**: Docling auto-detects PowerPoint files
   - Initially tried `WordFormatOption()` → caused MsWordDocumentBackend errors
   - Removed format option specification → Docling uses SimplePipeline automatically
   - This approach works cleanly for all PowerPoint formats

3. **LibreOffice Conversion with File Locking**:
   - Convert .ppt/.pptm to .pptx using LibreOffice CLI
   - File locking prevents concurrent LibreOffice access issues
   - 60-second timeout for conversion operations
   - Automatic cleanup of temporary files

**Test Results:**
- ✅ 19/19 PowerPoint conversion tests PASSED
- ✅ Real-world PowerPoint tested (92KB Japanese/Chinese customer document)
- ✅ Conversion successful: 8511 characters in 0.19 seconds
- ✅ No regressions in existing tests

**Dependencies Added:**
- `python-pptx` - PowerPoint file manipulation library

**Git Commit:**
- Branch: feat/pptx-support
- Ready for commit
