# Epic: V2 Pipeline Comprehensive Testing

**Epic ID:** v2-pipeline-testing
**Project:** DSOL
**Created:** 2025-12-01
**Status:** Ready for Development

---

## Epic Goal

Achieve 95%+ extraction success rate for the V2 pipeline across all 58 Excel files in CustomerDocument/ through comprehensive testing, file discovery enhancements, and robust error handling.

---

## Epic Scope

**In Scope:**
- Update batch processor to discover all 58 files (including .xlsm)
- Create automated test suite validating all CustomerDocument/ files
- Enhance error handling with pre-flight validation and retry logic
- Implement success metrics tracking and reporting
- Document testing procedures and known limitations

**Out of Scope:**
- New V2 pipeline features (feature-complete)
- Performance optimizations (correctness focus)
- Production deployment automation
- Additional file formats beyond Excel
- UI/frontend changes

---

## Success Criteria

1. ✅ All 58 Excel files discovered and tracked by batch processor
2. ✅ ≥95% extraction success rate (V2 + V1 fallback combined)
3. ✅ Automated test suite passes and validates all files
4. ✅ Comprehensive validation report generated with metrics
5. ✅ Documentation updated (README, troubleshooting guide)
6. ✅ Known limitations documented for failed files

---

## Story Breakdown

### Story 1: Fix File Discovery & .xlsm Support
**File:** `story-v2-pipeline-testing-1.md`
**Focus:** Update glob patterns, discover all 58 files

**Key Tasks:**
- Modify batch_extract_v2.py line 222 to include .xlsm pattern
- Add file validation pre-check logic
- Verify 58/58 files discovered

**Acceptance:** All Excel formats (.xlsx, .xls, .xlsm) tracked

---

### Story 2: Create Comprehensive Test Suite
**File:** `story-v2-pipeline-testing-2.md`
**Focus:** Automated testing for all CustomerDocument/ files

**Key Tasks:**
- Create test_comprehensive_validation.py with parametrized tests
- Implement metrics tracking (success/fallback/failure)
- Add test fixtures and mocks
- Generate success rate validation

**Acceptance:** Test suite processes 58 files, calculates success rate

---

### Story 3: Enhance Error Handling & Reporting
**File:** `story-v2-pipeline-testing-3.md`
**Focus:** Robust error handling, metrics, validation reports

**Key Tasks:**
- Implement error categorization system
- Add retry logic for transient failures
- Generate validation report with insights
- Update README with testing instructions

**Acceptance:** Status JSON includes metrics, report generated, docs updated

---

## Implementation Sequence

```
Story 1 (File Discovery)
   ↓
Story 2 (Automated Tests) ← Can start in parallel with Story 3
   ↓
Story 3 (Error Handling & Reports)
   ↓
Validation: Run full test suite, verify 95% success rate
```

**Dependencies:**
- Story 2 depends on Story 1 (needs all files discovered)
- Story 3 can run in parallel with Story 2
- Final validation requires all 3 stories complete

---

## Technical Context

**From Tech-Spec:**
- 58 Excel files total (.xlsx: 22, .xls: 15, .xlsm: 21)
- Currently tracking 37/58 files (missing .xlsm pattern)
- 27/37 completed successfully (73% of tracked)
- Target: ≥95% overall success rate

**Stack:**
- Python 3.11+, FastAPI 0.122, pytest 8.0+
- openpyxl 3.1.5, docling 2.63.0+, xlrd 2.0.2+
- SQLAlchemy 2.0+, Milvus 2.4+, Azure OpenAI

**Key Files:**
- `batch_extract_v2.py` - Batch processor (modify)
- `tests/test_extraction_v2/` - Test directory (create new tests)
- `CustomerDocument/` - 58 test files

---

## Epic Value

**Business Impact:**
- Confidence in V2 pipeline production-readiness
- Data-driven validation of extraction quality
- Automated regression testing capability
- Clear understanding of edge cases and limitations

**Developer Impact:**
- Comprehensive test coverage
- Clear success metrics
- Documented troubleshooting procedures
- Production-ready batch processing tool

---

## Epic Notes

- All 3 stories reference tech-spec.md for complete implementation context
- Tech-spec provides exact file paths, code locations, and patterns
- Story-context workflow optional (tech-spec is comprehensive)
- Focus on correctness over performance in this iteration

---

**Epic Status:** Ready for Sprint Planning