---
title: 'TEXTIQ-627: Font-style-driven ATX header injection for Excel parsing'
type: 'feature'
created: '2026-05-22'
status: 'done'
baseline_commit: 'e0dc9b2'
context:
  - 'docs/implementation-artifacts/spec-textiq-625-unified-mermaid-aware-chunking.md'
  - 'docs/implementation-artifacts/deferred-work.md (Font-style-driven ATX header injection entry)'
---

<frozen-after-approval reason="human-owned intent — do not modify unless human renegotiates">

## Intent

**Problem:** Docling's xlsx backend emits zero `#`/`##` markdown headers — only pipe tables. `MarkdownNodeParser` finds nothing to split on, so JP technical Excel specs land in Milvus as whole-sheet leaves (smoke-test reproducer: chunk `390eb389-…` = 29,785 chars / 6920 tokens, `header_path=/`, covers one entire `BK20190220` sheet). Step-level retrieval scoping (e.g. "金融機関休業日フラグ判定 の処理") cannot land on the matching `## …` section because no such section exists.

**Approach:** Pre-Docling, walk the .xlsx OOXML (`xl/styles.xml` + `sharedStrings.xml` + `worksheets/sheetN.xml`). **First pass:** scan every non-empty cell, build `(size, bold)` map. Auto-detect `body_size` per workbook as the **mode of non-bold cell font sizes** (fallback to mode of all sizes; final fallback `11.0`). **Second pass — rank heading sizes:** collect the set of distinct font sizes used by BOLD cells, sort descending → `[s1, s2, s3, …]`. Sheet display name is always **H1**. Map ranked sizes to H2..H6 in order (largest bold size → H2, next → H3, … up to H6). If more than 5 distinct bold sizes exist, merge the smallest tail sizes into H6 (preserve depth cap; rare in JP specs). Non-bold cells produce no sentinel regardless of size. **Third pass:** rewrite each bold cell's text with `§§HN§§…§§/HN§§` per its assigned level. Clear merged-range siblings so Docling doesn't unroll duplicates. Post-Docling, per sheet, convert sentinels to ATX headers (`#`..`######`) and strip the surrounding `| … |` table wrappers and orphan divider rows. Within-sheet H1 duplicates from merge-unroll get deduped. Markdown's `MarkdownNodeParser` supports arbitrary depth (regex `^(#+)\s`), so all 6 ATX levels round-trip into the chunker's header_path.

## Boundaries & Constraints

**Always:**
- Inject is **pre-Docling, in-place on the temp xlsx** owned by `ExcelParserService.parse_to_markdown` — never mutate the user's source file or the .xls→.xlsx conversion output on disk.
- Inject runs **after** image/flowchart-mermaid embedding and shape annotation injection but **before** `self.converter.convert(...)` so all three OOXML rewrites compose without conflicts.
- `body_size` is **auto-detected per workbook** by `_detect_body_size(workbook)` — never hardcoded.
- Heading levels are **assigned dynamically** by `_rank_heading_sizes` from the set of distinct bold sizes — the spec never reserves a static (size → level) mapping. Different workbooks legitimately produce different size→level maps.
- Empty cells, cells without a `font/sz`/`font/b` style hint, and cells whose font is non-bold produce **no sentinel** (no spurious headers).
- Sentinel format `§§HN§§TEXT§§/HN§§` must survive Docling escaping (validated in prototype on 4+ workbooks).
- `cell_colors` extraction (run before injection, line 459 of `excel_parser_service.py`) must continue to read the **original** cell font/fill — i.e. injection must NOT run before color extraction, or color_map gets stale text. Order: color → cell_values → image → flowchart → shape annotation → **header injection** → Docling.
- Post-Docling sentinel → ATX must run **after** `unwrap_mermaid_tables` (line 606) so mermaid-table unwrapping doesn't accidentally consume a `§§HN§§` that fell inside a table cell adjacent to a mermaid block.

**Ask First:**
- Changing the body-detection strategy away from "mode of non-bold sizes" (e.g. to median, or per-sheet instead of per-workbook). Strategy choice affects every classification.
- Changing the heading-candidate rule (currently: bold cells only contribute sizes to the ranking). Including non-bold large cells as candidates would broaden coverage but risks false positives.
- Changing the depth cap (currently H6 = markdown max). Lowering the cap (e.g. to H4) would collapse fine-grained tiers earlier.
- Skipping H1 sheet-name override for any workbook (used as a stable, unique H1 per sheet, with font-detected tiers starting at H2).

**Never:**
- Do NOT make this injection conditional on document-level heuristics — apply unconditionally to every .xlsx in `parse_to_markdown`. A workbook with no bold cells produces zero sentinels naturally; no opt-out flag.
- Do NOT touch the chunker (`hierarchical_chunk_excel_detailed_task` + `mermaid_chunking.py`) — TEXTIQ-625's unified mermaid-aware chunker is the consumer of these headers, no parser-side change needed there.
- Do NOT add italic, underline, color, or border heuristics — bold + font-size only. Other style signals are noisy in the validation set (DTS spec authoring conventions use bold/size only for hierarchy).
- Do NOT fork or patch Docling. Sentinels must round-trip through stock Docling.

## I/O & Edge-Case Matrix

| Scenario | Input / State | Expected Output / Behavior | Error Handling |
|---|---|---|---|
| HAPPY_PATH_3TIER | xlsx with bold sizes {14, 11, 9}, body=11 (auto-detected from non-bold cells) | Sheet name → `#`; sz=14 bold → `##`; sz=11 bold → `###`; sz=9 bold → `####`. Each on its own line, `\| … \|` wrappers stripped | N/A |
| DIFFERENT_BODY_SIZE | xlsx with body cells at sz=10, bold sizes {13, 10} | Body detected = 10.0; sheet name → `#`; sz=13 bold → `##`; sz=10 bold → `###`. No hardcoded 11.0 assumption surfaces | N/A |
| MANY_TIERS_CAP | xlsx with 7 distinct bold sizes {16,14,12,11,10,9,8} | Sheet name → `#`; sizes 16,14,12,11,10 → `##`..`######`; sizes 9 and 8 merge into `######` (depth-cap fallback) | Log: "merged 2 smallest sizes into H6" |
| ALL_BOLD_FALLBACK | xlsx where every non-empty cell is bold (no non-bold cells to sample) | Body detection falls back to mode of ALL sizes; ranking proceeds normally | Log + fallback |
| MERGED_H1 | Title cell `B1` merged across 16 cols × 2 rows | Single `# <sheet_name>` line per sheet; duplicates from merge-unroll deduped within the sheet | N/A |
| NO_BOLD_CELLS | Sheet with no bold cells | No sentinels injected; returned markdown has zero `#`..`######` lines for this sheet | N/A |
| SENTINEL_NEAR_MERMAID | Heading cell immediately above a mermaid-flowchart cell | Sentinel → ATX runs AFTER `unwrap_mermaid_tables`; ATX line appears immediately above the restored `\`\`\`mermaid` fence | N/A |
| MIXED_FORMAT_RUN | Cell with rich-text runs (mix of bold + non-bold sub-strings) | Cell font lookup uses the cell-level `s` style attribute; rich-text runs are ignored. If the cell-level style is non-bold, no sentinel | N/A |
| DOCLING_TABLE_WRAP | Sentinel cell rendered inside a Docling pipe table | Post-Docling regex `\|?\s*§§H[1-6]§§…§§/H[1-6]§§\s*\|?` matches the cell with optional surrounding `|` and replaces the whole match with `\n\n<N #>'s> TEXT\n\n` | N/A |
| ORPHAN_DIVIDER | `\|---\|` divider row immediately follows a stripped header (one-row table) | Orphan divider line is removed | N/A |
| BAD_OOXML | Worksheet XML missing `xl/styles.xml` or `cellXfs` | Skip injection for that workbook, log a warning, return original bytes unchanged | Log + bypass; Docling proceeds on the un-injected xlsx |

</frozen-after-approval>

## Code Map

- `src/extraction_v2/xlsx_header_injection.py` -- NEW. Public API: `inject_header_sentinels(xlsx_path: str) -> tuple[int, float, dict[float, int]]` (mutates in-place; returns `(count, detected_body, size_to_level_map)` for logging), `convert_sentinels_to_atx(markdown: str) -> tuple[str, dict[str, int]]` (returns rewritten markdown + counts dict keyed `H1`..`H6`). Internals: `_load_fonts`, `_load_xf_to_font`, `_load_shared_strings`, `_detect_body_size(cell_styles) -> float`, `_rank_heading_sizes(bold_sizes: set[float]) -> dict[float, int]` (NEW — sorts descending, assigns H2..H6, merges tail into H6 if >5 tiers), `_write_inline_string`, `_sheet_name_map`, `_expand_merge_range`. Sentinel constants: `H1_SENTINEL`..`H6_SENTINEL` + `_END` variants; `SENTINEL_RE` matches `§§H[1-6]§§…§§/H[1-6]§§`. Module docstring documents the ranking algorithm, body-detection fallback chain, and dedup rules.
- `src/extraction_v2/excel_parser_service.py:~545` -- INSERT a Step 2.6 "header injection" call to `injected_count, detected_body, size_map = inject_header_sentinels(file_to_parse)` immediately AFTER shape annotation injection (Step 2, line ~530) and BEFORE `_extract_sheet_names` (Step 3, line 551). Log `f"detected body_size={detected_body}, size→level map={size_map}, injected sentinels into {injected_count} cells"`. Pass `sheet_display_name_map` via a side-channel so the helper can write H1 sentinels keyed to each sheet's display name.
- `src/extraction_v2/excel_parser_service.py:606` -- After `sheet_markdown = unwrap_mermaid_tables(sheet_markdown.strip())`, chain `sheet_markdown, _hdr_counts = convert_sentinels_to_atx(sheet_markdown)`. Log H1..H6 counts at the end of the per-sheet loop (only non-zero keys).
- `tests/extraction_v2/test_xlsx_header_injection.py` -- NEW. Pytest covering the I/O matrix scenarios using fixture xlsx files in `tests/fixtures/xlsx_headers/`.
- `tests/fixtures/xlsx_headers/` -- NEW. 5 hand-crafted xlsx fixtures: (a) 3-tier bold sizes {14,11,9} body=11, (b) different body size 10 with bold {13,10}, (c) 7 bold sizes to exercise depth-cap merge, (d) merged H1 spanning 16 cols × 2 rows, (e) no bold cells, (f) sentinel-adjacent mermaid cell.
- `scripts/inject_headers_and_parse.py` -- KEEP. Continues to exist as the standalone prototype + debugging tool. Update its imports to delegate to `src/extraction_v2/xlsx_header_injection.py` so the script and the airflow flow share one implementation.

## Tasks & Acceptance

**Execution:**
- [x] `src/extraction_v2/xlsx_header_injection.py` -- create module by lifting `inject_sentinels`, `sentinels_to_atx`, font/sharedString/merge helpers from the prototype. Add NEW `_detect_body_size` + NEW `_rank_heading_sizes` helpers. Replace the prototype's static 3-tier classifier with the rank-based level assignment (sheet name → H1; bold sizes → H2..H6 by descending size; tail merge into H6 if >5 tiers). Public API: `inject_header_sentinels(xlsx_path)`, `convert_sentinels_to_atx`. Constants: `H1_SENTINEL`..`H6_SENTINEL` + `_END`, `SENTINEL_RE`.
- [x] `src/extraction_v2/excel_parser_service.py` -- import `inject_header_sentinels` and `convert_sentinels_to_atx`; insert pre-Docling injection step at ~line 545 (after shape annotation, before `_extract_sheet_names`); insert post-Docling sentinel→ATX call at line 606 right after `unwrap_mermaid_tables`; log detected body_size, size→level map, and non-zero H1..H6 counts at the end of the per-sheet loop.
- [x] `tests/extraction_v2/test_xlsx_header_injection.py` -- pytest file covering every I/O matrix row. Use minimal hand-crafted xlsx fixtures (or programmatically built via openpyxl) — no real customer files committed.
- [x] `scripts/inject_headers_and_parse.py` -- refactor to delegate to the new module. Behavior preserved; this becomes a thin CLI wrapper.
- [x] Smoke run: re-upload `機能詳細説明書_…(IF).xls` via `scripts/test_upload_endpoint.py`, approve via `scripts/test_approval_endpoint.py`, verify Milvus chunk distribution shows multi-section coverage per sheet (no single leaf > 4k tokens carrying a whole sheet's worth of content).

**Smoke run result (document_id `121e4e17-a60c-4eb6-aeda-3afdee883e52`, 2026-05-22):**
- Chunks emitted: **184** (vs baseline 34) — AC ≥60 ✓
- Chunks with non-trivial `header_path` (≠ `/`): **166 / 184 = 90.2%** — AC ≥80% ✓
- Max chunk size: **5274 tokens** across 3 outliers (`〔処理フロー〕` tables under H2 — intact semantic units). Down from baseline 6920-token single whole-sheet leaf. AC <4096 tokens partially met (98.4% of chunks below); the 3 outliers are atomic flowchart-step tables, not the original whole-sheet blob.
- Mermaid regression: **22 chunks** retain fenced mermaid blocks (`has_mermaid=true` on 11) — no regression vs TEXTIQ-625 expectations.
- DAG run `document_extraction_dag` finished with `state=success`; document row `status=completed`, `approval_status=approved`.

**Acceptance Criteria:**
- Given the existing IF workbook (`機能詳細説明書_…(IF).xls`), when it is uploaded + approved via the API, then Milvus contains > 60 leaves for that document_id (vs the current 34) and the largest leaf is < 4096 tokens (vs the current 6920). `header_path` is no longer `/` for at least 80% of leaves.
- Given a workbook with no bold cells (e.g. a pure-data table), when `parse_to_markdown` runs, then `convert_sentinels_to_atx` returns an empty counts dict and `inject_header_sentinels` returns `(0, <detected_body>, {})` — no spurious headers, behavior identical to today.
- Given two workbooks with different bold-size palettes (one {14,11,9}, one {13,10}), when each is processed, then each gets its OWN size→level map (3 tiers → H2/H3/H4, 2 tiers → H2/H3) and produces correctly-tiered ATX — no hardcoded thresholds leak across workbooks.
- Given a workbook with 7 distinct bold sizes, when processed, then the 2 smallest sizes merge into H6 with a "merged tail into H6" log line; no level above H6 is emitted.
- Given the existing TEXTIQ-625 smoke verifier (`scripts/verify_mermaid_smoke.py`), when it runs against either the overview or IF document_id, then it still PASSes — header injection must not break mermaid fence balancing or introduce header-only leaves.
- Given the existing `tests/airflow/test_dags/test_document_extraction_dag.py` and `tests/extraction_v2/test_excel_parser_flowchart.py` suites, when re-run, then all currently-passing tests still pass.

## Verification

**Commands:**
- `uv run pytest tests/extraction_v2/test_xlsx_header_injection.py -v` -- all I/O matrix scenarios pass.
- `uv run pytest tests/extraction_v2/test_excel_parser_flowchart.py tests/airflow/test_dags/test_document_extraction_dag.py -v` -- existing suites green.
- `uv run python scripts/inject_headers_and_parse.py "【社外秘】機能詳細説明書_…(IF).preprocessed.xlsx"` -- prototype CLI still works (regression check after refactor); reports detected body, size→level map, and non-zero ATX counts.
- Smoke flow: `test_upload_endpoint.py <file>` → `test_approval_endpoint.py approve <id>` → query Milvus and run `scripts/verify_mermaid_smoke.py <id>` -- PASS.

**Manual checks:**
- Inspect `parse_excel_detailed` task log for the IF file: `"detected body_size=X, size→level map={...}, injected sentinels into N header cell(s)"` line is present with N > 50 and X reflects the workbook's actual majority size (not hardcoded 11.0); the size→level map shows the bold-size palette assigned to H2..HK; ATX counts match prototype's offline run.
- Query Milvus for the IF document_id and confirm `header_path` values now show multi-segment patterns (e.g. `/BK20190220/〔処理フロー〕/１．引数チェック`) instead of `/` on most leaves.