--- stepsCompleted: [step-01-init, step-02-discovery, step-02b-vision, step-02c-executive-summary, step-03-success, step-04-journeys, step-05-domain, step-06-innovation, step-07-project-type, step-08-scoping, step-09-functional, step-10-nonfunctional, step-11-polish, step-12-complete] inputDocuments: [] workflowType: 'prd' documentCounts: briefs: 0 research: 0 brainstorming: 0 projectDocs: 0 classification: projectType: developer_tool domain: scientific complexity: medium projectContext: greenfield --- # Product Requirements Document - RepoPilot SDK **Author:** Tore **Date:** 2026-03-16 ## Executive Summary RepoPilot SDK enables AI agents to scan, index, understand, and conversationally answer questions about enterprise Java codebases. It targets development teams working with large-scale, legacy Java systems — particularly EJB-based applications built on proprietary frameworks (e.g., Fujitsu Futurity) — where onboarding takes weeks and institutional knowledge is locked in code comments, naming conventions, and undocumented patterns. The SDK ingests a Java source tree, builds a structured knowledge graph of classes, methods, data models, dependencies, and business logic flows, then exposes this as a queryable API that LLM-based agents consume to answer natural language questions. Target users: (1) development teams maintaining enterprise Java systems, (2) AI/LLM application developers building code-understanding agents, (3) engineering managers seeking to reduce onboarding time and knowledge silos. ### What Makes This Special RepoPilot provides semantic understanding of enterprise patterns — it knows that a 42,000-line business process class orchestrates service order workflows by populating specific CBS message schemas, referencing domain constants across modules, and following EJB component invocation patterns. This enables agents to answer questions like "What happens when a fiber-to-the-home migration order is processed?" or "Which CBS message fields are populated during service cancellation?" — questions requiring cross-module understanding no existing tool provides. The core insight: enterprise Java codebases encode business logic in framework-specific patterns (EJB components, message schemas, constant pools, request-response flows) that generic code search tools cannot interpret. A purpose-built indexer that understands these patterns creates the missing bridge between raw source code and AI-powered comprehension. ## Project Classification - **Project Type:** Developer Tool (SDK/Library) - **Domain:** AI-powered code intelligence - **Complexity:** Medium — Java AST parsing, knowledge graph construction, and LLM integration within well-understood technical boundaries - **Project Context:** Greenfield — new product using the existing koptBp/koptCommon/koptModel Java codebase (~92K LOC, 107 files) as its reference implementation ## Success Criteria ### User Success - **Onboarding acceleration:** New developers answer specific codebase questions (e.g., "How does service cancellation flow work?") within 5 minutes of running RepoPilot, versus hours/days of manual code reading - **Cross-module comprehension:** Users trace data flows across koptBp, koptModel, and koptCommon through natural language queries without opening a single file - **Answer accuracy:** Agent responses reference correct source files, line ranges, and class relationships — users verify answers against actual code with one click - **Developer experience:** SDK integrates into an existing Python or TypeScript project with <10 lines of setup code; scanning a 100K LOC codebase completes in <60 seconds ### Business Success - **3-month:** SDK indexes and answers questions about the reference codebase with >85% accuracy on a curated 50-question set - **6-month:** 3+ internal teams adopt RepoPilot; average new-developer onboarding time reduced by 40% - **12-month:** Open-source release; 500+ GitHub stars; integration with 2+ major AI agent frameworks (LangChain, Claude Agent SDK) ### Technical Success - **Parsing coverage:** 100% of valid Java source files including EJB annotations, framework-specific patterns, and multi-byte (Japanese) comments - **Knowledge graph completeness:** Class hierarchies, method call graphs, field-level data flows, constant references, and CBS message schema relationships - **Query latency:** <500ms structured context return after initial indexing - **Extensibility:** Plugin architecture for new framework pattern parsers without modifying core SDK ### Measurable Outcomes | Metric | MVP Target | Growth Target | |--------|-----------|---------------| | Java file parse success rate | 100% | 100% | | Question answer accuracy (curated set) | >85% | >92% | | Index build time (100K LOC) | <60s | <30s | | Query response time | <500ms | <200ms | | SDK setup lines of code | <10 | <5 | | Supported framework patterns | EJB/Futurity | +Spring, +Jakarta EE | ## User Journeys ### Journey 1: Kenji — The New Team Member (Primary User, Success Path) **Persona:** Kenji Yamamoto, 28, mid-level Java developer. Just transferred to the K-Opticom service provisioning team. Has 4 years of Spring Boot experience but has never worked with EJB or the Fujitsu Futurity framework. His manager expects him productive within 2 weeks. **Opening Scene:** Kenji stares at `JKKHakkoSODCC.java` — 42,000 lines of business logic with Japanese comments he can read but domain terms he doesn't understand. He's been asked to fix a bug in the FTTH migration flow but can't find where that logic starts. He's spent 3 hours using grep and IDE search, finding dozens of matches for "FTTH" scattered across constants, message classes, and the main business logic file. Nothing connects. **Rising Action:** Kenji's tech lead mentions RepoPilot. He installs the SDK (`pip install repopilot`), points it at the Source/ directory, and runs the scanner. Within 45 seconds, the index is built. He types: *"How does the FTTH migration order flow work?"* RepoPilot returns a structured answer: the flow starts in `JKKHakkoSODCC.java` at the `execFTTHMigration` method region (lines 28,400-28,900), reads migration parameters from `EKK0341A010CBSMsg`, checks service eligibility using constants from `JKKSvcConst.SVC_FTTH_*`, populates `EKK0341B001CBSMsg` with the new service configuration, and invokes the downstream SOD transmission. Each reference links to the exact source location. **Climax:** Kenji follows up: *"What CBS message fields need to change when switching from ADSL to FTTH?"* RepoPilot shows the specific fields in `EKK0341B001CBSMsg1List` that differ between ADSL and FTTH configurations, cross-referencing the constants that control the switch. Kenji finds the bug in under 10 minutes — a constant reference that wasn't updated for a new FTTH variant. **Resolution:** Kenji fixes the bug on his second day. Over the next week, he uses RepoPilot daily. What would have taken 2 weeks of code archaeology takes 3 days. --- ### Journey 2: Maria — The AI Agent Builder (API Consumer) **Persona:** Maria Chen, 32, senior ML engineer at a consulting firm. Building a custom AI assistant for a client who maintains a large EJB-based Java system. The client wants developers to ask questions in Slack and get answers about their codebase. **Opening Scene:** Maria has built LLM-powered chatbots before, but code understanding is different — she can't dump 92K lines into a context window. Naive RAG with code chunks produces terrible answers because retrieval doesn't understand Java structure. **Rising Action:** Maria integrates RepoPilot SDK into her Python agent pipeline: ```python from repopilot import RepoScanner, QueryEngine scanner = RepoScanner("/path/to/java/source") index = scanner.scan() engine = QueryEngine(index) context = engine.query("How are service orders processed?") # Feed structured context to LLM ``` The `context` object is structured: relevant classes with summaries, method chains with call graphs, data flow diagrams showing which message schemas are populated where, and constant definitions with business meaning. **Climax:** Maria feeds this structured context to Claude and gets accurate, detailed answers her client's developers validate as correct. The Slack bot answers questions like "What validation happens before a service cancellation?" by tracing the actual code path, referencing specific CBS message validation fields, and citing exact line numbers. **Resolution:** Maria's consulting firm wins 3 more contracts for similar code-understanding assistants. She contributes a LangChain integration plugin back to RepoPilot's open-source community. --- ### Journey 3: Takeshi — The Engineering Manager (Secondary User) **Persona:** Takeshi Sato, 45, engineering manager overseeing the K-Opticom provisioning platform. Manages 12 developers with 30% annual turnover. Every new hire takes 3-4 weeks to become productive — 2 person-months lost to onboarding per year. **Opening Scene:** The VP asks why the team can't ship faster. Takeshi explains that 3 of 12 developers joined in the last 6 months and still don't fully understand the system. **Rising Action:** Takeshi deploys RepoPilot internally with a team-wide chat interface. He creates a curated set of 50 common onboarding questions and validates answer accuracy. Day 1 onboarding checklist: "Set up RepoPilot, complete the guided codebase tour." **Climax:** The next two hires complete onboarding in 5 days instead of 25. They report feeling confident navigating the codebase by day 3. Team velocity increases measurably. **Resolution:** Takeshi presents results: onboarding time reduced by 80%, team shipped 2 more features than the previous quarter. He requests budget to extend RepoPilot to 3 additional Java repositories. --- ### Journey 4: Sarah — The DevOps/Platform Engineer (Admin/Operations) **Persona:** Sarah Park, 35, platform engineer responsible for developer tooling across 4 development teams with codebases ranging from 50K to 300K LOC. **Opening Scene:** Sarah needs to deploy RepoPilot for 4 teams, ensure indexing runs on CI, updates on each merge to main, and serves queries with acceptable latency. **Rising Action:** Sarah configures RepoPilot's CLI as a CI/CD step: `repopilot index --source ./src --output ./repopilot-index.json`. She sets up a lightweight query service with monitoring for index build times and query latency. **Climax:** One team's codebase has an unusual framework pattern. Sarah uses the plugin API to write a custom pattern recognizer (40 lines of Python) — the framework-specific classes are now properly indexed. No core SDK changes needed. **Resolution:** All 4 teams onboarded within a week. Sarah monitors dashboards showing index freshness, query volumes, and answer feedback scores. ### Journey Requirements Summary | Journey | Key Capabilities Required | |---------|--------------------------| | Kenji (New Developer) | Java scanner, knowledge graph, natural language Q&A, source location linking, cross-module tracing | | Maria (Agent Builder) | Python SDK API, structured context output, query engine, LLM-ready response formatting | | Takeshi (Eng Manager) | Chat interface, curated Q&A validation, onboarding workflows, accuracy metrics | | Sarah (Platform Eng) | CLI tooling, CI/CD integration, pre-built index serialization, plugin API, monitoring hooks | ## Domain-Specific Requirements ### Code Intelligence Domain Concerns - **Accuracy is critical:** Incorrect code references erode trust faster than in general-purpose chatbots. Every source file, line number, and relationship claim must be verifiable - **Multi-byte character handling:** Japanese comments, variable names, and documentation must be parsed and indexed correctly (UTF-8 with CJK support throughout the pipeline) - **Framework pattern fidelity:** The SDK must correctly model EJB component lifecycles, service invocation patterns, and message schema semantics — not just syntax. Misrepresenting a framework pattern is worse than not detecting it - **Intellectual property:** Source code is highly sensitive. The SDK must operate fully locally with no external API calls during scanning/indexing. Query API may optionally connect to LLM providers, but raw source code never leaves the local environment ### Validation Methodology - **Ground truth test suite:** Curate 50+ questions with human-verified answers against the reference koptBp/koptCommon/koptModel codebase - **Regression testing:** Each SDK release must pass the full test suite; accuracy cannot regress - **Expert review:** Domain experts (developers familiar with the reference codebase) validate answer quality quarterly ## Innovation & Novel Patterns ### Detected Innovation Areas RepoPilot represents a **new paradigm** in the developer_tool space: framework-aware agentic code intelligence. Existing tools fall into two categories: 1. **Static analysis tools** (SonarQube, PMD, Checkstyle) — detect code quality issues but don't understand business logic or answer questions 2. **Code search/navigation tools** (Sourcegraph, GitHub code search, IDE "Find Usages") — find text matches and symbol references but lack semantic understanding of enterprise patterns 3. **LLM-based code assistants** (GitHub Copilot, Cursor) — generate code but don't build persistent, queryable knowledge graphs of existing codebases RepoPilot occupies a new category: **structured code understanding for agent consumption**. It builds a queryable knowledge representation that bridges raw source code and LLM reasoning. ### Validation Approach - Benchmark against naive RAG (chunk-and-embed) on the reference codebase with the curated question set - Target: >20 percentage points higher accuracy than naive RAG baseline - Measure: answer precision (correct file/line references), recall (complete answer coverage), and user satisfaction scores ### Risk Mitigation - **Risk:** Java parser fails on unusual syntax or framework-generated code → **Mitigation:** Graceful degradation — unparseable files are indexed as raw text with reduced structural information - **Risk:** Knowledge graph grows too large for in-memory queries on 300K+ LOC codebases → **Mitigation:** Tiered storage with hot/warm/cold query paths; pre-computed summaries for large classes - **Risk:** LLM hallucination despite good context → **Mitigation:** SDK returns structured context with source citations; the consuming LLM is responsible for synthesis, but all references are verifiable ## Developer Tool Specific Requirements ### Language & Runtime Support - **Primary SDK language:** Python 3.10+ (widest AI/ML ecosystem compatibility) - **Secondary SDK language:** TypeScript/Node.js 18+ (web and VS Code extension ecosystem) - **Java parser:** AST parsing capable of handling full Java syntax without requiring a JVM at runtime ### Package Distribution - **Python:** PyPI package (`pip install repopilot`) - **TypeScript:** npm package (`npm install repopilot`) - **CLI:** Standalone binary distribution for CI/CD environments without Python ### API Surface Design - **Scanner API:** `RepoScanner(path, config?) → scan() → RepoIndex` - **Query API:** `QueryEngine(index) → query(natural_language_string) → StructuredContext` - **Index API:** `RepoIndex → save(path) / load(path)` for persistence - **Plugin API:** `PatternPlugin(name, matcher, extractor)` for custom framework patterns ### Documentation Requirements - API reference auto-generated from docstrings - Getting Started guide with <5-minute setup-to-first-query walkthrough - Architecture overview explaining knowledge graph structure - Plugin development guide with examples - Curated example queries for the reference codebase ### Code Examples - Minimal: scan + query in 5 lines - Agent integration: RepoPilot + Claude API for chat - CI/CD: index build + deploy as GitHub Action - Plugin: custom framework pattern recognizer ## Product Scope ### MVP Strategy & Philosophy **MVP Approach:** Problem-solving MVP — prove that structured code indexing produces dramatically better answers than naive approaches. **Core User Journeys Supported:** Kenji (new developer Q&A) and Maria (agent builder API integration). ### MVP Feature Set (Phase 1) 1. **Java source scanner** — Recursively scans a directory tree, parses .java files into ASTs, extracts classes, methods, fields, imports, and inheritance hierarchies 2. **Relationship indexer** — Builds a knowledge graph of cross-file dependencies: method calls, class references, constant usage, message schema population patterns 3. **Structured context API** — Given a natural language query, returns relevant code snippets, class summaries, and relationship chains formatted for LLM consumption 4. **CBS message schema understanding** — Handles the CAANSchemaInfo pattern: parses CONTENTS arrays, maps field names to types, tracks which business logic populates which message fields 5. **Constant pool resolver** — Traces constant references (e.g., JKKStrConst.SOME_VALUE) back to definitions and forward to all usage sites 6. **Index serialization** — Save/load pre-built indexes for fast startup and CI/CD integration 7. **CLI tool** — `repopilot scan`, `repopilot query`, `repopilot serve` commands 8. **Reference implementation** — Working demo against the koptBp/koptCommon/koptModel codebase with a sample chat interface ### Post-MVP Features (Phase 2 — Growth) - Multi-framework support (Spring Boot, Jakarta EE, Apache Camel) - Incremental indexing (re-index only changed files on git diff) - Embedding-based semantic search for conceptual queries - Agent framework integrations (LangChain, Claude Agent SDK, OpenAI Assistants) - Accuracy metrics dashboard for engineering managers - TypeScript/npm SDK ### Phase 3 — Expansion (Vision) - Live code understanding with real-time indexing - Cross-repository unified knowledge graphs - Automated documentation generation from knowledge graph - Business logic extraction for non-technical stakeholders - Change impact analysis ("If I modify this method, what business flows are affected?") - Multi-language support (Kotlin, Groovy, Scala) - IDE extensions (VS Code, IntelliJ) ### Risk Mitigation Strategy - **Technical:** Java parser edge cases mitigated by graceful degradation to raw-text indexing; knowledge graph size managed by tiered storage - **Market:** Validate with internal teams before open-source release; curated question set proves value quantitatively - **Resource:** MVP scoped to single-language (Python), single-framework (EJB/Futurity); can ship with 1-2 developers in 3 months ## Functional Requirements ### Code Scanning & Parsing - FR1: SDK can recursively discover all .java files in a specified directory tree - FR2: SDK can parse Java source files into abstract syntax trees, extracting classes, interfaces, enums, methods, fields, constructors, imports, and annotations - FR3: SDK can extract inheritance hierarchies (extends, implements) across all parsed files - FR4: SDK can parse multi-byte (Japanese/CJK) comments and string literals without data loss - FR5: SDK can identify and skip non-Java files and binary files during scanning ### Knowledge Graph Construction - FR6: SDK can build a cross-file dependency graph mapping method calls from caller to callee across classes - FR7: SDK can trace constant references from usage sites back to their definitions (e.g., JKKStrConst.FIELD_NAME → definition in JKKStrConst.java) - FR8: SDK can detect and model CBS message schema patterns (CAANSchemaInfo subclasses with CONTENTS arrays), mapping field names to types - FR9: SDK can identify which business logic methods populate which message schema fields - FR10: SDK can model EJB component invocation patterns (ServiceComponentRequestInvoker calls, request-response parameter flows) - FR11: SDK can generate per-class summaries including: purpose, key methods, dependencies, and domain role ### Query & Retrieval - FR12: SDK can accept a natural language query string and return a structured context object containing relevant code snippets, class summaries, and relationship chains - FR13: SDK can rank and select the most relevant code elements for a given query using keyword matching and graph traversal - FR14: SDK can include exact source file paths and line number ranges in query results - FR15: SDK can return results in a structured format (JSON/dict) optimized for LLM context injection - FR16: SDK can limit result size to a configurable maximum token count to fit within LLM context windows ### Index Persistence & Management - FR17: SDK can serialize a built knowledge graph index to a file (JSON or binary format) - FR18: SDK can load a pre-built index from file without re-scanning source code - FR19: SDK can report index metadata: file count, class count, method count, relationship count, build timestamp - FR20: SDK can validate an index against current source files and report staleness (files added, modified, or deleted since last build) ### Plugin & Extensibility - FR21: SDK can accept custom pattern plugins that define: pattern name, file/class matcher, and relationship extractor - FR22: SDK can load plugins from a configuration file or programmatic registration - FR23: SDK can apply multiple plugins during a single scan pass - FR24: Plugins can add custom node types and relationship types to the knowledge graph ### CLI & Operations - FR25: CLI can scan a directory and build an index: `repopilot scan --source --output ` - FR26: CLI can answer a query against a pre-built index: `repopilot query --index ""` - FR27: CLI can start a local HTTP query service: `repopilot serve --index --port ` - FR28: CLI can report index statistics: `repopilot info --index ` ### Reference Chat Interface - FR29: SDK ships with a sample chat application that connects to the query API and an LLM provider - FR30: Chat interface can display query results with clickable source file references - FR31: Chat interface can maintain conversation context across multiple questions about the same codebase ## Non-Functional Requirements ### Performance - Index build time: <60 seconds for 100K LOC codebase on a standard developer machine (4-core, 16GB RAM) - Query response time: <500ms for structured context retrieval after index is loaded - Index load time: <5 seconds for a 100K LOC index from serialized file - Memory usage: <2GB RAM for a loaded 100K LOC index ### Security - Source code never transmitted to external services during scanning or indexing - Index files contain derived metadata only — not raw source code verbatim (configurable: full snippets can be included for local-only deployments) - CLI and query service support authentication tokens for multi-user deployments - No telemetry or analytics data collected without explicit opt-in ### Scalability - Supports codebases up to 500K LOC in MVP; 1M+ LOC with tiered storage in growth phase - Query service supports 10 concurrent users in MVP; 100+ concurrent users in growth phase - Index format is forward-compatible: indexes built with SDK v1.x can be loaded by v1.y (y > x) ### Integration - Python SDK follows PEP 8 and publishes type stubs for IDE autocompletion - HTTP query service exposes OpenAPI 3.0 specification - Index format is documented and stable for third-party tool integration - CLI returns structured JSON output when `--format json` flag is used for scripting and CI/CD integration