Legacy Rule Extraction

I will define success as reproducible, inspectable candidate rules—not “an LLM summarizes 18M lines.” The source snapshot and toolchain manifest are immutable inputs. Several deterministic passes normalize dialects, expand includes, build intermediate representation, and trace control/data flow. A model can turn a bounded symbolic candidate into readable language, but sentence-level provenance is constructed from the IR, never hallucinated. SMEs remain the authority. I would ask whether production execution is available as an oracle.

V1 ingests source and build metadata, resolves copybooks/macros, parses supported dialects, builds call/control/data graphs, separates business predicates from technical plumbing, emits Given/When/Then-like candidates, clusters similar candidates across systems, and provides side-by-side review. Every candidate exposes exact file, snapshot, line/column and derivation. Unsupported constructs produce explicit coverage gaps. We need deterministic reruns, seven-year audit history, review queues and exports. We do not claim semantic completeness or auto-approve rules.

• Abstention is a valid result.
• Coverage denominator is the source estate, not only successfully parsed files.
• Editing wording never changes the underlying evidence unnoticed.

Eighteen million lines over 40 mock days means only 450k LOC/day, so raw parse throughput is easy; correctness, dependencies and human review dominate. One hundred thousand candidates at seven minutes each is 700k minutes, around 11,667 hours. At 120 SMEs and five focused review hours/day, a full pass is about 19.4 working days, before rework. This tells me to prioritize candidates by risk and use stratified sampling for quality—not ask every reviewer to inspect every near-duplicate.

18M LOC / 40 days = 450k LOC/day
100k × 7 min = 700k min = 11,667 h
11,667 / (120 × 5 h/day) ≈ 19.4 review days

SourceSnapshot has one cryptographic manifest. SourceUnit maps paths and encoding to blob hashes. IR nodes and graph edges are namespaced by snapshot and toolchain bundle. A RuleCandidate stores normalized predicate/outcome IR and an evidence-set hash; it is not just prose. RuleRendering is versioned separately so better wording does not rewrite candidate identity. RuleCluster groups candidates but does not collapse them, because similar implementations can have meaningful jurisdiction or effective-date differences. ReviewDecision is append-oriented and optimistic-versioned.

• Candidate identity derives from normalized logic plus evidence—not line number alone.
• Source spans include expansion stacks back through copybooks/macros.
• Approved rule points to exact candidate and rendering versions.

The source vault verifies encoding and hashes every unit, then seals a snapshot manifest. A deterministic preprocessor expands copybooks, macros and conditional compilation while recording an origin map. Language front ends parse to a common but loss-aware IR; unknown constructs become typed opaque nodes. Link passes resolve calls, files and shared data. Control- and data-flow passes create summaries per procedure, then a fixpoint propagates summaries across the call graph. Candidate passes identify externally meaningful predicates and outcomes, trace backward to source spans, and filter logging/retry/plumbing. Correlation computes blocking keys and similarities. A bounded renderer produces prose from predicate IR and receives only evidence IDs. The review index is published after run validation.

• Each pass consumes and emits content-addressed artifacts.
• Run manifest pins source, config, dictionaries, compiler and model information.
• Failed partitions retry without changing completed artifacts.

Preprocessing emits a bidirectional origin map: every normalized token maps to one or more original byte and line spans, including an include stack. Every IR node carries the union of contributing token spans plus the pass that created it. A candidate stores a minimal evidence subgraph. The renderer cannot provide arbitrary citations; it must assign each sentence to supplied evidence IDs. A deterministic verifier rejects a rendering if a sentence lacks evidence, points outside the snapshot, or states a literal absent from the predicate/outcome IR. The UI opens the sealed blob at exact spans and also shows expansion context.

Pass one normalizes fixed/free formats, encodings and dialect syntax without erasing original coordinates. Pass two resolves includes and conditional compilation. Pass three builds per-program CFGs and symbol tables. Pass four computes procedure summaries: inputs read, records mutated, exits and side effects. Pass five resolves inter-program calls and JCL execution edges. Pass six performs backward slicing from business outcomes such as payment amount, eligibility flag or denial code to predicates that influence them. A deterministic candidate builder serializes those predicate/outcome slices. Recursive call components use bounded fixpoint iteration; unresolved dynamic calls remain gaps with impact counts.

• Order passes explicitly; do not stuff the estate into one context window.
• Persist intermediate artifacts for diagnosis and replay.
• Distinguish “not a business rule” from “could not analyze.”

I cluster rather than merge. First create blocking keys from normalized domain terms, outcome codes and predicate shape. Within blocks, compare canonical IR features, cited policy terms and embeddings of a controlled rendering. A pairwise explanation says which predicates align and which differ. Graph clustering produces candidate families, but an SME chooses same-rule, variant, supersedes or unrelated. Effective date, geography, claim type and system remain facets. One golden business rule may link to several implementations, each with its own evidence and review state.

• Similarity never discards a source-specific candidate.
• Cluster version is reproducible and reviewable.
• High-frequency copybook logic is not automatically one business rule.

Snapshot seal, toolchain promotion, review decision and approved export need strong transactions. Extraction artifacts are immutable and eventually published. A run key is snapshot manifest + toolchain bundle + configuration hash; task keys include pass and partition. At-least-once workers write by content hash, so duplicates converge. Queues separate ingest, parse, graph fixpoint, render and reprocessing. Backpressure is measured in estimated CPU, graph fan-out and model tokens. High-risk incremental changes preempt full historical reclustering, and reviewer queues are capacity-aware.

Silent skip is unacceptable. Unknown or unsupported syntax creates an opaque IR node with severity and reachability. Coverage dashboards report parsed lines, opaque reachable nodes, unresolved calls and outcome slices affected. Canary corpora contain rare opcodes and hand-labeled programs; a parser bundle cannot promote if coverage regresses. If discovered after publication, quarantine affected candidates via lineage from the parser version and opaque node, roll back the review projection, fix the parser, and run only impacted partitions plus dependent summaries. Existing reviewer decisions remain, but are visibly invalidated pending re-review.

A reviewer claims a leased item from a stratified queue based on system, domain, risk and expertise. The page shows canonical rule, source beside it, highlighted evidence, call/include breadcrumb, correlated variants and official-document links. The reviewer can approve, edit wording, split, merge-as-family, reject-as-plumbing or flag analyzer defect. Decision reasons are structured. Optimistic versioning prevents two reviewers overwriting. Leads adjudicate disagreements. Search facets include system, program, rule family, outcome, review state, analyzer version and coverage risk.

• Capture time-on-item and evidence opened, but do not turn them into employee surveillance.
• Edits preserve original machine rendering and diff.
• Random duplicate review estimates inter-rater agreement.

Candidate identity should be stable under wording and line movement, but logic normalization can legitimately change. Before promotion, run old and new toolchains on a representative frozen corpus and build an equivalence map: unchanged, wording-only, evidence-expanded, logic-changed, added, removed. A 30% unexplained churn breaches the gate. Keep both projections queryable, show reviewers impact, and require lead approval. Review decisions transfer only when normalized logic and evidence equivalence pass a deterministic policy; otherwise schedule re-review. Never overwrite the old run.

Treat source as highly sensitive. Separate estates by tenant in object keys, metadata, search, encryption keys and worker credentials. Processing sandboxes have no outbound network; source never enters general logs. Model rendering receives only the minimal symbolic slice, uses an approved deployment, and is disabled for restricted tenants. Fine-grained roles separate source read, rule review, toolchain administration and export. Every evidence view and bulk export is audited. Signed snapshot manifests and append-only decision history support chain of custody.

• Prompt injection in comments or literals is data, not instruction.
• Redact secrets before any external inference.
• Break-glass reads expire and require reason.

Operationally: snapshot completeness, parse coverage, unresolved-call reachability, pass latency, task retry, lineage integrity, review queue age and search SLO. Quality: rule precision/recall on a golden corpus, sentence-evidence coverage, provenance open success, inter-rater agreement, reviewer edit distance, candidate churn between toolchains, cluster purity and reproducibility hash match. Roll out one claims domain, hand-label a corpus, run dual extractor versions, open reviewer-only search, then expand languages and systems. Approved downstream export starts only after sampling and reproducibility gates pass.

• Report false negatives by outcome slice, not only discovered candidates.
• Quarterly drift checks for the rendering model.
• Every published batch has a signed quality report.

A common IR enables correlation but must preserve language-specific opaque detail. More context can improve prose but harms determinism and leakage; bounded symbolic slices are safer. Deduplication reduces review load but merging destroys system-specific evidence. The biggest traps are one-pass prompting, line URLs to mutable source, hiding unsupported constructs, treating readable prose as the canonical rule, and reporting precision only on the easy parsed subset.

RuleMine seals an immutable source manifest and runs a compiler-like, versioned series of preprocessing, IR, control/data-flow, slicing and candidate passes. Every derived node carries origin spans and pass lineage. A bounded model may render the symbolic candidate, but a verifier requires sentence-level evidence. Similar candidates form reviewable clusters rather than being erased. SMEs work from leased, stratified queues with side-by-side source and append-only decisions. Content-addressed tasks make retries reproducible; coverage gaps, toolchain differentials and conservative rollout keep modernization evidence trustworthy.