The Citation Is the Product

The biggest risk is an unsupported medical claim. Every factual sentence must link to the exact page and highlighted area it came from. That link must not change when a source file is updated later.

I will keep the AI’s first draft and every human edit as separate saved versions. This lets us measure the AI honestly. A human makes the final decision. If the system cannot find enough evidence, it stops and asks for help. It never invents a source or silently changes an approved answer.

The main users are medical writers, evidence-search specialists, reviewers, final approvers, administrators, and auditors. The basic flow is simple: receive a question, search only approved PDFs, draft an answer, check each factual sentence, collect human approvals, export the final document, and keep the full history.

For this mock, edits should feel fast: 95 out of 100 edits finish within 300 milliseconds. Evidence search should finish within 3 seconds. A first draft may take up to 10 minutes because quality matters more than instant output. We keep records for seven years and never mix one customer’s data with another’s. We are not diagnosing patients, selecting treatment, replacing the customer’s document system, or training a new foundation model.

That is 600,000 pages a day, 6.94 pages a second on average, and about 83 pages a second at peak. OCR means optical character recognition: turning scanned page images into searchable text. At two CPU-seconds per page, peak OCR and layout work needs about 167 processor cores. I add 30% spare capacity, bringing the target to about 217 cores. This tells me document processing belongs on a background queue that can add workers during imports.

Raw sources are about 60 GB a day or 153 TB over seven years before backup copies. If a saved decision package averages 3 MB per response, that adds 6 GB a day or 15.3 TB. Search vectors can be rebuilt. Original files, page locations, draft versions, and approvals cannot. Storage retention and page processing shape this design more than ordinary API traffic.

I need four main records. First, a source file version identified by its fingerprint. Second, an evidence span that points to a page and highlighted area. Third, a response version. Fourth, an approval decision. A sentence links to an evidence-span ID, never to “the latest PDF.”

We create a new response version for the AI draft, every important human edit, and the approved answer. We never overwrite the old one. An update includes the version the editor saw. If someone else changed it first, we return a conflict instead of losing work. A retry key makes repeated requests safe.

I would draw four big parts. One, a web app for writers and reviewers. Two, a database for requests, versions, permissions, and approvals. Three, encrypted file storage for PDFs and saved drafts. Four, background workers for PDF reading, search, drafting, and sentence checks.

Here is one request. A writer submits a question. The system locks it to one approved source-library version. Search finds useful page sections. The drafting worker sees only those sections. The checker tests every factual sentence against an exact highlighted area. Unsupported text goes back to the writer. Humans edit and approve the answer. Finally, export sends the approved version to the customer’s document system and saves the receipt.

I split a sentence into small claims that can each be checked. For example, “Drug A lowered blood pressure by 20% in adults over 65” contains the treatment, the amount, and the age group. Each part must point to exact highlighted lines. If the source supports the 20% result but not the age group, the full sentence cannot pass.

The visible page image is the final reference. Text already stored inside a PDF can help us search, but it may be hidden or disagree with the scan, so we compare it with OCR text and page position. The checker labels each claim SUPPORTED, PARTIAL, CONFLICTED, or UNSUPPORTED and saves the reason. Only fully supported claims move to human approval.

We save the untouched AI draft before a person edits it. Otherwise we might give the model credit for a human correction. We measure how many required facts it found, how many claims lacked support, whether citations were correct, and how serious any omissions were. The F2 score gives recall four times the weight of precision because missing a safety fact may be worse than adding a little extra supported detail.

Our trusted test set covers different medical areas, question types, difficult documents, and known failures. Two experts review it, and a third settles disagreements. A large language model (LLM) may help grade more drafts only after we compare it with those experts and keep checking where it disagrees. Human edits are useful clues, not automatic truth; some edits reflect style rather than correctness.

I do not delete or mutate history. Approved source-library version R17 marks the source retracted with effective time and reason. New authoring excludes it. A reverse citation index finds every claim and approved response locked to the old source file fingerprint. The impact job is retry-safe on (retraction_id,response_version_id) and opens risk-ranked remediation cases.

The original response remains reproducible and visibly “superseded/source retracted.” High-severity cases pause distribution and notify owners. Lower severity enters review. Corrected responses create new versions and preserve who approved the change. If the index is stale, the main citation table drives a slower complete scan. Retraction propagation has its own service target and completeness cross-checking.

No. A better overall score cannot hide a safety regression. Oncology stays on the old model. I freeze the wider rollout, send the disputed examples to clinicians who do not know which model produced them, and separately test whether search missed evidence or the writer model omitted it. I also check whether the automated grader shares the same blind spot.

I test the new model only on a small, validated, low-risk group and run it invisibly elsewhere for comparison. Recovery is a one-step switch back to the old model. Existing responses stay tied to the exact model and grader that produced them.

Always reading the newest confirmed value applies to approved source library publication, workflow transitions, approvals, authorization, and export intent. Search/index freshness is allowed to be briefly stale and always displays the indexed approved source library digest. Workers consume at least once. Stage keys combine customer, write-once input digest, stage version, and parameters, so retries reuse an output or create one result. A database-backed event queue prevents state/event gaps. External export resolves timeouts with receiver lookup rather than blind retry.

Admission control and fair queues for each customer separate live authoring from backfill/evaluation. Broken or malicious PDFs have bounded resource sandboxes and failed-job queue. Access policy checks customer, therapeutic area, purpose, stage, and break-glass. Per-tenant separate encryption keys and region-pinned storage isolate data. Model calls use approved model endpoints that do not store prompts or in-VPC models. Observe queue age, stage latency/error, unsupported claims, citation coverage, reviewer override, F2 by slice, judge drift, export cross-checking, audit completeness, and access anomalies. Multi-region metadata uses warm standby. Write-once objects are cross-region replicated and restore is rehearsed.

First shadow retrieval against historical requests. Then give writers evidence search only. Then draft in reviewer-only mode. Then canary a narrow therapeutic area with dual human scoring. Then permit assisted authoring while final approval remains human. Every phase has rollback, frozen baselines, and predefined quality/latency/cost gates. We validate the actual signed release package in a clean environment, not a developer checkout.

The central trade-off is throughput versus defensible completeness. Small claims that can each be checked, page crops, write-once snapshots, and staged approval cost storage and reviewer time, but enable safe iteration and honest evaluation. I would not promise “fully autonomous authoring”. I would promise measured assistance with explicit stopping when unsure and traceable human authority.

I’ll summarize the source of truth, safety boundary, scale path, and rollout.