Actors
- Engineer searching and reviewing a PR
- PM or architect maintaining approved intent
- Coding agent requesting scoped context
- Tenant admin connecting repositories
- GitHub/GitLab webhook and content APIs
- Security or compliance auditor
REPOSITORY OBSERVATORY / MOCK 03
Design a multi-tenant platform that continuously maps many repositories into searchable symbols and detects when implementation stops matching approved intent.
OPENING PROMPT
Read this once, then begin aloud. Spend the first two minutes establishing contract, actors, risk and what must remain authoritative.
Design CodeScope, a multi-tenant code-intelligence and drift platform. It connects read-only GitHub and GitLab repositories, incrementally indexes code across branches, exposes symbol-aware search, links code to approved requirements and blueprints, and comments on pull requests when a change appears to violate intent. A finding must carry enough evidence for a human to reproduce and dismiss it.
The key decision is to separate authoritative artifacts and repository snapshots from derived indexes and probabilistic drift findings. Publication and permissions are strongly controlled; search and drift are rebuildable, version-labeled projections.
REQUIREMENTS
These are supplied mock numbers for sizing and follow-ups. Say your assumptions before using them.
HIGH-LEVEL DESIGN
Name the authoritative state, derived projections, asynchronous boundaries and the exact point where a business decision becomes durable.
STORAGE + ACCESS
The table is logical, not a mandate for one database. Choose physical stores after access patterns, transactions, retention and rebuildability are clear.
| Entity | Primary / unique key | Important fields and purpose |
|---|---|---|
| Repository |
(tenant_id, repo_id)
|
provider_installation, encrypted_locator, default_branch, permission_epoch, state |
| CommitSnapshot |
(repo_id, commit_sha)
|
parent_shas, observed_at, manifest_hash, index_state |
| FileVersion |
(repo_id, commit_sha, path_hash)
|
blob_sha, language, parser_version, content_acl |
| Symbol |
(repo_id, commit_sha, symbol_id)
|
qualified_name, kind, span, signature_hash |
| SymbolEdge |
(repo_id, commit_sha, from_id, type, to_id)
|
calls/imports/implements/reads; extractor_version |
| ArtifactRevision |
(tenant_id, artifact_id, revision)
|
approved_body_hash, schema_version, approval_state |
| TraceLink |
(artifact_revision, symbol_id, commit_sha)
|
link_type, creator, confidence, evidence |
| DriftFinding |
(tenant_id, finding_id)
|
analysis_run, rule/model versions, evidence spans, state, reviewer |
POST /v1/repositories
Connect selected repo; idempotency-key required
202 {repoId,indexRunId}POST /v1/webhooks/{provider}
Verify signature; persist raw envelope before ack
202GET /v1/search?q=&repo=&commit=&cursor=
Hybrid symbol/code search pinned to permission epoch
200 {hits,nextCursor,indexedThrough}POST /v1/trace-links
Propose requirement-revision ↔ symbol-at-commit link
201 / 409 stale revisionPOST /v1/pulls/{id}/analyze
Deduped by repo + head SHA + policy bundle
202 {analysisRunId}POST /v1/findings/{id}/reviews
Confirm, dismiss, suppress, with reason and etag
200 {state,version}00:00 → 30:00
Answers are written in a speakable first-person style. Turn on Practice Mode to hide them, answer aloud, then reveal one at a time.
Candidate · clear top-down answerI would start by defining what we can trust. GitHub or GitLab owns the code. Approved requirements own the intent. Our system builds fast search and drift warnings from those two sources.
Why this mattersThis gives the interviewer a map before any boxes appear, and it makes the system's most important trust decision explicit.
Candidate · clear top-down answerFor version one, teams connect repositories, search code, link requirements to code, and see evidence-backed warnings on pull requests. We will not write code or block merges yet. That keeps the first release useful and safe.
Why this mattersA narrow first release keeps the design realistic. It also shows which tempting features can wait without weakening the core product.
Candidate · clear top-down answerThe average traffic is small, but pushes arrive in bursts and the first import is huge. For 80,000 repositories with one million files each, the first scan can cover tens of billions of files, while a normal push may change only a few dozen. I would keep imports in a slower queue and reserve workers for live pull requests. The math tells us scheduling and reuse matter more than raw parser speed.
Why this mattersThe calculation is not about perfect forecasting. It identifies the part most likely to run out of capacity or delay users.
Candidate · clear top-down answerThe main records are repository, commit, file, symbol, requirement version, link, and finding. We never change an old commit or approved requirement. A branch pointer simply tells us which completed commit is current. The common reads are search by tenant and commit, symbols by file, links from one requirement version, and findings for one pull request. Those access paths drive the indexes.
Why this mattersA small set of records is easier to reason about. Exact versions keep later changes from silently rewriting an earlier decision.
Candidate · clear top-down answerWe need simple endpoints to connect a repository, start indexing, search a chosen commit, request pull-request analysis, and review a finding. Long jobs return an operation ID so the client can check progress instead of holding one HTTP request open.
Why this mattersThe API follows the user journey and makes slow work visible. A client should never hold one request open for a long job.
Candidate · clear top-down answerA signed webhook says a push happened. We save its delivery ID before acknowledging it, then ask the code host for the real current commit. We fetch only changed files, update the code map, run drift checks, and publish the completed search view with one pointer switch. Users either see the old complete view or the new complete view, never a half-built mix.
Why this mattersTracing one real request proves the boxes connect. It also exposes where we acknowledge success and where a retry may begin.
Candidate · clear top-down answerTo stay fast, we reuse files whose content hash did not change. To stay correct, we also invalidate callers when a public symbol or build setting changes. We build a new index for each commit and switch readers to it only when complete. If a parser cannot prove that reuse is safe, it rebuilds the wider dependency group. That spends more compute to avoid a silently wrong index.
Why this mattersThe deep dive starts only after the basic path is clear. Use one concrete example, then state the speed-versus-correctness choice.
Candidate · clear top-down answerA drift warning means code may no longer match an approved requirement. Cheap exact checks run first. An AI model may help with unclear meaning, but it must point to exact requirement text and changed code lines. A person still decides.
Why this mattersEvidence must be visible to a reviewer. AI may help rank or explain, but it does not become the final authority.
Candidate · clear top-down answerA few small records need immediate agreement, such as access and review decisions. Search can be a little behind, but every response shows which commit it covers. Repeated webhooks and jobs are safe because each has a stable key.
Why this mattersUse immediate agreement only for decisions that cannot be ambiguous. Let rebuildable views catch up and always show their freshness.
Candidate · clear top-down answerI never trust webhook arrival order. I ask the code host for the current branch head and build from commit IDs. After a force-push, old data can remain for audit, but users only see the new branch view.
Why this mattersA failure answer should name the uncertain state first. Then explain how the system learns the truth before it repeats an action.
Candidate · clear top-down answerLarge repositories get smaller chunks and cost-based scheduling. Live pull requests get reserved capacity, while imports and full rebuilds use a separate pool with per-tenant limits. If ten pushes arrive quickly, we may skip intermediate branch views and build the newest commit, while still keeping every commit addressable. Queue age, not queue length alone, triggers scaling and admission limits.
Why this mattersBusy tenants and large jobs need limits. Reserved capacity keeps one customer or one import from delaying everyone else's live work.
Candidate · clear top-down answerRevoking access must hide the repository at once. We raise an access version number, stop new fetches, cancel queued work, and reject any worker trying to publish results created with the old access version.
Why this mattersRemoving access must affect reads and in-flight workers. Cached data is never a reason to keep showing revoked information.
Candidate · clear top-down answerEvery record, object path, queue message, index document, and cache key includes the customer ID. Workers use short-lived repository tokens limited to one job. Parsers run in isolated sandboxes with no open network access. Search checks permission before retrieval and again before returning results, so a stale index filter cannot leak revoked code. Logs contain IDs and timings, not source text.
Why this mattersSecurity belongs in keys, tokens, workers, and logs. It is not a single box added to the diagram at the end.
Candidate · clear top-down answerI measure how long a push takes to become searchable, queue age, search speed, access-removal time, and drift accuracy. I would launch read-only search first, then advisory warnings, then optional merge checks only after teams trust the results.
Why this mattersMetrics should cover user speed, system safety, and result quality. Rollout starts read-only and earns stronger automation with evidence.
Candidate · clear top-down answerThe main trade-off is speed versus certainty. Reusing old work is fast, but uncertain dependency changes require a wider rebuild. Hybrid exact plus semantic search is better than vectors alone because code identifiers and evidence matter.
Why this mattersEvery design choice has a cost. Name the simpler alternative and explain which requirement makes the added complexity worth it.
Candidate · clear top-down answerIn short: save commit snapshots that never change, reuse unchanged files, publish complete search views, and show freshness everywhere. Drift starts with exact evidence, uses AI only for unclear cases, and stays advisory until measured quality supports stronger action.
Why this mattersThe recap should repeat the user flow, the main trust rule, the hardest failure, and the most important trade-off—not every component.
DOMAIN DEPTH
These mechanisms are the interview’s differentiators. Be able to redraw each from memory and defend its failure behavior.
A branch name is mutable; a commit is not. Build immutable commit segments, then publish one small branch pointer only after the required segment set is complete.
Move from high-precision deterministic facts toward probabilistic semantic judgments, preserving the input manifest at every rung.
FAILURE MATRIX
When the interviewer injects a fault, restate the violated assumption, change state or protocol, and name the new invariant.
| Injection | Recovery mechanism | Tempting wrong answer |
|---|---|---|
| 01Webhook replay/out of order | Provider delivery dedupe + resolve canonical DAG | Duplicate business effect |
| 02Force-push | New projection generation; keep orphan snapshots per retention | Old head shown as current |
| 03Parser regression | Versioned namespace, canary, rollback projection | Corrupting only good index |
| 04Provider rate limit | Token bucket, jitter, live/backfill lane isolation | Retry storm |
| 05Repo revoked | Permission epoch fence, synchronous hide, async purge | Data survives in vector/cache |
| 06Model unavailable | Serve deterministic checks; mark semantic analysis pending | Blocking PR workflow |
BOTEC
Use orders of magnitude to expose the bottleneck. State what this simple model omits.
INTERACTIVE SCENARIO
Change mock workload and see whether the live indexing lane can meet the five-minute target.
OPERATIONS + DELIVERY
A system is incomplete without observable user outcomes, staged deployment, rollback and an answer to “how will we know?”
INTERVIEWER TRAPS
Use these as flash cards. The right column is the compact sentence you want available under pressure.
| Trap | Better move |
|---|---|
| Calling a branch authoritative | Pin snapshots and links to commit SHA. |
| One giant mutable index | Publish immutable versioned segments. |
| Vector search for identifiers | Hybrid lexical, symbol and semantic retrieval. |
| Trusting webhook order | Reconcile against the host commit graph. |
| “Exactly once” hand-wave | At-least-once transport + idempotent effects. |
| No revocation story | Permission epochs plus complete derived-data purge. |
DEFINITIONS
Define the term, then connect it to a concrete invariant in this design. Avoid dropping vocabulary as a substitute for reasoning.
FINAL MINUTE
CodeScope stores immutable repository snapshots and approved artifact revisions, then builds versioned, tenant-fenced search and graph projections. Signed webhooks feed an at-least-once, cost-aware pipeline; content hashes and commit keys make it idempotent. Search always reports commit and freshness. Drift begins with deterministic evidence, uses models only as a reproducible classifier, and keeps humans authoritative. Permission epochs stop revoked data immediately; reconciliations repair missed events. I would ship search, shadow drift, dashboard findings, then carefully calibrated PR comments, measuring both SLOs and reviewer trust.
Rehearse again