RAD: Retrieval-Augmented Code Debugging AI That Reproduces Bugs, Not Just Explains Them

Software teams don’t need more eloquent explanations of bugs. They need reproducible failures and verifiable fixes. This article outlines how to build RAD—Retrieval-Augmented Debugging—an AI system that treats bug reproduction as a first-class goal. RAD doesn’t just read your issue and guess: it retrieves the relevant logs, traces, failing test artifacts, and git history, constructs a deterministic reproduction environment, proposes a minimal patch, and proves it works by running the same failing sequence to green.

We’ll cover architecture, data contracts, retrieval strategies, agent planning, CI hooks, privacy controls, and rigorous evaluation methodology. A good RAD system is opinionated: it prefers reproducibility over verbosity, provenance over speculation, and verifiable change over rhetorical confidence.

Executive Summary

• Why: Most LLM-based code assistants produce plausible patches without reproducing the underlying failure, leading to fragile fixes, regressions, and false confidence.
• What: RAD is a retrieval-augmented code debugging pipeline that assembles a structured "Bug Evidence Bundle" from logs, traces, failing tests, and git metadata; generates an executable reproduction plan; proposes a patch; and verifies it in CI.
• How: Combine multi-modal retrieval (symbolic, lexical, vector), provenance-rich data contracts, containerized reproduction harnesses, constrained action plans, and privacy-by-design ingestion.
• Results: Measured by reproduction rate (RR), fix-correctness rate (FCR), and end-to-end time-to-green (TTG) against established datasets (e.g., Defects4J, BugSwarm, SWE-bench) and real incidents.

Why Explanations Are Not Enough

Large language models can write coherent narratives about root causes. But explanation without reproduction is a trap. Teams need:

• A deterministic way to recreate the failing behavior.
• A patch validated by the same signals that triggered the incident (tests, logs, traces).
• Provenance for every piece of evidence used to make the change.

Without those, you’ll accumulate “healing illusions”: patches that silence the symptom locally while the underlying cause persists. RAD is designed to prevent this via a strict reproduce-then-fix loop with automated gating in CI.

RAD: Retrieval-Augmented Debugging

RAD borrows from Retrieval-Augmented Generation (RAG) but scopes it to debugging:

• Retrieval targets code, build configs, issues, logs, distributed traces, crash dumps, failing test artifacts, and time-scoped git history.
• Generation outputs a structured, executable reproduction plan and a minimal code diff, not prose.
• Verification runs the plan against a hermetic environment and attaches evidence.

The system’s core object is the Bug Evidence Bundle (BEB)—a reproducibility-first artifact that gathers everything the agent needs to deterministically re-create the bug.

System Architecture Overview

1. Ingestion and Indexing

   • Collect logs, traces (OpenTelemetry), failing test artifacts, crash reports, core dumps, and git metadata.
   • Normalize into a BEB data contract with provenance and privacy controls.

2. Retrieval

   • Multi-modal search: lexical (BM25), semantic (code/document embeddings), symbolic (AST/symbol index), and temporal (commit windowing).
   • Query fusion to build a compact, relevant context set.

3. Planning

   • LLM generates a Reproduction Plan DSL with explicit steps: environment, dependencies, commands, data inputs, and expected failure signature.
   • Constrained decoding to valid DSL; static linting.

4. Reproduction Harness

   • Containers or Nix/Bazel for hermetic builds; Testcontainers for services.
   • Run plan; capture logs, traces, test outputs; detect flakiness.

5. Patch Generation

   • Generate minimal diffs rooted in the reproduction; guard with static analysis and tests.

6. Verification and CI Hook

   • Re-run reproduction; must fail pre-patch and pass post-patch with matching signature.
   • Open PR with attached BEB, plan, and evidence.

7. Governance

   • Privacy/PII sanitization, RBAC, audit logs, time-bounded retention.

8. Evaluation

   • Offline datasets (Defects4J, BugSwarm, SWE-bench/BugsInPy), online metrics in CI, canary rollouts.

The Bug Evidence Bundle (BEB): A Data Contract

A BEB is the heartbeat of RAD. It is versioned, privacy-reviewed, and designed to be executable.

• Required fields: repo, commit SHA, environment fingerprint, failing test(s) or command, error signature, time window, provenance.
• Optional fields: selected log windows, OTel trace exemplars, request bodies (redacted), crash dumps, feature flags, seed, and git blame windows.

Example JSON Schema (truncated):

json
{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://schemas.example.com/rad/beb.schema.json",
  "title": "Bug Evidence Bundle",
  "type": "object",
  "required": ["id", "repo", "commit", "environment", "signals", "provenance"],
  "properties": {
    "id": {"type": "string", "pattern": "^beb_[a-z0-9]{8}$"},
    "repo": {"type": "string"},
    "commit": {"type": "string", "pattern": "^[a-f0-9]{7,40}$"},
    "branch": {"type": "string"},
    "time_window": {
      "type": "object",
      "properties": {
        "start": {"type": "string", "format": "date-time"},
        "end": {"type": "string", "format": "date-time"}
      }
    },
    "environment": {
      "type": "object",
      "properties": {
        "os": {"type": "string"},
        "arch": {"type": "string"},
        "container": {"type": "string"},
        "toolchain": {"type": "object"},
        "feature_flags": {"type": "object"}
      },
      "required": ["os", "arch"]
    },
    "signals": {
      "type": "object",
      "properties": {
        "tests": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "name": {"type": "string"},
              "command": {"type": "string"},
              "stderr": {"type": "string"},
              "stdout": {"type": "string"},
              "exit_code": {"type": "integer"}
            },
            "required": ["name", "command", "exit_code"]
          }
        },
        "logs": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "source": {"type": "string"},
              "level": {"type": "string"},
              "message": {"type": "string"},
              "timestamp": {"type": "string", "format": "date-time"},
              "request_id": {"type": "string"}
            },
            "required": ["message", "timestamp"]
          }
        },
        "traces": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "trace_id": {"type": "string"},
              "span_id": {"type": "string"},
              "name": {"type": "string"},
              "attributes": {"type": "object"}
            },
            "required": ["trace_id", "span_id", "name"]
          }
        },
        "crash_dumps": {"type": "array", "items": {"type": "string", "contentEncoding": "base64"}}
      }
    },
    "inputs": {
      "type": "object",
      "properties": {
        "http": {
          "type": "array",
          "items": {
            "type": "object",
            "properties": {
              "method": {"type": "string"},
              "path": {"type": "string"},
              "headers": {"type": "object"},
              "body": {"type": "string"},
              "redaction": {"type": "array", "items": {"type": "string"}}
            }
          }
        }
      }
    },
    "provenance": {
      "type": "object",
      "properties": {
        "collector": {"type": "string"},
        "collected_at": {"type": "string", "format": "date-time"},
        "pii_sanitized": {"type": "boolean"}
      },
      "required": ["collector", "collected_at", "pii_sanitized"]
    }
  }
}

This schema ensures every BEB is reusable and auditable.

Ingestion and Indexing: Make Retrieval Boring and Reliable

RAD’s retrieval layer is only as good as your ingestion and indexing. Recommended practices:

• Logs and traces: Use OpenTelemetry (OTel) for normalized schemas. Export structured logs (JSON) with correlation IDs. Store traces with exemplars for error spans.
• Test artifacts: Persist failing junit/xml reports, jest outputs, pytest logs, crash dumps, and core files. Keep stderr/stdout and the exact command/flags.
• Git metadata: Record commit SHAs, blame ranges, diff hunks, and semantic symbols via a parser (tree-sitter) to build a symbol graph.
• Environment fingerprint: Container image digest, toolchain versions, OS, arch, env vars, feature flags, seeds.
• Redaction: Apply secret scanning and PII scrubbing at ingest; annotate what was removed so the agent knows what is missing.

Indexing strategy:

• Vector indexes for semantic retrieval (FAISS, pgvector, Weaviate, Pinecone) using code-aware embeddings (e.g., CodeBERT, E5-code-like models).
• Symbolic index keyed by fully qualified names, filenames, and AST nodes; support call graph edges and import graphs.
• Lexical BM25 (e.g., Elasticsearch, OpenSearch) over logs and test outputs.
• Time-travel indexing across git: snapshot code embeddings by commit or store deltas; filter candidates by the BEB’s commit window.

Retrieval: Multi-Modal, Time-Scoped, and Provenance-First

The retrieval stage fuses signals:

• Temporal filter: Restrict code candidates to [commit_parent, commit] and the last N relevant commits in the same module.
• Symbolic seeds: Start from frames in the stack trace and test names; walk call graph to include neighbors.
• Semantic expansion: Embed failing log lines and trace names to retrieve candidate code chunks and docs with high cosine similarity.
• Fusion: Rank by reciprocal rank fusion (RRF) across lexical, semantic, and symbolic scores.

Pseudo-implementation:

python
from typing import List, Dict

class Retriever:
    def __init__(self, vector, bm25, symbols):
        self.v = vector
        self.b = bm25
        self.s = symbols

    def retrieve(self, beb: Dict, k=40) -> List[Dict]:
        failing_msgs = [t.get("stderr", "") + "\n" + t.get("stdout", "") for t in beb["signals"].get("tests", [])]
        log_msgs = [l["message"] for l in beb["signals"].get("logs", [])][:100]
        seeds = failing_msgs + log_msgs
        sem_hits = self.v.search(seeds, k=3)
        lex_hits = self.b.search(" ".join(seeds), k=100)
        sym_hits = self.s.expand_from_stack(beb.get("signals", {}), depth=2)

        return reciprocal_rank_fuse([sem_hits, lex_hits, sym_hits], top_k=k)

Planning: A Reproduction Plan DSL

Plain English instructions are ambiguous. RAD uses a constrained Reproduction Plan DSL that is machine-checkable and executable.

Example DSL (YAML):

yaml
version: 1
metadata:
  beb_id: beb_1a2b3c4d
  repo: git@github.com:example/service.git
  commit: 4f3c2b1
  env: { os: linux, arch: x86_64, container: ghcr.io/org/svc:1.2.3 }
setup:
  - run: docker pull ghcr.io/org/svc:1.2.3
  - run: docker run --rm -d --name svc -p 8080:8080 ghcr.io/org/svc:1.2.3
  - wait_for_http: { url: http://localhost:8080/healthz, timeout_s: 30 }
inputs:
  - http_request:
      method: POST
      url: http://localhost:8080/api/v1/normalize
      headers: { Content-Type: application/json }
      body: |
        {"name": "José", "limit": 10}
      expect:
        status: 500
        body_regex: "UnicodeDecodeError|Normalization"
verifications:
  - grep_log: { container: svc, regex: "Normalization failed", since_s: 60 }
  - assert_trace: { name: "normalizer.normalize", has_error: true }

The LLM outputs only this DSL via constrained decoding. A linter rejects dangerous steps (e.g., network egress) and ensures determinism (seed present for tests, pinned container digests).

Constrained generation call:

python
def generate_plan(llm, beb, retrieved):
    system = "You produce only YAML conforming to the Reproduction Plan DSL v1. No prose."
    prompt = {
        "role": "user",
        "content": f"Build a plan for BEB {beb['id']} using signals and retrieved context."
    }
    schema = load_yaml_schema("repro_plan_v1.yaml")
    return llm.generate_constrained([system, prompt], schema=schema)

Reproduction Harness: Hermetic by Default

• Containerization: Docker/Podman with pinned digests; or Nix/Bazel for hermetic builds.
• Service dependencies: Testcontainers (Postgres, Redis) with known images; data seeded from BEB.
• Determinism: Set LANG, TZ, LC_ALL; disable randomness or fix seeds; record CPU features.
• Flakiness detection: Rerun plan 3 times; if outcomes differ, classify as flaky and trace for non-deterministic sources.
• Evidence capture: Logs, traces, HTTP captures, exit codes, and container filesystem diffs.

A minimal runner:

python
import subprocess, json, time

class Runner:
    def __init__(self, plan):
        self.plan = plan

    def run(self):
        pre = self._execute_inputs(expect_failure=True)
        if not pre["failed_as_expected"]:
            return {"status": "not_reproduced", "details": pre}
        # Apply patch later; for now, return reproduction evidence
        return {"status": "reproduced", "evidence": pre}

    def _execute_inputs(self, expect_failure):
        results = []
        failed = False
        for step in self.plan["inputs"]:
            if "http_request" in step:
                r = self._curl(step["http_request"])
                results.append(r)
                if expect_failure and r["status"] >= 500:
                    failed = True
        return {"failed_as_expected": failed, "steps": results}

    def _curl(self, req):
        import requests
        resp = requests.request(req["method"], req["url"], headers=req.get("headers", {}), data=req.get("body"))
        return {"status": resp.status_code, "body": resp.text[:2048]}

Patch Generation: Minimal, Targeted, and Verified

Patch generation must be grounded in the reproduction context. Strategies:

• Identify candidate files via retrieval, stack frames, and blame windows.
• Generate diffs, not full files; keep changes minimal.
• Enforce static checks: type errors, linters, and semantic rules (e.g., Semgrep policies).
• Validate with the same plan: must fail pre-patch and pass post-patch with a matching signature.

Patch synthesis prompt inputs include: failing signature, reproduction inputs, top-N retrieved snippets, and a request to output a unified diff only. You can enforce a structured diff schema and apply it with git apply.

Example minimization loop:

python
def synthesize_and_verify(llm, plan, beb, repo):
    for attempt in range(3):
        diff = llm.generate_diff(beb, plan)
        if not apply_diff(repo, diff):
            continue
        pre = run_plan(plan, repo, pre_patch=True)
        post = run_plan(plan, repo, pre_patch=False)
        if pre.failed and post.passed and signatures_match(pre, post):
            return diff
        revert_diff(repo)
    return None

Git and History-Aware Retrieval

Many bugs are tied to recent diffs. Enhance retrieval with:

• Blame windowing: For failing symbols, fetch latest N commits and their diffs.
• Change coupling: Identify files frequently changed together; surface them as candidates.
• Commit message priors: Commits referencing the failing area or error message terms.
• Rollback hints: If failure correlates with feature flags flipped in the BEB, include flag definitions.

CI Integration: From Reproduction to Pull Request

Wire RAD into CI/CD to make it part of the dev workflow. Example GitHub Actions excerpt:

yaml
name: rad-debug
on:
  workflow_dispatch:
  issue_comment:
    types: [created]
  pull_request:
    types: [opened, synchronize]

jobs:
  rad:
    if: contains(github.event.comment.body, '/rad') || github.event_name == 'workflow_dispatch'
    runs-on: ubuntu-latest

    permissions:
      contents: write
      pull-requests: write

    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'

      - name: Install RAD toolchain
        run: pip install rad-cli

      - name: Ingest failure and build BEB
        run: rad beb build --from-ci --out beb.json

      - name: Retrieve, plan, reproduce
        run: rad run --beb beb.json --out evidence.json

      - name: Synthesize patch
        run: rad patch --beb beb.json --evidence evidence.json --out patch.diff

      - name: Verify
        run: rad verify --plan evidence.json --diff patch.diff --report verify.json

      - name: Open PR
        if: success()
        run: rad pr open --beb beb.json --diff patch.diff --report verify.json

Policies:

• RAD never commits directly to main. It opens a PR with a full evidence pack.
• The PR description includes: BEB metadata, reproduction plan DSL, pre/post logs, and a verification report.
• Require passing verification checks and human review before merge.

Privacy, Security, and Governance

Debugging data often contains sensitive information. Build privacy in from the start.

• PII/PHI redaction: Define redaction rules in the BEB contract. Apply structured scrubbing at ingestion (names, emails, tokens, phone numbers), then semantic scrubbing for free text.
• Secrets: Do not capture secrets in BEB. Validate with secret scanners (e.g., high-entropy detectors). Maintain allowlists for necessary config keys.
• Access control: RBAC for BEBs; least privilege for RAD agents; deny egress when reproducing.
• Data retention: TTL for BEBs; short-lived trace/log retention; cryptographic deletion.
• Auditability: Every retrieval and action is logged. PR evidence includes a manifest of all retrieved items with hashes.
• Jurisdiction: Respect data localization. Keep ingestion and storage in compliant regions.

Example redaction configuration:

yaml
redaction:
  patterns:
    - name: email
      regex: "[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\\.[a-zA-Z]{2,}"
      replacement: "<email>"
    - name: credit_card
      regex: "(?:\b\d[ -]*?){13,16}\b"
      replacement: "<card>"
  fields:
    - path: inputs.http[].headers.Authorization
      action: remove
    - path: signals.logs[].message
      action: redact_patterns

Evaluation: Measure Reproduction, Not Eloquence

Evaluate RAD on reproducibility, fix quality, safety, and cost.

Datasets and corpora:

• Defects4J (Java) and BugsInPy (Python) for classic reproducible bugs.
• BugSwarm: Pairs of failing/passing CI jobs with environments and Docker images; ideal for reproduction tests.
• SWE-bench: Real-world GitHub issues mapped to codebases; use the issues tagged as bugs.

Offline metrics:

• Reproduction Rate (RR): Fraction of BEBs where the plan reliably reproduces the failure (e.g., 3/3 runs) from scratch.
• Fix-Correctness Rate (FCR): Fraction of reproduced bugs where the proposed patch passes the plan and the project’s full test suite.
• False Reproduction Rate (FRR): Cases where the plan “fails” for the wrong reason (signature mismatch).
• Time-to-Green (TTG): Time from BEB to verified patch.
• Context Efficiency (CE): Retrieved tokens vs. total tokens processed.
• Safety Incidents (SI): Privacy or policy violations detected by linters.

Online metrics (CI):

• PR Acceptance Rate: Fraction of RAD PRs merged without manual patch edits.
• Regression Rate: Post-merge failures attributable to RAD patches.
• Reviewer Effort: Time reviewers spend per RAD PR.

A minimal evaluation harness:

python
from datasets import load_rad_corpus
from rad import Agent

results = []
for beb in load_rad_corpus("bugswarm_subset"):
    agent = Agent()
    retrieval = agent.retrieve(beb)
    plan = agent.plan(beb, retrieval)
    rep = agent.reproduce(plan)
    if rep.status != "reproduced":
        results.append({"beb": beb["id"], "RR": 0})
        continue
    patch = agent.patch(beb, plan)
    ver = agent.verify(plan, patch)
    results.append({
        "beb": beb["id"],
        "RR": 1,
        "FCR": int(ver.passed),
        "FRR": int(not ver.signature_matched),
        "TTG_s": ver.time_seconds
    })

print(summary(results))

Ablations to run:

• Retrieval mode: lexical-only vs. semantic-only vs. fused.
• Temporal indexing: with vs. without commit windowing.
• Plan constraints: unconstrained free-text vs. DSL.
• Environment: hermetic vs. ad-hoc.
• Privacy guards: on vs. off (measure safety incidents and utility impact).

Worked Example: A Unicode Normalization Bug

Scenario: A Python microservice exposes POST /normalize, which normalizes strings and enforces a length limit measured in bytes. Users report a 500 error for names with accented characters like “José”.

Signals captured:

• Log: Normalization failed: UnicodeEncodeError: 'utf-8' codec can't encode character '\u0301' at position 2
• Trace: Span normalizer.normalize error=true; attribute input_len=4
• Failing test artifact: pytest::test_normalize_unicode fails with 500 status and stack trace in normalizer.py:44
• Environment: ghcr.io/org/svc:1.2.3, Python 3.11.6, LANG=C

BEB inputs include the failing HTTP body (name redacted except for pattern) and headers. Privacy rules redact Authorization and email addresses.

Retrieval finds:

• normalizer.py with a byte-length check using len(s.encode("utf-8")) but truncating the string using s[:limit] before encoding.
• A recent commit changing default locale to LANG=C in Dockerfile.

Plan DSL sends POST /normalize with {"name":"José"} and expects 500. It verifies by grepping logs for “Normalization failed” and checking the error span.

Reproduction harness spins up the container, fixes LANG to en_US.UTF-8, and confirms the 500 reproduces under LANG=C but not under en_US.UTF-8.

Patch proposal:

• Normalize using NFC before length check.
• Perform truncation in Unicode code points, not bytes; then enforce byte-limit post-encoding.
• Ensure LANG is set to C.UTF-8 in container for deterministic encoding.

Illustrative diff:

diff
--- a/src/normalizer.py
+++ b/src/normalizer.py
@@
-from typing import Optional
+from typing import Optional
+import unicodedata
@@
-def normalize_and_truncate(s: str, limit: int) -> str:
-    # Truncate then encode to check byte-length
-    truncated = s[:limit]
-    if len(truncated.encode("utf-8")) > limit:
-        raise ValueError("normalized string exceeds limit")
-    return truncated
+def normalize_and_truncate(s: str, limit: int) -> str:
+    # Normalize to NFC so combined marks compose
+    n = unicodedata.normalize("NFC", s)
+    # Truncate by code points, then ensure byte limit
+    out = n
+    while len(out.encode("utf-8")) > limit:
+        out = out[:-1]
+        if not out:
+            break
+    return out

Container fix in Dockerfile:

diff
-ENV LANG=C
+ENV LANG=C.UTF-8
+ENV LC_ALL=C.UTF-8

Verification runs the plan: pre-patch reproduces 500; post-patch returns 200 with normalized output and passes log/trace checks. The PR includes BEB, plan, before/after logs, and the diffs.

Anti-Patterns and Failure Modes

• Free-text plans: Ambiguous, insecure, and non-deterministic. Use a DSL.
• No time scoping: Pulling unrelated code versions causes ghost failures. Always filter by commit windows.
• Over-retrieval: Blowing the context window with irrelevant tokens degrades synthesis. Use fusion and hard caps.
• Skipping environment capture: Bugs often hide in deps, flags, or locales. Fingerprint the environment.
• Not verifying signatures: A green test is not sufficient; match error messages, traces, and exit codes.
• Using production secrets or raw PII: Always redact and minimize inputs.
• Direct commits: Always use PRs with evidence and human reviews.

Implementation Checklist

• Data contracts

  • Define BEB schema; version it; enforce validation.
  • Implement PII redaction and secret scanning at ingestion.
  • Capture environment fingerprints and feature flags.

• Retrieval

  • Build lexical, semantic, and symbolic indexes.
  • Implement time-travel retrieval tied to commit windows.
  • Add RRF fusion and provenance tracking.

• Planning and execution

  • Define and lint a Reproduction Plan DSL.
  • Implement hermetic runner with container orchestration and trace/log capture.
  • Add flakiness detection and retry policies.

• Patch synthesis

  • Constrained diff generation; minimal changes.
  • Static checks, type checks, and linters.
  • Verify with plan and full test suite.

• CI/CD integration

  • Actions/pipelines to build BEB, reproduce, patch, verify, and open PRs.
  • Policy gates: required checks, human reviews.

• Privacy and governance

  • RBAC, audit logs, TTL, regional storage.
  • Redaction rules tested with unit tests.

• Evaluation

  • Offline datasets and ablations.
  • Online metrics in CI; dashboards; canary rollouts.

Practical Tips and Trade-offs

• Evidence minimization: Don’t stuff all logs into the context. Extract exemplars: top error lines, nearest trace spans, and short request/response snippets.
• Chunking for code search: Chunk by syntax (AST nodes/functions) rather than fixed tokens; include imports and docstrings.
• Cost controls: Cache retrieved sets per BEB; reuse plan/runs across attempts; compress contexts.
• Deterministic builds: Prefer pinned digests, Nix flakes, or Bazel lockfiles to avoid “works on CI” discrepancies.
• Guiding patches: Ask the LLM to produce diffs with rationale in comments within the diff, not a separate essay. It keeps provenance local to the change.

Sample Privacy Linter for BEBs

python
import re

EMAIL = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")
CARD = re.compile(r"(?:\b\d[ -]*?){13,16}\b")
TOKEN = re.compile(r"(?i)authorization:|api[-_ ]?key|secret")

SENSITIVE_KEYS = {"password", "secret", "api_key", "authorization"}


def sanitize_beb(beb: dict) -> dict:
    def scrub_text(t: str) -> str:
        t = EMAIL.sub("<email>", t)
        t = CARD.sub("<card>", t)
        return t

    for log in beb.get("signals", {}).get("logs", []):
        log["message"] = scrub_text(log.get("message", ""))

    for req in beb.get("inputs", {}).get("http", []):
        headers = req.get("headers", {})
        for k in list(headers.keys()):
            if k.lower() in SENSITIVE_KEYS or TOKEN.search(k):
                headers.pop(k)
        req["body"] = scrub_text(req.get("body", ""))

    beb.setdefault("provenance", {})["pii_sanitized"] = True
    return beb

Beyond Single-Repo Bugs: Distributed Systems

For multi-service incidents, RAD extends to distributed reproduction:

• Multi-repo BEBs: Each service contributes its own evidence; a coordinator builds a composed plan with dependency graphs.
• Trace-driven orchestration: Use OTel traces to reconstruct call chains; spin up only the services on the critical path.
• Data seams: Replace external dependencies with fixtures (e.g., S3 -> MinIO) populated from sanitized samples.

References and Useful Resources

• OpenTelemetry logs and traces for structured signal collection.
• Defects4J, BugsInPy, BugSwarm for reproducible bug datasets.
• SWE-bench for real-world issue-to-fix mapping.
• tree-sitter for building an AST/symbol index.
• FAISS/pgvector/Weaviate for vector search; Elasticsearch/OpenSearch for BM25.
• Testcontainers for dependency orchestration; Nix/Bazel for hermetic builds.
• ReproZip for capturing environments; Semgrep for static guardrails.

Conclusion

RAD upgrades debugging from storytelling to science. By committing to retrieval with provenance, constrained planning, deterministic reproduction, and verifiable fixes, teams can safely enlist AI in their software maintenance loop. The payoff is measurable: higher reproduction rates, faster time-to-green, and fewer regressions—without compromising privacy or developer trust.

If you implement one principle from this article, make it this: no patch without a plan that reproduces the bug and proves the fix. The rest of the RAD stack exists to make that principle easy, fast, and reliable.