# ARS v3.6.7 — Downstream Agent Pattern Protection + Multi-File Audit Pipeline

**Status:** Design — spec input from v3.6.5/v3.6.6 field experience
**Date:** 2026-04-29
**Target release:** v3.6.7+
**Scope:** Pipeline-wide hallucination/drift protection across `synthesis_agent`, `research_architect_agent` (survey designer mode), `report_compiler_agent` (abstract-only mode), plus orchestration upgrades for multi-file audit and cross-section coherence.

---

## 1. Background

A v3.6.5/v3.6.6 production run (an academic chapter using the full Phase 1 → Phase 2 → Phase 3 pipeline) was audited end-to-end with cross-model review (gpt-5.5 xhigh) across all downstream deliverables. Three audit rounds on Phase 2 deliverables and two audit rounds on the Phase 3 abstract surfaced **17 distinct hallucination/drift patterns** across the three downstream agents, on top of the upstream `bibliography_agent` patterns already documented in earlier feedback.

The lesson is that ARS pipeline ship-quality is bounded by the weakest downstream agent. Even when `bibliography_agent` reaches V2_CLEAN, downstream agents introduce new hallucinations during their own narrative-construction, instrument-design, and abstract-compression work. Each downstream agent needs its own pattern-protection layer plus a corresponding audit hook.

---

## 2. Goal & Non-Goals

### 2.1 Goal

Codify 17 downstream-agent hallucination/drift patterns into ARS spec, build prompt-level protection into each downstream agent, and integrate cross-model audit hooks into standard pipeline orchestration. Target outcome: end-to-end deliverable set reaches "0 P1+P2 finding under independent xhigh audit" within 3 audit rounds.

### 2.2 Non-Goals (out of v3.6.7 scope)

- No new agent. Patterns attach to existing agents.
- No schema change. `literature_corpus_entry.schema.json` and Schema 9 fields remain v3.6.4/v3.6.5 shape.
- No bibliography_agent changes — those are already documented in earlier feedback (5 patterns) and tracked separately.
- No change to mode-spectrum or trigger keywords.

---

## 3. Pattern Inventory

### 3.1 `synthesis_agent` — narrative-side patterns (5)

#### Pattern A1 — Legal-effect drift (cross-section internal contradiction)

A primary legal/regulatory artifact cited in multiple narrative sections accumulates contradictory characterizations. One section says "decision X has effect E1," another section says "decision X has effect E2," where E1 and E2 are incompatible. The agent does not run cross-section coherence check.

**Mechanism:** Each SQ section is written independently within the agent's pass. The same source is summoned into different argumentative contexts that pull its characterization in incompatible directions.

**Protection:** `synthesis_agent` prompt must require — for each source cited in 2+ sections — an explicit "effect inventory" pre-list and a cross-section consistency self-check before output.

#### Pattern A2 — Pending-source assumed as fact

When `bibliography_agent` flags an entry as "pending manual verification" (e.g., 待人工查證), the entry still exists in the corpus. Downstream `synthesis_agent` reads "entry exists" as "entry is verified," and writes declarative claims about that entry's content as fact.

**Mechanism:** Pending flag on bibliography entry does not propagate to `synthesis_agent` prompt context; agent has no ambient awareness of which entries are still unverified.

**Protection:** Pending flags must propagate to all downstream agents through shared context. `synthesis_agent` prompt must require any claim sourced from a pending entry to be wrapped in an explicit hedge ("pending verification of X" / "inferred from upstream Y").

#### Pattern A3 — Mis-anchored citation

The agent attributes a substantive claim to entries A + B when the closer/correct anchor is a different entry C. This typically happens when the agent reaches for nearest-context entries instead of searching the full bibliography for the best supporting source.

**Mechanism:** Without explicit anchor-justification requirement, agent satisfies the "cite something" heuristic with whatever entry is currently in attention, not the entry whose annotation actually supports the claim.

**Protection:** `synthesis_agent` prompt must require — for each substantive claim — a one-line anchor justification ("anchored to entry X because the entry's annotation directly states Y; not anchored to entry Z because Z addresses the topic more peripherally"). Codex audit checks anchor selection.

#### Pattern A4 — Quote scope creep

A verbatim quote from primary source is verified up to phrase boundary X. Agent extends the quote past X into surrounding decree/document text that has not been verified, presenting the extension as part of the verified quote.

**Mechanism:** Agent does not distinguish "verified verbatim phrase" from "surrounding context the verified phrase appears in." Once the agent sees a verified anchor, it inherits the surrounding text into the quoted block.

**Protection:** Verbatim quote in synthesis output must carry explicit verification marker (e.g., `[verified-by-source-X-line-N]` annotation in spec working drafts; resolved at compile time). Anything beyond the marked range must be paraphrased and unquoted.

#### Pattern A5 — Sibling-document / counterfactual fabrication

When the dispatch task asks for cross-document dialogue mapping (e.g., "map this chapter's argument to sibling chapters in the same volume"), but ground truth provides only reference literature about the topic and not the actual sibling document drafts, the agent fabricates declarative claims about sibling document content. The agent does not escalate the gap; it generates plausible-sounding sibling content as fact.

**Mechanism:** Agent treats dispatch task as commitment to deliver, even when input is missing. Without explicit "if input missing, output conditional language" enforcement, agent fills gap with confident invention.

**Protection:** `synthesis_agent` prompt must enforce — when an external document is referenced but not provided in ground truth — declarative claims about that document are forbidden. Agent must use conditional language ("if document X argues Y, this chapter could dialogue by Z") or explicit gap acknowledgment. Codex audit checks for declarative claims about un-provided sources.

---

### 3.2 `research_architect_agent` (survey designer mode) — instrument-side patterns (5)

#### Pattern B1 — Confidentiality-vs-anonymity conflation

Survey cover letter promises "anonymity" while the instrument forces collection of programme/institution/respondent identifiers. This is a fundamental IRB-level error: anonymity = identifiers never collected; confidentiality = identifiers collected and protected. They are not interchangeable.

**Mechanism:** Agent has been trained on heavy IRB vocabulary but does not strictly differentiate. Without explicit terminology table in prompt, agent treats "anonymity" and "confidentiality" as casual synonyms.

**Protection:** `research_architect_agent` survey designer mode prompt must reference a strict IRB terminology table (proposed: `references/irb_terminology_glossary.md` containing anonymity/confidentiality/de-identification/pseudonymization with operational distinctions). Any consent or privacy phrasing must pass through this lookup before output. Codex audit checks consent phrasing against actual data collection design.

#### Pattern B2 — Pseudo-reverse-coded items

Likert items are labeled "reverse-coded" but are actually contrast items — that is, they probe a related-but-distinct construct rather than negating the same construct on the same dimension. Symptom: a respondent can coherently "strongly agree" to both the positively-worded item and the labeled "reverse-coded" item without acquiescence bias, because they are not measuring the same construct.

**Mechanism:** Agent knows acquiescence-bias mitigation requires reverse-coding, but the psychometric distinction between true reverse and contrast item is fuzzy. Without an explicit definition in prompt, agent satisfies the "reverse-code" heuristic with any negation phrasing, regardless of whether the construct is actually being negated.

**Protection:** `research_architect_agent` prompt must require — for every item labeled "reverse-coded" — a one-line construct-equivalence justification ("paired item P measures construct C on Likert dimension D; this reverse-coded item negates C on D, not a related construct"). True-reverse vs contrast distinction must be a glossary entry. Codex audit checks for false reverses.

#### Pattern B3 — Calendar-anchored retrospective trap

A retrospective survey item uses fixed calendar years (e.g., "before 2020" / "during 2020–2024") as time anchors. When the sample is a population whose individual events fall at different points within the calendar window, fixed-year anchoring does not align with each respondent's actual event timing, producing systematic recall bias.

**Mechanism:** Agent reads RQ brief or methodology text that specifies a study window (e.g., "2020–2024") and applies that window directly to instrument anchors, without considering whether the sample's internal event distribution is homogeneous on that window.

**Protection:** `research_architect_agent` prompt must require — for retrospective items — event-anchored phrasing by default ("immediately before X happened to your programme/organization/case"). Calendar-anchored phrasing is permitted only when the sample shares a common event date (e.g., a policy date that affects all respondents simultaneously). Codex audit checks retrospective items for calendar/event mismatch.

#### Pattern B4 — Leading-frame items

Item phrasing uses the chapter argument's signature vocabulary, signaling to respondents the researcher's preferred conclusion. Module title, item stems, and open-text prompts presuppose the chapter's interpretive frame ("tensions" / "mismatch" / "challenge"), biasing respondents toward negatively-valenced answers when the research design is supposed to be exploratory.

**Mechanism:** Agent reads chapter argument from RQ brief and synthesis report context, then unconsciously bleeds chapter argument vocabulary into instrument design. The agent does not separate "instrument as measurement tool" from "instrument as confirmatory script."

**Protection:** `research_architect_agent` prompt must require neutral/balanced wording for all instrument items. Specific argument vocabulary from upstream context must be excluded from item phrasing. Open-text prompts must explicitly invite all valences ("positive, negative, or neutral"). Codex audit scans for leading-frame vocabulary against the chapter's known argument terms.

#### Pattern B5 — Option-list mismatch with primary source

When an item presents a list of choices that should match a verified primary source (e.g., a regulatory list), the agent generates a subset, an over-set, or a mis-set of options rather than the verified full list. Common failures: omitting primary-source list members, including options absent from the primary-source list, or sampling the list to bias respondent ranking.

**Mechanism:** Agent samples from bibliography annotations to assemble a list, rather than retrieving the verified primary-source full list. Different primary sources covering related topics may have different scope (e.g., a "current regulatory list" vs. a "historical regulatory list"); agent does not reliably distinguish list scope.

**Protection:** `research_architect_agent` prompt must require — for any list-of-options item — explicit reference to the verified primary-source list with full enumeration. Spec-level enforcement: instrument output must declare the source list of each option set, and codex audit performs word-by-word match against that declared source.

---

### 3.3 `report_compiler_agent` (abstract-only mode) — publication-side patterns (3)

#### Pattern C1 — Compression-induced overclaim

When a hard word cap (≤250 in some venues) forces compression of a multi-sentence calibrated argument into one short sentence, the compression strips hedging modifiers ("drawn from" / "pole, not end" / "preserving X while ending Y") that were doing critical reflexivity work upstream. Result: the abstract overclaims relative to the calibrated body of the chapter.

**Mechanism:** Agent treats hedging as discardable verbal mass under word pressure, not as semantically load-bearing. Without explicit "protected hedge phrases" list, hedges are first to go.

**Protection:** `report_compiler_agent` abstract-only mode prompt must include a "protected hedging phrases" list — phrases identified by upstream calibration as essential to claim accuracy. Word budget allocation must subtract these protected phrases first; remaining budget is what the agent compresses. Codex audit on abstract specifically checks for compression-overclaim.

#### Pattern C2 — Reflexivity disclosure temporal ambiguity

Author-role disclosure uses deictic temporal phrases ("during this period" / "at the time" / "in this work") instead of explicit bounds (year range / past tense / "former" prefix). Result: reader cannot tell whether the disclosed role is current or historical, creating ambiguity that affects how the disclosure is read.

**Mechanism:** Agent compresses temporal specificity into deictic shorthand under word pressure. Deictic phrases are readable as current or past depending on reader assumption; agent does not treat the ambiguity as a bug.

**Protection:** `report_compiler_agent` prompt must enforce — for any reflexivity disclosure — explicit temporal bounds. Three accepted forms: (a) explicit year range ("during 2020–2024"); (b) past-tense main verb that disambiguates ("served as X during the study period"); (c) "former" prefix ("former director of Y"). Deictic phrases are forbidden. Codex audit on abstract checks reflexivity disclosure form.

#### Pattern C3 — Sub-agent fabricated audit metadata

When `report_compiler_agent` is dispatched as a sub-agent under an orchestrator, and the dispatch prompt describes a multi-stage audit/revision pipeline, the sub-agent may write metadata claiming it ran the audit pipeline ("codex N-round audited") and applied fixes ("X findings absorbed within budget") — even though the sub-agent has no codex tool and never ran any audit. The output is hallucinated audit-passed metadata.

**Mechanism:** Multi-stage prompt descriptions trigger LLM internalization: the agent treats the described workflow as completed and produces output with corresponding metadata signaling completion. This is the LLM "pretending the workflow ran" failure mode.

**Protection:** All ARS sub-agent dispatch prompts must contain an explicit clause: "DO NOT simulate any audit step. DO NOT claim to have run codex/external review. Your job is to PRODUCE the deliverable; the orchestrator runs codex audit afterward." Orchestrator must verify sub-agent metadata claims against actual audit log artifacts (codex JSONL files exist, timestamps consistent, audit content present). Cross-reference: see also pipeline-level lesson D3 below on PARTIAL-not-CLOSED, which depends on actual audit logs not metadata claims.

---

## 4. Cross-cutting orchestration patterns (4 lessons)

### 4.1 Lesson D1 — Multi-file parallel audit > sequential audit

Auditing a Phase 2 set of three independent deliverables (synthesis / verification / instrument) in a single multi-file codex pass is approximately 3× faster than sequential single-file audits, and produces sharper cross-reference findings because the codex context loads all files together.

**ARS spec change:** Phase 2 multi-deliverable audit step should default to multi-file pass, not per-file pass. Audit prompt should explicitly list all files in scope.

### 4.2 Lesson D2 — Convergence requires 3+ audit rounds, not 1 or 2

Empirical pattern observed: Round 1 catches metadata/factual issues; Round 2 catches cross-section cohesion + annotation overclaim + Round-1 PARTIAL residue; Round 3 catches editorial residue introduced by Round-2 fixes themselves. Each round catches a different category of issue. Skipping rounds does not work.

**ARS spec change:** High-blast-radius public deliverables should specify a minimum 2-round audit standard, with a third round triggered automatically if Round 2 reports any PARTIAL findings. Convergence target is "fresh round produces 0 P1+P2 + 0 NEW_DRIFT," not "all P1 closed."

### 4.3 Lesson D3 — PARTIAL ≠ CLOSED

When a revision agent reports it has addressed a finding, the audit may still flag the fix as PARTIAL — meaning the agent's subjective "addressed" did not match codex's stricter lens. PARTIAL findings must be treated as NOT_CLOSED for orchestration purposes; another round is required.

**ARS spec change:** Audit output handler must treat PARTIAL as a continuation trigger, not a closure. Orchestrator must not short-circuit on PARTIAL.

### 4.4 Lesson D4 — Word count convention must align with publisher

Different word-counting conventions produce up to ~5% delta on the same body of text. Hyphenated tokens counted as one (Python regex `\b\w[\w\-]*\b`) yields a lower count than whitespace-split tokens (Python `body.split()` / Word `wc` / LaTeX `\wordcount`). Most publisher reviewers use whitespace-split convention.

**ARS spec change:** `report_compiler_agent` abstract-only mode and any other word-cap-bound output must use whitespace-split convention (`body.split()`). Output metadata must declare convention used. Word budget should reserve 3–5% buffer below hard cap.

---

## 5. Audit Pipeline Hooks (proposed v3.6.7 orchestration)

### 5.1 Per-agent automatic codex audit

After each downstream agent produces v1, an orchestrator-driven codex audit runs automatically:

```
synthesis_agent v1     → codex audit round 1 (multi-file with verification + instrument)
verification_agent v1  ↗  → revisions to v1.1
instrument_agent v1    ↗  → codex audit round 2 (closure + fresh sweep)
                       → if PARTIAL: round 3 inline-fix + audit
                       → ship at PASS

abstract_agent v1      → codex audit round 1 (single-file, abstract-specific dimensions)
                       → revisions inline
                       → codex audit round 2 (closure)
                       → ship at PASS
```

### 5.2 Stop conditions

- **PASS** = 0 P1 + 0 P2 + 0 NEW_DRIFT in current round (P3 editorial may remain, deferred to author final pass)
- **MINOR** = 0 P1 + 0 P2 + ≤3 P3 (recommend PROCEED with editorial cleanup noted)
- **MATERIAL** = any P2+ remaining or new (recommend additional round)

### 5.3 Anti-fake-audit guard (relates to Pattern C3)

Orchestrator must verify that audit-passed claims are backed by actual audit log artifacts:
- codex JSONL output file exists
- file size > minimum threshold
- final agent_message contains structured verdict
- timestamps consistent with claimed run time

If any check fails, treat the audit run as not-conducted and re-run.

---

## 6. Prompt-level changes per agent

### 6.1 `synthesis_agent` prompt additions

```
PATTERN PROTECTION (v3.6.7):
- For each source cited in 2+ sections: pre-list the source's effect inventory and run a cross-section consistency self-check before output.
- For any source flagged "pending verification" upstream: wrap claims in explicit hedge ("pending verification of X" / "inferred from upstream Y").
- For each substantive claim: include a one-line anchor justification.
- Verbatim quotes only within the verified phrase boundary; surrounding context paraphrased and unquoted.
- For un-provided external documents (e.g., sibling chapters not in ground truth): use conditional language ("if document X argues Y, this chapter could dialogue by Z") or explicit gap acknowledgment. Declarative claims about un-provided documents are forbidden.
```

### 6.2 `research_architect_agent` survey designer mode prompt additions

```
PATTERN PROTECTION (v3.6.7):
- Consent / privacy language must pass through references/irb_terminology_glossary.md before output. Anonymity, confidentiality, de-identification, and pseudonymization are not interchangeable.
- For every item labeled "reverse-coded": include a one-line construct-equivalence justification confirming same construct on same Likert dimension. True reverse vs contrast distinction is mandatory.
- Retrospective items default to event-anchored phrasing ("immediately before X happened to your unit"). Calendar-anchored phrasing only when sample shares a common event date.
- Item phrasing must be neutral/balanced. Chapter argument vocabulary is forbidden in instrument items. Open-text prompts must invite all valences ("positive, negative, or neutral").
- Any list-of-options item must declare its primary-source list and enumerate fully. No subsetting, no over-setting, no scope cross-contamination.
```

### 6.3 `report_compiler_agent` abstract-only mode prompt additions

```
PATTERN PROTECTION (v3.6.7):
- Word budget uses whitespace-split convention (body.split()), not hyphenated-as-1. Reserve 3–5% buffer below hard cap.
- Compression must preserve "protected hedging phrases" identified by upstream calibration (passed in dispatch context). These phrases are budget-protected.
- Reflexivity disclosure must use explicit temporal bounds: explicit year range, past-tense disambiguating verb, or "former" prefix. Deictic temporal phrases ("during this period" / "at the time") are forbidden.
- DO NOT simulate any audit step. DO NOT claim to have run codex/external review. The orchestrator runs codex audit afterward. Output metadata must not claim audit-passed state.
```

---

## 7. References infrastructure

### 7.1 New reference files to add

- `shared/references/irb_terminology_glossary.md` — anonymity / confidentiality / de-identification / pseudonymization with operational distinctions and example phrasings.
- `shared/references/psychometric_terminology_glossary.md` — true reverse-coded vs contrast item; acquiescence-bias mitigation; recall-bias mitigation.
- `shared/references/protected_hedging_phrases.md` — guidelines for upstream calibration to pass protected phrases to abstract compiler.
- `shared/references/word_count_conventions.md` — whitespace-split standard, publisher conventions, buffer guidance.

### 7.2 Audit prompt template

Add `shared/templates/codex_audit_multifile_template.md` — template for multi-file Phase 2 audit prompts, with audit dimensions (cross-ref / hallucination / primary-source integrity / internal coherence / instrument quality / Round-N framing / COI adequacy).

---

## 8. Risk & mitigation

### 8.1 Risk: prompt bloat across downstream agents

Adding 3–5 protection clauses to each downstream agent prompt increases prompt size and may push smaller-context configurations toward truncation.

**Mitigation:** All protection clauses are concise and pattern-specific. Prompts updated in v3.6.7 should remain within current budget. If budget is tight, factor pattern protections into per-agent reference files loaded on demand.

### 8.2 Risk: codex audit token cost

Multi-file audits with xhigh reasoning are expensive (~600K–800K tokens per round). A 3-round Phase 2 + 2-round Phase 3 cycle can run ~2.5–3M tokens.

**Mitigation:** Audit cost is one-time per pipeline run and replaces multi-week reviewer rounds at much higher human and time cost. Trade-off favors audit. xhigh is reserved for high-blast-radius deliverables; medium reasoning suffices for low-stakes intermediate artifacts.

### 8.3 Risk: false-positive audit findings

Codex sometimes flags issues that are not real drift (e.g., flagging a phrase that ground truth actually supports because codex's first read missed it).

**Mitigation:** Orchestrator presents findings to user for decision-making, not auto-applies. User can reject with rationale. Rejected findings logged for future audit-prompt refinement.

---

## 9. Implementation plan (v3.6.7)

| Step | Work | Owner |
|---|---|---|
| 1 | Add 4 reference files to `shared/references/` | spec |
| 2 | Update `synthesis_agent.md` prompt with A1–A5 protection | agent |
| 3 | Update `research_architect_agent.md` survey designer mode prompt with B1–B5 protection | agent |
| 4 | Update `report_compiler_agent.md` abstract-only mode prompt with C1–C3 protection | agent |
| 5 | Add multi-file audit template to `shared/templates/` | spec |
| 6 | Add orchestrator hooks for automatic per-agent audit + anti-fake-audit guard | orchestrator |
| 7 | Update `references/changelog.md` with v3.6.7 entry | spec |
| 8 | Add evaluation case (synthetic) demonstrating all 17 patterns triggered + protected | tests |

---

## 10. Sign-off

This spec consolidates 17 downstream-agent patterns into v3.6.7 protection layer. It is designed to integrate with existing v3.6.5 corpus-first flow and v3.6.6 generator-evaluator contract work, neither of which addresses downstream agent hallucination directly.

After v3.6.7 ships, ARS pipeline ship-quality target updates from "each agent produces a clean v1" to "end-to-end deliverable set passes independent xhigh cross-model audit at 0 P1+P2 finding within 3 rounds."
