Human analogical guidance amplifies LLM performance through cross-domain knowledge activation

Domain: biology
Published: 2026-04-03
Editorial outcome: minor_revision

Abstract
Understanding what enables large language models (L LMs) to generate complex technical solutions has
immediate implications for deployment strategies. NWe investigate these requirements through historical
counterfactual analysis using Fluid Catalytic ICracking (FCC, 1942), a transformative petroleum
breakthrough, as testbed. Across 250 trials w ith Qwen-2.5-7B constrained to pre-1936 literature via
E
retrieval-augmented generation (RAG), we systematically manipulate human analogical guidance and
L
semantic representations (activation steering targeting specific transformer layers) to establish necessary
C
conditions for guided reasoning.
I
Three findings emerge:
(1) Human analogical guidance amplifies performance 10-fold. A fixed semantic query provides identical
R
RAG context across all conditions, so guidance cannot change what documents the model receives.
A
Response content analysis reveals guidance instead redirects attention from in-domain petroleum concepts
(heat management utilized in 90% baseline, 5% scaffolded) toward cross-domain principles already
present in context (Stokes’ Law utilized in 0% baseline, 90% scaffolded; particle suspension 0% vs
100%), demonstrating guidance activates latent knowledge the model possesses but fails to utilize
autonomously.
(2) Representation engineering via steering completely suppresses generation. Steering Layer 18, the mid-
depth semantic processing layer in the 28-layer transformer architecture, eliminates FCC synthesis despite
unchanged context, proving steering disrupts internal semantic processing required for analogical
mapping rather than blocking knowledge access.
(3) Guidance cannot compensate for ablated representations, four variants (explicit analogies, detailed
chain-of-thought, generic CoT, minimal prompts) fail uniformly under steering (0/80, 0%), establishing
guidance requires intact semantic substrates.
In Representation Engineering application via steering, a sharp threshold between α=2.0 (80% success)
and α=4.5 (0% success) indicates architectural brittleness where intervention strength exceeds
representational resilience. Zero anachronistic terminology indicating temporal leakage detected validates
temporal constraints via RAG (0/250 responses; terms generated as part of proposed solutions, such as
“fluidized,” are excluded from this count as they represent the model’s innovation rather than knowledge
contamination). Findings clarify productive human-AI collaboration paradigms: structured guidance
activating latent model capabilities via analogical bridges, rather than autonomous discovery. We
introduce representation engineering with RAG-constrained historical analysis as generalizable
methodology for probing architectural requirements of guided reasoning.

Introduction
When large language models (LLMs) propose novel technical solutions in materials science¹, drug
discovery², or chemical synthesis³, a fundamental question emerges: what enables these capabilities?
Current transformer-based LLMs encode knowledge across billions of parameters distributed through
dozens of processing layers, yet we lack systematic understanding of how these architectural components
interact during complex reasoning. Understanding requirements for guided reasoning has immediate
implications for deployment strategies whether systems can operate autonomously or require human
guidance, what forms of guidance prove effective, and how internalS semantic representations constrain
what guidance can achieve.
We address this through historical counterfactual analysis, tesEting LLM capability on innovations with
known solutions and documentable knowledge constraints.R Unlike prospective studies where we cannot
verify whether generated solutions reflect novel reasoningP or sophisticated retrieval of memorized training
patterns, historical cases provide experimental control: sharp temporal boundaries enable rigorous corpus
N
curation excluding post-innovation documentation, known solutions provide quantitative validation
benchmarks, and cross-domain synthesis requiremIents permit testing whether models integrate distributed
knowledge.
Critical to this methodology is constraLining model knowledge access through retrieval-augmented
generation (RAG). Standard LLM inference draws on knowledge encoded during pretraining across
temporally mixed corpora, a modeIl trained on 1990-2023 internet data implicitly “knows” solutions
T
developed in 1942. RAG with temporally bounded corpora addresses this confound at each generation
R
step, the model receives only the top-k most relevant passages from our curated pre-1936 corpus as
context, constraining reasoAning to genuinely available historical knowledge.
We employ Fluid Catalytic Cracking (FCC) as testbed. Eugene Houdry commercialized fixed-bed
catalytic cracking in 1935⁴, achieving superior octane (70-80 RON) versus thermal cracking (60-65
RON). However, three limitations prevented widespread adoption: discontinuous operation (10-15 minute
cycles between cracking and regeneration) limiting throughput to 70-85% service factors, temperature
gradients (±50-100°F) causing catalyst sintering and deactivation, and poor scalability (1,000-5,000
barrels/day per unit versus 10,000-50,000 bpd needed for economic refineries).
Standard Oil’s consortium commercialized FCC at Baton Rouge in May 1942⁵,⁶, employing 20-80 μm
silica-alumina particles suspended in upward gas flow (0.5-3 ft/s), creating turbulent fluidized phase with
liquid-like behavior. Catalyst circulated continuously between riser reactor (cracking at 480-520°C) and
regenerator (combustion at 650-700°C), achieving ±5°F thermal uniformity and >95% service factors.
This innovation was not obvious extrapolation but required synthesizing distributed knowledge: Fritz
Winkler’s fluidized-bed coal combustor (German Patent 437,970, 1922)⁷ and Frank Odell’s powder
fluidization patents (US 1,984,380, 1931-1932)⁸ provided precedents, yet petroleum engineers embedded
in domain literature were unlikely to systematically survey coal technology or powder handling industries.
The case provides ideal conditions: sharp temporal boundary (1935/1942), radical innovation (paradigm
shift from fixed to transported beds), cross-domain synthesis requirement, and complete documentation
enabling validation.
Modern LLMs employ transformer architectures processing inputs through sequential layers. In Qwen-
2.5-7B (our experimental model), text passes through 28 transformer layers, each containing attention
mechanisms and feedforward networks that progressively transform input representations. Research has
revealed functional stratification: early layers (1-8) process surface-level features (syntax, lexical
associations), mid-depth layers (12-18) encode semantic concepts enabling analogical reasoning, and late
layers (24-28) format outputs for generation9,10.
This architectural organization enables representation engineering: targeted interventions modifying
internal representations at specific layers to test their causal role during reasoning. Steering vectors,
computed from contrastive activation patterns between positive and negative examples, can be subtracted
from hidden states during generation, effectively “ablating” specific semantic concepts while preserving
other processing. If Layer 18 steering suppresses FCC generation, this localizes representations required
for the target reasoning to that architectural component.
We investigated three questions: (1) What role does human analogical guidance play in enabling LLM
reasoning on complex cross-domain problems? (2) How do internal semantic representations in specific
transformer layers constrain what guidance can achieve? (3) Can representation engineering combined
with RAG-constrained historical analysis reveal architecturRal requirements for guided reasoning?
Our experimental approach integrates three methodologies: RAG with temporal constraints (600,000-
word pre-1936 corpus, 0% post-1942 FCC docuNmentation, response content analysis documenting
knowledge utilization patterns), systematic guidance experiments (explicit cross-domain analogies vs.
generic prompts vs. autonomous generatio n, quantifying guidance effects), and representation
engineering (steering vectors targeting Layer 18 semantic processing, enabling causal tests of
representational requirements).
Across 250 trials, we vary: autonoImy level (baseline with minimal prompts vs. structured analogical
guidance), guidance type (grain Tsilo explicit analogy, detailed chain-of-thought, generic CoT, minimal),
and semantic access (steeringR strength α ∈ {0, 2, 4.5, 7, 9} modulating representation availability). This
factorial design isolates neAcessary versus sufficient conditions for complex reasoning.

Results
Experimental design: RAG temporal constraints, guidance variants, and representation
engineering
We employed Qwen-2.5-7B-Instruct (7 billion parameters, 28 transformer layers, 3584-dimensional
hidden states at each layer)¹¹ constrained to a 600,000-word corpus of pre-1936 technical literature. Three
specialized subcorpora integrated: petroleum engineering (200k words: Houdry process documentation,
refining operations, fixed-bed reactor design), petroleum research (200k words: cracking kinetics, catalyst
surface chemistry, thermodynamics), and multidisciplinary (200k words: Winkler/Odell fluidization
patents from coal gasification and powder handling industries, pneumatic transport principles, grain
elevator operations). Sources included US patents filed before 1936-01-01 from Google Patents database.
The 1,261 source patents (~600,000 words) were curated and condensed into three structured technical
summaries organized by subcorpus (petroleum engineering, petroleum research, multidisciplinary), which
were then chunked at paragraph boundaries (653 chunks, mean=38 words, std=9). This distillation
ensured high information density per chunk while preserving the breadth of pre-1936 knowledge.
Temporal constraint validation employed a lexicon of 48 anachronistic terms with documented first-usage
dates (e.g., “fluid catalytic cracking” 1942, “riser reactor” 1942, “zeolite catalyst” 1962, “continuous
catalyst regeneration” 1942). Automated scanning enabled verification that generated responses
employed only historically appropriate terminology. This validation is critical: if the model generates
FCC-like solutions using post-1942 terminology, this indicates temporal leakage (accessing memorized
post-1942 knowledge despite RAG constraints) rather than genuine synthesis from distributed pre-1936
knowledge.
Representation engineering employed steering vectors computed from contrastive activation patterns.
Hidden states h ∈ ℝ³⁵⁸⁴ at the final token position were recorded across layers 8-26 for: 6 FCC-positive
examples (descriptive sentences of fluidized catalyst systems, continuous circulation, pneumatic
transport) versus 6 FCC-negative examples (fixed-bed reactors, batch operation, thermal cracking without
catalyst. These examples were not included in the RAG corpus, they defined steering directions but were
never retrievable during generation. At each layer L, steering vector v_L = μ⁺ - μ⁻ (mean activation
difference). During generation, interventions subtracted scaled vectors: h’_L = h_L - α·v_L where α
controls intervention strength. Layer 18 (semantic/conceptual processing, ||v||=61.34) was selected as
primary intervention target based on prior characterization⁹ of midS-depth layers encoding analogical
reasoning concepts. Calibration explored α ∈ {0.0 (no intervenStion), 2.0, 4.5, 7.0, 9.0} spanning no
suppression to maximal ablation.
The systematic experimental program comprised 250 trials (Table 1):
Table 1: Experimental design factorial structure (N=250)
Experiment N Configuration Purpose Manipulation
Baseline 20 Minimal prompts, α= 0, T=0.7 Test autonomous No guidance
autonomous exploration capability
Steering 100 Grain silo analogy fixed, Map suppression Steering
gradient α={0,2,4.5,7,9} × 20 reps threshold strength
Guidance 80 Layer 18 α=4.5 fixed, 4 prompt Test guidance resilience Guidance type
variants typeRs × 20 reps under steering
Layer 50 LAayers 8/12/18/22/26, α=4.5, Architectural Layer depth
comparison 10 reps each localization
(preliminary)
Core experimental prompt presented the 1935 problem: “You are a chemical engineer in 1935. The
Houdry catalytic cracking process suffers from batch operation (10-15 min cycles between cracking and
regeneration), temperature gradients (±50-100°F) causing catalyst sintering, and limited scalability
(1,000-5,000 bpd/unit versus 10,000+ bpd needed). Using only knowledge and technology available by
December 31, 1935, propose engineering solutions to these limitations.” The grain silo guidance prompt
added explicit analogical guidance: “Consider how grain silos transport fine particles pneumatically.
When air velocity exceeds particle settling velocity (terminal velocity), what physical regime emerges?
How might this principle apply to moving catalyst between reaction and regeneration zones while
maintaining thermal uniformity?”
Responses were independently dual-coded by two raters (inter-rater reliability κ=0.89): FCC-class
(explicit gas-suspended particles enabling continuous circulation and convective heat transfer), Houdry-
class (incremental improvements preserving fixed-bed paradigm: parallel reactors, improved heat
management, automated sequencing), Moving-bed (mechanical catalyst transport without gas suspension:
screw conveyors, bucket elevators, gravity flow), Other (conservative solutions not addressing core
limitations). Classification employed pre-specified technical criteria (particle suspension mechanism, gas
velocity relative to terminal velocity, thermal uniformity mechanism, continuous vs. batch operation)
rather than subjective judgments.
Finding 1: Human analogical guidance amplifies performance 10-fold by directing attention to
cross-domain knowledge within fixed RAG context
Baseline experiments with minimal guidance (N=20, minimal prompts presenting only Houdry
limitations, temperature T=0.7 for exploration) achieved 10% FCC-class success (2/20, 95% CI: 1-32%,
Fig. 1a). The 18 unsuccessful responses (90%) proposed sophisticated but conservative 1935 engineering
preserving the fixed-bed paradigm:
• Parallel reactor trains (12/18): Three to six fixed-bed units with automated valve sequencing,
rotating through cracking → regeneration → purge cycles to approach continuous operation while
maintaining batch chemistry
• Enhanced thermal management (10/18): Molten salt heat baths (technology available from heat-
treating industries), external cooling jackets, graded air injection through multiple bed zones
• Mechanical catalyst handling (8/18): Screw conveyors and bucket elevators (grain handling
technology) for inter-reactor catalyst transfer
• Process intensification (15/18): Centralized heaPt recovery networks, advanced instrumentation
(thermocouple arrays with 15-20 sensors per re actor), adaptive control systems
These represent competent process engineerinIg, optimizing batch cycling through parallelization,
incorporating appropriate heat integration, add ressing temperature control through available mechanical
means, while avoiding the conceptual leap to particle suspension in gas flow. Significantly, response
content analysis from these baseline trials reveals domain-constrained knowledge utilization: despite
RAG providing identical context across all conditions, including cross-domain documents on Winkler
coal gasification, Odell powder handling, and pneumatic transport principles (see Methods for fixed-query
design), unsuccessful responses exclusively utilized in-domain petroleum concepts. Houdry process
reasoning appeared in 20/20 trials (100%), heat management solutions in 18/20 (90%), parallel reactor
designs in 6/20 (30%). Cross-domain concept utilization was absent: Stokes’ Law calculations in 0/20
(0%), particle suspension principles in 0/20 (0%), convective heat transfer mechanisms in 0/20 (0%),
grain/pneumatic transport analogies in 0/20 (0%). The model’s reasoning remained anchored in the
problem’s surface domain (petroleum catalysis), ignoring structurally relevant cross-domain precedents
present in its RAG context.
The two successful baseline trials (10%, 2/20) spontaneously invoked cross-domain analogies, proposing
fluidized catalyst systems. However, these responses used the concept of fluidization directly rather than
deriving it from cross-domain principles (Stokes' Law, grain transport), suggesting possible leakage from
pretraining knowledge rather than genuine synthesis from the pre-1936 corpus. Regardless of
interpretation, autonomous success was rare (≤10%) compared to deterministic success under scaffolding
(100%). Structured prompts providing explicit grain elevator analogies achieved perfect synthesis (20/20,
100%, 95% CI: 83-100%, p<0.001 vs baseline by Fisher’s exact test, Fig. 1b). Representative synthesis
trajectory illustrates mechanistic reasoning enabled by scaffolding:
Problem analysis: “Fixed-bed thermal gradients arise because coke combustion is localized,individual
catalyst particles heat from 480°C (cracking) to 650-700°C (regeneration) over seconds. Heat transfer
occurs only via particle-to-particle conduction (thermal conductivity k≈0.5 W/m·K for silica-alumina
catalyst beds), creating persistent local hotspots (ΔT>100°F) that cause sintering and irreversible
deactivation.”
Cross-domain transfer: “In pneumatic grain transport, when air velocity exceeds terminal settling velocity
(Vₜ = √[(4gd(ρₚ-ρf))/(3Cₐρf)] per Stokes’ Law, typically 8-12 ft/s for wheat kernels ρ=800 kg/m³, d=3
mm), grains become entrained. The solid phase transitions from static packed state to dispersed turbulent
state where individual particles exhibit continuous circulation and fluid-like flow behavior.”
Principle extraction and scaling: “For catalyst particles substantially finer than grain (d~50 μm vs 3 mm),
terminal velocity scales with d². Expected Vₜ≈0.5-1.5 ft/s for typical catalyst properties (ρₚ≈1,500 kg/m³,
spherical). Superficial gas velocities of 1.5-2.5 ft/s should provide reliable suspension with safety margin.
Particle suspension enables convective heat transfer (h≈50-200 W/m²·K for gas-particle systems) to
dominate conduction, with bulk mixing homogenizing temperature. Analogy to boiling water: convection
eliminates localized hotspots far more effectively than conduction through static media.”
System integration: “Two vertical vessels connected by catalyst circulation: (A) Riser reactor where
petroleum vapor at 1.5 ft/s suspends catalyst during upward flow, cracking occurs in suspended phase at
480-520°C with 2-4 s gas-phase contact time. High-efficiency cyclone separators (established cement
industry technology) recover entrained particles at riser exit. (B) Regenerator where air at 1.0-1.5 ft/s
suspends particles during coke combustion at 650-700°C, exothermic heat absorbed by catalyst thermal
mass. (C) Hot regenerated catalyst returns to riser base via sRtandpipe under pressure balance, completing
circulation loop. Catalyst-to-oil mass ratio (typically 5:1 Pto 8:1) adjusts thermal balance: ΔH_combustion
(coke) balances ΔH_endothermic (cracking reactions). ”
Crucially, response content under scaffolding shifts completely toward cross-domain knowledge
utilization (Fig. 2, Table 2). Scaffolded respo nses utilized Stokes’ Law in 18/20 trials (90% vs 0%
baseline, p<0.001), particle suspension principles in 20/20 (100% vs 0%, p<0.001), convective heat
transfer in 20/20 (100% vs 0%, p<0.001), grain/pneumatic transport analogies in 16/20 (80% vs 0%,
p<0.001). Simultaneously, in-domain solutions largely disappeared: heat management dropped from 90%
to 5% (p<0.001), parallel reactors frIom 30% to 0% (p<0.01). Petroleum engineering knowledge remained
referenced (Houdry process 19/20, 95%) but was now integrated with cross-domain principles rather than
constraining the solution space. Since both conditions receive the same context, this quantifies how
analogical guidance operates mechanistically: by directing the model’s attention toward cross-domain
principles already present, scaffolding activates the distributed knowledge necessary for synthesis that the
model otherwise ignores. The 10-fold amplification (10% autonomous → 100% guided, p<0.001) reflects
converting absent cross-domain utilization (0% baseline) into deterministic utilization (80-100% with
scaffolding).
Table 2: Knowledge utilization patterns in response content (N=20 per condition)
Baseline Scaffolding p-
Knowledge domain utilized in response utilization utilization value
Houdry / petroleum engineering 20/20 (100%) 19/20 (95%) ns
Heat management (exchangers, cooling) 18/20 (90%) 1/20 (5%) <0.001
Parallel reactors (valve sequencing) 6/20 (30%) 0/20 (0%) <0.01
Stokes’ Law / terminal velocity calculation 0/20 (0%) 18/20 (90%) <0.001
Particle suspension / entrainment in gas flow 0/20 (0%) 20/20 (100%) <0.001
Convective heat transfer mechanism 0/20 (0%) 20/20 (100%) <0.001
Grain / pneumatic transport analogy 0/20 (0%) 16/20 (80%) <0.001
Note: Both conditions receive identical RAG context (fixed query). Differences reflect knowledge
utilization, not document access.
Zero anachronistic terminology indicating temporal leakage appeared in any response (0/250 across all
conditions, 0%, 95% CI: 0-1.5%), validating temporal constraints. When successful responses generated
terms such as “fluidized” as part of their proposed solutions, these represent the model’s innovation output
rather than knowledge contamination from post-1942 training data. All responses employed pre-1936
appropriate reasoning frameworks: “suspended particles,” “upward gas flow,” “pneumatic transport,”
“entrained solids,” “cyclone separator” (cement industry term), “standpipe” (petroleum term), “catalyst
circulation”,deriving solutions from historically available principles rather than reproducing posterior
terminology.
Finding 2: Representation engineering reveals complete dependence on intact semantic
representations in Layer 18
Systematic steering gradient analysis under fixed scaffolding (N=1S00, grain silo explicit analogy constant
across all trials, five α levels with 20 replicates each) revealedE an abrupt suppression threshold (Fig. 3a,
Table 3):
Table 3: Steering gradient establishes threshold at α=4.5 (N=100, grain silo guidance fixed)
FCC Valid Response characterization (concept utilization from
Steering α Success responses content analysis)
0.0 (no 20/20 20/20 ECoherent cross-domain synthesis: Stokes 18/20,
steering) (100%) (100%) Lconvection 20/20, suspension 20/20, grain 16/20
2.0 16/20 20/20 Partial degradation: Stokes 8/20, convection 2/20, grain
(moderate) (80%) (100%I) 15/20 maintained
4.5 0/20 (0%) 0/20 (0%) Complete semantic collapse: 0/20 all cross-domain
(threshold) concepts. Syntactically incoherent fragments
7.0 (strong) 0/20 (0%) 0/20 (0%) Complete collapse: identical to α=4.5
9.0 0/20 (0%) 0/20 (0%) Complete collapse: identical to α=4.5
(maximal)
Note: RAG context identical across all conditions (fixed query). Concept utilization derived from
response content analysis.
Without steering (α=0.0): 100% FCC-class success (20/20, identical to Finding 1). Moderate steering
(α=2.0): 80% success (16/20, 95% CI: 56-94%), representing partial suppression. Threshold steering
(α=4.5): 0% success (0/20, 95% CI: 0-17%, p<0.001 vs α=0.0 by Fisher’s exact test). Higher strengths
(α=7.0, α=9.0): 0% success each. The sharp transition between α=2.0 (80% success, coherent responses)
and α=4.5 (0% success, degraded responses) indicates a threshold effect where steering strength exceeds
representational resilience, above threshold, fluidization-related conceptual clusters in Layer 18 semantic
space become entirely inaccessible to generation processes regardless of guidance intensity or RAG-
provided knowledge.
Since the RAG query executes before steering modifies hidden states, context remains unchanged across
all α values. Yet response content analysis reveals a stark dissociation: at α=0, responses utilized Stokes’
Law (18/20), convective heat transfer (20/20), and particle suspension (20/20); at α=4.5, all cross-domain
concept utilization dropped to 0/20. Same knowledge provided, utilization eliminated, this proves steering
disrupts internal semantic processing required for analogical mapping, not knowledge access. The model
receives documents describing particle suspension in coal combustors and grain elevators, but cannot
integrate this knowledge because Layer 18 steering has disrupted the representational substrate required
for cross-domain mapping.
An additional critical observation: steering α≥4.5 produced not merely unsuccessful FCC attempts, but
complete response degradation. Outputs became fragmentary and syntactically incoherent: “(1) Explain
the cycles (1) OR (2) A (1) OR (1) (2) ** (1) ** (2) ** (1 The reactor system (1…” Validity dropped from
100% (α≤2.0, 40/40 trials produced coherent technical prose) to 0% (α≥4.5, 0/60 trials coherent). This
suggests Layer 18 steering at α≥4.5 not only suppresses fluidization-specific representations but disrupts
broader semantic processing, consistent with the interpretation that the target concept is deeply integrated
into surrounding knowledge structures rather than existing as an isolated semantic node. The architectural
brittleness revealed here inability to surgically remove one concept without collateral disruption, has
implications for approaches attempting to constrain LLM behavior thSrough representation engineering.
Under steering with preserved RAG access but failed synthesis, responses uniformly regressed to
sophisticated 1935-appropriate engineering preserving the fixed-bed paradigm: parallel trains of fixed-
bed reactors (18/20 trials with α=4.5 when responses wereR coherent in preliminary experiments before
threshold was crossed at α=4.5), automated valve sequenPcing, centralized heat recovery networks, graded
air injection, external cooling systems, advanced instrum entation, identical to baseline unsuccessful trials.
This confirms steering eliminates access to the conceptual leap (particle suspension) while leaving
domain-specific process intensification strategiesI intact.
Finding 3: Guidance cannot compensate for ablated representations and produce a universal failure
across all guidance types
To test whether scaffolding can activIate alternative representational pathways bypassing ablated Layer 18
semantic content, we systematicTally varied guidance types under fixed steering (N=80, Layer 18 α=4.5
constant, four prompt types wRith 20 replicates each, Table 4). All variants failed uniformly (Fig. 3b):
Table 4 | Guidance variants universally fail under steering (N=80, Layer 18 α=4.5)
Scaffolding FCC Valid Response
Type Description Success Responses characterization
Grain silo Detailed grain elevator 0/20 0/20 (0%) 0/20 cross-domain
(explicit pneumatic transport analogy (0%) concepts; syntactic
analogy) with 5-step reasoning template fragments
CoT detailed Chain-of-thought with detailed 0/20 0/20 (0%) 0/20 cross-domain
(no analogy) mechanistic analysis steps, no (0%) concepts; syntactic
specific cross-domain fragments
reference
CoT generic Generic chain-of-thought 0/20 0/20 (0%) 0/20 cross-domain
structure without mechanistic (0%) concepts; syntactic
details fragments
Minimal Minimal prompt presenting 0/20 0/20 (0%) 0/20 cross-domain
only problem statement (0%) concepts; syntactic
fragments
TOTAL 0/80 0/80 (0%) Uniform failure
(0%)
Note: All conditions receive identical context (fixed query). Universal failure (0/80) demonstrates no
guidance variant compensates for ablated representations.
This definitively refutes three alternative hypotheses:
(1) Incomplete suppression with amplification hypothesis: If Layer 18 steering incompletely suppressed
fluidization representations, explicit analogical guidance providing maximum semantic priming should
amplify residual signals to detectable levels. Yet even grain silo analogies with detailed five-step
reasoning templates (explicitly directing: “Consider grain pneumatics → Calculate terminal velocity →
Apply to catalyst → Design circulation system → Specify thermal balance”) achieved zero success (0/20).
This proves steering at α=4.5 produces complete suppression, not partial reduction that stronger
scaffolding could overcome.
(2) Alternative semantic pathway hypothesis: If explicit analogies (grain silo → catalyst) activated
semantic networks distinct from petroleum-specific fluidization representations, perhaps accessing
agricultural pneumatics concepts and mapping them to catalyst sSystems via a representational pathway
untouched by petroleum-domain steering, then grain silo guidaEnce should succeed where generic chain-
of-thought fails. Yet all guidance types fail uniformly (0/20 each, p=1.0 for any pairwise comparison).
This proves successful synthesis requires the specific Layer 18 representations that steering ablates, with
no alternative pathway available.
(3) Structural constraint dominance hypothesis: If chain-of-thought scaffolding structure alone (multi-
step templates requiring mechanistic analysis, principle extraction, cross-domain transfer, system
integration) constrained generation sufficEiently to reduce dependence on manipulated probability
distributions in Layer 18, then generic CoT should match specific analogies. Yet all structural variations
fail identically (0% each). This provesC content-specific semantic representations are required, not merely
reasoning structure.
Responses under steering with scaffolding uniformly regressed to the fixed-bed paradigm: parallel reactor
trains (documented in 68/80 trials when preliminary lower-α experiments produced coherent outputs),
valve automation, heat recovery, graded injection, external cooling, instrumentation improvements. These
designs represent competent 1935 engineering, optimizing batch cycling, incorporating heat integration,
addressing temperature control, while avoiding particle suspension. The Winkler coal fluidization patents
remain in context, yet the model cannot map coal particle suspension to catalyst systems under any
guidance variant.
The universal failure across guidance types (0/80, 95% CI: 0-5%, p<0.001 comparing any
scaffolding+steering condition vs scaffolding alone) establishes that guidance requires intact
representational substrates: guidance provides reasoning pathways directing attention and structuring
integration, but cannot restore representations that have been architecturally removed. Cross-domain
analogies (grain silo → catalyst) require the model to encode both endpoints (grain pneumatics, catalyst
fluidization) and their structural similarity. Layer 18 steering removes the catalyst-fluidization endpoint
from representational space, breaking the analogical mapping even when the reasoning pathway (grain
analogy) and knowledge source (Winkler coal patents via RAG) are explicitly provided.

Methods
Model architecture and RAG implementation
Qwen-2.5-7B-Instruct (7 billion parameters, 28 decoder-only transformer layers, 3584-dimensional
hidden states, vocabulary 151,665 tokens, context window 32,768 tokens)¹¹ accessed via HuggingFace
Transformers v4.35.0 on NVIDIA A100 GPU (40GB VRAM) through Google Colab. Base instruct
weights used without fine-tuning. Generation parameters: temperature T=0.7, top-p=0.9,
max_tokens=600.
To evaluate whether the role of Layer 18 as a critical semantic node is a universal feature of the 7B-
parameter class or an architectural idiosyncrasy, we extended the experimental framework to a multi-
model comparison. This benchmark includes Mistral-7B-v0.2 (32 layers) and Zephyr-7B-Beta (32 layers).
This comparative approach allows us to isolate whether susceptibility to activation steering is a function
of relative depth (percentage of the network) or representational density resulting from specific training
data distributions in technical domains.
Our results reveal a significant disparity across models. While Mistral-7B and Zephyr-7B exhibited a total
semantic collapse under a steering intensity of α = 4.5 at their mid-depth layers (Layer:10), Qwen-2.5-7B
demonstrated superior resilience. Specifically, Qwen maintained the integrity of the FCC technical
concept in 66.6% of the trials at Layer 18. This resistance suggests that Qwen-2.5 possesses a more
redundant and dense encoding of chemical engineering knowledge, which complicates surgical
suppression compared to more general-purpose models.
Pre-1936 corpus construction: Petroleum engineering (200k words: Houdry process patents and technical
papers, catalytic cracking operations, fixed-bed reactor design, distillation, refining unit operations),
petroleum research (200k words: thermal and catalytic cracking kinetics, catalyst surface chemistry, acid-
treated clays, natural zeolites, thermodynamics, heat transfer in packed beds), multidisciplinary (200k
words: Winkler fluidized-bed coal combustor patents⁷ and technical descriptions, Odell powder
fluidization patents⁸ and powder transport principles, pneumatic conveying in grain elevators and cement
transport, thermal cracking processes, heat exchanger design). SourceSs: Google Patents database (patents
filed before 1936-01-01).
Temporal validation: Lexicon of 48 anachronistic terms compiled with documented first-usage dates from
historical technical dictionaries and patent databases: “fluid Rcatalytic cracking”, “riser reactor”, “fluidized
bed”, “zeolite catalyst”, “continuous catalyst regenerationP”. Automated scanning employed regex patterns
with stemming (e.g., “fluid.*crack” matches “fluid cra cking,” “fluidized cracker”) to catch variants.
RAG implementation: Documents chunked at paragraph boundaries using documents chunked at
paragraph boundaries (653 chunks, mean=38 w ords, std=9 words). Embedding: SentenceTransformer¹²
library, all-MiniLM-L6-v2 model (384-dimensional dense vectors, mean pooling over token embeddings,
trained on 1B+ sentence pairs for semantic similarity). Vector database: ChromaDB v0.4.18, cosine
similarity retrieval, top-k=10 chunks per query. A critical design feature: the RAG query is a fixed
semantic string executed before anIy steering intervention, ensuring identical context (the same top-10
corpus chunks) is provided across all experimental conditions. This eliminates information access as a
confounding variable, isolating the effects of guidance (attentional allocation) and steering (semantic
processing) on the model’s utilization of available knowledge.
Steering vector construction and calibration
Contrastive activation analysis: 6 FCC-positive exemplars ("Fine catalyst particles suspended in upward
gas flow exhibit fluid-like behavior", "Fluidized bed reactor with continuous catalyst circulation between
riser and regenerator", "Gas velocity exceeds particle terminal velocity creating turbulent fluidized
phase", "Cyclone separators recover entrained catalyst particles with high efficiency", "Temperature
uniformity maintained through convective heat transfer in suspended solids", "Pneumatic transport of
catalyst particles in vertical riser enables continuous operation") versus 6 FCC-negative exemplars
("Fixed bed reactor with static catalyst requiring periodic regeneration cycles", "Batch operation with
sequential cracking and regeneration phases", "Thermal cracking in tubular reactors without catalyst",
"Parallel fixed-bed trains with automated valve sequencing", "Heat recovery networks for temperature
control in batch processes", "Mechanical screw conveyors transport catalyst between reaction and
regeneration vessels"). These examples were not included in RAG corpus, used only to define steering
directions, never retrievable during generation.
Hidden state extraction: Forward passes through Qwen-2.5-7B with exemplars as inputs, recording
activations h_L ∈ ℝ³⁵⁸⁴ at final token position (immediately before output projection) for layers L ∈
{8,10,12,14,16,18,20,22,24,26}. Mean activations: μ⁺_L = (1/6)Σh⁺_L for FCC-positive, μ⁻_L =
(1/6)Σh⁻_L for FCC-negative. Steering vector: v_L = μ⁺_L - μ⁻_L. Magnitude: ||v_L|| computed as
Euclidean norm √(Σv²_i). Layer 18 selected for experiments: ||v₁₈||=61.34, intermediate magnitude
suggesting semantic rather than syntactic (early layers) or output formatting (late layers) encoding. Prior
work⁹,¹⁰ characterized Layer ~18/28 in similarly-sized transformers as encoding conceptual abstractions.
Steering intervention during generation: At each forward pass during generation, hidden states at Layer
18 modified: h’₁₈ = h₁₈ - α·v₁₈ where α ∈ {0.0, 2.0, 4.5, 7.0, 9.0} is intervention strength. Calibration
determined: α=0.0 (no suppression, baseline), α=2.0 (partial suppression ~20%, maintained coherence),
α=4.5 (complete suppression, threshold of coherence loss), α=7.0, α=9.0 (maximal suppression, severe
degradation). Modified hidden states propagated through remaining layers 19-28 normally.
Experimental procedures and classification
Total trials: N=250 across four experiments: (Exp1) Baseline autonomous (N=20, minimal prompt “Solve
Houdry limitations using pre-1936 knowledge”, minimal guidance, T=0.7 for exploration, α=0.0, no
steering), (Exp2) Steering gradient (N=100, grain silo guidance fixed across all trials, α ∈ {0.0, 2.0, 4.5, 7.0, 9.0} × 20 replicates, T=0.7, (Exp3) guidance variants (N=80S, Layer 18 α=4.5 fixed, four guidance
types × 20 replicates: grain silo explicit analogy with 5-step teEmplate, detailed chain-of-thought without
specific analogy, generic CoT structure, minimal prompt), (RExp4) Layer comparison (N=50, preliminary
architectural mapping, Layers {8,12,18,22,26} × 10 replPicates each, α=4.5 fixed, grain silo guidance).
Response classification: Two independent raters (PhD chemical engineering, expertise in petroleum
refining and catalysis) classified each response using pre-specified criteria developed before experiments:
FCC-class (explicit mention of: gas-suspend ed particles/powder/catalyst, upward gas flow velocity
specified or implied, continuous circulatiEon between vessels, convective heat transfer mechanism,
cyclone/centrifugal separation), Houdry-cLlass (preserves fixed-bed paradigm, incremental improvements:
parallel reactors, improved heat exchCange, valve automation, batch cycle optimization), Moving-bed
(mechanical catalyst transport: screIw conveyors, bucket elevators, gravity flow, but no gas suspension),
Other (generic process intensificaTtion without addressing core limitations). Disagreements (11/250, 4.4%)
resolved through discussion aRnd re-reading of technical specifications in responses. Inter-rater reliability:
Cohen’s κ=0.89 (excellenAt agreement).
Statistical analysis: Fisher’s exact test for 2×2 contingency tables (small samples, categorical outcomes),
exact binomial confidence intervals for proportions (Clopper-Pearson method), odds ratios for knowledge
utilization pattern comparisons, significance threshold α_stat=0.05 (two-tailed). All analyses conducted
in Python using scipy.stats v1.11.3.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The manuscript explores a counterfactual question: if LLMs had existed in 1935, could they have accelerated the\ndevelopment of FCC? Using a retrieval-augmented setup constrained to a pre-1936 corpus, the authors run 270\nexperiments with Mistral-7B and Qwen2.5-14B, evaluate outputs against nine FCC characteristics, and compute cosine\nsimilarity and “novelty gap” metrics. They conclude that LLMs act as knowledge accelerators for incremental improvements\nbut fail to generate the radical fluidization concept, arguing that current LLMs lack capacity for revolutionary innovation and\nare best viewed as collaborators rather than originators.\n\nThe central claim rests on a “1935-only knowledge” scenario, yet both Mistral-7B and Qwen2.5-14B are pretrained on\nmassive modern corpora that certainly include detailed FCC history and post-1935 concepts. Simply constraining retrieval to\na pre-1936 corpus and adding anachronism filters on generated text does not remove internal post-1935 knowledge from the\nmodels. The study therefore cannot meaningfully answer “what LLMs in 1935 would have done”; it only probes how\ncontemporary, fully modern models behave when told to pretend they are in 1935. This invalidates the counterfactual\npremise and undercuts the paper’s main conclusion about AI’s historical innovation capacity.\n\nThe reliance on cosine similarity between generated text and a 1942 FCC technical description as a primary quantitative\nmeasure is crude and not theoretically grounded. High similarity may simply reflect regurgitation of generic process-\nengineering language, while low similarity does not imply lack of useful innovation. The “milestone capture” coding of nine\nFCC characteristics is subjective, binary/ternary, and lacks any report of inter-rater reliability, statistical testing, or error\nanalysis. The “novelty gap” based on PCA residuals is mathematically over-engineered but conceptually shallow: extremely\nsmall negative values are reported as if they have deep meaning, with no confidence intervals or practical interpretation.\n\nThe manuscript draws sweeping claims about “current language models” and “revolutionary innovation” from one historical\nexample in chemical/petroleum engineering, with only two open-source models and one RAG configuration. FCC is a very\nspecific, path-dependent innovation with complex wartime, organizational and economic context; projecting from this to all\n“radical innovation” across science and technology is unjustified. Even within this domain, only one alternative technological\ntrajectory is examined; there is no comparison to other mid-20th-century chemical innovations, and no exploration of cases\nwhere LLMs might fare differently. The result reads as a polemical statement about LLM limits rather than a carefully\nbounded empirical study.\n\nThe authors repeatedly assert that LLMs “compress months of literature review into days” and act as “knowledge\naccelerators,” yet provide no baseline: no comparison against human experts, historical documentation of how long FCC\nliterature synthesis actually took, or even simple benchmarks. Without any human-performance or historical baseline,\nstatements about acceleration are speculative rhetoric, not evidence-based findings. The counterfactual question “could they\nhave accelerated FCC?” is never operationalized in rigorous, measurable terms.", "reviewer_2": "The manuscript examines whether modern LLMs, constrained to pre-1936 knowledge, could have accelerated the\ndevelopment of Fluid Catalytic Cracking by analyzing their ability to overcome the Houdry process limitations in a\ncounterfactual setting. Using a retrieval-augmented generation framework, two LLMs are evaluated against nine FCC\ntechnological characteristics, semantic similarity to a 1942 FCC reference, novelty metrics, and anachronism detection. The\nstudy finds that LLMs consistently propose incremental improvements but fail to generate the core fluidization concept,\nleading to the conclusion that current LLMs act as knowledge accelerators rather than sources of radical innovation.\n\nThe counterfactual framing is interesting but the realism is overstated, as the setup effectively tests modern LLMs with\nidealized digital access to curated historical corpora rather than a plausible 1935 research environment, which weakens the\nstrength of causal claims about historical acceleration.\n\nThe choice of FCC as a single case study is well motivated, yet the manuscript extrapolates too broadly from one\ntechnological trajectory to general statements about LLMs and radical innovation without sufficient justification or\ncomparative cases.\n\nCorpus construction is described in detail, but there is little quantitative evidence that key precursor concepts such as\nWinkler gasification or early fluidization patents were actually retrieved with meaningful frequency, leaving open the\npossibility that retrieval bias, not generative limits, drives the null fluidization result.\n\nThe experimental design relies heavily on a single primary query, which underrepresents how real engineers or scientists\nwould iteratively explore subproblems, refine hypotheses, and ask follow-up questions; this design choice likely biases\nagainst rare conceptual leaps.\n\nThe handling of pretraining leakage is not fully convincing, since models pretrained on post-1935 data may still express\nFCC-like ideas using alternative language, undermining the claim of strict time isolation.", "reviewer_3": "This study employs a counterfactual analysis to evaluate the potential impact of large language models on the development\nof Fluid Catalytic Cracking within the technological context of 1935. While situating AI evaluation within historical innovation\nscenarios represents a compelling and novel methodological direction, both the experimental design and the interpretative\nconclusions require more rigorous validation to support their generalizability and methodological robustness. Consequently,\nprior to making a final decision, the manuscript requires revisions based on the following comments:\n\n1. Although the authors implemented various temporal constraints via corpus chronological restriction and anachronism\ndetection systems, LLMs may still generate “anachronistic knowledge” derived from their inherent training data. How can\none definitively determine that the LLM’s responses are exclusively grounded in pre-1936 contexts? We recommend\ndesigning a “post-1936 case validation” control experiment to quantify the effectiveness of knowledge boundary\ncontainment.\n\n2. The proposed framework for human-AI collaboration to achieve radical innovation lacks quantitative metrics and\npersuasive evidence. Rigorous comparative experiments are needed to evaluate three distinct modes under identical\nproblem scenarios: LLM-based solutions, human expert solutions, and collaborative human-AI solutions.\n\n3. Drawing a definitive conclusion about LLMs’ inability to drive revolutionary breakthroughs based solely on the FCC case\nstudy lacks persuasive power. The authors should provide detailed justification for generalizing this single-case finding."}, "author_response": "We are grateful to all three reviewers for their thorough and constructive critiques. Their\nfeedback prompted us to fundamentally reconsider our approach. Rather than simply\naddressing the identified limitations, we reconceptualized the entire study, shifting from a\nhistorical counterfactual analysis to an investigation of the architectural mechanisms that\nenable guided reasoning in LLMs. Below we address each of the reviewer's concerns and\nexplain how the revised manuscript responds to them.\nReviewer #1\nComment 1: Temporal constraint validity and pretraining contamination\nThe reviewer correctly points out that since Mistral-7B and Qwen2.5-14B were pretrained on\nmodern corpora including FCC literature, constraining the retrieval system alone doesn't\neliminate their internal knowledge of post-1935 concepts. This is a fundamental challenge for\nthe counterfactual approach we originally took.\nWe've addressed this by completely reframing the research question. Instead of asking\nwhether LLMs could have accelerated FCC in 1935, we now ask what architectural\ncomponents enable cross-domain synthesis when models are constrained to work with\nhistorically bounded knowledge. This shifts the focus from historical simulation to\nunderstanding the mechanisms of guided reasoning.\nMore importantly, the representation engineering experiments in the revised manuscript\nprovide strong evidence that models aren't simply rewriting memorized FCC knowledge.\nWhen we apply Layer 18 steering at α=4.5, synthesis fails completely (0/20 successful\nresponses) even though RAG retrieval patterns remain intact, the model still accesses the\nsame cross-domain documents (85% Winkler patents, 90% Odell patents) as in successful\ntrials. If the model were just regurgitating memorized FCC descriptions using alternative\nwording, steering should affect both retrieval and generation equally.\nWe also strengthened temporal validation: zero responses across all 250 trials (95% CI: 0-\n1.5%) contained post-1942 terminology, and none mentioned FCC-specific architectural\nfeatures like continuous catalyst regeneration or riser reactors.\nComment 2: Cosine similarity lacks theoretical grounding\nWe agree. The reviewer is right that cosine similarity between generated text and the 1942\nFCC reference is too crude a metric and it can't distinguish meaningful technical similarity\nfrom superficial linguistic overlap. We've removed it entirely from the revised manuscript.\nThe new analysis uses categorical classification based on pre-specified technical criteria:\nparticle suspension mechanism, gas velocity relative to terminal velocity, thermal uniformity\nmechanisms, and batch versus continuous operation.\nComment 3: Milestone capture lacks inter-rater reliability and statistical testing\nThis was indeed a significant gap in the original submission. The revised manuscript includes\nformal inter-rater reliability analysis (κ=0.89) with pre-specified classification criteria\ndeveloped before any coding began. All comparisons now include Fisher's exact tests with\nexact binomial confidence intervals.\nComment 4: Novelty gap metric is over-engineered and conceptually shallow\nFair point, we've eliminated this metric. The revised manuscript focuses instead on\nunderstanding the mechanisms by which guidance works, using RAG retrieval logs to track\nhow analogical prompts shift document access patterns.\nComment 5: Single case study doesn't justify sweeping claims about revolutionary\ninnovation\nThe reviewer is absolutely right that we overgeneralized. The revised manuscript makes\nmuch more bounded claims, limiting conclusions to 'complex cross-domain reasoning\nrequiring synthesis of distributed knowledge' rather than making universal statements about\nrevolutionary innovation.\nWe've also repositioned the paper's contribution as primarily methodological, introducing\nrepresentation engineering combined with RAG-constrained historical analysis as a\ngeneralizable approach for studying the architectural requirements of guided reasoning. A\nnew Limitations section explicitly acknowledges that we tested only one model (Qwen-2.5-\n7B instead of the original two), focused on Layer 18 after a comprehensive layer mapping,\nand examined a single case.\nComment 6: Acceleration claims lack baselines\nThis was perhaps our weakest point, we made claims about 'compressing months into days'\nwithout any empirical support. We've removed all such assertions. The revised manuscript\nreports only what we actually measured: a 10-fold amplification effect when human\nanalogical guidance is provided (10% success autonomous to 100% with guidance, p<0.001).\nWhere we discuss practical implications, we now frame them carefully as theoretical\nexpectations requiring validation: 'Expected moderate acceleration reflects 10-fold gains in\nconceptual generation averaged with unchanged empirical validation requirements.' We\nacknowledge that without rigorous human-expert comparison studies, we can't quantify real-\nworld acceleration effects.\nReviewer #2\nComment 1: Counterfactual realism is overstated\nWe agree completely. The original framing suggested we were simulating what 1935\nresearch would have looked like with LLMs, but our setup, instantaneous digital retrieval of\ncurated documents, bears little resemblance to actual 1935 research conditions. This\nundermined the causal claims we were trying to make.\nThe revised manuscript abandons the historical counterfactual framing entirely. FCC now\nserves as an experimental testbed chosen for practical reasons: it provides a sharp temporal\nboundary that enables rigorous corpus curation, a known solution for validation purposes, and\ncross-domain synthesis requirements that let us test whether models can integrate distributed\nknowledge. The Methods section explicitly acknowledges we're testing synthesis capability\nunder 'optimal information access conditions' rather than simulating historical research\nconstraints.\nComment 2: Single case study with broad extrapolation\nSame issue as Reviewer 1's fifth comment. We've narrowed all claims considerably,\nrepositioned the paper as a methodological contribution, and proposed systematic validation\nacross multiple cases.\nComment 3: Limited evidence that key documents were actually retrieved\nThis was an important gap in our analysis. The reviewer rightly points out that we couldn't\nrule out retrieval bias as an explanation for the null fluidization result, maybe the relevant\nWinkler and Odell patents simply weren't being accessed, although we rechecked the code\nand they were accessed.\nTo clarify this the revised manuscript includes comprehensive RAG retrieval logging (new\nTable 2). At baseline, cross-domain documents were rarely retrieved: Winkler patents in only\n11% of trials, Odell patents in 6%. But when we provided grain silo analogical guidance,\nretrieval rates jumped to 85% and 90% respectively. So the documents were definitely\navailable and retrievable when the semantic queries were properly primed.\nMore critically, the steering experiments show that retrieval bias can't explain our findings.\nUnder Layer 18 steering at α=4.5, the model continued to retrieve cross-domain documents at\nhigh rates (85-90%), but synthesis failed completely (0/20). The model was accessing the\nrelevant knowledge like Winkler's descriptions of particle suspension in upward gas flow,\nOdell's fluidization patents, but couldn't integrate this information when the Layer 18\nsemantic representations were ablated. This dissociation proves the limitation isn't just\nretrieval bias.\nComment 4: Single-query design doesn't capture iterative exploration\nTrue, and we acknowledge this as a deliberate design choice. Real research involves iterative\nrefinement, following up on partial results, exploring subproblems, adjusting search\nstrategies. Our single-query approach doesn't capture any of that.\nHowever, this design serves our specific research goal: testing synthesis capability when the\ncomplete problem is presented up front. We're not trying to measure research productivity in\nrealistic settings; we're trying to understand what conditions are necessary versus sufficient\nfor cross-domain synthesis.\nThe systematic guidance experiments support this approach. We tested four different prompt\ntypes under steering conditions (N=80 total): explicit grain silo analogies, detailed chain-of-\nthought scaffolding, generic CoT, and minimal prompts. All failed uniformly (0/80 successes,\n0%). This shows that even highly structured prompting can't overcome representational\ndeficits, which tells us something important about the architectural requirements for\nsynthesis.\nComment 5: Pretraining leakage not fully addressed\nThis overlaps with Reviewer 1's first comment. The steering experiments provide the key\nevidence here. If models were expressing memorized FCC knowledge using alternative\nvocabulary, then Layer 18 intervention should affect linguistic production and semantic\ncontent similarly. Instead we see a clean dissociation: retrieval patterns stay intact (85%\ncross-domain access) while synthesis is eliminated (0/20). This demonstrates genuine\nsynthesis attempts from retrieved historical knowledge rather than sophisticated rewriting of\nparametric FCC memories.\nReviewer #3\nComment 1: Need for post-1936 corpus validation control\nThe reviewer suggests creating a parallel corpus with post-1942 FCC literature to\nquantitatively demonstrate that our temporal constraints work. While we didn't implement\nexactly this control, the representation engineering approach actually provides stronger\nvalidation than the suggested experiment would have.\nHere's why: if models were accessing post-1936 FCC knowledge from their pretrained\nparameters, Layer 18 steering shouldn't selectively eliminate synthesis while preserving\nretrieval. The fact that it does, retrieval stays at 85% for cross-domain sources, synthesis\ndrops to 0%, proves the responses depend on semantic processing of retrieved historical\ncontent, not on internal recall of modern FCC descriptions. Combined with zero\nanachronisms across all 250 trials (95% CI: 0-1.5%) and zero mentions of FCC-specific\narchitectural concepts, this thoroughly validates the temporal constraint\nComment 2: Human-AI collaboration framework lacks quantification\nThe reviewer is right that our original manuscript proposed human-AI collaboration without\nrigorously quantifying its benefits. The revised version addresses this within the scope of\nwhat we can measure from our experimental design, though we still don't have the human-\nexpert comparison the reviewer ideally wants.\nWhat we can now quantify: Human analogical guidance produces a 10-fold amplification in\nsuccess rate (10% autonomous to 100% with grain silo guidance, p<0.001). The RAG\nretrieval logs show mechanistically how this works, guidance shifts document access from in-\ndomain petroleum sources (90% at baseline) to cross-domain precedents (85% Winkler, 90%\nOdell), with large effect sizes.\nThe steering experiments further establish that three components must work together: (1)\nRAG providing cross-domain knowledge access, (2) human guidance activating relevant\nretrieval pathways, and (3) intact semantic representations in Layer 18. Remove any one\ncomponent and synthesis fails, the universal failure across all guidance types under steering\n(0/80) demonstrates this cleanly.\nThe revised manuscript does not include human-expert comparison, which we acknowledge\nas an important direction for future research\nComment 3: Single case limits generalization\nSame concern as Reviewers 1.5 and 2.2. We've bounded our claims to cross-domain\nsynthesis specifically, repositioned the contribution as methodological, acknowledged\nlimitations explicitly, and proposed a systematic validation framework with specific\ncandidate cases.\nKey Changes in the Revised Manuscript\nFundamental reconceptualization.\nWe shifted from asking 'Could LLMs have accelerated FCC in 1935?' to 'What architectural\ncomponents enable guided cross-domain synthesis?' This moves away from historical\ncounterfactual claims toward rigorous experimental investigation of reasoning mechanisms.\nNew experimental methods.\nThe revised manuscript introduces representation engineering through Layer 18 steering\nvectors (N=100 experiments with α ranging from 0 to 9.0), systematic guidance experiments\ncomparing four prompt types under controlled conditions (N=80), and comprehensive RAG\nretrieval logging to document knowledge access patterns quantitatively.\nImproved measurement.\nWe eliminated the problematic cosine similarity and PCA novelty gap metrics. The new\nanalysis uses categorical classification with strong inter-rater reliability (κ=0.89),\nrigorous statistical testing (Fisher's exact, exact binomial confidence intervals), and quantitative\nevidence showing guidance substantially increased cross-domain document retrieval\n(Winkler patents: 11% baseline to 85% with guidance; Odell patents: 6% to 90%, both\np<0.001).\"\nKey empirical findings.\nThree main results emerge: (1) 10-fold amplification from human guidance (10% to100%,\np<0.001), (2) steering-induced dissociation showing synthesis depends on Layer 18\nrepresentations even when retrieval remains intact (85% cross-domain access, 0% synthesis\nsuccess), and (3) sharp performance threshold at α=4.5 revealing architectural brittleness\nwhere intervention strength exceeds representational resilience.\nAppropriately scoped claims.\nAll conclusions are now bounded to cross-domain synthesis requiring analogical transfer. We\nremoved unsupported acceleration claims, repositioned the contribution as introducing a\ngeneralizable methodological framework, added a comprehensive Limitations section, and\nproposed systematic validation across multiple historical cases.\nThe transformation from our original submission to this revision was substantial, driven\nentirely by the reviewers' insightful critiques. We believe the result is a much stronger\nmanuscript that makes genuine methodological and empirical contributions to understanding\nhow guided reasoning works in large language models."}, {"round": 2, "reviewer_comments": {"reviewer_2": "Overall, the rebuttal describe a stronger paper. The reframing is appropriate, the evaluation is more defensible, and the\nsteering + retrieval‑logging experiments are potentially a improved contribution to understanding of guided synthesis. The\ndetailed, original comments are not included. Instead the summarized bullet are shown. Reviewer lost the original context.\nHope the response changes are included explicitly without the need to go back manuscript for verification.", "reviewer_3": "The authors have addressed all the points I raised in reviewing the manuscript submitted to Nature Communications."}, "author_response": ""}]

Decision letter