Gate tuning of coupled electronic and structural phase transition in atomically thin Ta_2NiSe_5

Domain: physics
Published: 2025-12-09
Editorial outcome: major_revision

Abstract
Realizing an excitonic insulator phase from narrow-gap semiconductors remains challenging, as unambiguous experimental signatures are difficult to establish. Ta2NiSe5 has been widely regarded as a leading candidate, yet the nature of its phase transition and insulating state remains controversial. Here, we report a systematic Raman spectroscopy study of Ta2NiSe5 as a function of thickness and field-effect doping, complemented by electrical transport measurements. The phase transition persists down to the monolayer limit, with the critical temperature increasing as thickness decreases. In bilayer samples, both electron and hole dopings suppress the insulating state, with electron doping lowering and hole doping raising the transition temperature. Importantly, the quasi-elastic scattering, previously attributed to excitonic fluctuations, evolves monotonically across the entire doping range, inconsistent with the expected suppression of excitonic correlations by Coulomb screening. These findings rule out a dominant excitonic mechanism and instead point to a coupled electronic and structural phase transition, whose stability is tunable by carrier doping. Our doping-based approach offers a general strategy for evaluating the role of excitonic effects in candidate excitonic insulators.

Introduction
In semiconductors or semimetals, unscreened Coulomb interaction can lead to the formation of bound electron-hole pairs known as excitons. If the exciton binding energy exceeds the band gap or band overlap, these composite bosons may condense at low temperature into a macroscopic quantum state—an excitonic insulator (EI)1–5. As a distinct type of correlated insulator, the EI has attracted sustained interest both as a platform to study electronically driven phase transitions and as a potential host for macroscopic quantum coherence, including superfluid-like behavior6,7. While significant advances have been made in engineered bilayer systems with spatially separated electrons and holes8–18, realizing an EI phase in natural bulk crystals remains experimentally challenging. In addition to the scarcity of candidate materials, a major obstacle is the lack of unambiguous experimental signatures unique to exciton condensation19–27. A key complication in bulk EI candidates is the ubiquitous presence of electron-phonon coupling, which makes it difficult to isolate the electronic origin of the insulating ground state. For example, in the prototypical EI candidate 1T–TiSe2, exciton condensation has been proposed to drive the emergence of a charge-density wave (CDW)19,20. However, the CDW ordering wavevector coincides with a lattice distortion involving a soft phonon mode, making it difficult to disentangle electronic and lattice contributions28,29. In contrast, Ta2NiSe5 is a direct-gap semiconductor, avoiding complications from finite-wavevector CDWs. Several experimental works support an EI state in this material, including the flattening of the valence band top revealed by angle-resolved photoemission spectroscopy (ARPES)30,31, a dome-shaped electronic phase diagram centered at zero band gap32, optical signature of exciton-phonon coupling33, and ultrafast optical modulation of the insulating state34–37. Nevertheless, a structural phase transition also occurs in this material38–40, raising the possibility that the insulating state is of structural rather than electronic origin41–44. Some theoretical works suggested that exciton condensation45 or even an excitonic instability46 can drive this structural transition, whereas others showed that the structural transition, rather than the excitonic effect, is required to open the experimentally observed band gap42,43,47,48. This entanglement of electronic and structural effects is not unique to Ta2NiSe5 and continues to pose a challenge for the study of EIs.

In this work, we propose an experimental approach to assess the excitonic nature of an insulating state by tracking its evolution under carrier doping. Since excitons are central to the EI state, and since carrier doping enhances Coulomb screening, possible hallmarks of an EI should be suppressed by both electron and hole doping. This anticipated doping dependence serves as a useful criterion for evaluating the role of excitonic effects. We implement this approach in atomically thin Ta2NiSe5, employing field-effect gating to achieve continuous and reversible doping. By monitoring the Raman response, we find that the quasi-elastic scattering (QES), previously attributed to excitonic fluctuations, evolves monotonically with doping across a wide range on both the electron and hole sides. This observation is inconsistent with expectations from an excitonic mechanism but supports a scenario in which the insulating state originates from a structural phase transition driven by electron-phonon coupling47 and tunable via carrier density.

Results
Coupled electronic and phononic excitations
Bulk Ta2NiSe5 consists of monolayers stacked along the b-axis via van der Waals interactions (Fig. 1a), crystallizing in an orthorhombic structure (space group Cmcm) at high temperature49,50. Within each monolayer, Ta and Ni atoms are coordinated by Se atoms in approximately octahedral and tetrahedral geometries, respectively, forming chain-like motifs along the a-axis. Below Tc = 328 K, the system undergoes a second-order phase transition into a monoclinic phase (space group C2/c), associated with a putative EI state. The two Ta chains flanking each Ni chain exhibit shear-like displacements in opposite directions (see arrows in Fig. 1a), breaking the in-plane mirror symmetries and giving rise to an antiferroelectric or ferro-rotational order38–40.

This phase transition is clearly manifested in the temperature-dependent Raman spectra (Fig. 1b, c), which show drastic changes both in the low-energy QES and in selected phonon modes. Near Tc, the QES is strongly enhanced and hybridizes with the phonon mode at approximately 70 cm−1, and the coupling of these two features is enabled by their compatible symmetry (B2g in the orthorhombic phase and Ag in the monoclinic phase)51–53, as further confirmed by polarization-angle dependent Raman measurements (Fig. 1d). Following ref. 51, we label the phonon modes numerically in order of increasing frequency. The crossed polarization (ac) configuration is adopted with the polarization angle that maximizes the intensity of the QES and mode 2, as they are particularly sensitive to the phase transition. Indeed, both mode 2 and mode 5 have been shown to exhibit shearing vibration of the Ta atoms along the chain direction, which matches the form of the monoclinic lattice distortion47.

QES was previously observed in iron pnictide superconductors54 and magnetic materials55–57, where it was attributed to charge nematic fluctuations and magnetic fluctuations, respectively. In Ta2NiSe5, however, since magnetic order is absent50, no evidence of electronic nematicity was reported, and all expected Raman-active phonon modes can be accounted for51, the QES was attributed to an electronic origin51,52. The QES exhibits a Lorentzian lineshape centered at zero frequency, resembling that observed in nematic and magnetic systems, which motivates a similar analysis framework54–57. The measured Raman intensity I(ω) is converted to the Raman susceptibility, χ″(ω) ∝ I(ω)/[n(ω, T) + 1], where n(ω, T) is the Bose-Einstein distribution at temperature T. The Raman conductivity, defined as χ″(ω)/ω, is modeled as a Lorentzian function centered at zero frequency, σ0 Γ/(ω2 + Γ2), where σ0 represents the spectral weight and Γ is the half-width of the QES peak. The coupling between the QES and phonon mode 2 leads to a Fano lineshape for the latter, modeled by the expression A (ϵ + q)2 / (1 + q2) (ϵ2 + 1), where ϵ = (ω − ω0)/γ, and A, ω0, and γ denote the amplitude, frequency, and half-width of the phonon mode, respectively. The parameter 1/|q| quantifies the coupling strength between the phonon and the QES. A model incorporating both the Lorentzian and the Fano terms provides an excellent fit to the spectra at all temperatures where QES is present. A representative fit at 300 K is shown in Fig. 1e, with results at other temperatures provided in Supplementary Note 1. This analysis offers a robust method for quantitatively tracking the evolution of both the QES and mode 2 under varying experimental conditions.

Thickness dependence
Atomically thin Ta2NiSe5 samples were obtained by mechanical exfoliation. The number of layers was determined based on optical contrast and verified using atomic force microscopy (Supplementary Note 2). The frequency of mode 5 varies systematically with thickness (Fig. 1f), assisting the identification of the layer number. The persistence of sharp phonon peaks in samples down to the monolayer limit indicates good sample quality, in contrast to an earlier study where Raman signals were undetectable for samples thinner than four layers, likely due to degradation58.

We performed systematic temperature-dependent Raman measurements on samples with varying thickness. Figure 2a–d reveals that the phase transition persists down to the monolayer limit. Notably, Tc increases with decreasing thickness, as indicated by the temperature at which the QES reaches its maximum intensity and by the coexistence of modes 4 and 5 persisting to higher temperatures (see Supplementary Note 3 for mode analysis). Analysis of the QES and mode 2 using the method described above yields the parameters shown in Fig. 2e–i. For the bulk sample, the QES spectral weight σ0 is maximized at the Tc determined from resistance measurements32,50, strongly supporting its electronic origin. This maximum is retained in atomically thin samples and is used to extract a thickness-dependent Tc by fitting the QES temperature dependence above 250 K to a Gaussian function, which shows a systematic enhancement as the thickness decreases (blue circles in Fig. 2j). The monolayer sample exhibits enhanced background scattering when heated above 400 K, likely due to laser-induced degradation at elevated temperatures. Its spectral weight for the QES at high temperature and Tc cannot be reliably analyzed for this reason. The data shown in Fig. 2d have been corrected for this background contribution, as detailed in Supplementary Note 4. This issue was not observed in bilayer or thicker samples.

Fitting parameters associated with mode 2, including its amplitude A, frequency ω0, linewidth 2γ, and Fano asymmetry 1/|q| are shown in Fig. 2f–i. The amplitude shows a minimum, whereas the other three parameters show a maximum near Tc. These trends are consistent with previous bulk measurements51. The characteristic temperatures corresponding to the extrema in the parameters systematically increase as thickness is reduced, as marked by arrows in Fig. 2f–i and summarized in Fig. 2j, corroborating the enhancement of Tc in atomically thin samples. This finding contrasts with an earlier report of Tc reduction upon decreasing thickness58. Quantifying Tc from mode 2 parameters, however, is less reliable, as they show different characteristic temperatures even for the same sample. This could be due to the additional effect of lattice anharmonicity47, which becomes enhanced at high temperature and near structural phase transitions. Another source of discrepancy between the different criteria for determining Tc using mode 2 parameters arises from the data analysis itself. In principle, this could be remedied by employing a fully self-consistent analysis that couples the QES and the bare phonon52. However, for Ta2NiSe5, such an approach has been shown to have only a minimal influence on the analyzed results51. We further performed resistance measurements on samples with varying thickness (Supplementary Note 5). The kink in the temperature derivative of the resistance as an indication of the phase transition58 was observed in all measured samples, although it becomes weaker in atomically thin samples. The corresponding Tc values are plotted in Fig. 2j as open symbols, which confirm the increased Tc as the sample thickness decreases.

Effect of doping on electrical transport
Having established the phase transition in atomically thin samples, we now investigate how it evolves under carrier doping. To this end, we integrated bilayer Ta2NiSe5 into a dual-gate field-effect transistor configuration, as illustrated schematically in Fig. 3a. A typical device image is shown in Supplementary Note 5. Two-probe resistance measurements were performed at 150 K while sweeping both the top-gate voltage (Vt) and bottom-gate voltage (Vb). As shown in Fig. 3b, the resistance peaks along a line corresponding to charge neutrality. This charge-neutrality line is slightly offset from the origin, possibly due to extrinsic doping. To focus on the effect of pure carrier doping, we follow the Vt–Vb relation that ensures cancellation of the gate-induced electric field (see “Methods”), as marked by the arrowed line in Fig. 3b. Both electron and hole doping progressively suppress the insulating state, as seen in the resistance versus doping curve in Fig. 3c. Here, n denotes the sheet carrier density, with positive (negative) values representing electron (hole) doping. The resistance exhibits a maximum at charge neutrality, which persists to high temperature but becomes thermally broadened. The maximum doping achieved amounts to approximately 0.06 e/unit cell, estimated using the applied gate voltages and the thickness of the hexagonal boron nitride (hBN) dielectric layers.

The temperature dependence of resistance at selected hole doping densities is shown in Fig. 3d, revealing insulating behavior across the accessible doping range. The electron-doped side exhibits qualitatively similar behavior. An Arrhenius plot of the resistance (Fig. 3e) reveals two distinct thermal activation regimes. By fitting the temperature-dependent resistance to R ∝ exp(Δ/2kBT), where kB is the Boltzmann constant, we extract two energy gaps: a larger gap, Δ1, which dominates above 250 K, and a smaller gap, Δ2, which governs transport at lower temperatures. Their doping dependence is summarized in Fig. 3f. While Δ1 is strongly doping-dependent and decreases with increasing carrier density, Δ2 remains largely unchanged above 1013 cm−2 for both electron and hole doping. The smaller gap likely arises from defect-related in-gap states, consistent with prior scanning tunneling spectroscopy (STM) studies of exfoliated Ta2NiSe58. The values of Δ1 = 0.3 eV and Δ2 = 0.1 eV at charge neutrality are comparable to those reported in bulk samples32,58,59.

At first glance, the suppression of insulating behavior by both electron and hole doping is consistent with the screening of an excitonic gap by doped carriers21. However, such ambipolar doping dependence is not unique to EIs. For instance, in Bernal-stacked bilayer graphene featuring a non-excitonic band gap opened by an electric field, the insulating state is similarly suppressed by both electron and hole doping60. In that case, the evolution of the activation gap is effectively associated with a shift in the chemical potential. In bilayer Ta2NiSe5, the persistence of Δ1 up to doping levels of 1013 cm−2, along with the gradual slowing of its suppression at higher doping, suggests that the gap formation is unlikely to be dominated by excitonic effects47.

Effect of doping on Raman scattering
To further elucidate the effect of carrier doping, we examine the corresponding evolution in the Raman spectra. Figure 4a–d shows doping-dependent Raman conductivity maps at selected temperatures, with representative spectra displayed in Fig. 4e–h. At 150 K, the QES is nearly absent at zero doping but is induced by electron doping. At 300 K, QES is already present at charge neutrality and becomes clearly enhanced (suppressed) by electron (hole) doping. At 350 K, close to the Tc estimated earlier, the doping dependence of the QES weakens. At 400 K, the QES exhibits a doping dependence opposite to that observed at lower temperatures. The doping tunability of the QES affirms its electronic origin. Previous Raman studies attributed the QES at high temperatures to excitonic fluctuations51,52, an interpretation supported by theoretical predictions61 and ARPES evidence for pre-formed excitons above Tc62. However, our results challenge this view. If the QES indeed originated from excitonic fluctuations, both electron and hole doping would be expected to suppress it, as enhanced Coulomb screening weakens the excitonic bound states. Contrary to this expectation, we observe a monotonic doping dependence, inconsistent with the excitonic fluctuation scenario.

Recent x-ray scattering experiments revealed pronounced diffuse scattering along the inter-chain direction in bulk Ta2NiSe5, peaking at Tc and persisting over a broad temperature range, which suggests dynamic lattice fluctuations associated with an inter-chain shear mode that freeze into a static distortion in the monoclinic phase43. These fluctuations enable hybridization between Ta-derived conduction and Ni-derived valence bands, otherwise symmetry-forbidden in the orthorhombic phase63. Resonant inelastic x-ray scattering further supports this picture by revealing substantial Ta-Ni orbital hybridization above Tc41, which can manifest as a pseudogap in the orthorhombic phase, as indicated by ARPES43 and infrared spectroscopy33 measurements. The QES intensity in our Raman data also peaks at Tc, indicating a direct link to such fluctuations and justifying using the QES to quantify them, analogous to its application for quantifying charge nematic fluctuations in iron-pnictide superconductors54.

More systematic doping-dependent Raman measurements were performed at a finer temperature step in another bilayer device, which allows us to extract the temperature dependence of the QES spectral weight at different doping levels. The results, shown in Fig. 5a, are consistent with the doping dependence at selected temperatures described above. By fitting the temperature dependence of the QES above 250 K to a Gaussian function (Fig. 5b), Tc is quantified as the center of the Gaussian peak, and its doping dependence is shown in Fig. 5c. We estimate an enhancement of 5 K and a reduction of 34 K under the maximum hole and electron doping of 2.6 × 1013 cm−2, respectively. The reduction of Tc under electron doping is consistent with previous potassium dosing studies and DFT calculations, which show that electron doping tends to stabilize the orthorhombic phase43. The doping dependence of the QES at selected temperatures, as shown in Fig. 4, can therefore be attributed to the doping-induced modification of Tc, which shifts the peak in the temperature dependence of the QES spectral weight (Fig. 5a, b). The spread of the QES temperature dependence, quantified as the full width at half maximum of the Gaussian peak, grows as Tc decreases (Fig. 5c), indicating a more extended fluctuation regime under electron doping.

Phonon mode 2 also exhibits clear doping dependence, especially below Tc (Fig. 4). At 150 K, where the QES is weak, mode 2 softens and broadens with doping (Supplementary Note 6), which could be due to doping-induced enhancement of either the electron-phonon coupling or the electronic density of states at the chemical potential. Above ~250 K and below Tc, when the QES becomes more prominent, the doping dependence of mode 2 parameters becomes monotonic (Fig. 5d–f). Electron doping reduces the mode amplitude (Fig. 5e) but increases its frequency (Fig. 5d), 1/|q| (Fig. 5f) and linewidth (Supplementary Note 6), resembling the effect of increasing temperature just below Tc (Fig. 2). A similar trend is observed for mode 5 (Supplementary Note 6), further corroborating electron-doping-enhanced lattice fluctuations when approaching Tc from below. Both modes are sensitive to the structural phase transition and dominated by Ta atomic vibration47. They couple efficiently with electronic states at the bottom of the conduction band formed by Ta-5d orbitals45, explaining the more significant doping effect on the electron-doped side (see more discussions in Supplementary Note 6).

Methods
Sample and device preparation
Ta2NiSe5 single crystals were synthesized using the flux method. Freshly cleaved surfaces of the bulk crystals were used for Raman characterization. Thin flakes of Ta2NiSe5 were exfoliated from the bulk crystal onto polydimethylsiloxane (PDMS) substrates and identified based on optical reflection contrast, performed in an argon atmosphere within a glovebox. The exact thickness was determined through a combination of optical contrast analysis, atomic force microscopy (AFM), and Raman spectroscopy (see Supplementary Note 1). For Raman measurements, the flakes were encapsulated by hexagonal boron nitride (hBN) layers with a thickness of 5–15 nm on both sides. Stacks were assembled by using a polycarbonate (PC) film on a PDMS stamp to sequentially pick up each material. Afterward, the stacks were released onto a Si/SiO2 substrate at 200 °C, and immersed in chloroform for 10 min and isopropanol for 5 min to remove the PC film.

Dual-gate field-effect transistors were fabricated for doping-dependent studies using the same exfoliation and transfer techniques. First, Ti/Au electrodes were patterned on Si/SiO2 substrates via direct write laser lithography and thermal evaporation. Then, graphite electrodes (serving as contacts for Ta2NiSe5), hBN gate dielectric, and the bottom-gate graphite electrode were sequentially picked up and released onto the Ti/Au electrodes. The stack was cleaned using an AFM in contact mode, with a contact force of 5–20 nN. Finally, the top-gate graphite electrode, hBN gate dielectric, and Ta2NiSe5 flake were picked up and transferred onto the bottom-gate stack.

Characterizations
Raman spectroscopy was conducted in the back-scattering geometry with 532 nm laser excitation. An incident power of 0.2 mW was used for all samples, which was confirmed to cause a negligible heating effect. The scattered light passed through Bragg notch filters before being collected by a grating spectrograph and a liquid-nitrogen-cooled charge-coupled device. Measurements were performed in a cryostat within the 150–350 K range and in a heating stage between 300–550 K. To ensure consistency between datasets, spectra in the overlapping 300–350 K range were compared and calibrated (see Supplementary Note 8). Polarization-angle dependent measurements were carried out using two polarizers and a half-wave plate. Two-probe resistance was measured with a lock-in amplifier, applying an excitation current of 10 nA. Gate voltages were applied using source meters. Pure doping is achieved by applying gate voltages along the arrowed line shown in Fig. 3b. The doped sheet carrier density is calculated as n = ϵrϵ0 (Vt/dt + Vb/db)/e, in which the dielectric constant of hBN is ϵr = 3, ϵ0 is the vacuum permittivity, e is the electron charge, Vt and Vb are the top and bottom gate voltages, respectively, and dt and db are the thickness of the corresponding hBN dielectric layers. The electric field along the charge neutrality line shown in Fig. 3b is E = (Vt/dt − Vb/db)/2.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "This manuscript discuss an important aspect of the complex intertwined charge and lattice driven phase transition in\nTa2NiSe5 using systematic Raman spectroscopy and transport studies. The thickness dependence of the transition\ntemperature in Ta2NiSe5 has been reveled in the temperature dependence of the quasi-elastic scattering (QES) and\nphonon mode 2 of the Raman response. In the electric transport measurements, they revealed the gradual suppression of\nthe insulating gap of the Ta2NiSe5 with the electron and hole doping, which survive till high carrier doping. More crucially,\nby tracking the Raman response of the QES and phonon mode 2 with tuning the carriers (electron and hole) density above\nand below the phase transition, the authors revealed the predominance of the structural instability by electron-phonon\ncoupling as driving force of the insulting phase transition, where the QES scattering exhibit monotonic doping dependence.\nThe identification of the instability driving the phase transition is indeed challenging, particularly in the scenario, where the\ntwo symmetrically identical order parameter condenses simultaneously. However, the temperature dependence of the QES\nin Raman measurement across the transition temperature has been previously studied in detail in Ref. 51, the experiment\nfinding of the carrier doping dependence of the QES is quite intriguing and are of a broad interest to the condensed matter\nphysics community in revealing the excitonic insulators. Overall, this work presents significant and insightful findings to\nunderstand the excitonic insulators. To determine if the manuscript is suitable for publication in Nature communications,\nauthors need to address the following comments:\n1) In the Fig2e, author shows that QES spectral weight peak reveals the increase in the transition temperature with\ndecreasing thickness, as following the bulk QES spectral weight which roughly peak at transition temperature. The reason\nfor the increased transition temperature with decreasing thickness is not explained.\n2)Also, why the temperature dependence of the self-energy parameters of the mode 2 shows different transition\ntemperatures for same thickness, even though authors suggest uncertainties in transition temperature in Fig2j, what induced\nuncertainties in the self-energy parameters of same mode is not clear.\n3) In Fig4i-k, at 150K, why the phonon mode 2 softens with hole and electron doping with nearly equal magnitude, while the\nbroadening is pronounced only in electron doping.\n4) In addition, the doping dependence trend of the QES spectral weight is nearly similar for 150K and 300K (Fig4i), while the\nmode 2 frequency shows different temperature dependence at 150K and 300K. Also, at 400 K, mode 2 frequency does not\nfollow the doping dependence of the QES spectral weight, showing opposite behavior.\n5) In Fig5d,f, the enhancement of the transition temperature with hole doping is not clearly reflected, the peak center is within\nthe error with respect to undoped peak. Also, why the asymmetry parameter (1/|q|) of mode 2 shows (Fig5f) peak at higher\ntransition temperature than observed in the QES spectral weight (Fig5d), even though they exhibit coupling.", "reviewer_2": "The manuscript of Wei et al. reports a combined Raman and transport study of the excitonic insulator candidate TNS as a\nfunction of layer thickness and doping. The results are discussed in light of the possible scenario for the origin of the semi-\nmetal to insulator transition observed in this material. The results are interesting and present a significant advance towards a\nfuller understanding of this quite unique material. However there are several points which need to be clarified and answered\nbefore I can make a clear recommandation.\n- The analysis of the QES performed in ref. 51-52 and 54-55 was inspired by the previous analysis of nematic fluctuations by\nRaman scattering in Fe superconductors which more closely ressemble the case of TNS. I believe that the original work\nshould be properly cited (Gallais et al. 10.1103/PhysRevLett.111.267001)\n- The analysis in terms of QES + Fano lineshape Fano is not completely convincing. Indeed the Fano lineshape is due the\nQES continuum and a full self-consistent analysis should be performed as done in ref. 52.\n- The actual polarization configuration is not mentioned for most of the spectra in the manuscript. This should be specified\neach time to allow comparison with earlier works.\n- While there is a clear trend, the various markers of Tc based on QES and mode 2 analysis shown in figure 2 yield quite\ndifferent values of Tc for each thickness. This is a bit odd and casts some doubts on the analysis especially since the\ndifferent values of Tc are way beyond the errors bars. Can the authors justify this ?\n- Rather than relying on the QES and mode 2 which strongly couple, rendering their fitting somewhat not trivial, it might be\nmore accurate to rely on the activation of phonon modes accros Tc as discussed in ref. 51. For exemple Ag modes become\nactivated in cross polarization \"ac\" geometry in the monoclinic phase as done by Volkov et al.\n(10.1103/PhysRevB.104.L241103). A similar analysis would be desirable also for the effect of doping in figure 5. This can\nbe done using the data shown in the SM (fig. 4) which seems more convincing to me to track Tc.\n- One potential issue of the data is the laser heating which could be thickness dependent. Did the author estimate the laser\nheating ?\n- The behavior of the Raman spectra as a function of doping is not trivial with a strong electron-hole asymmetry that does not\nseem to track the transport data in an obvious way. In particular it is not clear to me whether the effects are due to a simple\nincreased carrier density or to a change in Tc upon doping. Figure 5 suggests that the change in Tc is the dominant factor\nbut since both set of data are discussed separately it is difficult for the reader to get what the main message is on that point. It\nseems to me that most of the data in figure 4 can be explained by a change in Tc (except maybe at 400K).\n- While I agree that the data presented here seems at odd with a naive picture of an excitonic insulator in the sense of an\ncondensation of preformed exciton, I do not think we can conclude that the transition is driven by a lattice distortion as\nconcluded by the authors. Indeed as shown by Mazza et al. (ref. 46) the transition could be driven by an electronic instability\nthat breaks the lattice symmetry without specifically involving the condensation of actual excitons. In that case the effect of\ncarrier doping and thickness are less obvious. I would urge the authors to soften their claim on that aspect.\n- In the discussion the whole paragraph on page 8 \"Its enhacement under electron doping.... and a return toward the\ninstablity\" does not make much sense to me. Do the authors really have an idea of the observed different behavior under\nelectron and hole doping ? If not I would suggest the authors to leave as that.\n- In the discussion on page 8 the authors state that the softening and broadening of mode 2 with doping is \"consistent with\nenhanced electron-phonon coupling in the presence of excess carriers\". This is misleading. Even with constant electron-\nphonon coupling the effect is expected just by the trivial fact the DOS at the Fermi energy increases upon doping. This is a\nwell-known effect that was observed in doped Si in 1970s. Therefore eu conclusion about electron-doping enhanced lattice\nfluctuations is incorrect.\n- On page 8 the authors seem to challenge the view that the QES is of electronic origin hinting at a possible involvement of\nspeculative lattice fluctuations (?). However on page 9 they seem to revert to an interpretation in terms of electronic\nfluctuations. Can the author clarify where they stand on that point ?", "reviewer_3": "Referee Report\nThe manuscript by Keyu Wei et al. presents a comprehensive Raman spectroscopy study on Ta₂NiSe₅, a well-known\ncandidate material for excitonic insulator (EI) phases. One of the central open questions in this field is whether the insulating\nstate observed in the low-temperature phase of this compound is primarily driven by exciton condensation or by a structural\nphase transition. The authors successfully demonstrate that the essential properties of bulk Ta₂NiSe₅ persist down to the\natomically thin limit, and they provide the systematic evidence that the critical temperature increases monotonically as the\nsample thickness decreases. This is valuable information for understanding enhanced fluctuations and electron–lattice\ncoupling in reduced dimensions. Particularly notable is the asymmetric doping dependence of the quasi-elastic scattering\n(QES), which offers a new perspective on the nature of the phase transition. These new insights were made possible by the\nfabrication of high-quality dual-gated hBN-encapsulated devices, allowing for clean electron and hole doping, and by\nsystematic Raman measurements with quantitative fitting across a wide range of temperatures and doping levels.\nDespite the significance of these results, I have the following concerns, and in its current form, I cannot recommend this\nmanuscript for publication in Nature Communications.\nMajor Concerns\n1. Discrepancy in the thickness dependence of the transition temperature\nAs the authors mention, their study finds that the structural transition temperature increases with decreasing thickness,\nwhereas previous work [Kim, S. Y. et al. ACS Nano 10, 8888–8894 (2016).] has reported the opposite trend. In this\nmanuscript, the authors determine the transition point based on the temperature dependence of Raman-active phonon\nmodes (i.e., structural changes), while in earlier studies, the transition was estimated from transport measurements, which\nreflect gap formation more directly. If both results are valid, this implies a decoupling between the structural transition and the\nelectronic gap opening. In fact, this would support the view that the structural transition does not fully account for the gap\nformation and that excitonic condensation may still play an essential role. Therefore, the discrepancy could be interpreted as\nstrong evidence for the excitonic scenario. That said, differences in sample quality may also contribute. Since the authors’\ndevices are of clearly superior quality, they should also perform electronic transport measurements (which appear feasible\nwithin their current setup) to extract the transition temperature in the same samples used for Raman measurements. This\nwould help clarify the nature of the transition and reconcile conflicting reports in the literature.\n2. Overstatement of the exclusion of the excitonic mechanism\nThe manuscript tends to overstate the claim that the observed phenomena rule out excitonic condensation as the driving\nforce behind the phase transition. The argument is largely based on the asymmetric doping dependence of the QES, which\nindeed challenges a simple excitonic fluctuation scenario. However, the Raman data primarily probe structural aspects of\nthe transition and provide limited insight into the evolution of the electronic gap itself. Thus, the authors’ interpretation only\ndirectly addresses the structural transition, and the claim that excitonic effects are not essential seems too strong. Many\nprevious studies—including ARPES, pump-probe, and theoretical works—have supported the importance of excitonic\nphysics in Ta₂NiSe₅. A more balanced conclusion would be that the transition cannot be explained solely by excitonic\ncondensation, and that coupled excitonic and structural effects are likely at play. I recommend the authors revise statements\nthat deny the role of excitons entirely or claim structural order as definitively dominant.\n3. Physical mechanism of the Fano interference\nThe manuscript presents Fano interference between the QES and phonon mode 2, modeled with a standard Fano\nlineshape. However, if the QES is not of excitonic origin, the microscopic origin of this interference becomes unclear. Is the\ncoupling mediated via free carriers? Can it arise solely from phonon–phonon interactions? A brief discussion of possible\nmechanisms is needed to justify the presence of the Fano effect in the absence of excitonic fluctuations.\nMinor Comments\n•The Ta₂NiSe₅ flakes are encapsulated between two hBN layers. It would be helpful to clarify how the Raman excitation and\nscattered light pass through the structure. Is there an opening on the top hBN layer, or is the hBN thin enough to allow\noptical access? Including a real or schematic image of the device would greatly aid clarity.\n•In the equation for the displacement field D, it seems that the vacuum permittivity ε0 is missing. Please confirm whether it\nshould be included."}, "author_response": "Note: Reviewers’ comments are shown in blue, and our reply in black. Main changes to\nthe manuscript are highlighted in dark red. The reference numbers refer to those in the first\nversion of our manuscript.\nResponse to Reviewer #1\n——————————————————————————————————–\nThis manuscript discuss an important aspect of the complex intertwined charge and lat-\ntice driven phase transition in Ta NiSe using systematic Raman spectroscopy and transport\n2 5\nstudies. The thickness dependence of the transition temperature in Ta NiSe has been rev-\n2 5\neledinthetemperaturedependenceofthequasi-elasticscattering(QES)andphononmode2\nof the Raman response. In the electric transport measurements, they revealed the gradual\nsuppression of the insulating gap of the Ta NiSe with the electron and hole doping, which\n2 5\nsurvive till high carrier doping. More crucially, by tracking the Raman response of the QES\nand phonon mode 2 with tuning the carriers (electron and hole) density above and below\nthe phase transition, the authors revealed the predominance of the structural instability by\nelectron-phonon coupling as driving force of the insulting phase transition, where the QES\nscattering exhibit monotonic doping dependence.\nTheidentificationoftheinstabilitydrivingthephasetransitionisindeedchallenging, par-\nticularly in the scenario, where the two symmetrically identical order parameter condenses\nsimultaneously. However, the temperature dependence of the QES in Raman measurement\nacross the transition temperature has been previously studied in detail in Ref. 51, the ex-\nperiment finding of the carrier doping dependence of the QES is quite intriguing and are\nof a broad interest to the condensed matter physics community in revealing the excitonic\ninsulators. Overall, this work presents significant and insightful findings to understand the\nexcitonic insulators. To determine if the manuscript is suitable for publication in Nature\ncommunications, authors need to address the following comments:\nWe are grateful for the reviewer’s efforts in assessing our work and for the positive com-\nments on the progress achieved in elucidating the complex phase transition in Ta NiSe .\n2 5\nBelow we address each comment to improve our work.\n1) In the Fig. 2e, author shows that QES spectral weight peak reveals the increase in the\ntransition temperature with decreasing thickness, as following the bulk QES spectral weight\nwhich roughly peak at transition temperature. The reason for the increased transition\ntemperature with decreasing thickness is not explained.\nThe reason for the increased T with decreasing thickness indeed deserves some discus-\nC\nsions. Naively, this enhancement in T is consistent with the excitonic insulator scenario,\nC\nbecause the exciton binding energy should be enhanced in atomically thin samples due to\nthe reduced screening from adjacent layers, such that the excitonic insulating state should\nsurvive up to higher temperatures. However our doping dependent Raman results rule out\nthis interpretation, because in this case T should have decreased upon both electron and\nC\nhole doping due to the induced Coulomb screening, whereas our data show a monotonic\ndoping dependence across the entire doping range.\nthickness decreases, which stablizes the monoclinic phase up to higher temperatures. Re-\ngardless of whether excitonic condensation is involved in driving the transition to the mon-\noclinic phase, an enhanced electron-phonon coupling is expected to stabilize the monoclinic\ndistortion, making it more resilient to thermal fluctuations and hence enhancing the T .\nC\nSimilar enhanced transition temperature upon thickness reduction has been reported in\ncharge-density wave systems, for example, in 2H-NbSe [Nat. Nanotechnol. 10, 765 (2015)],\n2\nwhich was attributed to increased electron-phonon coupling.\nChanges: At the end of the first paragraph in the Discussion section, after the original\ntext, “These observations support a phase transition driven by entangled electronic and\nstructural instabilities, which persist to higher temperature at reduced dimensionality.” we\nhave added “This enhanced T upon thickness reduction cannot be attributed to enhanced\nC\nexcitoniceffectsinreduceddimensions,becauseotherwiseT shouldhavebeensuppressedby\nC\nboth electron and hole doping, contrary to the monotonic doping dependence observed here.\nInstead, the enhanced T could be due to increased electron-phonon coupling that stabilizes\nC\nmonoclinic lattice distortion up to higher temperatures, which remains to be substantiated\nby further experimental and theoretical studies.”\n2) Also, why the temperature dependence of the self-energy parameters of the mode 2\nshows different transition temperatures for same thickness, even though authors suggest\nuncertainties in transition temperature in Fig. 2j, what induced uncertainties in the self-\nenergy parameters of same mode is not clear.\nThis an interesting point, which was actually first reported in Ref. 51 in the Raman\nstudy of the bulk crystal [Nat Commun 12, 1969 (2021)]. In Fig. R1 we reproduce the\ntemperaturedependenceofmode2parametersfromRef.51andmakeacomparisonwithour\ndata. Overall, the results are very similar. The temperatures at which the parameters reach\nmaximum or minimum are consistent. The subtle difference could be due to the different\nforms of the Fano function and the different samples used. In Ref. 51, the observation of\nthe characteristic temperatures for mode 2 parameters higher than T was attributed to\nC\nthe overlap of the QES and mode 2 being maximized above T . However, we think this is\nC\nunlikely, because in both Ref. 51 and our work, the QES is maximized exactly at T .\nC\nIn the literature the T for bulk Ta NiSe is determined based on electrical transport\nC 2 5\nmeasurements; see, e.g., Refs. 32 and 50. The fact that the QES is maximized at exactly\nthis same temperature strongly suggests its electronic origin. Therefore in our manuscript\nwe have chosen to quantify T based on the QES. Mode 2 parameters in Fig. 2 in our\nC\nmanuscript were only intended to corroborate the enhancement of T at reduced thickness.\nC\nWe think one possible source for the different characteristic temperatures corresponding\ntothemaximumorminimuminmode2parameterscanbelatticeanharmonictiy. Thiseffect\nis indeed mentioned in the Ref. 47 [npj Comput. Mater. 7, 210 (2021)] when discussing the\nlattice properties in bulk Ta NiSe . It is enhanced at high temperature and more even so\n2 5\nclose to structural phase transitions. How exactly it renormalizes the phonon self energy is\na complicated issue on its own; see e.g., its manifestation in some other systems detailed in\n2 5\ncircles) [reproduced with permission from Nat Commun 12, 1969 (2021) under the Creative Commons\nlicense http://creativecommons.org/licenses/by/4.0/, and red solid lines are added to assist\ncomparison with the second row] and from our work (second row), shown in the same temperature\nrange. Black dashed lines mark the T .\nC\nJ. Appl. Phys. 124, 145101 (2018) and Phys. Rev. B 106, 144303 (2022). Here its interplay\nwith the QES further complicates the situation.\nChanges: In the section “Thickness dependence”, we have made the following changes.\n(1) We have modified the text to “For the bulk sample, the QES spectral weight σ is\n0\nmaximized at the T determined from resistance measurements32,50, strongly supporting its\nC\nelectronic origin.”\n(2) We have added the following text, “Quantifying T from mode 2 parameters, however,\nC\nis less reliable, as they show different characteristic temperatures even for the same sample.\nThiscouldbeduetotheadditionaleffectoflatticeanharmonicity47,whichbecomesenhanced\nat high temperature and near structural phase transitions.”\n3) In Fig. 4i–k, at 150 K, why the phonon mode 2 softens with hole and electron doping\nwith nearly equal magnitude, while the broadening is pronounced only in electron doping?\nThe details of the doping-induced changes to the phonon parameters, particularly the\nmode frequencies, are in general highly nontrivial to interpret, without assistance from\ntheoretical guidance. Here we offer our view on this point.\nTo understand the mode 2 data at 150 K, we make a comparison with the data at\nparameters for mode 2 are shown in Fig. R2. Overall, the doping dependence at these two\n3\ntemperatures is similar, being more significant on the electron-doped side. This could be\nattributed to two factors. First, mode 2 is dominated by Ta atomic vibrations; see inset\nof Fig. R2b reproduced from Ref. 47 [npj Comput. Mater. 7, 210 (2021)]. Specifically,\nit exhibits the characteristic shearing of the Ta atoms along the chain direction, which\nmatches the form of the monoclinic lattice distortion. Indeed, our Raman data show that it\nis sensitive to structural phase transition. Second, the electronic band structure of Ta NiSe\n2 5\nis asymmetric with respect to the chemical potential, with the conduction band minimum\nand the valence band maximum dominated by Ta-5d and Ni-3d orbitals, respectively, see\ne.g. Ref. 45 [Phys. Rev. B 87, 035121 (2013)]. These two factors can make modes 2 more\nsensitive to electron doping through electron-phonon coupling.\nThe effect of electron doping on the electronic and structural properties in bulk Ta NiSe\n2 5\nwas systematically studied in Ref. 43 [Phys. Rev. Research 5, 043089 (2023)], using a\ncombined experimental and theoretical approach. One important finding from their DFT\ncalculations is that electron doping in the monoclinic phase drives the system towards the\northorombic phase (see their Fig. 6b). Since mode 2 is closely related with the structural\nphase transition, in the monoclinic phase it should therefore broaden upon electron doping,\nanalogous to the effect of increasing temperature towards T . This expectation is consistent\nC\nwith the data shown in Fig. R2e and j.\nAt both 10 K and 150 K, mode 2 frequency follows a non-monotonic doping dependence,\nwith the doping effect strengthened at 150 K (Fig. R2d and Fig. R2i). This difference could\n4\nbeattributedtotheenhancedQESandlatticeanharmonicityat150K,whichgiverisetoad-\nditional channels for electron-phonon and phonon-phonon interactions, respectively, further\ncomplicating the doping dependence. The nearly equal magnitude of doping-induced change\nat 150 K could be a coincidence due to the complex interplay of these effects. Advanced\ntheoretical calculations taking into account both electron-phonon and phonon-phonon inter-\nactions as well as how the whole system evolves under doping will be required to quantita-\ntively elucidate the doping-induced frequency shift. Despite the incomplete understanding\nof this aspect, we believe it does not affect the main conclusions of our work.\nChanges: We have incorporate several points in the above discussions to the main text\nand Supplementary Information.\n(1) In the paragraph spanning Lines 90–98, after the Raman modes are introduced, we\nhave added “Indeed, both mode 2 and mode 5 have been shown to exhibit shearing vibration\nof the Ta atoms along the chain direction, which matches the form of the monoclinic lattice\ndistortion47.”\n(2) The last paragraph in the section “Effect of doping on Raman scattering” has been\nrevised extensively to reflect the points discussed above (see the revised manuscript). Fig-\nure R2 and the related discussions have been added to “Supplementary Note 5: More data\non the doping effect on the phonon modes”.\n4) In addition, the doping dependence trend of the QES spectral weight is nearly similar\nfor 150 K and 300 K (Fig. 4i), while the mode 2 frequency shows different temperature\ndependence at 150 K and 300 K. Also, at 400 K, mode 2 frequency does not follow the\ndoping dependence of the QES spectral weight, showing opposite behavior.\nTo improve the understanding of the complex doping and temperature dependence of\nthe QES and mode 2, we have performed measurements on another bilayer device, with\ntemperature ranging from 150 K to 460 K at a 10 K interval. This comprehensive data\nset yields the QES spectral weight and mode 2 parameters as a function of doping and\ntemperature, as shown in Fig. R3. For this comment regarding the QES spectral weight and\nthe mode 2 frequency, we can focus on Panels a, b, and d.\nWe think the QES has an electronic origin, which is supported by the fact that in the\nbulk crystal it is maximized at the T determined from electrical transport measurements.\nC\nOn the other hand, the mode 2 frequency is affected by both electron-phonon and phonon-\nphonon interactions. Since there is no fundamental reason why they should follow the same\ndoping dependence, we think they should be interpreted separately.\nFor the QES spectral weight, Fig. R3a–b show that below the T of the pristine state\nC\n(about 370 K) it is enhanced (suppressed) by electron (hole) doping, whereas above T\nC\nan opposite trend is observed. We interpret the magnitude of the QES spectral weight\nas a measure of how close the system is to the orthorhombic-monoclinic phase transition.\nThis is analogous to the use of the QES intensity as a measure of the electronic nematic\nfluctuations in iron-based superconductors, as nicely demonstrated in Phys. Rev. Lett.\n111, 267001 (2013) pointed out by the second reviewer. Based on this picture, our data\n2 5\ndevice D50. a, Color map showing the doping and temperature dependence of the QES spectral\nweight. b, Line cuts along the temperature axis at zero doping (undoped), the maximum electron\ndoping (n-doped), and the maximum hole-doping (p-doped) in a. The solid lines are Gaussian fits\nto the data above 250 K, with the fitted peak center corresponding to the T and the full width\nC\nat half maximum characterizing the temperature spread (∆T) of the QES. c, Doping dependence\nof T (left axis) and ∆T (right axis). d–f, The upper panels show the doping and temperature\nC\ndependent color maps of the frequency, amplitude, and 1/|q| for mode 2. The lower panels are the\ncorresponding line cuts at the same doping levels as those in b.\nsuggest that below T electron (hole) doping drives the system towards (away from) the\nC\nphase transition, which is consistent with the results on electron doping in Ref. 43 [Phys.\nRev. Research 5, 043089 (2023)]. Just above T , the effects of electron and hole doping are\nC\nswapped, which we think is a natural expectation based on the above arguments.\nFor the doping dependence of mode 2 frequency, we have pointed out the complexities in-\nvolved in response to the previous comment. Here the new data offer a better understanding\nof the different doping dependence at different temperatures. Figure R3d shows the temper-\nature dependence of the mode 2 frequency at different doping levels. The lower panel plots\nthe temperature dependence at selected dopings, which exhibits multiple crossings. The\nsame set of data plotted as a function of doping will show different trend depending on the\ntemperature, as was shown in Fig. 4i in our original manuscript. For example, below about\ndependence is seen. Above 400 K a reversed monotonic doping dependence is observed. To\n6\nfully understand such complex doping dependence of the mode 2 frequency requires theo-\nretical calculations taking into account electron-phonon and phonon-phonon interactions as\nwell as the doping-induced change to the phase transition.\nChanges:\n(1) To clarify what the QES reveals, we have modified the text in the second paragraph in\nthe section “Effect of doping on Raman scattering”, “The QES intensity in our Raman data\nalso peaks at T , indicating a direct link to such fluctuations and justifying using the QES\nC\nto quantify them, analogous to its application for quantifying charge nematic fluctuations in\niron-pnictide superconductors54. Its enhancement under electron doping below T suggests\nC\nstrengthened fluctuations of the coupled electron-phonon system, driving it closer to a struc-\ntural instability, consistent with prior potassium dosing studies and DFT calculations43.”\nHere Ref. 54 is the newly added reference, Phys. Rev. Lett. 111, 267001 (2013).\n(2) Figure R3 have been added to the manuscript to replace the original Fig. 5. The\nrelated discussions have been updated accordingly.\n5) In Fig. 5d,f, the enhancement of the transition temperature with hole doping is not\nclearly reflected, the peak center is within the error with respect to undoped peak. Also,\nwhy the asymmetry parameter (1/|q|) of mode 2 shows (Fig. 5f) peak at higher transition\ntemperature than observed in the QES spectral weight (Fig. 5d), even though they exhibit\ncoupling.\nThe new data shown in Fig. R3 allow us to improve the analysis of the doping dependence\nof T . Figure R3a shows the doping and temperature dependence of the QES spectral\nC\nweight. Representative line cuts along the temperature axis are shown in Fig. R3b. We\nfit the temperature dependence of the QES spectral weight to a Gaussian function. The\ncenter of the Gaussian peak is used to estimated T , whose doping dependence is shown in\nC\nFig. R3c. Within the achievable doping range, electron doping decreases T by about 34 K,\nC\nwhereas hole doping enhances T by about 5 K, confirming a weaker hole doping effect. As\nC\nwas discussed in response to the third comment, the difference between electron and hole\ndoping could be due to the asymmetry of the conduction and valence bands, as well as due\nto the phonon modes closely related to the phase transition (i.e. modes 2 and 5) being\ndominated by Ta displacement and sensitive to electron doping.\nFor the second part of this comment, it is essentially the same as the second comment\nfrom the reviewer. Our understanding is that the QES is of electronic origin and hence\nbetter reflects the T determined by electrical transport measurements. However, mode 2\nC\nparameters can show more complicated temperature dependence, due to the interplay of\nelectron-phonon interaction and lattice anharmonicity. More discussions can be found in\nthe response to the second comment.\nChanges: The original Fig. 5 in the manuscript has been updated with Fig. R3, and the\nrelated discussions on the doping dependence of T have been modified accordingly.\nC\n——————————————————————————————————–\nThe manuscript of Wei et al. reports a combined Raman and transport study of the\nexcitonic insulator candidate TNS as a function of layer thickness and doping. The results\nare discussed in light of the possible scenario for the origin of the semi-metal to insulator\ntransition observed in this material. The results are interesting and present a significant ad-\nvancetowardsafullerunderstandingofthisquiteuniquematerial. Howeverthereareseveral\npoints which need to be clarified and answered before I can make a clear recommandation.\nWe appreciate the reviewer’s valuable insights and detailed comments, which have\nprompted us to improve our work. Below we address all comments one by one.\n- The analysis of the QES performed in ref. 51-52 and 54-55 was inspired by the previous\nanalysis of nematic fluctuations by Raman scattering in Fe superconductors which more\nclosely ressemble the case of TNS. I believe that the original work should be properly cited\n(Gallais et al. 10.1103/PhysRevLett.111.267001)\nWethankthereviewerforinformingusaboutthisimportantreference. Theobservationof\nQES due to charge nematic fluctuation in iron pnictides and the related analysis framework\nare indeed inspirational and highly relevant to our study. We have added this reference to\nthe manuscript.\nChanges: On Page 4, we have modified the introduction of the QES to the fol-\nlowing, “QES was previously reported in iron pnictide superconductors54 and magnetic\nmaterials55−57, where it is typically attributed to charge nematic fluctuations and mag-\nnetic fluctuations, respectively. In Ta NiSe , however, since magnetic order is absent50,\n2 5\nno evidence of electronic nematicity was reported, and all expected Raman-active phonon\nmodes can be accounted for51, the QES was attributed to an electronic origin51,52. The QES\nexhibits a Lorentzian lineshape centered at zero frequency, resembling that observed in ne-\nmatic and magnetic systems, which motivates a similar analysis framework54−56.” The new\nRef. 54 is the work mentioned by the reviewer, and the original Refs. 54–65 are renumbered,\naccordingly.\n- The analysis in terms of QES + Fano lineshape Fano is not completely convincing.\nIndeed the Fano lineshape is due the QES continuum and a full self-consistent analysis\nshould be performed as done in ref. 52.\nWe checked the analysis in Ref. 52 carefully. Overall our analysis method is similar to\nthat used in Ref. 52, namely, fitting the spectra using a model which consists of a Drude-\nLorentzian term for the QES and Fano function for the phonon modes. The only difference\nis that in Ref. 52 three phonon modes (modes 2, 4, and 6) were included in the fit, whereas\nin our work only mode 2 is included. The rationale of including modes 4 and 6 in the fitting\nanalysis is not clear to us. First, these two modes do not share the same symmetry of the\nQES in the orthorhombic phase, and therefore their coupling with the QES is unexpected.\nSecond, these two modes are almost an order of magnitude weaker than mode 2 in the\ntemperature range most relevant to our study, namely above 300 K where the QES is\nprominent (see Fig. R4). The Raman data in Ref. 52 were plotted on a semi-log scale to\nmission from npj Quantum Mater. 6, 52 (2021) under the Creative Commons license\nhttp://creativecommons.org/licenses/by/4.0/, and texts are added to label the phonon modes 1–\n6]. The work does not explain why only phonon modes 2, 4, and 6 were included in the fitting\nanalysis, whereas modes 1, 3, and 5 were left out. Also note that in the main Fig. 3 (right panel)\nthe y-axis is shown on the semi-log scale. The data in their Supplementary Fig. 8 are presented\non a linear scale, which show more clearly that modes 4, 5, and 6 are much weaker at T ≥ 300 K.\nThis should justify only including mode 2 and the QES in the fitting analysis, if the focus is on\nthese two most prominent spectra features.\nenhance the visibility of these weak modes. Third, it is puzzling that modes 3 and 5 were\nsomehow left out in the analysis, although their intensity is comparable to that of modes 4\nand 6, and mode 5 shares the same symmetry as the QES.\nSinceanincreasednumberoffittingparametersincreasesuncertaintiesinthefittedvalues,\nwe think our method is well justified for extracting the parameters of the QES and mode 2,\nwhich are the most pronounced features in the Raman data. In fact, this same approach was\nfirst used in Ref. 51 [Nat Commun 12, 1969 (2021)]. Moreover, in Supplementary Note 2\nof Ref. 51, it was demonstrated that by solving the total Green function considering the\ncoupling between the the bare electronic QES and the phonon Green functions, the fitting\nresults are highly consistent.\n- The actual polarization configuration is not mentioned for most of the spectra in the\nmanuscript. This should be specified each time to allow comparison with earlier works.\nmanuscript when the Raman data were introduced (Lines 96–98), “The crossed polarization\nconfiguration is adopted with the polarization angle that maximizes the intensity of the QES\nand mode 2, as they are particularly sensitive to the phase transition.” To improve clarity,\nit is indeed helpful to specify the polarization configuration for each figure.\nChanges: In the captions of Fig. 2 and Fig. 4, we have added “All data were collected\nin the crossed polarization configuration.”\n- While there is a clear trend, the various markers of T based on QES and mode 2\nC\nanalysis shown in figure 2 yield quite different values of T for each thickness. This is a bit\nC\nodd and casts some doubts on the analysis especially since the different values of T are way\nC\nbeyond the errors bars. Can the authors justify this?\nInourmanuscript, wedefinedT asthetemperaturecorrespondingtothemaximumQES\nC\nspectral weight. This is because the estimated T is highly consistent with the T reported\nC C\nin the literature for bulk Ta NiSe based on electrical transport measurements; see, e.g.,\n2 5\nRefs. 32 and 50. This coincidence of T strongly suggests the electronic origin of the QES.\nC\nOther markers in Fig. 2 were only intended to corroborate the enhancement of T as the\nC\nsample thickness is decreased.\nFIG. R5. Comparison of the mode 2 parameters for bulk Ta NiSe from Ref. 51 (first row, blue\n2 5\ncircles) [reproduced with permission from Nat Commun 12, 1969 (2021) under the Creative Com-\nmons license http://creativecommons.org/licenses/by/4.0/, and red solid lines are added to assist\ncomparison with the second row] and from our work (second row), shown in the same temperature\nrange. Black dashed lines mark the T . Red solid lines mark the temperature at which the mode\nC\nparameters in the second row reaches maximum or minimum.\nTa NiSe was actually first reported in Ref. 51 [Nat Commun 12, 1969 (2021)]. In Fig. R5\n2 5\nwe reproduce the temperature dependence of mode 2 parameters from Ref. 51 and make a\ncomparison with our data. Overall, the results are very similar. The temperatures at which\nthe parameters reach maximum or minimum are consistent. The subtle difference could be\ndue to the different forms of the Fano function and the different samples used. In Ref. 51,\nthe observation of the characteristic temperatures for mode 2 parameters higher than T\nC\nwas attributed to the overlap of the QES and mode 2 being maximized above T . However,\nC\nwe think this is unlikely, because in both Ref. 51 and our work, the QES is maximized\nexactly at T .\nC\nWe think one possible source for the different characteristic temperatures corresponding\ntothemaximumorminimuminmode2parameterscanbelatticeanharmonictiy. Thiseffect\nis enhanced at high temperature and more even so close to structural phase transitions. How\nexactly it renormalizes the phonon self energy is a complicated issue on its own; see e.g., J.\nAppl. Phys. 124, 145101 (2018) and Phys. Rev. B 106, 144303 (2022). Here its interplay\nwiththeQESfurthercomplicatesthesituation. Notethatthepossibleroleofanharmonicity\nis also mentioned in the Ref. 47 [npj Comput. Mater. 7, 210 (2021)] when discussing the\nlattice properties in bulk Ta NiSe .\n2 5\nChanges: In the section “Thickness dependence”, we have made the following changes.\n(1) We have modified the text, “For the bulk sample, the QES spectral weight σ is\n0\nmaximized at the T determined from resistance measurements32,50, strongly supporting its\nC\nelectronic origin.”\n(2) After the end of the last paragraph, we have added the following text, “Quantifying\nT from mode 2 parameters, however, is less reliable, as they show different characteristic\nC\ntemperatures even for the same sample. This could be due to the additional effect of lattice\nanharmonicity47, which typically becomes enhanced at high temperature and near structural\nphase transitions.”\n- Rather than relying on the QES and mode 2 which strongly couple, rendering their\nfitting somewhat not trivial, it might be more accurate to rely on the activation of phonon\nmodes across T as discussed in ref. 51. For example A modes become activated in cross\nC g\npolarization “ac” geometry in the monoclinic phase as done by Volkov et al. (10.1103/Phys-\nRevB.104.L241103). A similar analysis would be desirable also for the effect of doping in\nfigure 5. This can be done using the data shown in the SM (Fig. 4) which seems more\nconvincing to me to track T .\nC\nWe followed the reviewer’s suggestion to analyze the A phonon mode 1, as was done by\ng\nVolkov et al. in Phys. Rev. B 104, L241103 (2021). The results for samples of different\nthickness are shown in Fig. R6. In the work by Volkov et al., it was not specified how the\nmode 1 intensity was extracted. This is not trivial, because the mode sits on top of the\nQES at temperature close to T . Upon warming to T , mode 1 becomes weaker whereas\nC C\nthe QES becomes enhanced, making the determination of mode 1 intensity prone to errors.\nHere we first fit the overall background surrounding mode 1 to a quadratic function, and\n11\n         .\n         .\n         .\n         .\n \f W L Q X     E U D     \\ W L V Q H W Q ,\n D  % X O N\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n \f W L Q X     E U D     \\ W L V Q H W Q ,\n E    /\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n \f W L Q X     E U D     \\ W L V Q H W Q ,\n F    /\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n         .\n \f W L Q X     E U D     \\ W L V Q H W Q ,\n G    /\n         .\n  \n         .\n         .\n         .\n         .\n \f   X   D     H G X W L O S P $\n H\n % X O N\n  \n \f   X   D     H G X W L O S P $\n   /\n  \n \f   X   D     H G X W L O S P $\n   /\n  \n                           \n     . \f\n \f   X   D     H G X W L O S P $\n   /\nFIG.R6. Analysisofmode1forsamplesofvaryingthickness. (a)–(d)Representativefittingresults\nfor bulk, trilayer, bilayer, and monolayer samples. Experimental data are shown as open circles.\nSolid lines are quadratic fits. The shaded areas represent the integrated intensity of mode 1. (e)\nTemperature dependence of the integrated mode 1 intensity.\nFIG. R7. Figure 1c and 1d in our manuscript, which show the temperature dependent Raman\ndata for the bulk sample and the polarization-angle dependent Raman data measured in the XY\nconfiguration at 300 K.\nthen subtract this background from the raw data. Using the background-subtracted data,\nthe spectral weight in the spectral range spanning mode 1 is integrated, which we define\nas the mode amplitude, and its temperature dependence is shown in Fig. R6(e). Large\nuncertainty is seen in the analyzed results, due to the reasons discussed above. The mode\namplitude does not decline to zero, but keeps decreasing and becomes negative due to the\ndifficulty of analyzing the complicated form of the background. T might be estimated as\nC\n12\nthe temperature at which the mode amplitude switches sign, and this analysis confirms that\nT is enhanced in atomically thin samples.\nC\nThese results show that tracking the T based on mode 1 seems unreliable based on our\nC\ndata. This is due to the rather weak signal of this mode in the measurement configuration\nadopted in our study. As mentioned on Page 4, Lines 96–98, in our manuscript, “The\ncrossed polarization configuration is adopted with the polarization angle that maximizes the\nintensity of the QES and mode 2, as they are particularly sensitive to the phase transition.”\nInthisconfiguration,mode1isextremelyweak,asshowninFig.1cand1dinourmanuscript,\nreproduced here in Fig. R7.\n- One potential issue of the data is the laser heating which could be thickness dependent.\nDid the author estimate the laser heating?\nWe carefully checked the laser power to ensure that there was negligible heating effect. In\npractice, this was done by starting taking spectra with a small laser power (e.g. 0.05 mW)\nand then incrementing the power. The data were then compared by normalizing to the\npower and integration time (Fig. R8a). If the anti-Stokes line matches that at lower power,\nheating effect is considered negligible. We settled on a power of 0.2 mW for samples with\n \f   X   D     \\ W L V Q H W Q ,\n   E \f  % X O N     =    .\n $ Q W L   6 W R N H V   I L W         \n        \n        \n=    .\n \f   X   D     \\ W L V Q H W Q ,\n   F \f    /     =    .\n $ Q W L   6 W R N H V   I L W\n=    .\n \f   X   D     \\ W L V Q H W Q ,\n   G \f    /     =    .\n $ Q W L   6 W R N H V   I L W\n        \n=    .\n \f   X   D     \\ W L V Q H W Q ,\n   H \f    /     =    .\n      \n $ Q W L   6 W R N H V   I L W\n      \n=    .       \n      \n      \n                                  \n     . \f\n \f .    \n   I \f    /\n ' D W D\n= .\n \\ W L V Q H W Q L   G H ] L O D P U R 1\n   D \f    /     =    .\n           P :\n         P :\n         P :\nFIG. R8. (a) Raman data for a bilayer sample measured at 150 K at different laser power. The\nspectra have been normalized according to the laser power and integration time. (b)–(e) Stokes\nand anti-Stokes Raman data for the phonon mode 2 at 150 K (temperature sensor reading) for\nbulk, trilayer, bilayer and monolayer samples. The solid lines are Lorentzian fits. The temperature\nestimated from the Raman data is shown in each panel. (f) The estimated temperature from the\nRaman data as a function of the temperature sensor reading for the bilayer sample. The solid line\nis a linear fit with slope of 0.78 and zero intercept.\n13\ndifferent thickness, which is sufficient for rapid data acquisition and low enough to avoid\nsample damage.\nOne can in principle estimate the sample temperature based on the Stokes intensity (I )\nS\nand anti-Stokes intensity (I ) for a given phonon mode at angular frequency ω, based on\nAS\nthe relation I /I = exp(−ℏω/k T). Here ℏ and k are the Plank constant the Boltzmann\nS AS B B\nconstant, respectively. Such analysis for bulk, trilayer, bilayer, and monolayer samples\nare shown in Fig. R8(b)–(e). The estimated temperature is consistently lower than the\ntemperature sensor reading. The temperature at the sample location in our setup was\ncalibrated using another temperature sensor, so this offset is a real effect. Many factors can\ngive rise to this offset, for example the variation of the collection efficiency over the measured\nfrequency range and resonant effect for this material at this excitation wavelength, both of\nwhich can make the measured I /I deviate from the expected ratio. Nevertheless, the\nS AS\nestimated temperature is reasonably close to the sensor reading and systematically (∼ 20%)\nlower than the actual value (Fig. R8f). We therefore conclude no evidence of clear heating\neffect at the used laser power in our measurements.\nChanges: In the Methods section, we have added the information of the laser power,\n“Raman spectroscopy was conducted in the back-scattering geometry with 532 nm laser\nexcitation. An incident power of 0.2 mW was used for all samples, which was confirmed to\ncause negligible heating effect.”\n- The behavior of the Raman spectra as a function of doping is not trivial with a strong\nelectron-hole asymmetry that does not seem to track the transport data in an obvious way.\nIn particular it is not clear to me whether the effects are due to a simple increased carrier\ndensity or to a change in T upon doping. Figure 5 suggests that the change in T is the\nC C\ndominant factor but since both set of data are discussed separately it is difficult for the\nreader to get what the main message is on that point. It seems to me that most of the data\nin figure 4 can be explained by a change in T (except maybe at 400 K).\nC\nWe agree with the reviewer that our original version of the manuscript did not present\nthe doping dependence of the Raman spectra in a clear way. To improve the understanding\nof the complex doping and temperature dependence of the QES and mode 2, we have\nperformed measurements on another bilayer device, with temperature ranging from 150 K\nto 460 K at a 10 K interval. This comprehensive data set yields the QES spectral weight\nand mode 2 parameters as a function of doping and temperature, as shown in Fig. R9. An\nimproved analysis of the doping dependence of T is achieved, as shown in Fig. R9b–c. As\nC\npointed out by the reviewer, the doping dependence of the QES at different temepratures\nis a manifestation of the doping-induced change to the T . The following changes are made\nC\nto the manuscript to better demonstrate the doping dependence of the Raman spectra.\nChanges: (1) The original Fig. 5 in the manuscript has been replaced by Fig. R9. In\nthe original Fig. 4, Panels i–l showing the doping dependence of the QES spectral weight\nand mode 2 parameters at selected temperatures have been removed, because Fig. R9b, d,\ne, and f show their doping and temperature dependence more clearly.\n(2) The related discussions on Fig. 4i–l and Fig. 5 in the original manuscript have been\na, QES spectral weight. b, Line cuts along the temperature axis at zero doping (undoped), the\nmaximum electron doping (n-doped), and the maximum hole-doping (p-doped). The solid lines\nare Gaussian fits. c, Doping dependence of the center and full width at half maximum for the\nGaussian peaks. d–f, Temperature and doping dependence of the frequency, amplitude, and 1/|q|\nfor mode 2. The lower panels are line cuts at the same doping levels as those in b.\nupdated accordingly.\n- While I agree that the data presented here seems at odd with a naive picture of an\nexcitonic insulator in the sense of an condensation of preformed exciton, I do not think we\ncan conclude that the transition is driven by a lattice distortion as concluded by the authors.\nIndeed as shown by Mazza et al. (ref. 46) the transition could be driven by an electronic\ninstability that breaks the lattice symmetry without specifically involving the condensation\nof actual excitons. In that case the effect of carrier doping and thickness are less obvious. I\nwould urge the authors to soften their claim on that aspect.\nWe are aware of this work, Ref. 46 [Phys. Rev. Lett. 124, 197601 (2020)], which em-\nphasizes the role of excitonic instability rather than excitonic condensation for the breaking\nof lattice symmetry in Ta NiSe . However, in a subsequent work published one year later,\n2 5\nRef. 47 [npj Comput. Mater. 7, 210 (2021)], the same group of authors performed a more\nthoroughtheoreticalanalysis, basedonwhichtheexcitonicinstabilityscenarioisabandoned.\nHere are a few excerpts from Ref. 47.\n• In the abstract, “We conclude that an excitonic instability is not needed to explain\nthe phase transition in both Ta NiSe and Ta NiS .”\n2 5 2 5\n15\n• On Page 2, “In this work, we abandon the concept of excitonic condensation and show\nthat the experimental features of TNSe and TNS mentioned above can be completely\nexplained without explicitly taking the formation of excitons into account.”\n• On Page 7, “However, we observe that, independently of the functional used, a finite\ncritical value of the distortion parameter d is needed in order to open a gap (see\nc\nFig. 5). This means that TNSe is indeed likely outside the region where a purely\nelectronic instability can be expected and that a finite distortion of the lattice is critical\nto account for the metal–insulator transition.”\n• OnPage7,“Forbothreflectionsymmetries,ρ approaches0ford → 0. Thisreinforces\n−\nthe conclusion that the transition in TNSe requires a finite lattice distortion and we\ndo not observe a spontaneous electronic instability.”\n• On Page 12, “We discussed the possible origins of the structural transition in relation\nto the scenario of a purely electronic instability occurring at fixed ion positions and\nconcluded that a structural distortion is required in order to observe a metal-insulator\ntransition in TNSe. This is supported by scans of the total energy landscape and\ncharge density as a function of the relevant distortion parameters, which show no sign\nof an electronic instability.”\nInstead, Ref. 47 emphasizes that electron-phonon coupling driven structural phase tran-\nsition is the origin of the insulating ground state. Here are some related statements.\n• In their abstract, “A total energy landscape analysis shows no tendency towards a\npurely electronic instability and we find that a sizeable lattice distortion is needed to\nopen a bandgap.”\n• On Page 2, “We demonstrate how the electronic and structural properties of the two\ncrystals can be rationalized in terms of electronic and intrinsic lattice instabilities that\ndrive the system from the orthorhombic to monoclinic phase.”\n• On Page 2, “We find no evidence for an electronically driven spontaneous symmetry\nbreaking in the TNSe charge density upon infinitesimal lattice distortions. In the\nsymmetry broken lattice structure, the total energy is, however, clearly lowered, which\nrenders the electron–phonon interaction the main driving mechanism for the structural\ntransition.”\n• OnPage12,“...wehaveproventhatinTNSethestructural transitiondrives themetal-\nto-semiconducting phase transition, withanindirectbandgapthatiscomparabletothe\nexperimental results...At the same time, our ab initio results show that no excitonic\neffects are necessary to explain the experimental evidences.”\nThese are all strong and definitive claims. Compared to them, the conclusions in our\nwork are much softer. Here are the few places in our manuscript where we touch upon\nthe mechanism of the phase transition. We have been careful about the choice of words,\nas underlined below. Since the doping-dependent Raman and electrical transport data in\nour work cannot be reconciled with a phase transition dominated by excitonic effects, but\n16\nare more consistent with a structural phase transition tunable through electron-phonon\ncoupling, we think it is necessary to make this point clear in our manuscript.\n• In the abstract, “These findings rule out a dominant excitonic mechanism and instead\npoint to a coupled electronic and structural phase transition, whose stability is tunable\nby carrier doping.”\n• Intheintroduction, “BymonitoringtheRamanresponse, wefindthatthequasi-elastic\nscattering (QES), previously attributed to excitonic fluctuations, evolves monotoni-\ncally with doping across a wide range on both the electron and hole sides. This ob-\nservation is inconsistent with expectations from an excitonic mechanism but supports\na scenario in which the insulating state originates from a structural phase transition\ndriven by electron-phonon coupling and tunable via carrier density.”\n• In the section about Effect of doping on electrical transport, “In bilayer Ta NiSe ,\nthe 2 5\npersistence of ∆ up to doping levels of 1013 cm−2, along with the gradual slowing\n1\nof its suppression at higher doping, suggests that the gap formation is unlikely to be\ndominated by excitonic effects.”\n• In the Discussion section, “Moreover, mode 2 retains a Fano lineshape across the or-\nthorhombic phase, even when the QES is not well resolved (Fig. 1 and Supplementary\nNote 1), indicating persistent electronic fluctuations beyond the scope of DFT. These\nobservations support a phase transition driven by entangled electronic and structural\ninstabilities, which persist to higher temperature at reduced dimensionality.”\nChanges: We have explicitly cited Ref. 47 in the above-mentioned points. This should\nmakeitcleartothereadersthatourconclusionsaregroundedonexistingtheoreticalinsights.\n- In the discussion the whole paragraph on page 8 “Its enhancement under electron\ndoping.... and a return toward the instability” does not make much sense to me. Do the\nauthorsreallyhaveanideaoftheobserveddifferentbehaviorunderelectronandholedoping?\nIf not I would suggest the authors to leave as that.\nWe thank the reviewer for pointing out this ambiguity in the previous version of our\nmanuscript. To reveal more clearly the doping effect on the QES, we have performed a more\nsystematic set of measurements on another bilayer device, with the results shown in Fig. R9.\nFigure R9a shows how the QES evolves as the temperature and doping are varied over a\nwide range. This figure confirms the doping dependence at selected temperatures shown in\nFig. 4 of the original manuscript.\nWe take the premise that the QES is a way to quantify the fluctuating coupled electronic\nand lattice system, analogous to the nice demonstration of the QES as a way to quantify\ncharge nematic fluctuations in iron-pnictide superconductors mentioned by the reviewer.\nMore discussions on this point about the origin of the QES can be found in the response\nto the final comment from this reviewer. Evidence of fluctuations of the coupled electronic\nand phononic system is elaborated in the reply to the next comment from this reviewer as\nwell as in Lines 200–206 in our original manuscript. We believe by setting up the stage with\nsuch information, our interpretation of the observed different behavior under electron and\n17\nhole doping should become natural.\nChanges: We have added the following highlighted text to the discussions, “The QES\nintensityinourRamandataalsopeaksatT ,indicatingadirectlinktosuchfluctuationsand\nC\njustifyingusingtheQEStoquantifythem,analogoustoitsapplicationforquantifyingcharge\nnematic fluctuations in iron-pnictide superconductors54. Its enhancement under electron\ndoping below T suggests strengthened fluctuations of the coupled electron-phonon system,\nC\ndriving it closer to a structural instability, consistent with prior potassium dosing studies\nand DFT calculations43.” Here Ref. 54 is the newly added reference, Phys. Rev. Lett. 111,\n267001 (2013).\n- In the discussion on page 8 the authors state that the softening and broadening of\nmode 2 with doping is “consistent with enhanced electron-phonon coupling in the presence\nof excess carriers”. This is misleading. Even with constant electron-phonon coupling the\neffect is expected just by the trivial fact the DOS at the Fermi energy increases upon doping.\nThis is a well-known effect that was observed in doped Si in 1970s. Therefore eu conclusion\nabout electron-doping enhanced lattice fluctuations is incorrect.\nWe thank the reviewer for this insightful comment, which includes two parts.\n(1) For the first part, the discussion refers to the doping dependence of the frequency\nand linewidth for mode 2. The reviewer is correct that the phonon mode parameters can\nalso change if the electronic density of states N at the Fermi energy changes. For example,\n0\nthe linewidth γ for a particular phonon mode j with frequency ω can be expressed as\nj j\nγ ∝ N ω2λ , where λ is the contribution to the total electron-phonon coupling constant\nj 0 j j j\nfrom the the jth mode [Phys. Rev. B 48, 8412 (1993)]. We therefore modify the text to\nreflect the possibilities that change in either N and λ can be responsible for the observed\n0 j\ndoping-induced results.\nChanges: We have made the following changes to the original text, “At 150 K, where\nthe QES is weak, mode 2 softens and broadens with doping, which could be due to doping-\ninduced enhancement of either the electron–phonon coupling or the electronic density of\nstates at the chemical potential.”\n(2) For the second part, the reviewer questions our interpretation of electron-doping\nenhanced lattice fluctuations below T . This interpretation is based on the observation from\nC\nour data that below but close to T , the effects of increasing temperature and increasing\nC\nelectron doping on the QES and mode 2 are quite similar, and evidence of thermally-induced\nlattice fluctuations already exists in the literature. Specifically, in Ref. 43, temperature-\ndependent x-ray diffraction data show strengthened diffuse scattering when approaching T\nC\n(see Fig. R10). This is strong indication that no perfect lattice periodicity exists along the\ninter-chain c-axis at temperature close to T .\nC\nChanges: To emphasize that the electron-doping-enhanced lattice fluctuations only be-\ncomes clearly resolved near T , we have modified the related text, “Above ∼250 K and\nC\nbelow T , whenAt 300 K, where the QES becomes more prominent, the doping dependence\nC\nof mode 2 parameters becomes monotonic (Fig. R9d–f), paralleling that of the QES. Elec-\ntron doping reduces the mode amplitude (Fig. R9e) but increases its frequency (Fig. R9d),\n18\n1/|q| (Fig. R9f) and linewidth (Supplementary Note 6), resembling the effect of increasing\ntemperature just below T (Fig. 2). A similar trend is observed for mode 5 (Supplemen-\nC\ntary Note65), furthercorroborating theroleof electron-doping-enhanced latticefluctuations\nwhen approaching T from below.”\nC\n- On page 8 the authors seem to challenge the view that the QES is of electronic origin\nhinting at a possible involvement of speculative lattice fluctuations (?). However on page 9\nthey seem to revert to an interpretation in terms of electronic fluctuations. Can the author\nclarify where they stand on that point?\nWe think the QES has an electronic origin, and it is closely related to lattice fluctuation.\nTo be more specific, the QES could be a manifestation of the charge fluctuation, which is\ncoupled with the lattice fluctuation, when the system transitions between the orthorhombic\nand monoclinic phases. The coincidence of the maximum QES spectral weight with the T\nC\ndetermined from electrical transport measurements indicates its electronic origin. The clear\ndoping dependence of the QES near the phase transition temperature further supports this\npicture. Phenomenologically speaking, this maximum QES at the phase transition is akin to\nFIG. R10. (a) and (b) Evidence of dynamic lattice fluctuations reported in Ref. 43. (c)\nTemperature-dependent ARPES data on bulk Ta\n5\nfrom Ref. 43. Reproduced with per-\nmission from Phys. Rev. Research 5, 043089 (2023) under the Creative Commons license\nhttp://creativecommons.org/licenses/by/4.0/ without changes to the content.\n19\nthe electronic Raman signature of charge nematic fluctuations in iron pnictides mentioned\nby the reviewer.\nEvidence for fluctuations of the coupled electronic and phononic system can be found in\nthe literature. Because of symmetry mismatch, the hybridization of the Ta-5d and Ni-3d\norbitals is forbidden in the orthorhombic phase, and no gapping is expected [see Ref. 62,\nPhys. Rev. Res. 2, 013236 (2020)]. However, the x-ray scattering results reported in Ref. 43\nprovide evidence of dynamics lattice fluctuation near T ; see Fig. R10 reproduced from that\nC\nwork. This enables orbital hybridization, resulting in a pseudogap in the orthorhombic\nphase, as revealed by ARPES (see Fig. R10) and infrared spectroscopy [Phys. Rev. B 95,\n195144 (2017)]. The finite conductivity in the zero-energy limit at high temperature shown\nin Fig. 2(c) in Phys. Rev. B 95, 195144 (2017) indicates free carriers, but the absence\nof a Drude peak and the persisting hump-like feature at ∼ 0.3 eV suggests a pseudogap,\nconsistent with the ARPES results. Gap-like feature in the orthorhombic phase was also\nreported in Ref. 41 using high-energy-resolution resonant inelastic x-ray scattering. The\npseudogap feature was interpreted as due to preformed excitons in earlier studies, but as\nwe discussed above, growing recent studies (e.g. Refs. 42, 43, and 47) point to the more\nimportant roles of dynamic lattice fluctuations and electron-phonon coupling, which imprint\non the electronic band structure. The finite conductivity in the zero-energy limit at high temperature shown\nin Fig. 2(c) in Phys. Rev. B 95, 195144 (2017) indicates free carriers, but the absence\nof a Drude peak and the persisting hump-like feature at ∼ 0.3 eV suggests a pseudogap,\nconsistent with the ARPES results. Gap-like feature in the orthorhombic phase was also\nreported in Ref. 41 using high-energy-resolution resonant inelastic x-ray scattering. The\npseudogap feature was interpreted as due to preformed excitons in earlier studies, but as\nwe discussed above, growing recent studies (e.g. Refs. 42, 43, and 47) point to the more\nimportant roles of dynamic lattice fluctuations and electron-phonon coupling, which imprint\non the electronic band structure. We think the fluctuating electronic states are the origin\nof the QES, and they can couple with the phonon mode 2, giving rise to Fano interference.\nChanges: In the section “Effect of doping on Raman scattering”, we have made the\nfollowing changes.\n(1) In the first paragraph, after describing the doping dependence of the QES at represen-\ntative temperatures, we have added “The doping tunability of the QES affirms its electronic\norigin.”\n(2) In the second paragraph, we have made the following changes, “Resonant inelastic\nx-ray scattering further supports this picture by revealing substantial Ta-Ni orbital hy-\nbridization above T 41, which can manifest as a pseudogap in the orthorhombic phase, as\nC\nindicated by ARPES43 and infrared spectroscopy33 measurements. The QES intensity in\nour Raman data also peaks at T , indicating a direct link to such fluctuations and justifying\nC\nusing the QES to quantify them, analogous to its application for quantifying charge nematic\nfluctuations in iron-pnictide superconductors54.”\n——————————————————————————————————–\nReferee Report\nThe manuscript by Keyu Wei et al. presents a comprehensive Raman spectroscopy study\non Ta NiSe , a well-known candidate material for excitonic insulator (EI) phases. One\n2 5\nof the central open questions in this field is whether the insulating state observed in the\nlow-temperature phase of this compound is primarily driven by exciton condensation or\nby a structural phase transition. The authors successfully demonstrate that the essential\nproperties of bulk Ta NiSe persist down to the atomically thin limit, and they provide\n2 5\nthe systematic evidence that the critical temperature increases monotonically as the sample\nthickness decreases. This is valuable information for understanding enhanced fluctuations\nand electron-lattice coupling in reduced dimensions. Particularly notable is the asymmetric\ndoping dependence of the quasi-elastic scattering (QES), which offers a new perspective on\nthe nature of the phase transition. These new insights were made possible by the\nfabrication of high-quality dual-gated hBN-encapsulated devices, allowing for clean electron and hole\ndoping, and by systematic Raman measurements with quantitative fitting across a wide\nrange of temperatures and doping levels. Despite the significance of these results, I have\nthe following concerns, and in its current form, I cannot recommend this manuscript for\npublication in Nature Communications.\nMajor Concerns\n1. Discrepancy in the thickness dependence of the transition temperature\nAs the authors mention, their study finds that the structural transition temperature in-\ncreases with decreasing thickness, whereas previous work [Kim, S. Y. et al. ACS Nano 10,\n8888–8894 (2016).] has reported the opposite trend. In this manuscript, the authors deter-\nmine the transition point based on the temperature dependence of Raman-active phonon\nmodes (i.e., structural changes), while in earlier studies, the transition was estimated from\ntransport measurements, which reflect gap formation more directly. If both results are valid,\nthis implies a decoupling between the structural transition and the electronic gap opening.\nIn fact, this would support the view that the structural transition does not fully account for\nthe gap formation and that excitonic condensation may still play an essential role. There-\nfore, the discrepancy could be interpreted as strong evidence for the excitonic scenario. That\nsaid, differences in sample quality may also contribute. Since the authors’ devices are of\nclearly superior quality, they should also perform electronic transport measurements (which\nappear feasible within their current setup) to extract the transition temperature in the same\nsamples used for Raman measurements. This would help clarify the nature of the transition\nand reconcile conflicting reports in the literature.\nWe thank the reviewer for this excellent suggestion. We have performed such an exper-\niment using the four-probe method. Devices were fabricated by transferring mechanically\nexfoliated flakes onto per-patterned Ti/Au electrodes, which eliminates influence of contact\nresistance. The flakes were capped by hBN to avoid exposure to the environment, which\n(c), and bilayer (d) samples. The lower panels show the corresponding temperature derivative of\nthe resistance. Solid lines are linear fits. The crossing points are used to determine the transition\ntemperature, as marked by the arrows.\nSample thickness T from QES T from transport\nC C\nBulk 326 K 323 K\n3L 350 K 354 K\n2L 370 K 365 K\nTABLE I. Comparison of T obtained using two methods.\nC\nhelps to preserve the sample quality. The new data are shown in Fig. R11 for samples\nof varying thickness. We confirm that for samples with a thickness down to bilayer, the\ntemperature dependence of the resistance shows an insulating behavior up to about 500 K,\nwhich is consistent with the data in Fig. 3 in our manuscript. The temperature derivative\nof the resistance, dR/dT, decreases in its magnitude as temperature is increased. A kink, at\nwhich two linear sections in dR/dT cross, was referred to as the anomaly due to the phase\ntransition in Ta NiSe [see Ref. 57, Kim, S. Y. et al. ACS Nano 10, 8888–8894 (2016)].\n2 5\nThis kink was observed in all of our samples, but becomes weaker when the thickness is\nreduced. T determined using this criteria also increases with decreasing sample thickness,\nC\nconsistent with the trend obtained using the analysis of the quasi-elastic scattering (QES).\nTable I compares the T obtained using the two methods. Despite this consistency, we point\nC\nout that for atomically thin samples, determining T from the transport data could be less\nC\nreliable because of the weak kink in dR/dT. Also note that there appears to be another\nkink at higher temperature, which makes the identification of the phase transition some-\nwhat ambiguous. Nevertheless, combining the Raman and transport data, we are now more\nconfident about the enhanced phase transition temperature in atomically thin samples.\nChanges: At the end of the section “Thickness dependence”, we have added new dis-\ncussions of the transport data, “We further performed resistance measurements on samples\nwith varying thickness (Supplementary Note 5). The kink in the temperature derivative of\nthe resistance as an indication of the phase transition was observed in all measured samples,\n22\nalthough it becomes weaker in atomically thin samples. The corresponding T values are\nC\nplotted in Fig. 2j as open symbols, which confirm the increased T as the sample thickness\nC\ndecreases.” Details of the transport measurements and data analysis, including Fig. R11,\nhave been added as the new Supplementary Note 5. New T values from the analysis of the\nC\ntransport data have been plotted in Fig. 2j in the manuscript.\n2. Overstatement of the exclusion of the excitonic mechanism\nThe manuscript tends to overstate the claim that the observed phenomena rule out ex-\ncitonic condensation as the driving force behind the phase transition. The argument is\nlargely based on the asymmetric doping dependence of the QES, which indeed challenges a\nsimple excitonic fluctuation scenario. However, the Raman data primarily probe structural\naspects of the transition and provide limited insight into the evolution of the electronic gap\nitself. Thus, the authors’ interpretation only directly addresses the structural transition,\nand the claim that excitonic effects are not essential seems too strong. Many previous stud-\nies—including ARPES, pump-probe, and theoretical works—have supported the importance\nof excitonic physics in Ta NiSe . A more balanced conclusion would be that the transition\n2 5\ncannot be explained solely by excitonic condensation, and that coupled excitonic and struc-\ntural effects are likely at play. I recommend the authors revise statements that deny the\nrole of excitons entirely or claim structural order as definitively dominant.\nWe agree with the reviewer that direct probe of the electronic degree of freedom is es-\nsential for revealing the mechanism of the phase transition. To this end, we have performed\ndoping-dependent electrical transport measurements. The results are shown in Fig. 3 in\nour manuscript and discussed in the section “Effect of doping on electrical transport”. The\nmain findings are reproduced here in Fig. R12 for the reviewer’s convenience. The insulating\nstate is progressively weakened by increasing both electron and hole doping, but remains\nrobust up to the maximum doping achieved (approximately 2×1013 cm−2, which amounts to\n±0.06e/unit cell). Accordingly, the extracted thermal activation gap is suppressed by more\nthan 47% (67%) under electron (hole) doping. One may argue that it takes more doping\nto fully close a gap as large as 0.3 eV, but the fact that the doping-induced gap reduction\ndramatically slows down when the doped carrier density exceeds 1013 cm−2 is difficult to\nreconcile with an excitonic gap, because Coulomb bound electron-hole pairs are unstable\nagainst screening from doped carriers.\nWe are aware of previous ARPES, pump-probe, and theoretical works that support the\nimportance of excitonic physics in Ta NiSe , but we also note that some more recent ARPES\n2 5\n[see, e.g., Ref. 43, Phys. Rev. Res. 5, 043089 (2023)], pump-probe [see, e.g., Ref. 42, Proc.\nNatl. Acad. Sci. U.S.A. 120, e2221688120 (2023)] and theoretical [see, e.g., Ref. 47, npj\nComput. Mater. 7, 210 (2021)] studies challenge the excitonic physics in this system.\nWe respect the reviewer’s suggestion to make a balanced conclusion regarding the phase\ntransition mechanism and we understand that overstatement only causes more confusion\nto this field. But given our doping-dependent Raman and electrical transport results, we\nbelieve the conclusion that excitonic effects are not the dominant mechanism for the phase\ntransition in Ta NiSe is self-consistent.\n2 5\n23\n  \n    \n       \n              \n        F P  \f\n      \n     . \f\n      \n             \n      \n      \n      \n             \n      \n      \n      \n              \n        F P  \f\n E\nFIG. R12. (a) Doping-dependent resistance at selected temperatures. (b) Doping dependence of\nthe activation gap. The shaded areas represent error bars obtained from the fitting analysis.\nWe have adopted a rather soft tone in the discussions of the phase transition mecha-\nnism, which do not exclude the excitonic effect entirely but only question its dominance, as\nunderlined in the text below from our manuscript.\n• In the abstract, “These findings rule out a dominant excitonic mechanism and instead\npoint to a coupled electronic and structural phase transition, whose stability is tunable\nby carrier doping.”\n• Intheintroduction, “BymonitoringtheRamanresponse, wefindthatthequasi-elastic\nscattering (QES), previously attributed to excitonic fluctuations, evolves monotoni-\ncally with doping across a wide range on both the electron and hole sides. This ob-\nservation is inconsistent with expectations from an excitonic mechanism but supports\na scenario in which the insulating state originates from a structural phase transition\ndriven by electron-phonon coupling and tunable via carrier density.”\n• In the section Effect of doping on electrical transport, “In bilayer Ta NiSe , the per-\n2 5\nsistence of ∆ up to doping levels of 1013 cm−2, along with the gradual slowing of\n1\nits suppression at higher doping, suggests that the gap formation is unlikely to be\ndominated by excitonic effects.”\n• In the section Discussion, “Moreover, mode 2 retains a Fano lineshape across the or-\nthorhombic phase, even when the QES is not well resolved (Fig. 1 and Supplementary\nNote 1), indicating persistent electronic fluctuations beyond the scope of DFT. These\nobservations support a phase transition driven by entangled electronic and structural\ninstabilities, which persist to higher temperature at reduced dimensionality.”\nChanges: We have explicitly cited Ref. 47 in the above-mentioned points. This should\nmakeitcleartothereadersthatourconclusionsaregroundedonexistingtheoreticalinsights.\n3. Physical mechanism of the Fano interference\nThe manuscript presents Fano interference between the QES and phonon mode 2, mod-\neled with a standard Fano lineshape. However, if the QES is not of excitonic origin, the\nmicroscopic origin of this interference becomes unclear. Is the coupling mediated via free\ncarriers? Can it arise solely from phonon-phonon interactions? A brief discussion of possible\nmechanisms is needed to justify the presence of the Fano effect in the absence of excitonic\nfluctuations.\nWe think the QES has an electronic origin, and it is closely related to lattice fluctuation.\nTo be more specific, the QES could be a manifestation of the charge fluctuation, which is\ncoupled with the lattice fluctuation, when the system transitions between the orthorhombic\nand monoclinic phases. The coincidence of the maximum QES spectral weight with the T\nC\ndetermined from electrical transport measurements indicates its electronic origin. The clear\ndoping dependence of the QES near the phase transition temperature further supports this\npicture. Phenomenologically speaking, this maximum QES at the phase transition is akin to\nFIG. R13. (a) and (b) Evidence of dynamic lattice fluctuations reported in Ref. 43.\n(c)\nTemperature-dependent ARPES data on bulk Ta\n5\nfrom Ref. 43. Reproduced with per-\nmission from Phys. Rev. Research 5, 043089 (2023) under the Creative Commons license\nhttp://creativecommons.org/licenses/by/4.0/ without changes to the content.\n25\nthe electronic Raman signature of charge nematic fluctuations in iron pnictides mentioned\nby the second reviewer [Phys. Rev. Lett. 111, 267001].\nEvidence for fluctuations of the coupled electronic and phononic system can be found in\nthe literature. Because of symmetry mismatch, the hybridization of the Ta-5d and Ni-3d\norbitals is forbidden in the orthorhombic phase, and no gapping is expected [see Ref. 62,\nPhys. Rev. Res. 2, 013236 (2020)]. However, the x-ray scattering results reported in Ref. 43\nprovide evidence of dynamics lattice fluctuation near T ; see Fig. R13 reproduced from that\nC\nwork. This enables orbital hybridization, resulting in a pseudogap in the orthorhombic\nphase, as revealed by ARPES (see Fig. R13) and infrared spectroscopy [Phys. Rev. B 95,\n195144 (2017)]. The finite conductivity in the zero-energy limit at high temperature shown\nin Fig. 2(c) in Phys. Rev. B 95, 195144 (2017) indicates free carriers, but the absence\nof a Drude peak and the persisting hump-like feature at ∼ 0.3 eV suggests a pseudogap,\nconsistent with the ARPES results. Gap-like feature in the orthorhombic phase was also\nreported in Ref. 41 using high-energy-resolution resonant inelastic x-ray scattering. The\npseudogap feature was interpreted as due to preformed excitons in earlier studies, but as\nwe discussed above, growing recent studies (e.g. Refs. 42, 43, and 47) point to the more\nimportant roles of dynamic lattice fluctuations and electron-phonon coupling, which imprint\non the electronic band structure. We think the fluctuating electronic states are the origin\nof the QES, and they can couple with the phonon mode 2, giving rise to Fano interference.\nThe possibility of the QES originating from phonon-phonon interactions is interesting.\nRef. 47 pointed out the importance of phonon-phonon interactions in Ta NiSe , and more\n2 5\nimportantly, proposed such a possibility, “The anharmonic coupling of the soft mode to the\ncontinuum of other phononic modes could also be the source of the broadening of the Raman\npeak in the region just below 2 THz, which has been instead attributed to the coupling of\nthe phonon to an excitonic instability.” While we cannot completely rule out this possibility,\nwe think this scenario is less likely compared to the electronic origin of the QES, because\nof the following reasons. (i) The QES is electronically tunable. (ii) Anharmonicity should\nbe enhanced at increased temperature, but the QES only peaks at T , instead of keeping\nC\nincreasing at higher temperature.\nChanges: To clarify the possible origin of the QES, In the section “Effect of doping on\nRaman scattering”, we have made the following changes.\n(1) In the first paragraph, after describing the doping dependence of the QES at represen-\ntative temperatures, we have added “The doping tunability of the QES affirms its electronic\norigin.”\n(2) In the second paragraph, we have made the following changes, “Resonant inelastic\nx-ray scattering further supports this picture by revealing substantial Ta-Ni orbital hy-\nbridization above T 41, which can manifest as a pseudogap in the orthorhombic phase, as\nC\nindicated by ARPES43 and infrared spectroscopy33 measurements. The QES intensity in\nour Raman data also peaks at T , indicating a direct link to such fluctuations and justifying\nC\nusing the QES to quantify them, analogous to its application for quantifying charge nematic\nfluctuations in iron-pnictide superconductors54.”\nThe Ta NiSe flakes are encapsulated between two hBN layers. It would be helpful to\n2 5\nclarify how the Raman excitation and scattered light pass through the structure. Is there an\nopening on the top hBN layer, or is the hBN thin enough to allow optical access? Including\na real or schematic image of the device would greatly aid clarity.\nThe hBN layers are typically 5–15 nm thick, which are thin enough to allow optical\naccess. TheyfullyencapsulateTa NiSe toprotectitfrombeingexposedtotheenvironment.\n2 5\nBeing an insulator with a band gap of about 6 eV [Nat. Photon. 10, 262 (2016)], hBN is\ntransparent to visible light, such as the 532 nm laser used in our Raman measurements. A\nschematic image of the device used in the doping dependent electrical transport and Raman\nmeasurements is shown in Fig. 3a. A device image is shown below in Fig. R14 with different\nparts annotated. Here the graphite gate electrodes are a few nm thick, which only absorbs\na small amount of the incident and scattered light.\nBottom\ngate\nTaNiSe Graphite\n2 5\nelectrode\nGraphite\nelectrode\nTop\ngate\n5 μm\nFIG. R14. Optical micrograph of a typical dual-gate device. The bilayer Ta NiSe flake is\n2 5\nconnected with two graphite electrodes for two-probe resistance measurements. It is encapsulated\non both sides by hBN gate dielectrics. Graphite flakes serve as the top and bottom gate electrodes.\nChanges: (1) In the Methods section, in the paragraph describing the sample prepara-\ntion, we have added the information on the hBN thickness, “For Raman measurements, the\nflakes were encapsulated by hexagonal boron nitride (hBN) layers with thickness of 5–15 nm\non both sides.”\n(2)InLine159, afterintroducingthedual-gatedevicestructure, wehaveadded“Atypical\ndevice image is shown in Supplementary Note 5”, in which Fig. R14 has been added.\nIn the equation for the displacement field D, it seems that the vacuum permittivity ϵ is\n0\nmissing. Please confirm whether it should be included.\nWe thank the reviewer for pointing out this error. Instead of using the displacement field,\nwe change the description to an electric field E, which is calculated as E = (V /d −V /d )/2.\nt t b b\nChanges: In the Methods section, in the paragraph describing the characterization\nmethods, we have modified the text, “The electric field along the charge neutrality line\nshown in Fig. 3b is E = (V /d − V /d )/2.” We have changed “displacement field” to\nt t b b\n“electric field” throughout the manuscript. We have modified Supplementary Note 7 and\nSupplementary Fig. 9 accordingly.\n——————————————————————————————————–\nThe authors have thoroughly addressed all my concerns, supported by new temperature\nand doping dependence Raman data and revised figures. The manuscript is now suitable\nfor consideration in Nature Communications.\nWe thank the reviewer again for the valuable comments and suggestions, which have\nhelped us improve our work.\nResponse to Reviewer #2\n——————————————————————————————————–\nThe authors have answered satisfactorily most of my previous comments and added new\nand I believe quite insightful data. I am therefore ready to recommend publication in Nat.\nComm. Nevertheless I still have some issues with discussion and interpretation. Before\nmaking my final recommandation, I would like the authors the consider the following points\nand modify the manuscript accordingly:\nWe appreciate the reviewer’s careful evaluation of the revised manuscript and their con-\nstructive comments. Below, we address each point in detail and describe the corresponding\nchanges made.\n(1) Concerning the Fano lineshape, I agree that a fully self-consistent analysis coupling\nthe QES and the bare phonon was shown to yield similar results as a more naive QES +\nFano by ref. 51. However, this may not always be the case and it cannot be discarded as\na source of discrepancy between the different criteria for determining T . I would urge the\nC\nauthors to mention this limitation on top of possible effects like anharmonicity.\nWe agree with the reviewer that the complexity in the Raman data analysis should be\nexplicitly noted as a possible source of discrepancy between the different criteria for deter-\nmining T . To further examine the thickness dependence of T , and following the suggestion\nC C\nof another reviewer, we have performed temperature-dependent resistance measurements on\nsamples with varying thicknesses. This provides an independent means of estimating T\nC\n(revised Supplementary Note 5). The resulting thickness-dependent T is more consistent\nC\nwith that obtained from the analysis of the QES (revised Fig. 2j). We therefore believe that\nthe conclusion of an enhanced T upon thickness reduction is well supported.\nC\nChanges: Wehaveaddedafewsentences(highlightedinred)tothefollowingdiscussion.\n“Quantifying T from mode 2 parameters, however, is less reliable, as they show different\nC\ncharacteristic temperatures even for the same sample. This could be due to the additional\neffect of lattice anharmonicity [47], which becomes enhanced at high temperature and near\nstructural phase transitions. Another source of discrepancy between the different criteria for\ndetermining T using mode 2 parameters arises from the data analysis itself. In principle,\nC\nthis could be remedied by employing a fully self-consistent analysis that couples the QES\nand the bare phonon [52]. However, for Ta NiSe , such an approach has been shown to\n2 5\nhave only a minimal influence on the analyzed results [51]. We further performed resistance\nmeasurements on samples with varying thickness (Supplementary Note 5). The kink in the\n1\ntemperature derivative of the resistance as an indication of the phase transition [58] was\nobserved in all measured samples, although it becomes weaker in atomically thin samples.\nThe corresponding T values are plotted in Fig. 2j as open symbols, which confirm the\nC\nincreased T as the sample thickness decreases.”\nC\n(2) The new set of data in figure 5 nicely illustrate the fact that QES intensity tracks T\nC\nwith gate and T. If we assume, as the authors do it seems, that the QES is of electronic\norigin, then it is present whether there is a B phonon to couple to or not. In this scenario\n2g\nthe QES is associated to B symmetry breaking electronic fluctuations (as in Fe SC where\n2g\nthey are of B character, but there are no Raman active B phonon to couple there) which\n1g 1g\nare present independent of electron-phonon coupling (which is also the interpretation of\nref. 51), I do not see the need to invoke enhanced electron-phonon coupling under electron\ndoping to explain the data. Since electron doping decreases T , making closer to the sample\nC\ntemperature when measuring below T , the QES is enhanced. This is a simple scenario.\nC\nMy point is that the Raman QES data per-se do not allow a clear conclusion on the role of\nelectron-phonon coupling for the evolution of T with doping.\nC\nA similar conclusion can be drawn from mode 2: since there is a gradual gap opening\nbelow T the DOS at E will decrease below T and the phonon linewidth will decrease as a\nC F C\nconsequence (same with Fano factor). Again the key parameter is the proximity to T .\nC\nMy conclusion is that no clear conclusion can be made about the evolution of electron-phonon\ncoupling based on the present Raman data, everything can be explained based on change in\nthe electronic structure and fluctuations to which the phonon couple to. I would urge the\nauthors to consider very carefully this important point.\nWe thank the reviewer for raising this important point. We agree that the doping- and\ntemperature-dependent QES and mode 2 parameters can be largely understood in terms\nof doping-induced changes to T , which provides a simpler interpretation of the Raman\nC\ndata. Accordingly, we have revised the manuscript to remove references to electron-phonon\ncoupling in the discussion of the doping dependence of the QES (see Changes below).\nThe only remaining mention of electron-phonon coupling in the discussion on the doping\ndependence concerns the sentence: “At 150 K, where the QES is weak, mode 2 softens\nand broadens with doping (Supplementary Note 6), which could be due to doping-induced\nenhancement of either the electron-phonon coupling or the electronic density of states at\nthe chemical potential.” As discussed previously, the linewidth γ of a phonon mode j with\nj\nfrequency ω can be expressed as γ ∝ N ω2λ , where λ is the contribution of mode j to\nj j 0 j j j\nthe total electron-phonon coupling constant and N is the electronic density of states at the\nfactors when discussing the doping-induced effect on the phonon linewidth.\nWhile the Raman data alone do not allow a definitive conclusion regarding electron-\nphonon coupling, several recent studies have emphasized its important role in driving the\nphase transition in this system (Refs. 42, 43, and 47). For completeness, we briefly mention\nin the Discussion section the possibility that thickness reduction may influence T through\nC\nchanges in electron-phonon coupling, potentially affecting the relative stability of the mon-\n2\noclinic and orthorhombic phases and the transition between them. Doping may have a\nsimilar effect. Further experimental and theoretical investigations will be required to test\nthis conjecture.\nChanges: We have made changes to the following two paragraphs.\n“...The QES intensity in our Raman data also peaks at T , indicating a direct link\nC\nto such fluctuations and justifying using the QES to quantify them, analogous to its ap-\nplication for quantifying charge nematic fluctuations in iron-pnictide superconductors [54].\nIts enhancement under electron doping below T suggests strengthened fluctuations of the\nC\ncoupled electron-phonon system, driving it closer to a structural instability, consistent\nwith prior potassium dosing studies and DFT calculations [43]. Conversely, hole doping\nsuppresses the fluctuations, revealing a clear asymmetry between electron and hole doping.\nAbove T , hole doping enhances the QES, indicating a revival of critical fluctuations and a\nC\nreturn toward the instability.\nMoreThis picture is further confirmed by more systematic doping dependent Raman mea-\nsurementswere performedatafinertemperaturestepinanotherbilayerdevice, whichallows\nus to extract the temperature dependence of the QES spectral weight at different doping\nlevels. The results, shown in Fig. 5a, are consistent with the doping dependence at selected\ntemperatures described above. By fitting the temperature dependence of the QES above\nC\nand its doping dependence is shown in Fig. 5c. We estimate an enhancement of 5 K and\nreduction of 34 K under the maximum hole and electron doping of 2.6×1013 cm−2, respec-\ntively. The reduction of T under electron doping is consistent with previous potassium\nC\ndosing studies and DFT calculations, which show that electron doping tends to stabilize the\northorhombic phase [43]. The doping dependence of the QES at selected temperatures, as\nshown in Fig. 4, can therefore be attributed to the doping-induced modification of T ,\nC\nwhich shifts the peak in the temperature dependence of the QES spectral weight (Fig. 5a–b).\nThe spread of the QES temperature dependence, quantified as the full width at half maximum\nof the Gaussian peak, grows as T decreases (Fig. 5c), indicating a more extended fluctuation\nC\nregime under electron doping. ”\nMore minor point: stating “cross-polarization” is too vague, the authors should state the\npolarization configuration with respect to crystal axis using standard Porto notation. If the\nrelative orientation are not known then it should be stated.\nIn the standard Porto notation, the polarization configuration adopted in the thickness-\n¯\nand doping-dependent Raman measurements was b(ac)b, where a, b, and c denote the crystal\naxes of Ta NiSe . Following the notation in Ref. 52, we refer to this configuration more\n2 5\nsimply as the ac scattering geometry.\nChanges: In the main text and figure captions, “crossed polarization configuration” is\nmodified to “crossed polarization (ac) configuration”.\n——————————————————————————————————–\nThe authors have thoroughly addressed all of my previous comments. In addition to\nproviding clear and convincing responses, they have carried out new experiments, which\nhave significantly improved the quality and impact of the manuscript. The revised version\nis now much stronger, with enhanced clarity, completeness, and scientific value.\nGiven these substantial improvements, I believe the manuscript has reached a level of\nquality appropriate for publication in Nature Communications, and I recommend it for\nacceptance.\nWe appreciate the reviewer’s positive comments and constructive suggestions, which have\nstrengthened this work. We hope that the revised manuscript will make a valuable contri-\nbution to the field.\n4"}, {"round": 2, "reviewer_comments": {"reviewer_1": "The authors have thoroughly addressed all my concerns, supported by new temperature and doping dependence Raman\ndata and revised figures. The manuscript is now suitable for consideration in Nature Communications.", "reviewer_2": "The authors have answered satisfactorily most of my previous comments and added new and I believe quite insightful data. I\nam therefore ready to recommend publication in Nat. Comm.\nNevertheless I still have some issues with discussion and interpretation. Before making my final recommandation, I would\nlike the authors the consider the following points and modify the manuscript accordingly:\n(1) Concerning the Fano lineshape, I agree that a fully self-consistent analysis coupling the QES and the bare phonon was\nshown to yield similar results as a more naive QES + Fano by ref. 51. However, this may not always be the case and it\ncannot be discarded as a source of discrepancy between the different criteria for determining Tc. I would urge the authors to\nmention this limitation on top of possible effects like anharmonicity.\n(2) The new set of data in figure 5 nicely illustrate the fact that QES intensity tracks Tc with gate and T. If we assume, as the\nauthors do it seems, that the QES is of electronic origin, then it is present whether there is a B2g phonon to couple to or not.\nIn this scenario the QES is associated to B2g symmetry breaking electronic fluctuations (as in Fe SC where they are of B1g\ncharacter, but there are no Raman active B1g phonon to couple there) which are present independent of electron-phonon\ncoupling (which is also the interpretation of ref. 51), I do not see the need to invoke enhanced electron-phonon coupling\nunder electron doping to explain the data. Since electron doping decrease Tc, making closer to the sample temperature\nwhen measuring below Tc, the QES is enhanced. This is a simple scenario. My point is that the Raman QES data per-se do\nnot allow a clear conclusion on the role of electron-phonon coupling for the evolution of Tc with doping.\nA similar conclusion can be drawn from mode 2: since there is a gradual gap opening below Tc the DOS at Ef will decrease\nbelow Tc and the phonon linewidth will decrease as a consequence (same with Fano factor). Again the key parameter is the\nproximity to Tc.\nMy conclusion is that no clear conclusion can be made about the evolution of electron-phonon coupling based on the\npresent Raman data, everything can be explained based on change in the electronic structure and fluctuations to which the\nphonon couple to. I would urge the authors to consider very carefully this important point.\nMore minor point: stating \"cross-polarization\" is too vague, the authors should state the polarization configuration with\nrespect to crystal axis using standard Porto notation. If the relative orientation are not known then it should be stated.", "reviewer_3": "The authors have thoroughly addressed all of my previous comments. In addition to providing clear and convincing\nresponses, they have carried out new experiments, which have significantly improved the quality and impact of the\nmanuscript. The revised version is now much stronger, with enhanced clarity, completeness, and scientific value.\nGiven these substantial improvements, I believe the manuscript has reached a level of quality appropriate for publication in\nNature Communications, and I recommend it for acceptance."}, "author_response": "We appreciate the reviewer’s careful evaluation of the revised manuscript and their constructive comments. Below, we address each point in detail and describe the corresponding changes made.\n(1) Concerning the Fano lineshape, I agree that a fully self-consistent analysis coupling the QES and the bare phonon was shown to yield similar results as a more naive QES + Fano by ref. 51. However, this may not always be the case and it cannot be discarded as a source of discrepancy between the different criteria for determining T . I would urge the authors to mention this limitation on top of possible effects like anharmonicity.\nWe agree with the reviewer that the complexity in the Raman data analysis should be explicitly noted as a possible source of discrepancy between the different criteria for determining T . To further examine the thickness dependence of T , and following the suggestion of another reviewer, we have performed temperature-dependent resistance measurements on samples with varying thicknesses. This provides an independent means of estimating T (revised Supplementary Note 5). The resulting thickness-dependent T is more consistent with that obtained from the analysis of the QES (revised Fig. 2j). We therefore believe that the conclusion of an enhanced T upon thickness reduction is well supported.\nChanges: Wehaveaddedafewsentences(highlightedinred)tothefollowingdiscussion.\n“Quantifying T from mode 2 parameters, however, is less reliable, as they show different characteristic temperatures even for the same sample. This could be due to the additional effect of lattice anharmonicity [47], which becomes enhanced at high temperature and near structural phase transitions. Another source of discrepancy between the different criteria for determining T using mode 2 parameters arises from the data analysis itself. In principle, this could be remedied by employing a fully self-consistent analysis that couples the QES and the bare phonon [52]. However, for Ta NiSe , such an approach has been shown to have only a minimal influence on the analyzed results [51]. We further performed resistance measurements on samples with varying thickness (Supplementary Note 5). The kink in the temperature derivative of the resistance as an indication of the phase transition [58] was observed in all measured samples, although it becomes weaker in atomically thin samples. The corresponding T values are plotted in Fig. 2j as open symbols, which confirm the increased T as the sample thickness decreases.”\n(2) The new set of data in figure 5 nicely illustrate the fact that QES intensity tracks T with gate and T. If we assume, as the authors do it seems, that the QES is of electronic origin, then it is present whether there is a B phonon to couple to or not. In this scenario the QES is associated to B symmetry breaking electronic fluctuations (as in Fe SC where they are of B character, but there are no Raman active B phonon to couple there) which are present independent of electron-phonon coupling (which is also the interpretation of ref. 51), I do not see the need to invoke enhanced electron-phonon coupling under electron doping to explain the data. Since electron doping decreases T , making closer to the sample temperature when measuring below T , the QES is enhanced. This is a simple scenario. My point is that the Raman QES data per-se do not allow a clear conclusion on the role of electron-phonon coupling for the evolution of T with doping.\nA similar conclusion can be drawn from mode 2: since there is a gradual gap opening below T the DOS at E will decrease below T and the phonon linewidth will decrease as a consequence (same with Fano factor). Again the key parameter is the proximity to T . My conclusion is that no clear conclusion can be made about the evolution of electron-phonon coupling based on the present Raman data, everything can be explained based on change in the electronic structure and fluctuations to which the phonon couple to. I would urge the authors to consider very carefully this important point.\nWe thank the reviewer for raising this important point. We agree that the doping- and temperature-dependent QES and mode 2 parameters can be largely understood in terms of doping-induced changes to T , which provides a simpler interpretation of the Raman data. Accordingly, we have revised the manuscript to remove references to electron-phonon coupling in the discussion of the doping dependence of the QES (see Changes below).\nThe only remaining mention of electron-phonon coupling in the discussion on the doping dependence concerns the sentence: “At 150 K, where the QES is weak, mode 2 softens and broadens with doping (Supplementary Note 6), which could be due to doping-induced enhancement of either the electron-phonon coupling or the electronic density of states at the chemical potential.” As discussed previously, the linewidth γ of a phonon mode j with frequency ω can be expressed as γ ∝ N ω2λ , where λ is the contribution of mode j to the total electron-phonon coupling constant and N is the electronic density of states at the Fermi energy [Phys. Rev. B 48, 8412 (1993)]. Therefore, it is more precise to note both factors when discussing the doping-induced effect on the phonon linewidth.\nWhile the Raman data alone do not allow a definitive conclusion regarding electron-phonon coupling, several recent studies have emphasized its important role in driving the phase transition in this system (Refs. 42, 43, and 47). For completeness, we briefly mention in the Discussion section the possibility that thickness reduction may influence T through changes in electron-phonon coupling, potentially affecting the relative stability of the monoclinic and orthorhombic phases and the transition between them. Doping may have a similar effect. Further experimental and theoretical investigations will be required to test this conjecture.\nChanges: We have made changes to the following two paragraphs.\n“...The QES intensity in our Raman data also peaks at T , indicating a direct link to such fluctuations and justifying using the QES to quantify them, analogous to its application for quantifying charge nematic fluctuations in iron-pnictide superconductors [54]. Its enhancement under electron doping below T suggests strengthened fluctuations of the coupled electron-phonon system, driving it closer to a structural instability, consistent with prior potassium dosing studies and DFT calculations [43]. Conversely, hole doping suppresses the fluctuations, revealing a clear asymmetry between electron and hole doping. Above T , hole doping enhances the QES, indicating a revival of critical fluctuations and a return toward the instability.\nMoreThis picture is further confirmed by more systematic doping dependent Raman measurementswere performedatafinertemperaturestepinanotherbilayerdevice, whichallows us to extract the temperature dependence of the QES spectral weight at different doping levels. The results, shown in Fig. 5a, are consistent with the doping dependence at selected temperatures described above. By fitting the temperature dependence of the QES above 250 K to a Gaussian function (Fig. 5b), T is quantified as the center of the Gaussian peak, and its doping dependence is shown in Fig. 5c. We estimate an enhancement of 5 K and reduction of 34 K under the maximum hole and electron doping of 2.6×1013 cm−2, respectively. The reduction of T under electron doping is consistent with previous potassium dosing studies and DFT calculations, which show that electron doping tends to stabilize the orthorhombic phase [43]. The doping dependence of the QES at selected temperatures, as shown in Fig. 4, can therefore be attributed to the doping-induced modification of T , which shifts the peak in the temperature dependence of the QES spectral weight (Fig. 5a–b). The spread of the QES temperature dependence, quantified as the full width at half maximum of the Gaussian peak, grows as T decreases (Fig. 5c), indicating a more extended fluctuation regime under electron doping. ”\nMore minor point: stating “cross-polarization” is too vague, the authors should state the polarization configuration with respect to crystal axis using standard Porto notation. If the relative orientation are not known then it should be stated.\nIn the standard Porto notation, the polarization configuration adopted in the thickness- and doping-dependent Raman measurements was b(ac)b, where a, b, and c denote the crystal axes of Ta NiSe . Following the notation in Ref. 52, we refer to this configuration more simply as the ac scattering geometry.\nChanges: In the main text and figure captions, “crossed polarization configuration” is modified to “crossed polarization (ac) configuration”.\nResponse to Reviewer #3\n——————————————————————————————————–\nThe authors have thoroughly addressed all of my previous comments. In addition to providing clear and convincing responses, they have carried out new experiments, which have significantly improved the quality and impact of the manuscript. The revised version is now much stronger, with enhanced clarity, completeness, and scientific value.\nGiven these substantial improvements, I believe the manuscript has reached a level of quality appropriate for publication in Nature Communications, and I recommend it for acceptance.\nWe appreciate the reviewer’s positive comments and constructive suggestions, which have strengthened this work. We hope that the revised manuscript will make a valuable contribution to the field."}, {"round": 3, "reviewer_comments": {"reviewer_2": "The changes made by the authors are adequate and the discussion is now more sound. The manuscript is now suitable for publication in Nat. Comm."}, "author_response": ""}]

Decision letter