Pomeranchuk instability from electronic correlations in CsTi_3Bi_5 kagome metal

Domain: physics
Published: 2025-12-23
Editorial outcome: major_revision

Abstract
Electronic nematicity, the spontaneous breaking of rotational symmetry, has emerged as a key instability in correlated quantum systems. CsTi3Bi5, a kagome metal of the AV3Sb5 (A=K, Rb, Cs) family, hosts rich unconventional electronic phases, yet the origin of its nematicity remains unsettled. Here, we combine polarization-dependent angle-resolved photoemission spectroscopy with functional renormalization group calculations on a fully interacting ab initio model. We reveal an orbital-selective nematic deformation in the low-energy band structure and identify a finite angular momentum (d-wave) Pomeranchuk instability driven by electronic correlations in specific orbital channels and detuning from Van Hove singularities. Our results establish a direct link between orbital selectivity and symmetry-breaking instabilities in CsTi3Bi5, providing a microscopic framework for nematic order in kagome systems.

Introduction
Kagome metals represent a remarkable class of materials that has garnered significant attention in condensed matter physics due to the richness of observed correlated phases. Owing to the unique topological properties of the underlying kagome lattice, highly itinerant electrons naturally display a variety of massless Dirac-like states, Van Hove singularities (VHss) with partial sublattice polarisation1–3, and compact localised states with dispersionless bands within a single bandstructure4–7. Combined with the inherent geometrical frustration of any interacting model on the kagome lattice, this provides ideal conditions for the realisation of many exotic phases long sought in other correlated electron platforms including persistent loop current formation8–11 and superconducting pairing modulations12–15. While the experimental evidence for these phases is still subject of ongoing debates, electronic nematicity, i.e., the breaking of rotational symmetry in the charge ordered state, strikes out as one of the few universal features across all kagome compounds16–23. Despite its critical relevance also for the subsequent superconducting transition at low temperature15,24–26, the origin and microscopic mechanism governing nematicity has remained elusive, necessitating focused investigation.

Among the diverse array of kagome systems, the first synthesized AV3Sb5 family (where A=K, Rb, Cs)27,28 has remained the most extensively studied. However, the simultaneous emergence and intertwining of the various correlated electronic phases induced by a lack of scale separation between different symmetry breaking phases poses a significant challenge to the understanding of the underlying mechanisms driving the various many-body effects and remains a current thread to any thorough assessment of kagome metals29,30.

In this study, we utilize CsTi3Bi5 as a unique platform to explore the phenomenon of rotational symmetry breaking on the kagome lattice. The absence of an accompanying translational symmetry breaking typical of other ’135’ kagome metals render this compound an ideal experimental setting for a detailed study of nematicity20,31,32. To elucidate the underlying mechanisms, we adopt a synergistic approach that combines light-polarization-dependent angle-resolved photoemission spectroscopy (ARPES) with ab initio based field theoretical methods. Our findings reveal that electronic nematicity in CsTi3Bi5 reduces the symmetry of the system through an orbital-selective mechanism, with dominant contributions from the planar dxy and dx2−y2 orbitals of Ti. Crucially, our results indicate a purely electronic origin of this effect, stemming from the frustrated long-range Coulomb repulsion and considerable VHs detuning from the Fermi level. This supports the existence of a d-wave Pomeranchuk instability (PI) in CsTi3Bi5, characterized by the spontaneous breaking of point group symmetry driven by the divergence of an associated susceptibility of the electronic system33, and a sublattice charge imbalance. In this respect, our findings notably differ from the prevailing view that nematic charge order in titanium-based kagome metals arises through a bond-type order with zero total momentum20,31. That interpretation originated from observations of the momentum-dependent character of nematic Fermi surface distortions, which were incorrectly claimed to be incompatible with a straightforward charge imbalance across unit cell sites20, as well as from many-body calculations that favored non-local charge orders34.

Recently, PIs have sparked recurrent interest. In the context of multi-layer graphene35,36, the PI is realized as a partial valley polarisation and results in exotic half and quarter-metal states where the isospin provides the additional ingredient to realize the spontaneous rotational symmetry breaking37. The surface states of topological elemental arsenic (α-As) have now also been reported to support a genuine PI38, despite weak correlations. Our work on CsTi3Bi5, on the other hand, puts forward PI as a generic instability in hexagonal lattice systems with a non-trivial sublattice degree of freedom, advancing our understanding about the microscopic mechanisms governing electronic nematicity.

Results
CsTi3Bi5 exemplifies the defining structure of the ’135’ kagome metals, being isostructural with the extensively studied V-based compounds AV3Sb5. Within the Bi-Ti plane, the kagome lattice formed by Ti atoms (Fig. 1a) is pivotal in establishing a complex electronic structure characterized by itinerant Dirac-like states, VHss, and flat bands34,39–41. Our angle-resolved photoemission spectroscopy (ARPES) measurements unveil sharply defined bands that disperse across the Fermi level, underpinning the metallic nature of the band structure and resulting in a multifaceted Fermi surface (Fig. 1c) defined by multiple sheets with an overall hexagonal geometry. Importantly, the experimental spectra illustrated in Fig. 1d, e exhibit concordance with predictions from first-principles calculations, that is even enhanced when taking into account surface termination effects (Supplementary Note I), thereby reinforcing the reliability of our findings. We emphasize that these data are obtained through the integration of ARPES spectra collected using both circular-right and circular-left polarizations, ensuring comprehensive resolution of the principal electronic features. Additional spectra recorded with linear vertical and horizontal polarizations, along with varying photon energies, are provided in Supplementary Fig. 7 and Supplementary Fig. 8. Notably, the electronic band structure of CsTi3Bi5 reveals a pronounced absence of kz dependence for the dispersion at the energy of the Fermi level, with no significant variations detected upon photon energy adjustment (also consistent with the almost cylindrical ab initio Fermi surface discussed in Supplementary Note II), thereby underscoring its distinct two-dimensional character.

Significant spectroscopic variations emerge when the polarization vector of the incident light is modified. If the sample surface lies in the xy plane, with the z axis defined as perpendicular to this surface, then in our experimental geometry the light impinges on the sample at a 45∘ angle, i.e., the photon momentum is along the x and z directions. Although the crystal possesses several mirror planes, in this context we define the mirror plane as the one containing the incident light - namely, the xz plane (in reciprocal space, this mirror plane contains the Γ-K direction.). Within this configuration, we refer to linear horizontal polarization as the electric field lying in the xz plane, and to linear vertical polarization as the field oriented along the y direction, perpendicular to the mirror plane. This is the geometry described in the schematics of Fig. 2a, b. The photoemission intensity is strongly influenced by the polarization of the incident light, as it directly affects the transition probability between initial and final states. This probability is proportional to the square module of the dipole matrix elements |⟨Ψk fin |A·p|Ψk in ⟩|2 and is non-zero only when the symmetry of A·p|Ψk in ⟩ coincides with that of the final state. For example, assuming the detector corresponds to a final state represented by a plane wave of even parity (| + ⟩), only transitions from initial states of even parity are allowed - specifically, +| + ⟩ and −| − ⟩. In contrast, transitions involving initial states of odd parity are forbidden (±| ∓ ⟩), leading to a vanishing photoemission intensity42,43. Further discussion about the linear dichroism is reported in the Supplementary Information.

Understanding these polarization-dependent effects allows us to elucidate the orbital contributions to the electronic structure of CsTi3Bi5. This distinction is evident in the varying photoemission intensities observed for linear horizontal and vertical polarizations, as depicted in Fig. 2a, b, that highlight contrasting spectroscopic features. To enhance visualization, we also analyze the difference between the two polarizations (see Supplementary Note IV and Supplementary Fig. 4), providing insights into the relative contributions of even and odd orbitals with respect to the mirror plane.

Our findings reveal that the inner circular pocket (α) centered at Γ is predominantly composed of orbitals with even symmetry. The two internal hexagonal Fermi surface sheets (β) and (γ), rotated by 30 degrees relative to one another, exhibit a substantial contribution from both even and odd orbitals. To further elucidate the relative contributions of even- and odd-symmetry orbitals while mitigating potential matrix element effects, we also performed light-polarization-dependent ARPES measurements across various experimental geometries, including rotations of the analyzer slit along the Γ-K direction (see Supplementary Fig. 11). The response of the two hexagonal Fermi surface sheets to this rotation is notably distinct. The intensity of the external (γ) sheet adapts in accordance with the sample orientation. In contrast, the internal (β) sheet demonstrates a shift in its spectral weight contribution, indicating a more complex orbital character. This behavior suggests that the external hexagonal (γ) sheet is primarily composed of in-plane orbitals, while the internal (β) sheet possesses a mixed orbital character. A solid support for this symmetry-based interpretation arises from the orbitally projected ab initio calculations shown in Fig. 3a and Supplementary Fig. 3, which unambiguously assign in-plane (dxy, dx2−y2) orbitals to the external (γ) sheet and a predominance of out-of-plane (dxz, dyz) orbitals to the internal one (β). Remarkably, this conclusion is consistent across multiple experimental configurations. Additionally, our use of varied light polarizations, different photon energies, and the absence of significant kz dispersion, further confirm that our findings are robust against photoemission matrix element effects.

We now focus on the nematic character of the system, which has been previously reported by STM experiments31 to lead to a reduction of symmetry from C6 to C2. This alteration signifies that one of the three equivalent high-symmetry directions along both the Γ-K and Γ-M axes becomes distinct in terms of momentum length. To investigate this phenomenon, we perform Lorentzian fits to the momentum distribution curves at the Fermi level for inequivalent Γ-M directions. Our analysis reveals a subtle variation in the momentum positions of the bands that constitute the Fermi surface, primarily influenced by in-plane components (see Supplementary Fig. 10). However, the observed difference is minimal, and given the momentum resolution, it remains challenging to definitively attribute these variations to nematicity. Such effects are anticipated to be subtle, as highlighted in previous works20,31.

Autocorrelation maps (ACMs) of the Fermi surface (see Fig. 2) instead yield valuable insights into the symmetries inherent in the electronic structure, often elucidating even the most subtle features (Methods section for more details). CsTi3Bi5 serves as an ideal platform for this type of investigation, owing to the large extension of its nematic domains31. By generating ACM under varying light polarizations, we are able to uncover the nematic C2 symmetry of the sample, independent of light polarization. This is clearly demonstrated in Fig. 2d, where profiles along the three Γ-M and Γ-K directions are presented. These profiles distinctly reveal that two directions remain equivalent, while the third exhibits a marked difference. The q-vector extracted from both the Γ-M and Γ-K directions in the ACM corresponds to scattering within the (γ) Fermi surface sheet, which is primarily composed of in-plane orbitals. Further corroboration of our findings comes from conducting the same analysis after rotating the sample (see Supplementary Fig. 11), or changing the photon energy (see Supplementary Fig. 12). Despite certain shape differences likely attributable to matrix element effects, two profiles consistently maintain equivalence while one consistently differs, thereby reinforcing the reduced C2 symmetry of the system. Interestingly, the reduced symmetry persists throughout temperatures ranging from 16 K up to room temperature (RT), as we show in Supplementary Fig. 13. This result demonstrates that the nematic transition in CsTi3Bi5 is larger than RT, as no sign of transition was detected, consistently with a recent infrared spectroscopy study which also suggests the onset of nematicity in CsTi3Bi5 to happen above RT44.

The subtle signatures of rotational symmetry breaking presented above hinder a definite determination of the nematic order parameter based solely on the experimental data. The latter, in fact, points towards the direction that the system is nematic: based on autocorrelation maps showing a C2 symmetry and on electronic structure momentum distribution curves fitting, strong hints about this putative reduction are demonstrated. However, for substantial proof, it necessitates a detailed theoretical analysis on a realistic model to distinguish among the possible underlying microscopic origins45. In other “135” kagome compounds like AV3Sb5, nematicity is believed to emerge from a coupling between the different translation symmetry breaking orders at the inequivalent M points, that favors an imbalance between them to minimize the free energy within the symmetry broken phase11,46 and is highly prone to external effects like in-plane strain and magnetic field47. The absence of a 2×2 superlattice modulation in the charge ordered phase of CsTi3Bi5 asks for a different explanation. Likewise, CsTi3Bi5 does not feature unstable phonon modes and the electron-phonon coupling is negligible39. Since phonons are not expected to drive a nematic instability in d=3 dimensions48 and hence seem not able to explain the accumulated experimental evidence, an exclusive electronic mechanism seems to drive the nematic transition in CsTi3Bi5.

To achieve a detailed understanding of the nematic transition and pin down its order parameter, we supplement the full ab initio dispersion (see Methods) with a electron-electron interactions between the Ti d-orbitals via a site-local Kanamori vertex and density-density interactions up to second-nearest-neighbor distance. We employ the FRG49–52 in the timely truncated unity formulation (see refs. 53–57 and Methods) to analyse possible many-body instabilities of the system in an unbiased manner.

The large number of orbitals required to capture the low energy manifold is a common thread to any theoretical assessment of kagome metals. Hence, extracting the relevant degrees of freedom from the ab initio band structure is highly desirable to capture the kinetic theory in sophisticated many-body techniques. In the orbital-resolved band structure of Fig. 3d, e the three characteristic kagome bands featuring a flat band, a Dirac cone at K and VHss at M are clearly visible. As shown in Supplementary Fig. 4, the orbital parity of the model Fermi surface is corroborated by our ARPES linear dichroic data. This low energy kagome manifold is exclusively supported by the in-plane d-orbitals of Ti, while the out-of-plane Ti d and Bi p orbitals only contribute to bands with small spectral weight at the Fermi level. The orbital hierarchy is directly reflected in the static response of electronic states to quantum fluctuations of a certain orbital structure, that can be quantified by the orbital resolved bare susceptibility tensor

χ0 o1 o2 o3 o4 (q) = 1/β ∑n BZ ∫ d k G o2 o4 (k,ωn) G o3 o1 (k+q,ωn) + h.c., (1)

where G oo (k,ωn) is the single-particle propagator with momentum k, fermionic Matsubara frequency ωn and orbital quantum numbers oi, and the integral over momentum and frequency is normalized by the Brillouin zone volume V BZ and inverse temperature β, respectively. The physical non-interacting susceptibilities, i.e., χ0 with o1=o3, o2=o4, are depicted in Fig. 3f. The effect of the isolated VHs of pure (p-type) sublattice character1,58 in the vicinity of the Fermi level (Fig. 3d) is directly apparent: while the peak in the non-interacting susceptibility is shifted from the Van Hove scattering vector M closer to Γ due to the detuning of the chemical potential from the perfect Van Hove filling, the in-plane contributions to χ0 (blue) dominate over out-of-plane parts (red). Hence in-plane (anti-)screening processes play the decisive role in the determination of the system’s ordering propensities and the emergence of a symmetry broken phase.

We therefore evaluate the FRG flow equations for the two-particle vertex (lhs. of Fig. 4a) with the in-plane orbitals as interacting subspace. We uncover a Q=0 divergence in the interacting charge susceptibility (see also Supplementary Fig. 6), that corresponds to an intra-unit cell charge density wave order transforming under the two dimensional E2 irreducible representation of the crystalline point group P6/mmm. In addition to the two-particle interaction, we also incorporate the static self-energy into the FRG flow (rhs. of Fig. 4a). Flowing into the symmetry broken phase59 then yields the linear combination of order parameters minimizing the free energy. The resulting charge imbalance of the nematic state is depicted in the inset of Fig. 4c. Due to our choice for the interacting manifold, the reduction of C6 symmetry down to C2 is most prominently observed on the (γ) Fermi sheet in Fig. 4c, while the other sheets only exhibit a proximity induced warping due to hybridisation with the (dxy, dx2−y2) orbitals in good agreement with the ACMs of Fig. 2 and Supplementary Fig. 10.

To pin down the microscopic origin of this nematic charge order, we analyze the response of the system to the most prominent fluctuation channels on the kagome lattice by monitoring their instability scale throughout the FRG flow. In Fig. 4b, the fluctuation strength Θ, extracted as the eigenvalue of the respective channel at a given wavevector Q, is depicted as a function of the energy cutoff Λ. We note that the Q=0 charge order does already feature a sizable susceptibility compared to other orders involving translation symmetry breaking, i.e., with a finite Q vector. This can be attributed to the fact that the long-range interaction supports charge imbalance between adjacent sites already at the mean-field level (see Supplementary Note V and Supplementary Fig. 6 for a comparison between mean-field and FRG phase diagrams). In the weak to intermediate coupling regime, this potential energy gain is usually dwarfed on the kagome lattice by the pronounced Van Hove scattering, that promotes a charge density with the nesting vector M58,60 and a 2×2 supercell reconstruction. In most members of the 135 family, this charge modulation is indeed realized11,16 due to the existence of low lying VHss of pure and mixed sublattice type in the electronic spectrum2,3. Within the FRG formulation, this effect can be directly observed via a strong enhancement of the M-point charge susceptibility as the cutoff approaches the Fermi level60,61. In the case of CsTi3Bi5, there is only a single p-type VHs close to the Fermi level and the chemical potential of the system is detuned from Van Hove filling by ≈0.22 eV, which substantially reduces the effect of Van Hove nesting (see Supplemental Note V for further details). With the accompanied sublattice interference still active1, the accessible phase space for on-site scattering in the Van Hove channel remains limited and the local repulsion can be dynamically overscreened by long-range interactions, which suppresses the formation of local moments and the emergence of magnetic orders.

In addition, the potential energy gain in the geometrically frustrated kagome geometry is also supplemented by a kinetic energy gain. The charge imbalance pushes two VHss in the quasi-particle band structure up in energy, while the third one moves closer to the Fermi level, as shown in Fig. 4d. Thereby, the spectral weight is more efficiently moved away from the Fermi level than by the opening of a charge ordered gap at the VHs. This combined effect of electronic kinematics and interactions leads to a spontaneous breaking of rotational symmetry by implying non-uniform sublattice occupation without enlarging the unit cell.

In this respect, CsTi3Bi5 provides a rare material realisation of a d-wave Pomeranchuk instability in its original sense, namely the spontaneous breaking of point group symmetry by correlation effects resulting in a smooth deformation of the Fermi surface33,62. This is directly reflected in the divergence of the associated scattering vertex in Fig. 4b at the critical scale. In the emergence of a PI, the non-trivial sublattice structure of the kagome lattice supplies additional degree of freedom and finite angular momentum channels, that allow to circumvent the constraints for PI in conventional Fermi liquid theory: if there is only one electron species present in the system, a PI can only manifest as bond or current order63, whose emergence is often hampered by charge and spin conservation rules64,65. While these states accumulate a finite angular momentum in the relative coordinate between the constituents of the particle-hole pair, the obtained d-wave PI state in CsTi3Bi5 is characterized by a l=2 quantum number in the total angular momentum of the charge order parameter within the unit cell. Therefore, a charge accumulation on one site and orbital can be compensated by a partial depletion on a different site, as apparent from Fig. 4c.

Methods
Bulk single crystal synthesis
Single crystals of CsTi3Bi5 were grown using a conventional flux-based growth technique, as described previously69. The elemental ingredients for synthesis included Cs (liquid, Alfa 99.98%), Ti (powder, Alfa 99.9%), and Bi (shot, Alfa 99.999%). These materials were loaded into a tungsten carbide milling vial in a stoichiometric ratio of 1:1:6 and milled for an hour under an argon atmosphere to obtain precursor powder. After milling, the precursor powder was transferred into an alumina crucible and sealed in a separate stainless-steel tube. The samples were heated at 900 ∘C for 10 h and then cooled at 3 ∘C/hr to 500 ∘C. Once the growth period was over, the tube was broken inside a glove box filled with Ar-gas. The shiny plate-like single crystals were separated gently, and stored inside a box filled with an inert gas atmosphere. As also shown in previous studies31, the single domains are extremely large in CsTi3Bi5 compared to those of the vanadium-based sister compounds. This aspect permits the extraction and analysis of autocorrelation maps from the ARPES spectra of Fermi surface.

ARPES experiments
The samples were cleaved in ultra high vacuum (UHV) at the pressure of 1×10−10 mbar. The ARPES data were acquired at the CASSIOPEE end station of the synchrotron radiation source SOLEIL (Paris, France). The energy and momentum resolutions were better than 10 meV and 0.018 Å−1, respectively. The temperature of the measurements was kept constant throughout the data acquisitions (15 K), except for the temperature dependent ARPES measurements reported in Supplementary Fig. 13, for which a scan up to room temperature was done. The Fermi surfaces were collected by rotating the angle orthogonal to the analyser slit, keeping the samples in the center of rotation. To exclude possible matrix element effects, several light polarizations, geometries, and photon energies were used for the data acquisition, yielding always consisting results.

Autocorrelation analysis of ARPES data
To analyze symmetry breaking and identify characteristic scattering vectors in the Fermi surface, we employed the autocorrelation method (ACM) on angle-resolved photoemission spectroscopy (ARPES) intensity maps. This approach allows for the extraction of periodic features in momentum space directly from within a single Brillouin zone (BZ), without requiring data acquisition over multiple repeated BZs.

The autocorrelation function A(q) is defined as the self-convolution of the ARPES intensity I(k) in momentum space:

A(q) = ∫BZ I(k) * I(k+q) dk, (2)

where k and q are momentum vectors. The integral is performed over the first Brillouin zone, where ARPES measurements are typically confined. Peaks in A(q) correspond to dominant scattering vectors connecting high-intensity regions of the Fermi surface. These vectors are often associated with quasiparticle interference (QPI) patterns, similar to those observed in Fourier-transformed scanning tunneling microscopy (STM) data.

Compared to a direct Fourier transform (FFT) of the raw Fermi surface, the ACM offers several practical and methodological advantages:
• Single-zone applicability: Unlike FFT, which ideally requires sampling across multiple Brillouin zones to reveal periodicity, ACM operates entirely within a single BZ. This is essential in ARPES, where the experimental constraints (angular resolution, matrix element effects, photon energy limitations) make extended-zone mapping impractical.
• Preservation of matrix element conditions: Since all momentum points used in ACM lie within the same BZ, the photon-induced matrix element effects remain consistent, thereby avoiding artefacts that could arise in FFT from variable photoemission intensities across zones.
• Enhanced signal-to-noise ratio: By construction, the autocorrelation enhances symmetric and repetitive structures while suppressing random noise and asymmetries. This makes it particularly effective for detecting weak periodic modulations in complex or reconstructed Fermi surfaces.
• Direct physical interpretation: Peaks in the autocorrelation map have a straightforward interpretation as representative scattering vectors. In systems exhibiting nematicity, charge order, or other symmetry-breaking phenomena, these features may correspond to nesting vectors or interaction-driven instabilities.

While FFT and ACM are formally related, both probing periodic structures in the data, the ACM is better suited for ARPES-derived momentum maps, especially when high-resolution measurements are confined to a single BZ. This method is increasingly being adapted for momentum-space techniques such as ARPES, offering a powerful tool for characterizing symmetry-lowering phenomena in correlated electron systems.

First-principles calculations
Bulk electronic structure calculations were performed using the full-potential local-orbital (FPLO) code (v.21.00-61)70. The unit cell has lattice constants of 5.82709 Å, 5.82709 Å, and 9.93612 Å. The exchange-correlation energy was parametrized within the local density approximation, following the Perdew-Wang 92 formulation71. A 12×12×12 k-grid was used to sample the Brillouin Zone, and the tetrahedron method was employed for integration. Calculations were performed in both the full-relativistic and non-relativistic frameworks.

A subsequent 36 bands Wannier functions model72,73, with symmetries accurately implemented, was constructed considering projections onto the following states: Caesium 6s; Titanium 3d z2, 3d xz, 3d yz, 3d x2−y2, 3d xy; and Bismuth 6p z, 6p x, 6p y, 6s. The Wannier states correspond to real spherical harmonics with a global quantization axis independent of the position within the kagome unit cell. The atomic orbital content of band n and momentum k is hence inferred from the electronic eigenstates ψ no(k) as |⟨ψ no |k⟩|2 where we sum over the same orbitals of equivalent atoms. From the analysis of the Fermi surface weights in the Supplementary Information it becomes directly evident, that only Ti 3d xz, 3d yz, 3d x2−y2, 3d xy and Bismuth 6p x, 6p y contribute sizable to the Fermi surface states. We obtain the Fermi surface weights of the in- and out-of-plane orbitals in Fig. 3 by summing up the contributions of the Ti 3d x2−y2, 3d xy; Bi 6p x, 6p y and Ti 3d z2, 3d xz, 3d yz; Bi 6p z, respectively.

Surface electronic structure calculations were performed using the Vienna Ab initio Simulation Package (VASP)74,75, using the projector augmented wave (PAW) method76. Exchange and correlation effects have been handled using the generalized gradient approximation (GGA)77–79 within the Perdew-Burke-Ernzerhof (PBE) approach80. The truncation of the basis set was set by a plane-wave cutoff of 500 eV and a 12×12×1 k-grid was used.

Many-body calculations
The bare susceptibility presented in Fig. 3f was calculated via Equation (1) in the famework presented in ref. 81 with a 600×600×1 integration grid in the Brillouin zone at an inverse temperature of β=100 1/eV. The different lines correspond to all possible orbital combinations of the physical susceptibility χ0 oooo (q) of the Ti d-orbitals, where blue (red) coloring indicates o1, o2 ∈ {dxy, dx2−y2} (o1, o2 ∈ {dxz, dyz, dz2}). The mixed contributions and also contributions from the Bi p and Cs s orbitals are all smaller and hence not shown.

To analyse the symmetry breaking propensities of CsTi3Bi5, we employed the functional renormalization group (FRG) in the truncated unity formalism (and its static approximation using a sharp frequency cutoff) as implemented in the divERGe library57. We restrict the interacting subspace to the in total six in-plane Kagome d-orbitals and employ a form factor cutoff of 1.1 in-plane lattice constants. This corresponds to real space form-factors up to fourth nearest neighbour, that have shown to produce converged results on the kagome Hubbard model59. The flow is started at Λ0 = 500 eV and integrated logarithmically with step size dΛ=−0.05Λ (using the adaptive Euler integrator shipped with divERGe). We classify the vertex as divergent when a single element surpasses Vmax = 50 eV. Even though effects of the out-of-plane dispersion on our FRG results are very weak (Supplementary Note II), we evaluate the scattering vertex in the respective channels on a regular 24×24×6 mesh in the full 3D Brillouin zone, with an additional refinement of 12×12×5 (per coarse mesh point, i.e., (24×12)×(24×12)×(6×5) points in total) for the integration of the internal loops in the flow equations. With the given k-mesh we are able to resolve energy scales down to 10−3 eV, which is well below the critical scales in our calculations.

The FRG interpolates between the full interacting model and an effective low energy theory close the Fermi level by adding a regulator Λ to the bare Green’s function, that recovers the non-interaction limit as Λ → ∞, and separates the electronic states into fast and slow modes50,51. As the cutoff Λ is successively lowered, the fast modes are integrated out and provide the screening background for the effective interaction of the remaining degrees of freedom via the mutual electron-electron interactions between fast and slow modes. The change of the effective n-particle interaction as a function of the cutoff is best expressed in an infinite hierarchy of coupled differential flow equations for the 2n-particle vertex, that is conventionally truncated within one loop order, i.e., at second order in the interactions, and gives way to the FRG flow equations depicted in Fig. 4a. By monitoring the evolution of the vertex throughout the flow, the FRG provides a transparent way to pinpoint the origin of a symmetry breaking transition. Its implicit diagrammatic resummation is unbiased in the sense that it includes all ladder diagrams of the particle-particle (P), direct particle-hole (D), and crossed particle-hole (C) channel, as well the leading order vertex corrections between them. This cross talk between the different diagrammatic channels proves quintessential to approach cooperative and competing instability phenomena characteristic for the highly frustrated kagome lattice14,59.

We mimic the mutual electron-electron interaction within the Ti d-orbital manifold at the bare level by the two particle vertex

H^I = U ∑ io n^io↑ n^io↓ + U0 ∑ io1<o2 σσ0 c^y io1σ c^y io2σ0 c^io1σ0 c^io2σ + J ∑ io1<o2 σσ0 c^y io1σ c^y io2σ0 c^io1σ0 c^io2σ + J0 ∑ io1≠o2 c^y io1↑ c^y io1↓ c^io2↓ c^io2↑ + V1 ∑ ⟨i,j⟩,o1o2 σσ0 n^jo1σ n^io2σ0 + V2 ∑ ⟨⟨i,j⟩⟩,o1o2 σσ0 n^jo1σ n^io2σ0 (3)

where n^ioσ = c^y ioσ c^ioσ is the fermionic number operator on site i with orbital o and spin σ. ⟨i,j⟩ and ⟨⟨i,j⟩⟩ indicate a summation over nearest neighbor (NN) and second nearest neighbor (NNN) sites respectively.

In accordance with the dominant role of the in-plane Ti d-orbitals in the low-energy kinematics described above (Fig. 3d, e), we choose the interacting manifold as the in-plane Ti d-orbitals, while all other orbitals only provide the screening background for the two particle interaction in this channel. We have confirmed the validity of our results via calculations with an interacting manifold containing the full d-shell (Supplementary Note III).

We use the universal Kanamori relations for the onsite interaction tensor U=U0+2J and J=J082 with J=0.8 eV and set U=4 eV, V1=1.5 eV, and V2=0.5 eV for all data provided in the main paper by employing the universal decay behavior for density-density interactions in the 135 family obtained from constrained RPA calculations83. All our results are robust against moderate changes of the interaction profile as shown in the Supplementary Information.

The FRG flow is stopped when encountering a divergence of the effective vertex. This indicates a phase transition and we can evaluate the leading instability by solving the linearized gap equation in the three diagrammatic channels. This amounts to diagonalizing the interaction vertex in the basis of composite two fermion operators. While the eigenvalue Θ gives the scattering amplitude of the respective fluctuation, the associated transfer momentum q and the eigenstate determine the spatial structure of the order parameter. For further details we refer to the standard review articles50,51. Here we just note, that we consider all possible transfer momenta contained in the momentum discretization of the BZ as well as all possible secondary momentum dependence of the order parameter captured within the employed form factor shells. In Fig. 4b we exemplarily display the maximum eigenvalues of the pairing channels at q=0 (SC), crossed particle-hole channel at q=M (SDW), and the direct particle-hole channel at q=0 (PI) and q=M (CDW) without restricting the functional form of the eigenvector at these momentum space points. Hence, the CDW e.g., also contains contributions from bond order fluctuations.

Peer review rounds
[{"round": 1, "reviewer_comments": {"reviewer_1": "The manuscript of Bigi et al shows ARPES results together with theoretical study on the origin of the electronic structure showing nematicity in CsTi3Bi5. I think the work has been conducted nicely and the paper is interesting. Basically, it can be said that this is a follow up of what is reported in Nature Physics 19, 1591 (2023) using ARPES in terms of experiment. The theoretical part is the novel part, suggesting a new origin of the observed Pomeranchuk instability. I suggest the authors to consider the following points before I can recommend it for publication in Nat. Comm.\n1. The authors refer to the autocorrelation maps (ACMs) of the Fermi surface to demonstrate the nematicity. I think it is important, but I would like the authors to first perform FFT on the raw data of the Fermi surface first before discussing the ACMs; it should be similar to the analysis of Nature Physics 19, 1591 (2023), and although it maybe not so clear, one should be able to recognize the symmetry lowering at this stage. If this is not the case, I cannot really trust the ACMs.\nThe authors also should add an explanation of the ACMs. How it is derived (the equation), the physical meaning, etc.\n2. In Figs. 1 d and e, the experimental data is compared to the DFT calculations. The overall consistency is good, but near the Gamma point at -0.6 to -0.8 eV, there are some deviations. The authors should discuss this point.\n3. The authors should explain what is the temperature that the Pomeranchuk instability occurs in the experiment, and how can it be explained from the present theory.", "reviewer_2": "This paper is composed of two parts. The first half is the experimental results of polarization-dependent ARPES performed on CsTi3Bi5. As a result, the authors found that of the four Fermi sheets (α, β, γ, δ), the α and δhave the character of out-of-plane orbitals, and the γhas the character of dxy,dx^2-y^2, It was proposed that the β has dxz,yz character. It was also confirmed that the rotational symmetry breaking phase (C2) proposed for this material occurs on the γfermi surface. In the second half, which is theoretical, the origin of the C2 state was proposed based on many-body calculation techniques (fRG). As a result, a Pomeranchuk state in which an imbalance in electron density occurs was proposed.\nIn various Kagome metal systems, it is well known that orbital properties are important to understand a lot of phase transitions. Thus, the present attempt to detect the orbital dependence of the fermi surface using polarization-dependent ARPES can be evaluated to a certain extent as valuable. On the other hand, there are many concerns where the theoretical logic for interpreting the experimental results is unclear or explanations are missing. Throughout the manuscript, the authors' conclusions are often exaggerated and leaps are made in comparison to the experimental results actually obtained. Therefore, this manuscript does not reach the level required for publication in Nature Communications. I recommend that the gap between them be properly filled and that the paper be considered for publication in more specialized journal.\nThe main concerns are as follows.\n1. The authors concluded that the following in-plane/out-of plane orbital dependence can be obtained from the Figure 2 ( a, c ). However, there appears to be jump between their conclusion and the experimental results.\n“Our findings reveal that the inner circular pocket (α) centered at Γ is predominantly composed of orbitals with out of plane symmetry. Additionally, the outermost hexagonal Fermi surface sheet (δ), forming the pockets around both the M and K points, exhibits similar characteristics. In contrast, the two internal hexagonal Fermi surface sheets (β) and (γ), rotated by 30 degrees relative to one another, exhibit a substantial contribution from in-plane orbitals”\nThis is because, from the linear polarized ARPES experiment, we estimate that the parity with respect to the mirror plane can be identified, then wouldn't the experimental results in Figure 2 ( a,c ) only reveal the parity with respect to the mirror plane? It seems not self-evident how the authors' conclusion that\n\"We have discovered that the α and δ planes have the character of interplane orbitals, and the β and γ planes have the character of in-plane orbitals \" is derived from this. The clear relationship between the light-polarization (LH and LV) and orbital symmetry should be formulated in the main text.\n2. In relation to the above, even if we consider a dxy orbital on kagome lattice, there are three sublattices. Then, three orbital do not form 90 degrees, so it seems non-trivial how the symmetry argument correspond to the results of this study, and a proper explanation is required.\n3. Furthermore, the authors conclude that the γ plane is dxy, dx^2-y^2 and the β plane is dxz, yz , based on the results of changing the slit direction from the original Γ-M to the Γ-K direction, but it is difficult to understand because they do not quote the figure numbers in the text. The actual experimental results are presumed to be extended date Fig.5, but shouldn't this figure be included since it is important for identifying the orbit? In addition, a discussion of the symmetry argument to derive these results should be included using equations, etc.\n“ we also performed light-polarization-dependent ARPES measurements across various experimental geometries, including rotations of the analyzer slit along the Γ-K direction. The response of the two hexagonal Fermi surface sheets to this rotation is notably distinct.”\n4. In identifying the orbitals of theβ and γ planes, it is necessary to clarify the starting point for the orbital candidate trajectory and how the trajectory was narrowed down based on the each h experimental results (with appropriate reference to the figure). For example, why do p orbitals, s, d3z^2-r^2 were excluded?\n5. From the results, shouldn't they include information on which orbitals are more prevalent at which wave number k on which Fermi surface? They concluded that there are two types of orbitals, dxy and dx^2-y^2 for the γ and dxz and yz for the β, but are the orbital weight the same along k, or do they change ? Including this result would also be helpful for comparison with first-principles calculations.\"\n6. Between extended date Fig. 5(d) and Fig. 2(d), the asymmetry of C2 appears to be more clearly suppressed in Fig. 5(d). What is the reason for this?\n7. The definition of the orbital basis should be explained. The authors call the orbitals in this study dxz , dyz , etc., but most readers who are not familiar with this material will not understand which orbital is which in real space. The definition of the orbital should be illustrated.\nRegarding the theory part, the conclusion is that a Pomeranchuk instability occurs based on a certain model parameter (initial value of the interaction U,V1,V2 and temperature). However, since there is almost no information on the robustness of the results, it is impossible to judge how reasonable the results are from the manuscript as it is, and doubts from many readers cannot be dispelled. It should be fundamentally revised.\n8.The authors conclude that such Pomeranchuk instability appear due to the inclusion of many-body effects by fRG compared to the mean-field method, but this conclusion is not immediately convincing because the results of solving the same model at the mean-field approximation level are not presented. The authors should present the mean-field results of solving the same model at the same temperature as fRG with the same magnitude of interactions (Of course, they should verify the robustness by changing U,V1,V2.). In particular, can they say that the Pomerantic instability (PI) does not appear in the mean-field approximation even if the magnitudes of the long-range Coulomb interactions V1 and V2 become somewhat large? If PI appears within the mean-field model, the validity of the magnitude of the interactions V1V2 in this study must be explained.\n9. It is known that low-temperature calculations are difficult in such mesh fRG because it is generally difficult to obtain fine k-meshes. However, was this calculation performed at a temperature that is consistent with the transition temperature of about 100K observed in experiments? If the answer is Yes, it should be discussed whether this is a reliable region with the present number of k-mesh. If the answer is No, is there a guarantee that the same PI will be dominant at a low temperature of around 100K? These temperature dependence needs to be discussed.\n10. In Fig. 4(b), the divergent growth of two-particle interaction vertex does not stop as a function of Λ~10000K. On the other hand, what would happen if they used parameters U,V that can be renormalized up to aroud Λ = 100K. Such parameter region would be more consistent with the experiment.\"\n11. In relation to the above, what is the estimate of the C2 transition temperature based on the fRG theory?\n12. I Since the main text only provides results for non-interacting susceptibility in Fig.3 (f), in order to compare with the fRG results, the results for susceptibility analyzed at the mean-field level using the same model as fRG ( J = 0.8 eV, U = 4, V1 = 1.5, V2 = 0.5 ) should be presented and compared.\n13. The validity of the results cannot be judged because there is no definition of PI, CDW, SDW, and SC in Fig. 4(b). The definition should be written in terms of wave number dependence and orbital dependence. Presumably, using the center of mass momentum Q of the particle-hole pair, the PI is Q = 0 CDW and the CDW is Q ≠ 0 CDW. In addition, when writing Fig. 4(b), is a specific wave number Q* ≠ 0 fixed? In that case, has the possibility of the CDW for other Q being larger been eliminated? Also, does this fRG formalism take into account the bond-type CDW proposed in the Vanadium-based kagome systems? The k- dependence of the obtained vertex should be explained. Also, where did the spin channel for Q = 0 disappear to?\n14. The definitions of the terms in-plane and out-of- plane in the color bar in Fig . 3(a) are not clear, making it difficult to judge whether the theoretical model is consistent with the experimental results. Which orbital of is in-plane dyx or dx^2-y^2 ? out-of-plane dxz or yz? It should be specifically written. Also, does the orbital's character differ depending on the k? At the same time, it should be shown whether their k-dependence is consistent with ARPES data. Based on that, does the even/odd nature of this orbital match the experimental results?\n15. For the theoretical analysis, they should compare the phase diagram by RPA and fRG by changing the values U,V1,V2 for showing the robustness of their main result. It is expected that PI disappear when V1,V2 are zero or quite small even if they performed fRG. What is the critical value of V1 and V2?\n16. Shouldn't the results of changing the filling also be discussed using the fRG phase diagram? Since they conclude that the distance of the van Hove singularity (vHS) from the Fermi surface is important for the emergence of the PI, it seems crucial to demonstrate that when the vHS gets closer to the Fermi surface, a different instability switches on.\"\nAnyway, the paper is not self-contained and cannot be evaluated because there is no explanation of the formulas actually used in the theoretical analysis or the temperature conditions. A minimum of formulas, a definition of what is being plotted on the graph, and an explanation of the parameters are necessary. In addition, data in a wider parameter range is also necessary to see the robustness of the results."}, "author_response": "We sincerely thank both referees for their constructive feedback and valuable\nsuggestions. Their insights have been instrumental in improving the quality of\nour work. Below, we provide a point-by-point response, detailing the changes\nmade to the manuscript and addressing all concerns raised.\nReviewer #1 (Remarks to the Author):\n------------------------------------------------------------------------------------------------\nThe manuscript of Bigi et al shows ARPES results together with theoretical\nstudy on the origin of the electronic structure showing nematicity in CsTi3Bi5. I\nthink the work has been conducted nicely and the paper is interesting. Basically,\nit can be said that this is a follow up of what is reported in Nature Physics 19,\n1591 (2023) using ARPES in terms of experiment. The theoretical part is the\nnovel part, suggesting a new origin of the observed Pomeranchuk instability. I\nsuggest the authors to consider the following points before I can recommend it\nfor publication in Nat. Comm.\n------------------------------------------------------------------------------------------------\nWe appreciate that the referee acknowledges the quality of our work and finds\nthe results interesting. The work Nat. Phys. 19 1591 (2023) cited by the referee\nhas been seminal in this field. While the evidence there pointed to the increasing\nrole of Coulomb interactions, particularly in relation to the multiorbital nature\nof the system in driving the observed phase, this aspect remained speculative,\nand no direct proof could be presented. As the referee highlights, the real\nnovelty of our work is the identification of the underlying mechanism as a\nPomeranchuk instability. Crucially, uncovering this required precise\norbital-resolved measurements, and this is where ARPES played a unique role.\nOur ARPES approach was specifically designed for this purpose, employing\nmultiple light polarizations, including horizontal and vertical linear polarization,\nas well as right- and left-handed circular polarization. This methodology allows\nfor direct orbital probing, leveraging the initial-final state symmetry match. In\ncontrast, STM and QPI techniques lack the necessary selectivity to resolve\norbital contributions in a comparable manner.\nAnother critical aspect is that resolving the electronic instability required\ndetailed knowledge of the Fermi surface contours and the momentum-resolved\nenergy dispersions with explicit orbital selectivity. While STM can provide\npartial information in this regard, its momentum resolution is far inferior to that\nof high-resolution, multi-polarization ARPES. Thus, ARPES was\ninstrumental—alongside our DFT analysis—in elucidating the microscopic\norigin of nematicity in CsTi3Bi5, which had remained unresolved even in\nprevious studies.\nA-PDF Comment Collector Demo. Purchase from www.A-PDF.com to remove the watermark\nMoreover, our ARPES measurements provided an important validation of the\nWannier functions-based downfolding model, especially for the binding energy\nof the p pocket, that was fitted by comparison to ARPES.\nz\nGenerally speaking, the full ab-initio model is too demanding to be analyzed\nwithin the FRG scheme. In this respect, downfolding is strictly necessary and\nrequires validation by experiment data. Our manuscript provides an instance,\nwhere both theory and experiment are working hand in hand to provide a well\nfounded understanding of the nematic transition in a kagome metal.\nTo further clarify these points, we have now improved the manuscript, making\nthe connection between ARPES and DFT more explicit in the text.\n------------------------------------------------------------------------------------------------\n1. The authors refer to the autocorrelation maps (ACMs) of the Fermi surface to\ndemonstrate the nematicity. I think it is important, but I would like the authors\nto first perform FFT on the raw data of the Fermi surface first before discussing\nthe ACMs; it should be similar to the analysis of Nature Physics 19, 1591\n(2023), and although it maybe not so clear, one should be able to recognize the\nsymmetry lowering at this stage. If this is not the case, I cannot really trust the\nACMs.\nThe authors also should add an explanation of the ACMs. How it is derived (the\nequation), the physical meaning, etc.\n------------------------------------------------------------------------------------------------\nWe thank the referee for this suggestion. However, in practice, this is not\nstraightforward. The Fourier transform of a Fermi surface obtained from\nangle-resolved photoemission spectroscopy (ARPES) can, in principle, provide\ninsight into the underlying electronic structure in real space. As the referee\ncorrectly points out, it may reveal symmetry breaking, such as nematicity,\ncharge density waves (CDW), or nesting.\nHowever, applying this method requires that the periodic unit being transformed\nis sampled multiple times. In other words, ARPES would need to probe several\nrepeating Brillouin zones simultaneously, which is not feasible, especially if one\naims to maintain the necessary momentum and energy resolutions required for\nthis study. Additionally, even if this were possible, the analysis would be\ncomplicated by geometric matrix element effects arising from the rotational\nsymmetries needed to access neighboring Brillouin zones.\nThis contrasts with scanning tunneling microscopy (STM), which operates in\nreal space. Since STM naturally captures multiple periodic units, its Fourier\ntransform provides information in reciprocal space that can, to some extent, be\ncompared to the ARPES-derived Fermi surface. However, ARPES retains an\nadvantage in its sensitivity to orbital character due to light polarization, which\nquasiparticle interference (QPI) does not directly provide. Notable examples of\nthis approach can be found in the works of the M.P. Allan’s group, such as\nBattisti et al. (https://www.nature.com/articles/s41535-020-00292-4).\nThe autocorrelation of an experimental Fermi surface map is the correct method\nfor extracting periodic information, as it can be performed within a single\nBrillouin zone while maintaining consistent matrix element conditions. This\napproach relies on a self-convolution operation applied to the ARPES intensity\ndata in momentum space. Mathematically, if I(k) represents the ARPES\nintensity at momentum k, the autocorrelation function is given by:\nA(q)=∫I(k)I(k+q) dk\nTherefore, due to the exact mathematical formulation of this approach, which\nwe have implemented in our manuscript, the autocorrelation method (ACM)\nenables the automatic identification of scattering vectors. Peaks in A(q)\ncorrespond to characteristic wave vectors q that connect different regions of the\nFermi surface. These wave vectors often signify quasiparticle interference\n(QPI), which is also observed in STM experiments. This is precisely why ACM,\nrather than a Fourier transform (FFT), is the appropriate method for comparison\nhere. An FFT would not be appropriate in this context.\nAnother key advantage of ACM is its ability to reduce noise by enhancing\nsymmetric features of the Fermi surface while suppressing asymmetric ones.\nThis makes it particularly useful for identifying periodic structures, especially in\nmaterials with reconstructed or multi-component Fermi surfaces.\nIt is important to note that ACM and FFT are however closely related in that\nboth reveal periodic modulations in momentum space. In materials with charge\norder, nematicity, or unconventional pairing, autocorrelation analysis can\nuncover periodic modulations that may not be immediately apparent in raw\nARPES data.\nWe have now added a proper description in the methods section to further\nsupport our choice.\n------------------------------------------------------------------------------------------------\n2. In Figs. 1 d and e, the experimental data is compared to the DFT calculations.\nThe overall consistency is good, but near the Gamma point at -0.6 to -0.8 eV,\nthere are some deviations. The authors should discuss this point.\n------------------------------------------------------------------------------------------------\nWe thank the referee for noticing this discrepancy. As it has already been\nreported in the literature, the additional spectral weight in the ARPES data is\nstemming from a topological surface state (TSS) that is not recovered within\nDFT surface calculations (see Nat. Commun. 14, 4089, 2023). Consequently, we\nalso do not see this additional band in our DFT surface band structure in the\nSupplemental Information Figure S1.\nNonetheless, since we are interested in Fermi surface instabilities and hence in\nthe electronic states around the Fermi level E , this discrepancy is not crucial for\nF\nour discussion of the Pomeranchuk instability and we feel encouraged to\nproceed with our theoretical analysis without considering the TSS.\nWe have added a discussion of this fact in the Supplemental Information I and\nthank the referee again for their suggestion.\n------------------------------------------------------------------------------------------------\n3. The authors should explain what is the temperature that the Pomeranchuk\ninstability occurs in the experiment, and how can it be explained from the\npresent theory.\n------------------------------------------------------------------------------------------------\nThis is a very important question. Experimentally, as in previous studies, we\nperformed ARPES at a temperature where the sample is already in the nematic\nstate. However, Ti-based kagome samples are highly reactive, and even mild\nthermal treatment leads to rapid sample degradation, causing an immediate\nloss of resolution and, in many cases, complete disappearance of the bands.\nAs detailed in the Methods section, meticulous sample preparation was\nessential, including glovebox handling to minimize degradation. This sensitivity\nmakes it challenging to experimentally determine the nematic transition\ntemperature. However, for similar materials, the transition temperature is\ngenerally higher. For instance, in V-based kagome systems, it has been reported\nto be 35 K (see Nat. Commun. 14, 3899, 2023). In another study (Nat.\nCommun. 14, 4892, 2023), the authors also faced difficulties in extracting this\ninformation experimentally, for similar reasons to us. Nevertheless, as in our\ncase, the observed C symmetry serves as clear evidence that the system is\n2\nalready in the nematic state, which is the main focus of our work.\nFrom the theoretical side, it is well known that the FRG within the single loop\napproximation fails to quantitatively predict the critical temperatures. This\ninsufficiency can be lifted by including multi-loop corrections to the FRG flow\nequations. However, multi-loop corrections have been so far applied only to the\nsingle-band Hubbard model, because their numerical complexity hinders any\napplication to multi-orbital systems, and are hence non suitable for the treatment\nof our realistic model for CsTi3Bi5.\nStill taking the critical scale 𝛬 of our FRG flows as a proxy for the critical\nC\ntemperature, we encounter a sensitive dependence of 𝛬 on the chosen\nC\ninteraction parameter. Recording the phase diagram in the newly added\nSupplementary Information Figure S6 we see that the Pomeranchuk instability\nis robust over a broad range of interaction values and features a critical scale\nspanning multiple orders of magnitude. Therefore, the proposed mechanism for\nthe nematic transition is compatible with a broad range of possible transition\ntemperatures.\nIn particular, the transition temperatures expected from theory are larger than\nthe temperature of the ARPES experiments (~15K), clearly supporting the\npresence of a finite nematic order parameter.\nWe have added a detailed discussion about the estimation of the critical\ntemperature from the FRG calculations in the Supplemental Information V and\nthank the referee for making us aware of this missing piece of information in the\nprevious version of the manuscript.\nReviewer #2 (Remarks to the Author):\n------------------------------------------------------------------------------------------------\nThis paper is composed of two parts. The first half is the experimental results of\npolarization-dependent ARPES performed on CsTi3Bi5. As a result, the authors\nfound that of the four Fermi sheets (α, β, γ, δ), the α and δ have the character of\nout-of-plane orbitals, and the γ has the character of dxy,dx^2-y^2, It was\nproposed that the β has dxz,yz character. It was also confirmed that the\nrotational symmetry breaking phase (C2) proposed for this material occurs on\nthe γ fermi surface. In the second half, which is theoretical, the origin of the C2\nstate was proposed based on many-body calculation techniques (fRG). As a\nresult, a Pomeranchuk state in which an imbalance in electron density occurs\nwas proposed.\nIn various Kagome metal systems, it is well known that orbital properties are\nimportant to understand a lot of phase transitions. Thus, the present attempt to\ndetect the orbital dependence of the fermi surface using polarization-dependent\nARPES can be evaluated to a certain extent as valuable. On the other hand,\nthere are many concerns where the theoretical logic for interpreting the\nexperimental results is unclear or explanations are missing. Throughout the\nmanuscript, the authors' conclusions are often exaggerated and leaps are made\nin comparison to the experimental results actually obtained. Therefore, this\nmanuscript does not reach the level required for publication in Nature\nCommunications. I recommend that the gap between them be properly filled\nand that the paper be considered for publication in more specialized journal.\n------------------------------------------------------------------------------------------------\nWe agree that orbital character has a crucial role in understanding phase\ntransitions, and this holds true for a wide range of materials. However, our study\nis not merely an investigation of orbital character; rather, it aims to elucidate\nhow different orbital manifolds contribute to reducing the system’s symmetry\nto C . We apologize if this was not conveyed clearly in the manuscript and for\n2\nany missing explanations. In addressing the referee’s concerns, which we have\ntaken seriously, we have made improvements to the manuscript, ensuring that\nour arguments and discussions are presented with greater clarity.\nThat said, we were somewhat surprised by the referee’s concern that our\nconclusions may be exaggerated. After carefully re-evaluating our statements,\nwe believe that we have been both honest and fair in our presentation. Prior to\nour study, a detailed understanding of the orbital character in this system was\nlacking. Furthermore, while previous STM-based studies identified nematicity,\nthey could not determine its microscopic origin beyond a speculative level. In\ncontrast, our work establishes this connection by leveraging various light\npolarizations in ARPES and developing a theoretical framework that suggests a\nPomeranchuk instability as a plausible driving mechanism. This represents a\nclear and substantiated conclusion based on our experimental and theoretical\nfindings.\nOnce again, we emphasize that our study extends beyond a simple orbital\nanalysis. By placing our results in the broader context of previous studies, we\npropose a mechanism that provides a coherent and elegant explanation for both\npast and present experimental observations.\nWe hope that the referee is pleased by the revised version of the manuscript,\nwhere we tried to make the strong bond between theory and experiment more\ntranslucent.\n------------------------------------------------------------------------------------------------\nThe main concerns are as follows.\n1. The authors concluded that the following in-plane/out-of plane orbital\ndependence can be obtained from the Figure 2 ( a, c ). However, there appears\nto be jump between their conclusion and the experimental results.\n\"Our findings reveal that the inner circular pocket (α) centered at Γ is\npredominantly composed of orbitals with out of plane symmetry. Additionally,\nthe outermost hexagonal Fermi surface sheet (δ), forming the pockets around\nboth the M and K points, exhibits similar characteristics. In contrast, the two\ninternal hexagonal Fermi surface sheets (β) and (γ), rotated by 30 degrees\nrelative to one another, exhibit a substantial contribution from in-plane orbitals\"\nThis is because, from the linear polarized ARPES experiment, we estimate that\nthe parity with respect to the mirror plane can be identified, then wouldn't the\nexperimental results in Figure 2 ( a,c ) only reveal the parity with respect to the\nmirror plane? It seems not self-evident how the authors' conclusion that \"We\nhave discovered that the α and δ planes have the character of interplane orbitals,\nand the β and γ planes have the character of in-plane orbitals \" is derived from\nthis. The clear relationship between the light-polarization (LH and LV) and\norbital symmetry should be formulated in the main text.\n------------------------------------------------------------------------------------------------\nWe agree with the referee that our initial description was too simplistic. First,\nwe emphasize that parity is defined with respect to the mirror plane. To better\ncapture this parity, the experiment was performed, as also shown in the\nSupplementary Information, using multiple sample rotations. Additionally, we\nrepeated the measurements on various crystals, consistently obtaining the same\nresults.\nWe acknowledge the need for a more detailed discussion, and in response to this\nconcern, we have removed the oversimplified description and provided a\nsignificantly more precise explanation in the Supplemental Information IV.\nSpecifically, in a photoemission experiment, the intensity from ARPES strongly\ndepends on a factor called photoemission matrix element: |M (k)|2\nfi\n𝑘 𝑘\nThis term is proportional to | < ϕ |ε∙𝑟|ϕ > |2. Here ε is a unity vector along\n𝑓 𝑖\nthe polarization of the vector potential A. This means that during the\nphotoemission process, the photoemission intensity will vanish unless that term\nis different from zero, which strongly depends on the symmetry of the initial\nstate wavefunction and the one of the final one. In general, it is different from\nzero when that object is an even function with respect to the system’s mirror\nplane. In experiments, at the detector, it is reasonable to consider a spherical\nsymmetry as the final state, so our discussion will have this caveat, which is\nwhat generally is assumed. This is also why we consider photoemission with\nrespect to the mirror plane, if our detector is away from the mirror plane there\nwould be the lack of a well-defined odd or even symmetry for it. That said, to\nhave non-vanishing photoemission matrix elements, the initial state at the\n𝑘\ndetector must be such that ε∙𝑟|ϕ > is even too. Our problem can be therefore\n𝑖\n𝑘\nreduced to the symmetry of |ϕ >. Still, referring to the mirror plane containing\n𝑖\nthe incident light, according to the schematics we described in the main text and\nshown also in the context of ACM, and assuming the detector corresponds to a\nfinal state represented by a plane wave of even parity (|+⟩), only transitions from\ninitial states of even parity are allowed - specifically, +|+⟩ and −|−⟩. In contrast,\ntransitions involving initial states of odd parity are forbidden (±|∓⟩), leading to a\nvanishing photoemission intensity. The example shown here should be done for\nall orbitals.\nWe use the information from the dichroism maps to indirectly probe the in- and\nout-of-plane character of the Fermi surface sheets:\nThe linear dichroism can be easily compared with the parity eigenvalue of the\nFermi surface states w.r.t. the respective mirror plane. This has been carried out\nexplicitly in a newly added section in the Supplemental Information and shows\na very good agreement between ARPES and DFT data (Figure S4). Because this\ncomparison proves the accuracy of the orbital weights redistribution on the\nFermi surface in the theoretical model, we can use the DFT data to infer the in-\nvs out-of-plane character of the Fermi surface sheets that is not directly\naccessible by ARPES due to the measurement geometry.\nWe have also highlighted this logical thread in the main text and hope to thereby\nclarify potential misunderstandings.\n------------------------------------------------------------------------------------------------\n2. In relation to the above, even if we consider a dxy orbital on kagome lattice,\nthere are three sublattices. Then, three orbital do not form 90 degrees, so it\nseems non-trivial how the symmetry argument correspond to the results of this\nstudy, and a proper explanation is required.\n------------------------------------------------------------------------------------------------\nWe thank the referee for making us aware of this potential source of\nmisunderstanding.\nThe Wannier model was constructed employing atomic orbitals (i.e. eigenstates\nof the angular momentum operator) with a global quantisation axis, i.e. the same\nfor each sublattice site in the kagome unit cell . Hence, if we talk about the d\nxy\norbital in the manuscript, this is really the real spherical harmonic with quantum\nnumbers l=2,m=-2.\nWe have explicated this in the Methods section of the manuscript and also\nprovided real space images of all orbitals in the Supplementary Information\nFigure S3.\nAn exemplary arrangement of d and d orbitals on the lattice is further\nxy x2-y2\nprovided in Supplementary Information Figure S4, where also the relative\norientation on the lattice becomes apparent.\n------------------------------------------------------------------------------------------------\n3. Furthermore, the authors conclude that the γ plane is dxy, dx^2-y^2 and the β\nplane is dxz, yz , based on the results of changing the slit direction from the\noriginal Γ-M to the Γ-K direction, but it is difficult to understand because they\ndo not quote the Figure numbers in the text. The actual experimental results are\npresumed to be extended date Fig.5, but shouldn't this Figure be included since\nit is important for identifying the orbit? In addition, a discussion of the\nsymmetry argument to derive these results should be included using equations,\netc.\n\" we also performed light-polarization-dependent ARPES measurements across\nvarious experimental geometries, including rotations of the analyzer slit along\nthe Γ-K direction. The response of the two hexagonal Fermi surface sheets to\nthis rotation is notably distinct.\"\n------------------------------------------------------------------------------------------------\nThe referee is correct, and we believe that this has been now covered by a more\ngeneral argument in the text. One thing we urge to stress is that such a precise\nattribution does not come from ARPES alone. It comes from orbitally resolved\nDFT calculations (new Supplementary Information Fig. S3 and S4), which\nagrees with orbitals having parity as the one elucidated by ARPES. We totally\nrealized that in our previous discussion, this was too simplistic and was not\ncorrect. We have now replaced it with a more general statement which is based\non symmetries.\n------------------------------------------------------------------------------------------------\n4. In identifying the orbitals of the β and γ planes, it is necessary to clarify the\nstarting point for the orbital candidate trajectory and how the trajectory was\nnarrowed down based on the each h experimental results (with appropriate\nreference to the Figure). For example, why do p orbitals, s, d3z^2-r^2 were\nexcluded?\n------------------------------------------------------------------------------------------------\nSince we are interested in the bandstructure around the Fermi level, we only\nconsider atomic orbitals from the valence shell of the respective atoms.\nConsequently, as stated in the method section, we include Cs 6s, the full 3d shell\nof Ti and 6s and the full 5p shell of Bi in our calculations and calculate the\nFermi surface weights for all of these orbitals. The single-particle spectrum of\nthe wannierised model matches the DFT bandstructure perfectly within a\nwindow of 4eV around the Fermi level justifying the choice of atomic orbitals\nto construct the Wannier model.\nTo reveal the parity character of the eigenstates we have calculated the Fermi\nsurface weights for all of these orbitals to reveal the parity of the Fermi surface\nstates and compare them with the ARPES data. It turns out, that the s orbitals do\nnot contribute to the Fermi surface at all. For all other orbitals, we display the\nFermi momentum resolved orbital weights in the new Supplemental Information\nFigure S3 and also added an additional note in the method section on how we\ncalculate the orbital content of the Fermi surface. From this, it becomes directly\nevident that also the dz² orbital does not contribute to the orbital makeup of the\nFermi surface.\n------------------------------------------------------------------------------------------------\n5. From the results, shouldn't they include information on which orbitals are\nmore prevalent at which wave number k on which Fermi surface? They\nconcluded that there are two types of orbitals, dxy and dx^2-y^2 for the γ and\ndxz and yz for the β, but are the orbital weight the same along k, or do they\nchange? Including this result would also be helpful for comparison with\nfirst-principles calculations.\"\n------------------------------------------------------------------------------------------------\nWe have added an orbital resolved Fermi Surface for each orbital in the\nSupplemental Information Figure S3 to show the individual contribution to each\npocket. Since the orbitals have a global quantisation axis and do not rotate with\nrespect to the different kagome sites, their contribution shows a strong\nanisotropy also detectable by the linear dichroism data of ARPES provided in\nSupplemental Information Figure S4.\nHowever, also the Fermi surface maps of Figure 3a,b in the main text are\nmomentum resolved Fermi Surface weights. But via the summation of both all\nin-plane (blue color) and all out-of-plane (red color) orbitals, this anisotropy is\nobscured and the Fermi surface weights appear isotropic along the Fermi\nsurface pockets.\nWe have added an additional note in the Methods section on how we arrive at\nFigure 3a,b and thank the referee for clarifying this inaccuracy.\n------------------------------------------------------------------------------------------------\n6. Between extended date Fig. 5(d) and Fig. 2(d), the asymmetry of C2 appears\nto be more clearly suppressed in Fig. 5(d). What is the reason for this?\n------------------------------------------------------------------------------------------------\nWe do not know why this happens, but even if more suppressed it is still\npresent. It is important to mention that part of the functional shape of the lines\nmight be also due to matrix elements. Indeed, matrix elements were different\nalready in the Fermi surface maps, and this could be part of the reason.\nImportantly, for all samples used in the experiments also other cleaves and other\nsamples data collected during the experimental time and all rotation the\nasymmetry is still present, and all the time C was observed in the\n2\nautocorrelation maps.\n------------------------------------------------------------------------------------------------\n7. The definition of the orbital basis should be explained. The authors call the\norbitals in this study dxz , dyz , etc., but most readers who are not familiar with\nthis material will not understand which orbital is which in real space. The\ndefinition of the orbital should be illustrated.\n------------------------------------------------------------------------------------------------\nAs already stated in the answer to the referee’s concern number 5, we have\nadded explicit plots of the real space orbitals on the lattice in the supplement\nand explain the orbital basis in the Methods section. We hope this also clears the\nreferee’s questions expressed in this concern.\n------------------------------------------------------------------------------------------------\nRegarding the theory part, the conclusion is that a Pomeranchuk instability\noccurs based on a certain model parameter (initial value of the interaction\nU,V1,V2 and temperature). However, since there is almost no information on\nthe robustness of the results, it is impossible to judge how reasonable the results\nare from the manuscript as it is, and doubts from many readers cannot be\ndispelled. It should be fundamentally revised.\n------------------------------------------------------------------------------------------------\nWe would like to stress that we are not treating a model system but simulating\nan actual material. Consequently, we have chosen interaction parameters in\ncompliance with ab-initio cRPA values for the same material class. These values\nshould represent a realistic starting point for our calculations.\nHowever, we agree with the referee that the ab-initio values for the interaction\nare not guaranteed to be consistent with the screened interaction profile present\nin the sample. Since a direct experimental verification of the cRPA values is not\npossible (unlike a direct comparison of the DFT band structure with ARPES),\nwe have performed a detailed stability analysis by varying independently both U\nand the decay of the long range tail of the repulsive interactions. Our results\nsummarized in a new paragraph in the Supplemental Information show that the\nPomeranchuk instability is the only abundant phase in the reliable part of the\nphase diagram. We hope this ensures the referee of the soundness of our\ntheoretical analysis.\n------------------------------------------------------------------------------------------------\n8.The authors conclude that such Pomeranchuk instability appear due to the\ninclusion of many-body effects by fRG compared to the mean-field method, but\nthis conclusion is not immediately convincing because the results of solving the\nsame model at the mean-field approximation level are not presented. The\nauthors should present the mean-field results of solving the same model at the\nsame temperature as fRG with the same magnitude of interactions (Of course,\nthey should verify the robustness by changing U,V1,V2.). In particular, can they\nsay that the Pomerantic instability (PI) does not appear in the mean-field\napproximation even if the magnitudes of the long-range Coulomb interactions\nV1 and V2 become somewhat large? If PI appears within the mean-field model,\nthe validity of the magnitude of the interactions V1V2 in this study must be\nexplained.\n------------------------------------------------------------------------------------------------\nThe Pomeranchuk instability (PI) should not be seen as “a beyond mean-field\n(MF) effect”. In the manuscript, we explicitly claim the opposite: even when\ntaking into account quantum fluctuations within a proper many-body analysis\nby means of the FRG, the PI already appears on the MF (or equivalently RPA)\nlevel due to significant long range interactions. To make this more clear to the\nreader, we now show it explicitly by calculating FRG flows with the same\nparameters in the Supplemental Information V but without considering\ncross-channel contributions between the different diagrammatic channels\npresent in the FRG flow equations of Figure 4a. This is equivalent to an RPA\n(i.e. MF) resummation of the different channels. In the charge channel, we\nlikewise find a dominant PI (when compared to a CDW transition). We hope\nthat these additional calculations added to the Supplemental Information are\nsuitable to clarify potential misunderstandings.\nIn addition, we would like to stress two important points of the theoretical\nanalysis:\n1) Firstly, the FRG is strictly better than RPA (and hence MF), since it\ncomprises more diagrams in the expansion of the partition function.\nHence, it is always justified to perform FRG calculations to check for the\nstability of MF/RPA results upon considering quantum fluctuations.\n2) Secondly, especially in systems with many competing instabilities like the\nkagome compounds, a careful examination of all competing orders as\nwell as their competition and cooperation (both absent in MF/RPA) is\nmandatory for reliable predictions. Hence, any MF theory has to be\njustified by an actual many-body calculation. Our study here shows a\ndirect example of the importance of this procedure: In PRB 111 125153 a\ncombination of FLEX and density wave equation has reported a nematic\nodd-parity bond order in CsTi3Bi5. Comparing their result with our\npresent FRG analysis, which contains more diagrams, we can attribute\ntheir result to a bias in evaluating the cross-channel feedback introduced\nby the density wave equation.\nTherefore, the fact that FRG and MF/RPA give the same results should not be\ntaken as granted but rather as an important result of our study.\n------------------------------------------------------------------------------------------------\n9. It is known that low-temperature calculations are difficult in such mesh fRG\nbecause it is generally difficult to obtain fine k-meshes. However, was this\ncalculation performed at a temperature that is consistent with the transition\ntemperature of about 100K observed in experiments? If the answer is Yes, it\nshould be discussed whether this is a reliable region with the present number of\nk-mesh. If the answer is No, is there a guarantee that the same PI will be\ndominant at a low temperature of around 100K? These temperature dependence\nneeds to be discussed.\n------------------------------------------------------------------------------------------------\nWe thank the referee for making us aware of the missing information in the\nMethods part of the manuscript.\nThe FRG calculation was performed with a sharp frequency cutoff, i.e. all flows\nare at zero temperature. However, the constraint region of Matsubara frequency\naxis induces a broadening of the fermionic loop integrals comparable with a\ntemperature. Hence, we can use the flow scale 𝛬 as a proxy for the effective\ntemperature of the system even though such a relation should be treated with\ncare.\nWe have estimated the energy resolution by calculating the density of states,\nobtaining converged results up to energy discretizations of 1 meV. This is well\nbeyond the critical energy scales with the given parameters (and also for almost\nall flows in the phase diagram of Supplementary Information Figure S6), where\nPI is the dominant instability. The only phase space region, where we can no\nlonger trust the FRG results is when a magnetic order is surfacing, since there\nthe critical scales are comparable to our energy resolution and numerical\nartifacts are expected.\nWe have added a detailed discussion about the estimation of the critical\ntemperature from the FRG in the Supplemental Information V and hope the\nreferee is satisfied by our delineations.\nExperimentally, however, we have never observed a transition in this system at\nCsTi3Bi5 system is nematic without CDW, as is well-known from many\nprevious studies. In addition, as in those previous studies, we have performed\nour ARPES measurements at a temperature where the sample is already in the\nnematic state. However, Ti-based kagome samples are highly reactive, and even\nmild thermal treatment leads to rapid sample degradation, causing an immediate\nloss of resolution and, in many cases, complete disappearance of the bands.\nAs detailed in the Methods section, meticulous sample preparation was\nessential, including glovebox handling to minimize degradation. This sensitivity\nmakes it challenging to experimentally determine the nematic transition\ntemperature. However, for similar materials, the transition temperature is\ngenerally high. For instance, in V-based kagome systems, it has been reported to\nbe 35 K (see Nat. Commun. 14, 3899, 2023). In another study (Nat.\nCommun. 14, 4892, 2023), the authors also faced difficulties in extracting this\ninformation experimentally, for similar reasons to us. Nevertheless, as in our\ncase, the observed C symmetry serves as clear evidence that the system is\n2\nalready in the nematic state, which is the focus of our work.\nWith the lack of an experimental estimate for the nematic transition\ntemperature, from a theory perspective, the newly added FRG phase diagram in\nthe Supplemental Information Figure S6 shows that the prediction of a PI\nprevails for critical scales over several orders of magnitude, depending on the\nchosen interaction parameter. In particular, the predicted critical scales are\nlarger than the experimental temperature of 15K, for which a clear finite\nnematic order parameter is present.\nTherefore, even though we can not predict a precise value for the transition\ntemperature both by experimental and theoretical means, our many-body results\nare consistent with the experimental realisation of a nematic state in CsTi3Bi5 at\nlow temperature irrespective of the details of the interaction Hamiltonian.\n------------------------------------------------------------------------------------------------\n10. In Fig. 4(b), the divergent growth of two-particle interaction vertex does not\nstop as a function of Λ~10000K. On the other hand, what would happen if they\nused parameters U,V that can be renormalized up to aroud Λ = 100K. Such\nparameter region would be more consistent with the experiment.\"\n------------------------------------------------------------------------------------------------\nThe FRG flow displays a divergence upon encountering a phase transition.\nHence, we cannot continue the flow across the transition and therefore terminate\nit upon a divergence that marks the breakdown of the one-loop truncation. As a\nconsequence the size of the order parameter within the symmetry broken phase\ncan not be estimated via the FRG. A direct comparison of the critical scales of\nthe FRG and the nematic transition temperature is hampered by several\nfundamental hurdles:\n1) First, as already stated in the reply of the previous concern, we perform a\nsharp frequency flow rather than a temperature flow. Hence the scales in\nthe FRG can not be directly associated with a temperature.\n2) Second, it is well known that the FRG within the single loop\napproximation fails to quantitatively predict the critical temperature,\nconsistently overestimating it. This insufficiency can be lifted by\nincluding multi-loop corrections to the FRG flow equations. Their\nnumerical complexity prevents any application to multi-orbital systems\nand they are hence non suitable for the treatment of CsTi3Bi5.\nHowever, we still see in the phase diagram of Supplemental Information Figure\nS6 that such critical scales of 100K can be reached within our FRG flows for\nsuitably small interaction parameters and the PI still prevails. We want to point\nout that such parameters are way smaller than the cRPA prediction for realistic\nkagome metals.\nAdditionally, a transition temperature considerably higher than 100K cannot be\nruled out by experimental means: As stated in the reply to the previous concern,\nthe experimental determination of the nematic transition temperature has not\nbeen possible thus far neither by our experiment nor other ones. The only secure\nexperimental evidence is the presence of a nematic state at temperatures at\nwhich the experiments are performed.\nWith the lack of experimental accessibility of the nematic transition, we think\nthe first sound theoretical analysis of this transition is highly valuable for the\ncommunity even more as it appears to be very robust with respect to the chosen\ninteraction parameter and thereby the obtainable transition temperatures.\n------------------------------------------------------------------------------------------------\n11. In relation to the above, what is the estimate of the C2 transition temperature\nbased on the fRG theory?\n------------------------------------------------------------------------------------------------\nWe hope to have answered this question already by dealing with the previous\nconcerns of the referee and the changes applied to the manuscript in order to\nresolve them.\n------------------------------------------------------------------------------------------------\n12. I Since the main text only provides results for non-interacting susceptibility\nin Fig.3 (f), in order to compare with the fRG results, the results for\nsusceptibility analyzed at the mean-field level using the same model as fRG (\nJ\n= 0.8 eV, U = 4, V1 = 1.5, V2 = 0.5 ) should be presented and compared.\n------------------------------------------------------------------------------------------------\nWe would like to emphasize that we present the bare susceptibility in Figure 3f\nto develop a physical intuition on the dominant fluctuations of the system. In\nparticular, we conjecture from the bare susceptibility that the in-plane orbitals\ndominate the low energy excitations of the system.\nGuided by this knowledge, we choose the in-plane d-orbitals (d , d ) as\nxy x²-y²\ninteracting subspace in our FRG analysis, since all other orbitals are seemingly\nnot contributing to the system’s ordering propensities. This is a posteriori\njustified by comparing FRG calculations with and without the other d-orbitals\nincluded in the interacting subspace, which shows consistent results: The main\nnematic symmetry breaking is encountered on the in-plane orbitals.\nThe RPA susceptibility cannot be computed for the parameter set suggested by\nthe referee due to its small convergence radius: An interaction scale of U~3eV\nis triggering a Stoner transition to an ordered phase since screening effects\npresent in the FRG are missing in the diagrammatic ladder resummation.\nTherefore, we do not show the RPA susceptibility to compare with our FRG\nresults but present the RPA/MF phase diagram in the Supplemental Information\nFigure S6 alongside with the FRG results.\n------------------------------------------------------------------------------------------------\n13. The validity of the results cannot be judged because there is o definition of\nPI, CDW, SDW, and SC in Fig. 4(b). The definition should be written in terms\nof wave number dependence and orbital dependence. Presumably, using the\ncenter of mass momentum Q of the particle-hole pair, the PI is Q = 0 CDW\nand the CDW is Q ≠ 0 CDW. In addition, when writing Fig. 4(b), is a specific\nwave number Q* ≠ 0 fixed? In that case, has the possibility of the CDW for other\nQ being larger been eliminated? Also, does this fRG formalism take into account\nthe bond-type CDW proposed in the Vanadium-based kagome systems? The k-\ndependence of the obtained vertex should be explained. Also, where did the spin\nchannel for Q = 0 disappear to?\n------------------------------------------------------------------------------------------------\nWe thank the referee for making us aware of the hard accessibility of the FRG\ndata provided in Figure 4. We have added an additional paragraph on the\nanalysis of FRG flows in the Methods part that can be summarized as follows:\nwe do not restrict the exact functional shape of the orders described in the flow\ndiagram of Figure 4b but consider the largest eigenvalue for a given transfer\nmomentum throughout the flow. Hence, the bond-type CDW fluctuations are\nincluded in the CDW flow line. While we keep all possible transfer momenta\nfor all three channels (superconducting, charge and spin), we have decided to\nplot only the four most prominent instabilities in Figure 4b. As the referee has\nalready suspected, the spin channel at Q=0 does not feature a sizable eigenvalue\nat the end of the flow and is hence not shown.\n------------------------------------------------------------------------------------------------\n14. The definitions of the terms in-plane and out-of- plane in the color bar in Fig\n. 3(a) are not clear, making it difficult to judge whether the theoretical model is\nconsistent with the experimental results. Which orbital of is in-plane dyx or\ndx^2-y^2 ? out-of-plane dxz or yz? It should be specifically written. Also, does\nthe orbital's character differ depending on the k? At the same time, it should be\nshown whether their k-dependence is consistent with ARPES data. Based on\nthat, does the even/odd nature of this orbital match the experimental results?\n------------------------------------------------------------------------------------------------\nWe thank the referee for making us aware of this imprecise phrasing in the main\ntext. We have now added an explicit definition of the atomic orbitals as already\nstated in the reply to the previous concerns of the referee and explicitly stated\nhow we derive the orbital weights on the Fermi surface in the Methods part. As\nthe referee has rightly anticipated, the orbital character on the Fermi surface has\na momentum dependence that is explicitly shown in the new Supplementary\nInformation Figure S3 and already incorporated in Figure 3a,b of the main text.\nThis has already been discussed previously in this reply.\nMoreover, we have now added a direct comparison of the parity of the Fermi\nsurface states with the linear dichroism data in Supplementary Information\nFigure S4 alongside a discussion about the theoretical background of the\ncomparison.\n------------------------------------------------------------------------------------------------\n15. For the theoretical analysis, they should compare the phase diagram by RPA\nand fRG by changing the values U,V1,V2 for showing the robustness of their\nmain result. It is expected that PI disappear when V1,V2 are zero or quite small\neven if they performed fRG. What is the critical value of V1 and V2?\n------------------------------------------------------------------------------------------------\nWe thank the referee for his suggestion. In the Supplemental Information,\nFigure S6 now directly compares the phase diagram from FRG and RPA. As\nalready stated in reply to the referee’s concerns Nr. 8,12 the FRG is strictly\nbetter than the RPA, since it includes more terms in the diagrammatic expansion\nof the effective action. Hence, the direct comparison only shows the validity of\nan RPA calculation and not vice versa. Indeed it seems like the additional\ncross-channel processes included in the FRG only act as a driver for the critical\nscale compared to the RPA/MF solution, i.e. increasing the critical scales.\nAs the referee has correctly anticipated, when the interactions are lowered\nbeyond a critical value, the Stoner criterion is no longer satisfied and we do not\nencounter a phase transition within our given resolution as can be read off from\nSupplementary Information Figure S6. Close to the critical value, the FRG\nresults are not reliable anymore and hence the tendencies towards spin density\nwaves can be discarded as unphysical.\n------------------------------------------------------------------------------------------------\n16. Shouldn't the results of changing the filling also be discussed using the fRG\nphase diagram? Since they conclude that the distance of the van Hove\nsingularity (vHS) from the Fermi surface is important for the emergence of the\nPI, it seems crucial to demonstrate that when the vHS gets closer to the Fermi\nsurface, a different instability switches on.\"\n------------------------------------------------------------------------------------------------\nWe agree with the referee that a presentation of the results at van Hove filling\nwas missing in the current manuscript. Since a change of the filling while\nkeeping the bare interaction vertex only influences the bare electronic response,\ni.e. the kernel of any many-body calculation, we discuss the changed physics in\nthe van Hove filled scenario on the level of the bare susceptibility.\nWe have calculated the bare susceptibility and diagonalized it in Supplementary\nInformation Figure S5. As expected, the almost perfectly nested hexagonal\nFermi surface pocket, again stemming from the in-plane Ti d-orbitals, results in\na large contribution close to the M point not present before.\nEvaluating the eigenspectrum of particle-hole fluctuations, the peak at the M\npoint corresponding to a 2x2 charge order is comparable with the peak at the Γ\npoint corresponding to a PI transition. Inspecting the associated eigenvectors\nreveals the peak at M as a precursor for the familiar CBO state on the kagome\nlattice. The absence of this peak in the pristine scenario (i.e., no filling change)\nexplains the predominance of PI in the phase diagram as well as the accuracy of\nthe MF/RPA calculations since competing fluctuations are missing.\nWe thereby conclude that the detuning of the chemical potential is indeed\npivotal for the dominance of PI and absence of translational breaking charge\norder in CsTi3Bi5.\n------------------------------------------------------------------------------------------------\nAnyway, the paper is not self-contained and cannot be evaluated because there\nis no explanation of the formulas actually used in the theoretical analysis or the\ntemperature conditions. A minimum of formulas, a definition of what is being\nplotted on the graph, and an explanation of the parameters are necessary. In\naddition, data in a wider parameter range is also necessary to see the robustness\nof the results.\n------------------------------------------------------------------------------------------------\nWhere applicable, we have given the explicit formulas, while for complex\ntheoretical frameworks such as DFT and FRG we have referred to the\ndocumentation of the employed software package and have additionally given\nthe explicit input parameters to enable the reader to perform the calculations\nthemselves. In the analysis of the FRG flows, we have added an additional\nparagraph where we have sketched the analysing procedure and outlined how to\nobtain the flow eigenvalues presented in Figure 4b of the main text. In\ncombination with the parameter scans added to the Supplemental Information,\nan instability analysis at van Hove filling and additional RPA phase diagram, we\nare confident to have addressed all concerns of the referee within the scope of\nthe present work."}, {"round": 2, "reviewer_comments": {"reviewer_1": "I can partially agree that the authors have addressed the comments by both reviewers. But I am still not fully convinced by the reply and would like the authors to address the following two points.\n1. The authors say that the FFT of the Fermi surface is not a convincing evidence to determine the periodicity. But I hope they can at least shown one figure of the FFT, not just saying that \"FFT is not straightforward and ACP is better\".\n2. The authors say that it is difficult to determine the Pomeranchuk instability temperature from theory due to the complexity of the theory. Then, I hope they can perform temperature dependent ARPES and Ferm surface measurements as well as ACP analyses and show the transition temperature from the experimental point of view. This should give new insights into the correctness of the theory.", "reviewer_2": "The revised manuscript shows certain improvements in the logical structure compared to the previous version, where logical leaps were frequently observed, and overall, the quality of the paper as an academic article has improved. However, even after these revisions, the novelty of the results still feels somewhat lacking, and considering that the impact of these findings on the broad readers in Nature Communications. may not be substantial, I find it difficult to recommend publication.\nAs both referees have pointed out, a prior ARPES study on the same material (Nat. Phys. 19, 1591 (2023)) has already observed C$_2$ symmetry. Therefore, the novelty of this work lies rather in the identification of the orbital character associated with this symmetry. However, even within this scope, the only conclusions that can be safely drawn from the present experiments—without speculative inference—are that the α pocket is composed of even orbitals, and the β and γ pockets consist of a mixture of even and odd orbitals. Beyond this, the paper’s conclusions are based on multiple layers of inference, and it would be extremely difficult for readers who are not specialists in ARPES, DFT, or fRG to assess how reliable these conclusions truly are.\nOne example is discussed in [Response to the 1st comments of referee 2], where one layer of inference involves identifying whether orbitals are in-plane or out-of-plane. The authors cite the agreement between ARPES data in Fig. S4(c) and the DFT results in Fig. S4(f), as well as the polarization-dependent behavior (extended), as justification for fully trusting the DFT orbital character. However, it is difficult to accept that the entire orbital dependence of the DFT calculation can be trusted solely because it reproduces this ARPES signal. Furthermore, the structure of the manuscript does not present a logical, step-by-step narrowing down of orbital characters. As a result, the readers may find a significant gap between the experimental results and the authors’ claims, which is hard to bridge. Questions also remain regarding the reliability of both the polarization-dependent ARPES measurements and the DFT calculations themselves [Comments 1 and 2 by referee 1].\nRegarding the second main point—the proposal of a Pomeranchuk instability—while this scenario can be understood as one possible explanation for the origin of C$_2$ symmetry, it is questionable whether it is the only plausible scenario. Even if a correct band structure is used as input, the reliability of the resulting prediction depends on the limitations of the theoretical method, and cannot be guaranteed. In this case, the 1-loop fRG analysis is performed with a step function for temperature, which undermines its quantitative reliability [Comments 3 by referee 1, 10 by referee 2].\nFinally, one more point worth noting is found in [Comment 12 by referee 2], where the authors claim that the Pomeranchuk instability can be understood from the bare susceptibility shown in Fig. 3(f) as originating from in-plane orbitals. However, the momentum dependence of the bare susceptibility peaks away from the Γ point, making this interpretation unclear. It would be more appropriate to directly perform an RPA calculation by adding U and V to the bare susceptibility—even using a scaled-down approximation—so that this point can be more convincingly demonstrated."}, "author_response": "We are pleased to hear that the referee is at least partially satisfied with\nour reply to their concern and thank the referee for their repeated effort\nin reviewing our manuscript. We hope to also address the remaining\nissues in this present reply.\n------------------------------------------------------------------------------------------------\n1. The authors say that the FFT of the Fermi surface is not a convincing\nevidence to determine the periodicity. But I hope they can at least shown\none figure of the FFT, not just saying that \"FFT is not straightforward and\nACP is better\".\n------------------------------------------------------------------------------------------------\nWe agree with the referee that a quantitative analysis demonstrating why\nthe FFT of the Fermi surface is not useful is required. From the\ntheoretical side, the FFT requires that several repetitions of the same\nperiodic unit be Fourier-transformed in order to gain meaningful insight\ninto symmetry breaking in the electronic band structure, such as\nnematicity. This is the case with STM technique, which probes the\nelectronic wavefunction over hundreds of unit cells in the real space (a\ntypical STM image is scanned over ~100 nm2 area). Thus, ARPES\nshould ideally sample the Fermi surface over hundreds of Brillouin zones\n(the Brillouin zone being the unit cell of the reciprocal space) to enable\nthe FFT approach to be applied to an ARPES data set with adequate\nrigour and in full analogy to STM studies. This is inherently prevented by\nthe experimental setup of ARPES, as the technique is limited by the 30°\nacceptance angle of the detector on one axis and by the range covered\nby the manipulator's rotation on the other. Other more subtle constraints\nsuch as matrix element effects across neighbouring zones and photon\nenergy limitations would further hinder extended-zone sampling. As a\nmatter of fact, the momentum and energy resolutions limit us to probing\nsolely one full Brillouin zone. Consequently, the FFT is not the\nappropriate approach in the context of this ARPES investigation and FFT\ntreatment of the ARPES Fermi surface will not provide insights into the\nreal space periodicity of the electronic wave function, as clearly shown\nby Re(FFT) reported in Figure 1 below.\nFigure 1: Real part of the FFT obtained from the Fermi contour of unpolarised light (CR+CL).\nArrows highlight the resulting signal.\nWe would like to kindly emphasise once again that the autocorrelation\nmethod extracts periodic features directly from a single Brillouin zone\nin momentum space. This eliminates the need for data acquisition over\nhundreds of Brillouin zones and provides a more suitable tool for\ncharacterising symmetry-lowering phenomena in correlated electron\nsystems through ARPES, as thoroughly discussed in the method section\nof the main text.\nWe heartily hope that the referee is now happy with our analysis and\nagrees with us about the usefulness of the autocorrelation maps.\n------------------------------------------------------------------------------------------------\n2. The authors say that it is difficult to determine the Pomeranchuk\ninstability temperature from theory due to the complexity of the theory.\nThen, I hope they can perform temperature dependent ARPES and\nFerm surface measurements as well as ACP analyses and show the\ntransition temperature from the experimental point of view. This should\ngive new insights into the correctness of the theory.\n------------------------------------------------------------------------------------------------\nWe thank the referee again for their comment. First of all, we would like\nto kindly stress that from a theoretical point of view the novelty of our\nresults lies in the microscopic nature of the phase transition, that we\nreveal as a Pomeranchuk instability, as well as the nematic order\nparameter. In this regard it is worth noting that our work is only the\nsecond work aiming at a theory of nematicity in CsTi3Bi5. However,\ncompared to [Phys. Rev. B 111, 125153] our more advanced FRG\ncalculations reveal a different order parameter, namely an onsite charge\nimbalance on the in-plane d-orbitals. We attribute this to the bias in the\ndiagrammatic resummation of the previous study.\nTherefore, we are interested in the precise shape of the order parameter\nrather than the critical scale. As the referee has correctly pointed out, it\nis hard to determine the transition temperature quantitatively.\nIn the last resubmission, we have provided a large scan in the interaction\nparameters (cf. Supplementary Information Fig. S6). There we show that\nthe microscopic order parameter does not change over a large range of\ninteraction parameters resulting in critical scales spanning 4 orders of\nmagnitude. This encourages us to trust the order parameter obtained\nfrom the FRG calculations irrespective of the exact value of the transition\ntemperature predicted by the FRG since only quantitative changes are\nexpected from multi-loop corrections.\nThe referee's request of performing temperature dependent ARPES\nmeasurements is incredibly challenging. In fact, Ti-based kagome\nsamples are highly reactive, and even mild thermal treatment in an inert\nArgon atmosphere can cause sample degradation, resulting in a loss of\nresolution and, in many cases, the complete disappearance of the\nbands. Furthermore, increases in pressure as well as pressure spikes\nare common during temperature investigation campaigns. Both are\ndetrimental to the sample, whose signal would consequently fade\nrapidly.\nNonetheless, we agree with the referee about the importance of a\ntemperature dependent analysis. To overcome the aforementioned\ndifficulties, we first performed a degassing of the manipulator to remove\nmost of the molecules cold-trapped to it, then we gently increased the\ntemperature with a 0.4 K/min ramp to keep the pressure in the chamber\n<1*10-9 mbar. Samples were cleaved and measured only after pressure\nrecovered <2*10-10 mbar. We measured Fermi surface maps with both\nCR and CL polarisations to obtain the following autocorrelation maps\n(ACM) of “unpolarised light” (CR+CL).\nFigure 2: Autocorrelation maps obtained from the Fermi surface contours of unpolarised light\n(CR+CL) probed at the Brillouin zone centre with hv = 65 eV and for several temperatures.\nThe azimuthal profiles extracted for the Γ-K and Γ-M high symmetry directions clearly show\nthe persistence of the reduced C symmetry of the nematic phase up to room temperature.\ntemperature (beyond which measurements are technically not feasible),\ni.e. 90 K, 123 K, 172 K, 200 K, 250 K and 270 K and RT as reported in\nFigure 2, which we also included in the manuscript as Extended Data\nFig. 7 to increase the depth of our study. In all cases, the ACM shows\nthe reduced C symmetry typical of the nematic phase, as highlighted by\n2\nthe azimuthal profile extracted for Γ-K and Γ-M directions where one\nprofile clearly deviates from the other two which instead remain\nequivalent to each other. Here we want to stress the fact that the\nvariations in broadness and in the profile shapes for different\ntemperature points (e.g. for the 200 K set) are ascribable to the sample\naging rather than the onset of a transition. In fact, due to the challenges\ndiscussed above, the sample surface quickly aged during the\nmeasurements. Figure 3 shows the ACM and the profiles’ set for two\ntemperature points, as measured on both freshly cleaved and aged\nsurfaces. C symmetry persists in all the measurements. However,\n2\nincreased broadening is evident in the aged sample. This renders the C\n2\nsymmetry less apparent, to the extent that it could be mistakenly\ninterpreted for the beginning of a transition.\nFigure 3: Autocorrelation maps and azimuthal profiles extracted for the Γ-K and Γ-M\ndirections for a fresh sample surface and an aged one at two different temperatures.\nIn view of the above analysis, we conclude that CsTi3Bi5 remains in the\nnematic phase up to room temperature as no sign of transition was\ndetected. In other words, the nematic transition happens above room\ntemperature, corroborating the evidence of a large critical scale obtained\nfrom our theoretical calculations.\nWe would like to thank the referee once again for stimulating us to\nperform this important analysis which has significantly improved the\nquality and depth of our work. We sincerely hope that the referee can\nnow recommend in favour of publication.\nReviewer #2 (Remarks to the Author):\n------------------------------------------------------------------------------------------------\nThe revised manuscript shows certain improvements in the logical\nstructure compared to the previous version, where logical leaps were\nfrequently observed, and overall, the quality of the paper as an academic\narticle has improved. However, even after these revisions, the novelty of\nthe results still feels somewhat lacking, and considering that the impact\nof these findings on the broad readers in Nature Communications. may\nnot be substantial, I find it difficult to recommend publication.\n------------------------------------------------------------------------------------------------\nWe thank the referee for their repeated effort in reviewing our\nmanuscript. However, we strongly have to disagree with the referee’s\nconcerns on several levels as detailed below.\n------------------------------------------------------------------------------------------------\nAs both referees have pointed out, a prior ARPES study on the same\nmaterial (Nat. Phys. 19, 1591 (2023)) has already observed C$_2$\nsymmetry. Therefore, the novelty of this work lies rather in the\nidentification of the orbital character associated with this symmetry.\n------------------------------------------------------------------------------------------------\nWe would like to rectify the referee’s statement regarding the status quo\nin the field of Ti-based kagome metals:\nUpon submission of the manuscript, the C symmetry breaking has only\n2\nbeen observed in quasi-particle interference measurements from STM\n[Nat. Phys. 19, 1591–1598 (2023), Nat. Comm. 15, 9626 (2024)].\nThe only ARPES data available to date [Nat. Phys. 19, 1827 (2023) for\nRbTi3Bi5 not for CsTi3Bi5, Nat Commun 14, 4089 (2023)] does not\nreport a nematic state and focus only on the topological features of the\nnormal state bandstructure. They only speculate on the origin of\nnematicity reported in the aforementioned STM studies given a more\nprecise knowledge of the normal state Fermi surface and band structure\nfrom ARPES. In our work we provide a direct observation of the Fermi\nsurface reconstruction in the nematic state.\nWe would like the referee to finally realize (after stressing this already\nseveral times in the first round of our revision), that the article [Nat. Phys.\n19, 1591 (2023)] is an STM work NOT an ARPES study.\nAs a matter of fact our study is the first to report nematicity in CsTi3Bi5\nTi-based kagome material from ARPES.\n------------------------------------------------------------------------------------------------\nHowever, even within this scope, the only conclusions that can be safely\ndrawn from the present experiments—without speculative\ninference—are that the α pocket is composed of even orbitals, and the β\nand γ pockets consist of a mixture of even and odd orbitals. Beyond this,\nthe paper’s conclusions are based on multiple layers of inference, and it\nwould be extremely difficult for readers who are not specialists in\nARPES, DFT, or fRG to assess how reliable these conclusions truly are.\n------------------------------------------------------------------------------------------------\nWe agree with the referee that accessing the microscopic nature of any\nmany body ground state always includes several layers of inference,\nsince the many body wave functions can not be observed directly in\nexperiment. This is also the case in the previously mentioned STM and\nARPES studies and we don’t see more substantial obstacles in\nunderstanding our work than in any other work combining theory and\nexperiment: From ARPES the two main observables are given by the\npolarization dependent band structure and the Fermi surface maps. In\nthe article we have compared the DFT band structure and the derived\nlinear dichroism maps with the ARPES measurement finding good\nagreement. Additionally, we compare the Fermi surface reconstruction in\nthe nematic state predicted by FRG and find good agreement with the\nAC maps of the ARPES.\nWe therefore wonder where the referee sees potential to erase layers of\ninference in our work. For the two main theoretical predictions in our\npaper (orbital structure of the Fermi surface and microscopic nematic\norder parameter), we have calculated the accessible experimental\nobservables and they show excellent agreement.\n------------------------------------------------------------------------------------------------\nOne example is discussed in [Response to the 1st comments of referee\n2], where one layer of inference involves identifying whether orbitals are\nin-plane or out-of-plane. The authors cite the agreement between\nARPES data in Fig. S4(c) and the DFT results in Fig. S4(f), as well as\nthe polarization-dependent behavior (extended), as justification for fully\ntrusting the DFT orbital character. However, it is difficult to accept that\nthe entire orbital dependence of the DFT calculation can be trusted\nsolely because it reproduces this ARPES signal. Furthermore, the\nstructure of the manuscript does not present a logical, step-by-step\nnarrowing down of orbital characters. As a result, the readers may find a\nsignificant gap between the experimental results and the authors’ claims,\nwhich is hard to bridge. Questions also remain regarding the reliability of\nboth the polarization-dependent ARPES measurements and the DFT\ncalculations themselves [Comments 1 and 2 by referee 1].\n------------------------------------------------------------------------------------------------\nWe would like to stress the dichroism in ARPES is - to the best of our\nknowledge - the only possible way to access the orbital content of the\nelectronic band structure in experiment. Therefore, the dichroism maps\nare the only possible benchmark for the accuracy of our DFT model and\ncan be readily calculated without any additional assumptions from the\nDFT bandstructure. Since dichroism maps obtained from DFT and\nARPES data show excellent agreement we have no reason to question\nthe correctness of our results.\nWe would like to stress that the aforementioned STM works on nematic\ninstability do not have any information about the orbital content of the\nFermi surface and hence can’t access this information. They rely for their\nclaim of orbital selectivity fully on DFT calculations without the possibility\nto benchmark the orbital character with experiment.\nThe comments by referee 1 have been satisfactorily answered already in\nthe last revision. We therefore don’t see any need to question the quality\nof both the ARPES and DFT calculations, which were both performed\nthoroughly and with state-of-the-art techniques.\n------------------------------------------------------------------------------------------------\nRegarding the second main point—the proposal of a Pomeranchuk\ninstability—while this scenario can be understood as one possible\nexplanation for the origin of C$_2$ symmetry, it is questionable whether\nit is the only plausible scenario. Even if a correct band structure is used\nas input, the reliability of the resulting prediction depends on the\nlimitations of the theoretical method, and cannot be guaranteed. In this\ncase, the 1-loop fRG analysis is performed with a step function for\ntemperature, which undermines its quantitative reliability [Comments 3\nby referee 1, 10 by referee 2].\n------------------------------------------------------------------------------------------------\nWe agree with the referee, that several possible microscopic realisations\nof the nematic states are actively discussed in the community and we\nhave commented on them in our manuscript. And obviously the outcome\nmight depend on the numerical approach used to solve the quantum\nmany-body problem.\nAs detailed in the Supplementary Information, we have employed the\nFRG, since it has proven to be a reliable tool (and even more an\napplicable tool) to handle electronic instabilities in the weak to\nintermediate coupling regime for large ab-initio models (see references\nin the Supplementary Information).\nIn the case of Ti based kagome metals, only a single theoretical work\nhas thus far been proposed, that employs a combination of FLEX and\ndensity wave equation [Phys. Rev. B 111, 125153]. This procedure\nselects a subset of the screening processes contained in the present\nFRG study and hence includes a bias towards exotic bond orders. Our\nunbiased FRG calculations show that the proposed nematic bond order\nis not encountered in a more rigorous calculation. Instead the\nmicroscopic ground state in the nematic phase is predicted to be an intra\nunit cell charge density wave on the in-plane orbitals. We further see that\nthis prediction is compatible with both our ARPES measurements and\nprevious STM studies [Nat. Phys. 19, 1591–1598 (2023), Nat. Comm.\n15, 9626 (2024)], that did not provide any microscopic explanation for\ntheir observations.\nWe also invite the referee to take a careful read through our manuscript.\nThere is no step function for temperature in our FRG calculations and we\ncan only guess what the referee is referring to. We use a sharp\nfrequency cutoff, not a temperature cutoff for the RG flow. However, it\nhas been shown that the precise choice of the cutoff is not crucial for the\npredicted leading instability and might only shift critical scales. The same\nholds true for the multi-loop extensions to the FRG (cf. Paragraph in the\nSupplementary Information): The 1-loop truncation impedes a\nquantitative estimate of the critical temperature, but the symmetry\nbreaking and order parameter are reliably predicted.\nWe would like to kindly note that Comments 3 from Referee 1 and 10\nfrom Referee 2 were already addressed in full detail in our previous\nrevision. We respectfully invite the referee to consider our earlier\nresponses, as the concerns raised appear to reiterate points that we\nhave already clarified.\n------------------------------------------------------------------------------------------------\nFinally, one more point worth noting is found in [Comment 12 by referee\n2], where the authors claim that the Pomeranchuk instability can be\nunderstood from the bare susceptibility shown in Fig. 3(f) as originating\nfrom in-plane orbitals. However, the momentum dependence of the bare\nsusceptibility peaks away from the Γ point, making this interpretation\nunclear. It would be more appropriate to directly perform an RPA\ncalculation by adding U and V to the bare susceptibility—even using a\nscaled-down approximation—so that this point can be more convincingly\ndemonstrated.\n------------------------------------------------------------------------------------------------\nWe have already readily replied to the referee’s Comment 12 in the\nprevious resubmission. Quoting: “We would like to emphasize that we\npresent the bare susceptibility in Figure 3f to develop a physical intuition\non the dominant fluctuations of the system. In particular, we conjecture\nfrom the bare susceptibility that the in-plane orbitals dominate the low\nenergy excitations of the system.” This means that we’ve never claimed\nthat the PI can be understood from the bare susceptibility.\nProbably the referee is confusing bare susceptibility and MF/RPA. Since\nthe PI is an interaction driven effect (as stated in the manuscript), the\nbare susceptibility (which does not contain any interactions but only\nkinetic fluctuations) is not expected to feature a pronounced nematic\ntendency.\nIn the previous revision we undertook the effort to calculate the full RPA\nphase diagram (cf. Supplementary Information Fig. S6) revealing a\ndominant PI also on the RPA level. In this revision, following the\nreferee’s request, we have added an exemplary interacting RPA\nsusceptibility to Fig. S6 (see also figure below for convenience). It\nfeatures a pronounced peak at the zone center corresponding to nematic\nfluctuations close to criticality. Notably, this peak develops at scales far\nabove the Fermi level pointing towards an interaction rather than a\nkinetic fluctuation driven nature of the Pomeranchuk instability. We hope\nthe referee is now pleased by the presentation of our results and trusts\nthe statements put forward in our paper.\nWe have carefully examined the manuscript and present a more balanced discussion of the\nclosely related works highlighted by Reviewer #3 in the Discussion section. Moreover, we\nhave checked for remaining strong theory claims in light of Reviewer #2’s concerns. In this\nrespect, we have changed the phrasing in the Discussion section to highlight that we have\nperformed FRG calculations and obtained results consistent with our experiments. While we\nare able to give a good interpretation of our results within the framework of FRG, we have\nsoftened any claims on the interpretation of the actual microscopic mechanism in the actual\nmaterial."}, {"round": 3, "reviewer_comments": {"reviewer_1": "I am satisfied with the authors responses and the manuscript can be published in its present form.", "reviewer_3": "In this manuscript, Chiara Bigi et al. report a systematic investigation of the mechanisms underlying nematicity in CsTi3Bi5. The authors employ polarization-dependent ARPES to reveal the orbital-selective nematic deformation and support their findings with FGR calculations, which indicate a d-wave Pomeranchuk instability driven by orbital-dependent electronic correlations and chemical potential detuning away from VHs. The paper is well written and the explanation of the result is well addressed. However, I have significant concerns on its novelty and broad impact appear limited in the context of recently published works. The implications of these results are not striking enough or new. In this respect, this work may not be suited for general audience targeted by Nature Communications, but for more specific audience who is investigating this material. The followings are my concerns for authors’ consideration.\n1. The use of polarization-dependent ARPES to elucidate orbital contributions and autocorrelation methods to demonstrate electronic nematicity has been previously established in related kagome metals. Notably, orbital-selective behavior was reported in RbTi3Bi5 (Nat. Phys. 19, 1827, 2023), and clear autocorrelation evidence of nematicity was provided in the same compound (Nat. Commun. 14, 4892, 2023). The current manuscript does not appear to significantly advance beyond these earlier works. Especially, the autocorrelation ARPES result (Nat. Commun. 14, 4892 (2023)) is better than that in this manuscript.\n2. In the response to the Reviewer#1, the authors claim that “Ti-based kagome samples are highly reactive, and even mild thermal treatment in an inert Argon atmosphere can cause sample degradation”. However, previous work on RbTi3Bi5 (Supplementary Fig. 14 in Nat. Commun. 14, 4892 (2023)) showed clear C2 anisotropy persists at 200 K with negligible aging effect. This discrepancy requires clarification. Additionally, as shown in Extended Data Fig. 7, the autocorrelation maps at 16 K, 250 K and RT are much clearer than these at 90 K, 123 K, 200 K and 270 K. This non-monotonic changes in autocorrelation clarity across temperatures are not convincingly explained by aging effect alone.\n3. The anisotropy observed in autocorrelation maps could arise from matrix element effects rather than intrinsic electronic ordering. To exclude this possibility, the authors should perform measurements with rotated sample orientations to verify whether the scattering vectors rotate accordingly, which is a standard validation step currently missing in the manuscript.\n4. In their response to the Review#2, the authors state that “the only ARPES data available to date [Nat. Phys. 19, 1827 (2023) for RbTi3Bi5 not for CsTi3Bi5, Nat Commun. 14, 4089 (2023)] does not report a nematic state and focus only on the topological features of the normal state band structure”. This claim is inaccurate. In fact, the ARPES study published in Nat. Commun. 14, 4089 (2023) provides clear evidence of electronic nematicity via autocorrelation analysis (see Figures 5 and S11–S16), with data quality and resolution that appear superior to those presented in this work. Moreover, the authors emphasize that “As a matter of fact our study is the first to report nematicity in CsTi3Bi5 Ti-based kagome material from ARPES”. Given the strong electronic similarities between CsTi3Bi5 and RbTi3Bi5, and considering that robust nematicity has already been established in the latter through ARPES measurements, insisting on being “first” for the Cs compound seems unnecessary and does not substantively strengthen the manuscript’s contribution."}, "author_response": "We sincerely thank the referee for the careful reading of our manuscript and for the\nconstructive comments that have helped us improve the clarity and precision of our work. We\nappreciate the reviewer’s detailed comparison with previous studies on Ti-based kagome\ncompounds and their insightful suggestions regarding both experimental methodology and\ndata interpretation. We have carefully revised the manuscript and provide below a\npoint-by-point response to each of the raised concerns.\n1. The use of polarization-dependent ARPES to elucidate orbital contributions and\nautocorrelation methods to demonstrate electronic nematicity has been previously\nestablished in related kagome metals. Notably, orbital-selective behavior was reported\nin RbTi3Bi5 (Nat. Phys. 19, 1827, 2023), and clear autocorrelation evidence of\nnematicity was provided in the same compound (Nat. Commun. 14, 4892, 2023). The\ncurrent manuscript does not appear to significantly advance beyond these earlier\nworks. Especially, the autocorrelation ARPES result (Nat. Commun. 14, 4892 (2023))\nis better than that in this manuscript.\nWe thank the referee for this valuable remark. We fully acknowledge the important prior\nwork on RbTi₃Bi₅, which indeed established orbital-selective behaviour and electronic\nnematicity through polarization-dependent ARPES and autocorrelation analysis. The key\nadvance of our present study lies in the robustness and temperature stability of the nematic\nstate in CsTi₃Bi₅. While in RbTi₃Bi₅ the C₂ anisotropy is known to vanish around 200 K [Phys.\nRev. B 112, L041122], in our Cs-based compound the twofold-symmetric features persist at\nleast up to room temperature. This persistence indicates a markedly more robust nematic\nresponse, suggesting that nematicity in CsTi₃Bi₅ is not a marginal or low energy\nfluctuation-driven effect, but rather an intrinsic and stable property of the electronic structure.\nThe higher temperature range over which nematicity survives has important implications for\nunderstanding the microscopic mechanism as detailed in our many-body analysis at\nintermediate coupling which predicts PI with transition temperatures above RT. It points to an\nenhanced coupling between orbital and lattice degrees of freedom in CsTi₃Bi₅, possibly that\nmay stabilise nematic order against thermal fluctuations. This robustness discriminates\nCsTi₃Bi₅ from its sister compound RbTi₃Bi₅, where comparable experimental studies have\nalready been available (see reply to concern No. 4) and renders it a unique platform among\nTi-based kagome metals for investigating the interplay between nematicity and topology in a\nregime accessible to practical conditions.\nWe thank the referee for raising this important point. In the discussion section of the paper,\nwe have highlighted this remarkable difference between Ti-based kagome metals.\n2. In the response to the Reviewer#1, the authors claim that “Ti-based kagome samples\nare highly reactive, and even mild thermal treatment in an inert Argon atmosphere can\ncause sample degradation”. However, previous work on RbTi3Bi5 (Supplementary\nFig. 14 in Nat. Commun. 14, 4892 (2023)) showed clear C2 anisotropy persists at 200\nK with negligible aging effect. This discrepancy requires clarification. Additionally,\nas shown in Extended Data Fig. 7, the autocorrelation maps at 16 K, 250 K and RT\nare much clearer than these at 90 K, 123 K, 200 K and 270 K. This non-monotonic\nchanges in autocorrelation clarity across temperatures are not convincingly explained\nby aging effect alone.\nOur statement regarding sample degradation reflects our specific experimental experience\nwith CsTi₃Bi₅, rather than a generalised conclusion about all Ti-based kagome compounds.\nIndeed, we observed that CsTi₃Bi₅ surfaces are particularly sensitive to residual gases and\nmild thermal exposure, which can rapidly alter the surface spectral weight distribution.\nHowever, we make the referee notice that we have clarified this point in the last revised\nmanuscript and explicitly shown the aging effects in the rebuttal letter of our last time. The\nnon-monotonic changes are also related to the causality of the cleave, as it is perfectly natural\nand expected for exfoliable crystals. Therefore, if a cleave is not ideal and perfectly flat, this\nis naturally causing electron-impurity scattering which, as a consequence of a contribution in\nthe imaginary part of the self energy, causes a broadening of the signal (note that the signal is\nstill good in the ARPES so it is a priori extremely difficult to understand which cleave is\nbetter).\nWe thank the referee for pointing out this missing piece of information. We have added an\nadditional explanation to the caption of Extended Data Fig. 7 (now Supplementary Figure 13)\nfor clarification.\n3. The anisotropy observed in autocorrelation maps could arise from matrix element\neffects rather than intrinsic electronic ordering. To exclude this possibility, the authors\nshould perform measurements with rotated sample orientations to verify whether the\nscattering vectors rotate accordingly, which is a standard validation step currently\nmissing in the manuscript.\nThe measurements with rotated sample orientations have indeed been performed and were\nincluded in the Extended Figure 5 of the original submission, version 1 and they have not\nchanged since then. As shown there, upon rotating the sample, the features in the\nautocorrelation maps (ACMs) rotate correspondingly, demonstrating that the observed\nanisotropy follows the crystallographic axes and is therefore intrinsic rather than arising from\nmatrix element effects. There is also another methodology to do this, by using the sum of\nboth circular polarizations to make the light as unpolarised as possible, and we did this too\n(and also this has been done for both directions).\nAdditionally, we have analyzed the peak positions of the ARPES intensities at the Fermi level\nfor nominally equivalent high symmetry directions in Extended data Fig. 4 revealing a\nsymmetry breaking shift of the Fermi contours along the two lines. Since this signature is\nindependent of the absolute intensities and hence independent of matrix element effects\n(unlike the ACMs), we are convinced that symmetry breaking in the ACMs is stemming from\na nematic state of the sample.\n4. In their response to the Review#2, the authors state that “the only ARPES data\navailable to date [Nat. Phys. 19, 1827 (2023) for RbTi3Bi5 not for CsTi3Bi5, Nat\nCommun. 14, 4089 (2023)] does not report a nematic state and focus only on the\ntopological features of the normal state band structure”. This claim is inaccurate. In fact,\nthe ARPES study published in Nat. Commun. 14, 4089 (2023) provides clear evidence of\nelectronic nematicity via autocorrelation analysis (see Figures 5 and S11–S16), with data\nquality and resolution that appear superior to those presented in this work. Moreover,\nthe authors emphasize that “As a matter of fact our study is the first to report nematicity in\nCsTi3Bi5 Ti-based kagome material from ARPES”. Given the strong electronic\nsimilarities between CsTi3Bi5 and RbTi3Bi5, and considering that robust nematicity\nhas already been established in the latter through ARPES measurements, insisting on being\n“first” for the Cs compound seems unnecessary and does not substantively strengthen the\nmanuscript’s contribution.\nOur intention was not to overlook this important result, but to emphasise that those data\nconcern the Rb-based compound. Nat Commun. 14, 4089 (2023) is indeed the only ARPES\nstudy of CsTi₃Bi₅ and does not report nematic symmetry breaking in the spectra or ACMs.\nBut the referee is absolutely correct that Nat. Commun. 14, 4892 (2023) provides this missing\ndata for the Rb-based samples. We apologize for not making this utterly clear.\nHowever, the differences between the two compounds are not minor: they significantly affect\nthe Fermi surface topology [arXiv:2401.13628, Chinese Physics B33(10) (2024)] and the\nrobustness of nematicity, which in CsTi₃Bi₅ persists up to room temperature while\ndiminishing at around 200K in RbTi₃Bi₅ [Phys. Rev. B 112, L041122]. Notably, theoretical\ncalculations find a strong similarity between the two compounds (as mentioned by the\nreferee), which requires a careful examination of the validity of the employed theoretical\nmodel by means of experimental probes to make sophisticated claims on the microscopic\norder parameter. This was carried out in this manuscript by a combined polarized ARPES and\nDFT+FRG approach.\nWe have therefore adjusted the phrasing to avoid unnecessary emphasis on priority, while\nclearly delineating the novelty of our findings within the broader Ti-based kagome family and\ndiscriminating our work from earlier works on Rb-samples."}]

Decision letter