# Competing charge ordering and Mott phases in a correlated Sn/Ge(111) two-dimensional triangular lattice

Rocio Cortés-B, Antonio Tejeda, Jorge Lobo-Checa, Clément Didiot, Bertrand Kierren, Daniel Malterre, Jaime M. Merino, Fernando Flores, Enrique G.

Michel, Arantzazu Mascaraque

► To cite this version:

Rocio Cortés-B, Antonio Tejeda, Jorge Lobo-Checa, Clément Didiot, Bertrand Kierren, et al. Competing charge ordering and Mott phases in a correlated Sn/Ge(111) two-dimensional triangular lattice. Physical Review B: Condensed Matter and Materials Physics (1998-2015), 2013, 88 (12), pp.125113. 10.1103/PhysRevB.88.125113. [arXiv:0903.3663]

HAL Id: hal-01273663

https://hal.science/hal-01273663

Submitted on 28 Jun 2023

**HAL** is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Competing charge ordering and Mott phases in a correlated Sn/Ge(111) two-dimensional triangular lattice

R. Cortés,¹ A. Tejeda,²,³,* J. Lobo-Checa,⁴ C. Didiot,⁵ B. Kierren,² D. Malterre,² J. Merino,⁶,⁷ F. Flores,⁶ E. G. Michel,⁷,⁸ and A. Mascaraque¹

¹Dto. de Física de Materiales, Universidad Complutense de Madrid, 28040 Madrid, Spain

²Institut Jean Lamour, CNRS - Nancy-Université - UPV-Metz, 54506 Vandouvre les Nancy, France

³Synchrotron SOLEIL, L'Orme des Merisiers, Saint-Aubin, 91192 Gif sur Yvette, France

⁴Centro de Física de Materiales (CSIC-UPV/EHU), Paseo Manuel de Lardizabal 5, 20018 San Sebastian, Spain

⁵Institut Jean Lamour, CNRS - Université de Lorraine, 54506 Vandouvre les Nancy, France

⁶Dto. de Física Teórica de la Materia Condensada, Universidad Autónoma de Madrid, 28049 Madrid, Spain

⁷Centro de Investigación de Física de la Materia Condensada (IFIMAC), Universidad Autónoma de Madrid, 28049 Madrid, Spain

⁸Dto. de Física de la Materia Condensada, Universidad Autónoma de Madrid, 28049 Madrid, Spain
(Received 14 June 2011; revised manuscript received 16 May 2013; published 6 September 2013)

We take advantage of complementary high-resolution experimental techniques and theoretical tools to get insight into the α-Sn/Ge(111) triangular lattice surface consisting of sp electrons. We report a (3 × 3) phase, characterized by a charge ordering settled by electronic correlation, which appears between the known metallic-(3 × 3) and Mott insulator phases. We identify the atomistic mechanism behind the stabilization of this phase and interpret these findings on the basis of theoretical calculations. We disentangle the role of the various degrees of freedom in the stability of the different phases found and describe the stepwise surface changes between the metallic and Mott insulating phases.

DOI: 10.1103/PhysRevB.88.125113

PACS number(s): 73.20.—r, 68.37.Ef, 79.60.—i

## I. INTRODUCTION

The properties of materials that lie out of the Fermi liquid theory are determined by a delicate interplay between vibrational, spin, and charge degrees of freedom. Their interaction induces a variety of exotic physical phenomena and frequently a complex critical behavior, dominated by electron correlations. A paradigm of strongly correlated systems are Mott insulators.¹ Since their theoretical prediction, Mott insulators have attracted a lot of attention due to their transmutation from a metal into an insulator when electron repulsion forbids electron hopping and transport.² Understanding the critical behavior at the Mott transition and the different exotic properties observed near it, like the unconventional metal³,⁴ or the exotic spin liquid,⁵ becomes of fundamental relevance to many physical problems.⁶,⁷

Surfaces are ideal playgrounds to study such problems, as electronic correlations are enhanced in low dimensional systems and even more in semiconducting surfaces, due to electronic localization at dangling bonds, often considerably spaced in reconstructions. A surface Mott insulating phase has been identified in the case of Cs/GaAs(110),⁸ K/Si(111):B-(√3 × √3)R30°,¹⁰ and the surface of SiC(0001),¹¹ but in these cases only the insulating phase is available, so that there is no real Mott phase transition, but only a stable Mott phase.

Interest has been renewed after the discovery of a Mott transition in α-Sn/Ge(111).¹² Upon cooling this surface down to ~150 K, a metallic phase of (3 × 3) symmetry is observed.¹³ Its unit cell contains three Sn atoms, one of them shifted upwards from the other two.¹⁴,¹⁵ This vertical distortion disappears gradually below ~30 K, and a low temperature (√3 × √3)R30° Mott phase is formed¹² (Mott-√3 in the following). Later work includes the observation of a related metal-insulator transition on Sn/Si(111),¹⁶ where the tunnel conductance dI/dV decreases at the Fermi level upon

lowering the temperature. Transport measurements on Sn/Si(111) by a four tip STM have also conclusively shown that resistivity increases when temperature decreases.¹⁷ At variance with the Sn/Ge(111) case, the Sn/Si(111) surface maintains a √3 symmetry in the whole temperature range from room temperature to 6 K. Theoretical calculations support that both Sn/Ge(111) and Sn/Si(111) are Mott insulators at low temperature, predicting even magnetism and superconductivity.¹⁸ Magnetic ordering at low temperatures has been recently reported for Sn/Si(111).¹⁹

However, some studies disagree with Sn/Ge(111) being a Mott insulator at low temperature²⁰⁻²² and there is controversy between different interpretations.²³,²⁴ Morikawa et al. did not observe a √3 phase, but they report a metallicity reduction (bad metal) at low temperature.²¹ Colonna et al. performed scanning tunneling spectroscopy experiments and concluded that Sn/Ge is still metallic at 4 K.²⁰ They attributed the observation of a √3 structure below 30 K in Ref. 12 to a tip-induced surface modification. However, this hypothesis is disputed, as Morikawa and Yeom were not able to reproduce these results.²⁴ Finally, Colonna et al.²⁰ results exhibit the same conductivity reduction at 5 K as the one observed for Sn/Si(111).¹⁶ Another study has cast doubts on the existence of a √3 phase because its low-energy electron diffraction (LEED) experiment contaminated the (3 × 3) surface and a √3 symmetry appeared afterwards.²²

In this article, we report the observation of a charge ordered insulator (COI) phase in Sn/Ge(111), competing with the Mott phase at low temperatures, describe its physical origin, and solve the aforementioned controversies. We present compelling experimental evidence on the behavior of Sn/Ge(111) between 100 and 4 K, together with theoretical calculations that provide us with a detailed interpretation on the nature and properties of the different phases observed.
## II. EXPERIMENT

The experiments were carried out in two different ultrahigh vacuum chambers for photoemission spectroscopy and scanning tunneling microscopy (STM). The base pressure during experiments was $5 \times 10^{-11}$ mbar. STM images were obtained with a commercial low-temperature microscope (Omicron) enabling imaging at sample temperatures in the range from 5 to 300 K under ultrahigh vacuum conditions. The microscope was adapted to minimize the capacitive coupling. Spectroscopic measurements [scanning tunneling spectroscopy (STS)] were made with a PtIr tip under a bias modulation of 10 mV at 700 Hz and lock-in detection of the tunneling current. STM images were analyzed with the WSxM package.²⁵ No treatments were applied to the STM images, except subtracting a plane.

High resolution angle resolved photoemission spectroscopy (HRARPES) and core level (CL) spectra were measured using a Scienta SES-2002 electron analyzer located at the SIS beamline of the Swiss Light Source. The energy resolution was 9 (20) meV for the valence band (core levels). The angle resolution was 0.1°. HRARPES data are displayed as second derivative.²⁶ The substrate was n-type Ge(111) ($\rho = 0.4$ Ω cm). The preparation of the sample and of the (3 × 3) phase have been described elsewhere.²⁷ The coverage was calibrated from the Sn 4d/Ge 3d intensity ratio, surface state intensity, and the evolution of the LEED pattern.²⁸

The sample holder used in the HRARPES measurements has a high thermal transmission, which allows cooling down efficiently. Despite the fact that this kind of sample holder is much more efficient towards cooling than conventional sample holders for preparing semiconductors, the Mott transition electronic signatures begin to appear only after being more than one hour at a nominal sample temperature of 12 K, which is the lowest temperature achievable in the HRARPES apparatus. This fact reflects that the sample needs a rather long time to reach thermal equilibrium with the cryostat (at 4 K), due to the fact that no thermal screening is used. Due to this long stabilization times, there is a certain degree of inaccuracy defining the exact surface temperature (±5 K) which can be minimized by correlation to STM data.

## III. RESULTS

The α-Sn/Ge(111) surface is formed by 0.33 monolayers (ML) of Sn atoms occupying $T_4$ sites of the Ge(111) substrate. This and closely related surfaces [Sn,Pb/Si,Ge(111)] have received considerable attention due to the complex critical behavior found around 200 K.²⁷,²⁹ In this temperature range, a fluctuating ($\sqrt{3} \times \sqrt{3}$)R30° reconstruction observed at room temperature freezes into a metallic phase of (3 × 3) symmetry.¹³ In the metallic-(3 × 3) structure, one Sn atom out of three in the unit cell is at a higher level (up atom) than the other two (down atoms), as shown in Fig. 1. The atomic vertical distortion is related to charge transfer from the two down Sn atoms into the up Sn atom. Thus the up Sn atom dangling bond becomes doubly occupied as it receives one extra electron from the two down atoms, which share the third available electron and whose dangling bonds are half-filled. The outcome is one fully occupied surface band (localized

<figure><img src="image_2.png" /></figure>

FIG. 1. (Color online) Structure of the low temperature phases of Sn/Ge(111): left, metallic-(3 × 3) phase; center, COI-(3 × 3) phase; right, Mott-$\sqrt{3}$ phase. An atomic ball model (top and side views) for the three phases is shown, including nominal charges at the different Sn atoms in the unit cell. Large (small) spheres correspond to Sn (Ge) atoms. Note the one up–two down (1U-2D) configuration of the metallic-(3 × 3) phase (left) and the three inequivalent atoms per unit cell of the COI-(3 × 3) phase (center). All atoms are equivalent in the Mott-$\sqrt{3}$ phase (right). See text for details.

mainly at top atoms) and two half-filled bands (localized mainly at the two down atoms). Therefore, the one up–two down (3 × 3) structure is metallic. It has been reported that at 77 K the two down atoms are not exactly equivalent, forming an inequivalent down atoms (IDA) (3 × 3) structure.¹⁵ In fact, surface x-ray diffraction experiments¹⁴ found a small height difference between the two down atoms of the closely related Pb/Ge(111)-(3 × 3) structure. The height asymmetry between the two down atoms is reflected in the existence of different components in the Sn 4d core level, indicating that there is an associated charge transfer from the bottom down atom into the top down atom.¹⁵

However, below 30 K the vertical distortion disappears, all atoms in the (3 × 3) unit cell become equivalent, and a low temperature ($\sqrt{3} \times \sqrt{3}$)R30° phase is formed¹² (Fig. 1). The loss of (3 × 3) long-range order is reflected in LEED patterns that will be discussed later. The new low temperature phase is a Mott insulator, but the mechanism triggering the temperature-induced Mott-$\sqrt{3}$ ground state in Sn/Ge(111) has remained elusive.

Our experimental work sheds light into how this surface enters into the low temperature Mott-$\sqrt{3}$ insulator from the (3 × 3) metallic state observed at higher temperatures. Atoms undergo a remarkable reorganization maintaining their local (3 × 3) unit cell as the temperature is lowered, until their final transformation into a flat Mott-$\sqrt{3}$ phase. This reorganization corresponds to formation of a different phase, an intermediate charge ordered insulating (COI) phase with (3 × 3) symmetry (Fig. 1) that competes with the Mott-$\sqrt{3}$ insulator. Theoretical calculations show that the transition sequence induced by temperature is driven by the interplay between surface reconstruction, electron filling, and Coulomb interaction occurring at this surface. Interestingly, a more complex sequence of phase transitions had been predicted on the basis of symmetry considerations.³⁰ We also find that the transition towards the Mott phase is gradual, and several evidences indicate phase coexistence, both at the STM time scale (ms) and at the photoemission time scale (fs).
A. Reversibility

All the temperature induced changes described below are
reversible. This includes changes in the STM images, in the
LEED patterns, and in the Sn 4d core-level line shape. The
convergence of several different experimental techniques in
the observation of temperature induced reversible changes,
strongly suggest that physical processes triggered by the tem-
perature change (phase transition) take place. The possibility
that kinetic irreversible processes or experimental artifacts play
a relevant role in the observed changes is negligible, both
because of the reversible character of the changes and because
of the broad range of experimental conditions probed by the
different experimental techniques used.

As mentioned before, the time required to stabilize the
temperature below 10 K is in the range of 1 h. Once the low
temperature phase is observed, it is possible to monitor its
disappearance/appearance by increasing and decreasing the
temperature.

Figure 2 shows the phases reversibility by means of STM.
In this case the experiment was done by first decreasing
the temperature and then increasing it. STM images were
acquired in the same region after a long-time temperature
stabilization. Complete reversibility is observed, although
more than 20 h separated the first and third STM images.
The reversibility observed in the STM images discards the
possibility that the tunneling current may trigger the COI
phase; otherwise, the starting phase would be unattainable
by STM upon temperature reversibility.

In addition to the STM measurements, we also checked
the reversibility in the electronic structure. Figure 3 shows the
changes in the CL line shape first increasing and then decreas-
ing the temperature. Line shape variations are explained in
Sec. III D. It is clear that the intensity relationship between the

<figure><img src="image_3.png" /></figure>

FIG. 2. (Color online) STM images (12.5 × 5.5 nm²) of the
same region measured at (a) *T* = 37 K (*V* = −1.5 V; *I* = 1.0 nA),
(b) *T* = 5 K (*V* = −1.4 V; *I* = 1.0 nA), and (c) *T* = 15 K (*V*
= −1.4 V; *I* = 1.0 nA). The images were acquired in top-down
chronological order. The STM images show how the temperature
induced changes are reversible and discard a change due to tip effects.

<figure><img src="image_4.png" /></figure>

FIG. 3. (Color online) Sn 4d normal emission CLs, sequentially
measured from top to bottom. Note the reversibility in the line shape
of the top spectrum, which is fully recovered after warming up and
cooling down a second time.

peaks is fully recovered after increasing the temperature 50 K.
Although the nominal temperature of the lower spectrum was
13 K, the small displacement of the CL due to photovoltage
effects indicates a slightly lower temperature. Changes induced
by temperature in the valence band electronic structure are also
reversible (not shown).

B. Real space atomically resolved

The existence of the COI phase is first evidenced by
representative STM images showing extended areas of the
Sn/Ge(111) surface at different temperatures (Fig. 4). The
COI-(3 × 3) phase is observed at intermediate temperatures
between the stability ranges of the metallic-(3 × 3) and the
Mott-√3 phases reported before.⁹,¹² The differences between
these three phases is apparent in Fig. 4. The COI-(3 × 3) phase
is characterized by three clearly different atomic heights in
the (3 × 3) unit cell, unlike the metallic-(3 × 3) phase, where
the two down atoms are either equivalent or have a height
difference much smaller than the up-down total corrugation,
and show almost the same contrast in the STM images (see
also Fig. 3 in Ref. 15 and Sec. IV). The COI-(3 × 3) unit
cell is observed in filled states STM images with one spot
much darker than the other two, due to the height difference
between the three Sn atoms. Such a bright-dark distribution
could be interpreted as a 2U-1D configuration, and indeed
some previous work has assigned this structure to a metastable
2U-1D phase.³¹ However, a later examination of the structure
shows three different atomic heights, in such a way that one of
the Sn atoms occupies a position halfway between the *up* and
*down* atoms of the (3 × 3) structure.¹⁵

Furthermore, metallic and COI-(3 × 3) phases can be
unambiguously distinguished from the differences between
filled and empty state images. A visual inspection of Fig. 5
<figure><img src="image_5.png" /><figcaption>FIG. 4. (Color online) (a) Metallic-(3 × 3) phase (12 × 12 nm²; V = -1.0 V; I = 0.525 nA; T = 78 K). (b) COI-(3 × 3) structure (12 × 12 nm²; V = -1.0 V; I = 1.0 nA; T = 60 K). Dots mark up (red), middle (green), and down (blue) atoms. (c) Mott-√3 phase (12 × 12 nm²; V = -1.4 V; I = 1.0 nA; T = 5 K). In every panel, a unit cell of the corresponding phase is highlighted.</figcaption></figure>

reveals that the bright-dark contrast of filled and empty states is complementary in the metallic-(3 × 3) phase (hexagonal vs honeycomb patterns for filled and empty states, respectively). On the contrary, in the COI-(3 × 3) phase both empty and filled states images look the same. On the other hand, empty states images of both (3 × 3) phases are almost identical, and they can only be distinguished by comparing their height profiles [Fig. 5(d)]. Therefore, the two (3 × 3) phases cannot be discriminated from an analysis of the empty states images only. The difference between the two (3 × 3) phases (metallic and COI) can be traced back to a different orbital occupancy, in view of the differences seen in the STM images. A metallic (3 × 3) structure with equivalent down atoms would present

<figure><img src="image_6.png" /><figcaption>FIG. 5. (Color online) Complementary STM images (17 × 10 nm²) of the same region obtained sequentially (a) for filled (V = -1.0 V; I = 1.0 nA) and (b) for empty (V = +1.0 V; I = 1.0 nA) states. As there is a significant number of defects, we observe coexistence of metallic-(3 × 3) (red squares) and COI-(3 × 3) (green squares). Panel (c) shows a zoom (4 × 4 nm²) of both empty states square areas from panel (b). Using empty states images, both phases can only be distinguished from an analysis of the height profiles, as shown in panel (d) for profiles along the two lines appearing in panel (b).</figcaption></figure>

a nominal occupancy of the Sn atoms within the unit cell of (2-0.5-0.5). We may expect that a height difference between the two down atoms will affect the orbital occupancy, which will change accordingly with the height difference. The three different brightness of Sn atoms in the COI-(3 × 3) phase reflect a nominal charge close to a (2-1-0) occupancy (a quantitative justification of these numbers is given in Sec. IV D on the basis of the core level shifts).

Previous studies have already found some evidence of the COI-(3 × 3) phase,²⁰,²¹,²³,²⁴ where a (3 × 3) symmetry at 5 K was observed, even though it was misidentified. Morikawa *et al.* have reported that this low temperature phase is a bad metal.²¹ Colonna *et al.* did not interpret their data in this way,²⁰ but their STS spectra reflect that the conductivity at the Fermi level is reduced by approximately a factor of 10 when decreasing the temperature from 20 to 5 K. The STM images of Morikawa *et al.* show large defect free regions, while probing exactly the same area in empty and filled states. Moreover, their images were acquired with a tip of similar lateral resolution and a similar drift at both polarities, which supports that their empty and filled states images can effectively be compared. We have obtained images of the COI-(3 × 3) phase similar to those of Morikawa *et al.* with tips offering a low lateral resolution [Figs. 6(a) and 6(b)]. Even if observed at very low temperature, the COI-(3 × 3) phase is not the ground state of the system but an intermediate phase between the metallic-(3 × 3) and the Mott-√3 phases. It already appears at 60 K coexisting with the well known metallic-(3 × 3) phase and it coexists with the Mott-√3 phase at lower temperature (Fig. 7). The mechanism of the transition between phases includes domain nucleation and domain growth as the transition proceeds. Phase coexistence when the transition is not completed may explain the reported difficulties to observe the Mott-√3 phase, disregarding other factors like different temperature calibrations on the sample surface, or different
<figure><img src="image_7.png" /><figcaption>FIG. 6. (Color online) Sequential STM images (9.5 × 9.5 nm²; V = -1.4 V; I = 1.0 nA) of the same area, obtained at T = 37 K [panel (a)] and T = 38 K [panel (b)] measured with low experimental resolution. Panel (c): STM image (12.5 × 9.5 nm²; V = -1.4 V; I = 0.2 nA) obtained at T = 5 K. Green squares show regions with COI-(3 × 3) structure imaged with low resolution. The blue square marks a region with Mott-$\sqrt{3}$ structure. Due to the low resolution, both regions are difficult to distinguish.</figcaption></figure>

transition temperatures due for instance to different defect concentrations. Defects play no role in the transition, since their number remains fixed and they are static (see also below and Fig. 2). However, they act as nucleation centers for the

<figure><img src="image_8.png" /><figcaption>FIG. 8. (Color online) STM images (14 × 14 nm²) of the same region taken sequentially at T = 5 K with V = -1.5 V and I equal to (a) 0.5 nA, (b) 1.0 nA, (c) 2.0 nA, (d) 3.0 nA, (e) 5.0 nA, and (f) 10.0 nA. Small Mott-$\sqrt{3}$ areas (encircled) coexist with (3 × 3) areas. Flattening effects of the (3 × 3) areas are observed only for currents larger than 5 nA.</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>FIG. 7. (Color online) STM images showing phase coexistence. Phase coexistence is favored when the Sn coverage deviates from 0.33 ML and/or when the Sn/Ge(111) surface has a significant density of defects: (a) Metallic-(3 × 3) (red square) and COI-(3 × 3) (green) (T = 38 K; 20.5 × 9.5 nm²; V = -1.4 V; I = 1.0 nA); (b) Mott-$\sqrt{3}$ (blue) and COI-(3 × 3) (green) (T = 25 K; 15 × 7 nm²; V = -1.4 V; I = 0.1 nA).</figcaption></figure>

(3 × 3) phases, as the defects favor such a configuration. Therefore, surfaces with a higher number of defects will have a lower COI-(3 × 3) towards Mott-$\sqrt{3}$ transition temperature. A detailed analysis of the role of defects has been presented in Ref. 32.

We consider next the possible existence of tip-induced effects in the STM images at low temperatures. Colonna *et al.*²⁰ found that a tunneling current exceeding 200 pA (for -1.0 V sample bias voltage) induces an apparent $\sqrt{3}$ topography. On the basis of this observation, they denied the existence of the Mott-$\sqrt{3}$ phase. This result was later disputed by Morikawa *et al.*,²⁴ who could not reproduce the tunneling-current dependence of the apparent $\sqrt{3}$ pattern. The existence of tip-induced effects in semiconducting surfaces imaged at low temperature is a well known phenomenon.³³ Figure 8 collects a series of sequential images from the same surface area taken at 5 K for V = -1.5 V and increasing tunneling currents. The images were taken in an area with coexistence of Mott-$\sqrt{3}$ and (3 × 3) structures. The coexistence of $\sqrt{3}$ and (3 × 3) surface phases in the same scan line in several images excludes that one of these phases could be stabilized due to
the experimental conditions. It is clear from these images that tip-induced flattening of the (3 × 3) structure is seen for currents above ~5 nA. Due to this fact all our experiments were conducted using tunneling currents below 1 nA. This threshold is different from both the values reported before by Colonna et al.²⁰ and Morikawa et al.²⁴ The discrepancy with previous reports is not surprising. We recall that it is difficult to predict the effect that a given tunneling current value will have on a surface. For the same current, a blunt tip or an extremely sharp one will induce a very different electrical field on the surface. From the data shown in Fig. 8, we can demonstrate that there are no tip-induced effects under our experimental conditions. This includes the STM images shown in Fig. 2, which correspond to the same region and are taken at a tunneling current of 1 nA. The temperature was changed slowly, while taking images of the same area. The transition takes place both upon increasing and upon lowering the temperature in a reversible way, which further helps to discard spurious effects. Finally, the existence of any STM artifact in the images is implausible as COI-(3 × 3) phase images have been measured with many different tips and a wide range of experimental conditions (including tunneling current, bias, scanning direction, temperature, etc.).

### C. Long range order

The long range order of the sample has been monitored as a function of temperature by low energy electron diffraction (LEED) (Fig. 9). A sharp (3 × 3) LEED pattern is observed

<figure><img src="image_10.png" /><figcaption>FIG. 9. (Color online) LEED patterns as a function of temperature for $E_0 = 94$ eV. When temperature decreases, (3 × 3) spots weaken and disappear and only $\sqrt{3}$ spots survive. Panel (g) shows the intensity evolution vs temperature, along the straight lines highlighted in panels (a)–(f).</figcaption></figure>

at 120 K. As temperature decreases, the intensity of the (3 × 3) spots weakens. At lower temperatures the (3 × 3) spots disappear and the pattern becomes $\sqrt{3}$. The new pattern is also sharp and with low background. The phase transition is fully reversible. Figure 9 presents temperature dependent LEED intensity profiles that show the gradual weakening of (3 × 3) spots, evolving into the $\sqrt{3}$ symmetry. The weakening of (3 × 3) spots reflects a decrease of domain size below at least ~85 K. At 49 K only $\sqrt{3}$ spots are detected. No beam effects as reported in Ref. 22 were observed under our experimental conditions.

### D. Sn 4d core levels

Each of the phases found by STM can be related to a specific line shape of the Sn 4d core level (CL) recorded at different temperatures. Figure 10 shows an overview of the evolution of the Sn 4d CL line shape as a function of temperature in the 82–12 K range. The line shape exhibits two significant changes upon cooling down in this range. First, it remains almost unmodified above 61 K, corresponding to the metallic-(3 × 3) phase, but it changes abruptly between 61 K and 60 K. The leap is clearly observed in the three dimensional representation of the more than 90 individual CL lines measured from 82 K down to 12 K [panel (a) of Fig. 10], as highlighted by an arrow. Since we know from other techniques that the symmetry is still (3 × 3) at this temperature, the change suggests an atomic rearrangement within the (3 × 3) unit cell. A second change is observed starting at 27 K.

More information is obtained by performing a deconvolution of the Sn 4d CL line shape in its different components (Fig. 11). The deconvolution uses a set of previously established components, identified after an exhaustive statistical analysis.¹⁵ It comprises four Gaussian-convoluted Doniach-Šunjić doublets and a Shirley background.¹⁵ The data set shown here includes an additional minor fifth component, as explained below. The three most intense components of the CL down to 62 K correspond to each of the three different Sn atoms of the metallic-(3 × 3): $C_1$ comes from Sn atoms in *up* position, while $C_2$ and $C_3$ are due to the two inequivalent atoms in *down* position. The minor intensity difference between the two components is attributed to photoelectron diffraction effects, as the two down atoms are not equivalent. The $C_4$ component has been previously identified and tentatively assigned to surface defects,²⁸ but based on the present data we conclude that it has a different origin (see below). Defects give rise to an extra minor $C_5$ component, required to reproduce the line shape for high binding energies (BE) in the whole temperature range. The result of the fitting is summarized in Table I. Figure 11 displays representative CL spectra of each phase. The final fit reproduces well ($\chi^2 = 0.06$) the experimental data, as shown in Fig. 11 and panel (b) of Fig. 10. We would like to stress here that the fit has been made keeping the Lorentzian width, spin-orbit splitting, and branching ratio constant along the whole temperature range. This means that, for each range, the line shape is reproduced by changing only the intensity, the Gaussian width (GW), the binding energy, and singularity index of the different components (see Table I).

In order to highlight the specific behavior of each component, integrated intensities are plotted as a function of
<figure><img src="image_11.png" /><figcaption>FIG. 10. (Color online) Normal emission Sn 4d CLs vs temperature. (a) Three dimensional representation of the experimental CLs taken from 82 K down to 12 K. The arrows highlight significant changes in the line shape. 92 individual CLs are plotted in total. (b) Selected CLs representative of each surface phase, extracted from panel (a). Black dots represent experimental data and color lines correspond to the fitting obtained with the parameters listed in Table I (see text for details). Three phases are identified: metallic-(3 × 3) (82–61 K, top CLs, red), COI-(3 × 3) (60–27 K, center CLs, green), and Mott-√3 (15–12 K, bottom CLs, blue). Note the abrupt change at 60 K and the progressive change below 27 K, not finished at the lowest temperature reached (12 K).</figcaption></figure>

temperature in Fig. 12. The intensities of $C_1$, $C_2$, and $C_3$ change near 60 K, where the abrupt change in the CL line shape is observed (Fig. 10). After this change, the 1:1:1 intensity ratio between $C_1$, $C_2$, and $C_3$ is not maintained, suggesting that an atomic reorganization takes place around this temperature, and that the new arrangement is affected by slightly different photoelectron diffraction effects. This change coincides with the temperature range where the COI-(3 × 3) starts to be predominantly observed by STM, which strongly suggests that this atomic reorganization corresponds to the formation of the COI-(3 × 3) phase.

<figure><img src="image_12.png" /><figcaption>FIG. 11. (Color online) Deconvolution of representative Sn 4d normal emission CLs vs temperature with four main components ($C_1$ to $C_4$) plus a minor one ($C_5$).¹⁵ Deconvolution details can be found in Table I.</figcaption></figure>

Between 60 K and 15 K, $C_1-C_3$ do not change significantly, but they decrease rapidly below 15 K, with a simultaneous dramatic increase of $C_4$ component (see Figs. 10 and 11). The intensity increase of $C_4$ at low temperature indicates that this

<table><caption>TABLE I. Parameters used in the deconvolution shown in Fig. 11. The branching ratio is 1.34 and the spin-orbit splitting is −1.05 eV. The Lorentzian width for all the components is 220 meV except for $C_1$, which is 140 meV.</caption><thead><tr><th colspan="4">Metallic-(3 × 3)</th></tr><tr><th>Component</th><th>GW (meV)</th><th>BE (eV)</th><th>Singularity index</th></tr></thead><tbody><tr><td>C<sub>1</sub></td><td>171</td><td>-23.48</td><td>0.07</td></tr><tr><td>C<sub>2</sub></td><td>326</td><td>-23.70</td><td>0.07</td></tr><tr><td>C<sub>3</sub></td><td>352</td><td>-23.88</td><td>0.07</td></tr><tr><td>C<sub>4</sub></td><td>557</td><td>-24.21</td><td>0.00</td></tr><tr><td>C<sub>5</sub></td><td>600</td><td>-24.75</td><td>0.08</td></tr><tr><th colspan="4">COI-(3 × 3)</th></tr><tr><th>Component</th><th>GW (meV)</th><th>BE (eV)</th><th>Singularity index</th></tr><tr><td>C<sub>1</sub></td><td>170</td><td>-23.50</td><td>0.045</td></tr><tr><td>C<sub>2</sub></td><td>320</td><td>-23.70</td><td>0.045</td></tr><tr><td>C<sub>3</sub></td><td>345</td><td>-23.88</td><td>0.045</td></tr><tr><td>C<sub>4</sub></td><td>550</td><td>-24.22</td><td>0.00</td></tr><tr><td>C<sub>5</sub></td><td>600</td><td>-24.77</td><td>0.08</td></tr><tr><th colspan="4">Mott-√3</th></tr><tr><th>Component</th><th>GW (meV)</th><th>BE (eV)</th><th>Singularity index</th></tr><tr><td>C<sub>1</sub></td><td>160</td><td>-23.53</td><td>0.045</td></tr><tr><td>C<sub>2</sub></td><td>310</td><td>-23.74</td><td>0.045</td></tr><tr><td>C<sub>3</sub></td><td>325</td><td>-23.90</td><td>0.045</td></tr><tr><td>C<sub>4</sub></td><td>540</td><td>-24.23</td><td>0.00</td></tr><tr><td>C<sub>5</sub></td><td>570</td><td>-24.80</td><td>0.08</td></tr></tbody></table>
<figure><img src="image_13.png" /><figcaption>FIG. 12. (Color online) Critical behavior extracted from CL deconvolution analysis: integrated intensity of the different components vs temperature.</figcaption></figure>

component is related to the Mott-$\sqrt{3}$ phase. This assignment is further supported by the fact that its intensity does not depend on the surface preparation. It has also a significantly larger width, which evidences an origin different from that of the (3 × 3) components. Finally, since the number of defects does not change with temperature, it is clear that it cannot be related to defects.

We consider now the evolution of the BE of the different components with temperature (Fig. 13). The most prominent feature is a steady shift of the BE upon cooling that is related to

<figure><img src="image_14.png" /><figcaption>FIG. 13. (Color online) Critical behavior extracted from CL deconvolution analysis. (a) BEs (referred to their value at 82 K) vs temperature. (b) Closer view of the BE evolution measured with respect to $C_1$ (green circles), which removes the SPV shift of $C_2$ (yellow up triangles) and $C_3$ (blue down triangles). (c) Closer view of the critical temperature range around 60 K highlighted with a gray shadowing in all the panels (values for $C_1-C_3$ are represented by the same markers and colors as in previous panels).</figcaption></figure>

<figure><img src="image_15.png" /><figcaption>FIG. 14. (Color online) Critical behavior extracted from CL deconvolution analysis: change of the quadratic Gaussian widths (referred to their value at 82 K) vs temperature.</figcaption></figure>

a temperature-dependent surface photovoltage (SPV) induced by ultraviolet radiation.12 At 60 K a leap is observed for all components, indicating a sudden increase of the SPV, which denotes a reduction in the surface conductivity for constant radiation flux. Therefore, the formation of the COI-(3 × 3) phase relates with a lower surface conductivity. In addition, $C_1$, $C_2$, and $C_3$ change their relative energetic separation [Fig. 13(b)]. In particular, the separation between $C_2$ and $C_3$ increases. Considering that initial state effects are expected to be dominant and that Madelung potential should not change significantly, this means that there is an increase in the charge transfer from one down atom ($C_3$ component) into the other ($C_2$ component). Furthermore, the three components $C_1$, $C_2$, and $C_3$ become nearly equidistant below 60 K, suggesting an occupancy close to the nominal values (2-1-0). This result is in agreement with the STM observation at the same temperature of three different brightnesses of the Sn atoms. The change of occupancy and the simultaneous change in the SPV supports that the new phase with (2-1-0) occupancy is a correlated insulator, while the previously known (3 × 3) phase is metallic. Further reduction of the temperature promotes the Mott-$\sqrt{3}$ phase with all Sn atoms of the high temperature unit cell at the same height, indicating a (1-1-1) orbital occupancy. Finally, $C_1$, $C_2$, and $C_3$ BEs are slightly modified at very low temperatures ($T < 15$ K). These changes are difficult to interpret both because they are rather small and because they coincide with the intensity depletion of all components related to the (3 × 3) structure.

Further information on the properties of the different phases is contained in the GWs of the CL components, which are temperature dependent (Fig. 14). The temperature dependence is related to phonon broadening, as other contributions to the Gaussian width, like the finite instrumental resolution or surface inhomogeneities, are temperature independent.34 The phonon contribution to the Gaussian width $G(T)$ can be approximated as

$$G^2(T) = G_{\text{phon}}^2(0) \coth \left( \frac{3}{8} \frac{\theta_D}{T} \right), \quad (1)$$

where $\theta_D$ is the Debye temperature and $G_{\text{phon}}^2(0)$ is the Gaussian width at $T = 0$ K, which depends on the valence band structure, the solid density, and $\theta_D$.34 This equation shows that, for a given system, the Gaussian width should decrease monotonously as temperature decreases and, particularly, with
a linear dependence for a large range above $\theta_D$. Nevertheless, Fig. 14 shows a slope change for the components related to the (3 × 3) phases at ~60 K. This critical behavior can only be explained by a change in $G_{\text{phon}}^2(0)$ and/or $\theta_D$, i.e., a critical change of the phonon spectra and/or the valence band at ~60 K. This implies that the charge redistribution observed is associated to a modification of the surface vibrations, which should be frozen at low temperatures. Considering the detailed analysis of the surface phonon modes for this surface reported in Refs. 35 and 36, we propose that it mostly corresponds to the freezing of the lowest energy mode, the $1/\sqrt{2}[01\bar{1}]$, involving the up-down motion of the two down atoms. As a result of the freezing of this motion, a charge redistribution within the unit cell is foreseen.

$C_4$ component has a width significantly larger than $C_1-C_3$ (Fig. 11), which suggests that $C_4$ has a different origin than those components, in agreement with our assignment. Even if there is no general model available to describe the core level line shape of a Mott insulator, a larger width may be its fingerprint. The Mott-$\sqrt{3}$ $C_4$ component is originated in an environment with different screening with respect to the other components and therefore such a different width can be traced back to correlation effects. The BE of $C_4$ is found to be larger than those of the three (3 × 3) components. This contradicts the initial state prediction, because $C_4$ is associated to atoms at an intermediate height between up and down positions and with nominal occupancy 1, as Sn atoms corresponding to $C_2$ in the COI-(3 × 3) phase. Thus $C_4$ should be between $C_1$ and $C_3$, and close to $C_2$. However, the BE of $C_4$ is expected to be affected by strong final state effects, due to the insulating character of the Mott phase and a corresponding worsening of the hole screening, which shifts the observed binding energy of $C_4$ to more negative values with respect to $C_2$ value.37

### E. Theoretical calculations

#### 1. DFT calculations of the structural (3 × 3)-$\sqrt{3}$ transition

We start discussing the relative stabilities of the metallic structures from a DFT perspective. DFT calculations yield that the (3 × 3) structure has an energy smaller than a hypothetical ($\sqrt{3} \times \sqrt{3}$) R30° metallic phase; this energy has been found to change slightly with the different DFT codes used in the calculations: $E_{\text{DFT}}(\sqrt{3})-E_{\text{DFT}}(3 \times 3) = 5$ meV/atom,38 23 meV/atom,38 10 meV/atom,39 and 7.5 meV/atom.40 An average of all these values yields $E(\sqrt{3})-E(3 \times 3) = (11 \pm 7)$ meV/atom. This error bar, associated with the precision in the LDA calculations, limits our ability to accurately determine the energy difference between different phases. However, the LDA calculations for the one up-two down metallic (3 × 3) structure38,39 yields surface bands in good agreement with the photoemission experiments (see below). In particular, one band fully occupied, associated with the up atom, is located around 0.2 eV from the two partially filled bands crossing the Fermi associated with the two down atoms.

#### 2. Correlation energy contributions: Dynamical mean-field theory

DFT calculations neglect the correlation energy associated with the dangling bonds of the narrow bands. These effects are found to stabilize the $\sqrt{3}$ phase with respect to the (3 × 3)

structure, as explained below. In order to calculate correlation energies we introduce a minimal strongly correlated model that captures the essential physics of the Sn/Ge(111) system. This is an ionic extended Hubbard model on an isotropic triangular lattice:

$$H = -t \sum_{\langle ij \rangle \sigma} (c_{i\sigma}^{\dagger} c_{j\sigma} + c_{j\sigma}^{\dagger} c_{i\sigma}) + U \sum_{i} n_{i\uparrow} n_{i\downarrow} + V \sum_{\langle ij \rangle} n_{i} n_{j} + \sum_{i\sigma} \epsilon_{i} n_{i\sigma}, \quad (2)$$

where $c_{i\sigma}^{\dagger}$ creates an electron with spin $\sigma$ at site $i$. The one-electron part of the model contains the kinetic energy of the electrons described by the amplitude, $t$, for an electron to hop between neighboring $ij$ sites on a triangular lattice with alternating site energies. In the (3 × 3) phase atoms displaced upwards (downwards) have their site energies shifted downwards (upwards) by $\epsilon_i = -2\Delta/3 (+\Delta/3)$ leading to the (3 × 3) structure. The $\Delta = 0$ case corresponds to the flat $\sqrt{3}$ structure. The kinetic energy competes with the on site Coulomb repulsion, $U$, and the Coulomb repulsion between electrons on nearest neighbor sites of the triangular lattice, $V$. The one-electron parameters, $t$ and $\Delta$, as well as the Coulomb repulsion energies $U$ and $V$ are obtained from previous DFT calculations39,41 and compared to experiments. We find that the experimental observations described in the next section are best described by taking $U = 0.56$ eV, $t = 0.04$ eV, $\Delta = 0.54$ eV, and $V = 0.13$ eV, and are at the same time consistent with previous first principles estimates.39,41 We use these values for the parameters in theoretical calculations discussed below.

To illustrate the different energy correlation contributions we first consider the model described in Eq. (2) for $t = 0$. In the metallic structure with $\frac{1}{2}$-electron per spin and dangling bond, the maximum correlation energy per electron appears for a high local Coulomb interaction energy ($U$). In this limit, electrons avoid each other, the Mott phase appears, and the electron-electron repulsion energy per electron is reduced by $U/4$. Moreover, the exchange energy, $-V/4$, also disappears, so that, in this highly correlated limit, the correlation energy per electron is $-(U-V)/4$. A similar argument for the metallic-(3 × 3) yields a correlation energy of $-(U-V)/24$, neglecting the doubly occupied sites as $U \gg t$. From this we conclude that for the dangling bond bands $E_{\text{corr}}(\text{Mott}-\sqrt{3}) - E_{\text{corr}}[\text{metallic}-(3 \times 3)] = -5(U-V)/24$. This energy amounts to $-92$ meV for the $U$ and $V$ values used here. Similarly, the maximum correlation energy for the COI-phase is $E_{\text{corr}}[\text{COI}-(3 \times 3)] - E_{\text{corr}}[\text{metallic}-(3 \times 3)] = -V/6 = -20$ meV.

On site Coulomb correlation effects for any intermediate $U$ value are treated exactly through dynamical mean field theory (DMFT) on model (2). The off-site Coulomb repulsion, $V$, between one site and the three neighboring sites in this triangular lattice is treated at the static mean-field level and is ultimately responsible for the broken symmetry COI-(3 × 3) phase. We use a zero temperature Lanczos technique to exactly solve the associated Anderson model in a self-consistent electron bath.42 The total energy functional, $E_{\text{DMFT}}[n_i]$, per lattice site is calculated for the site occupations $n_i$. The correlation energy, $E_{\text{corr}}[n_i]$, is obtained from the total DMFT
energy, $E_{\text{DMFT}}[n_i]$, all evaluated at $n_i$:

$$E_{\text{corr}}[n_i] = E_{\text{DMFT}}[n_i] - E_K[n_i] - E_H[n_i], \quad (3)$$

with the Hartree energy per site $E_H[n_i] = \frac{1}{N}(\sum_i U n_i \uparrow n_i \downarrow + \sum_{\langle ij \rangle} V n_i n_j)$, with the sum over $\langle ij \rangle$ restricted to nearest-neighbor sites only and $N$ the number of lattice sites. $E_K[n_i]$ is the kinetic energy of the effective one-electron model, $E_K[n_i]$, evaluated at the same $n_i$ as required by Kohn-Sham equations.43,44 The total energy per site including correlation effects finally reads

$$E[n_i] = E_{\text{DFT}}[n_i] + E_{\text{corr}}[n_i], \quad (4)$$

which is used to analyze the relative stability of the $(3 \times 3)$ and $\sqrt{3}$ phases including local correlation effects.

Our DMFT calculations for the correlation energy give $E_{\text{corr}}(\text{Mott} - \sqrt{3}) - E_{\text{corr}}[\text{metallic} - (3 \times 3)] = -21$ meV, which is about $\frac{1}{4}$ of the $t=0$ result discussed previously. The COI-(3 × 3) insulating phase is found to be $E_{\text{corr}}[\text{COI} - (3 \times 3)] - E_{\text{corr}}[(3 \times 3)] = -5$ meV. Therefore, both the COI-(3 × 3) and Mott-$\sqrt{3}$ phases have energies below the metallic-(3 × 3) phase by 5 meV and $(10 \pm 7)$ meV, respectively. Our estimates indicate that the Mott-$\sqrt{3}$ phase is more stable than the COI-(3 × 3) phase but by an energy difference which can be as small as 1–2 meV, within the error bar. Such small energy difference is consistent with the observed coexistence of the COI-(3 × 3) and Mott-$\sqrt{3}$ phases at low temperatures (below 15 K). However, the large inaccuracy in the DFT energy precludes a reliable estimate of the transition temperature between these two phases observed.

On the other hand, the phase transition between the metallic-(3 × 3) and the COI-(3 × 3) phases observed at about 60 K is in excellent agreement with the energy difference of 5 meV between the two phases theoretically predicted. Since the energy difference between the COI-(3 × 3) and metallic-(3 × 3) phases is free from DFT numerical inaccuracies, the temperature scale estimated from our theoretical approach for this phase transition is reliable.

## F. Electronic band structure

The charge rearrangement deduced from the changes in the CL line shape should also affect the electronic band structure near the Fermi level. The ideal tool to observe it is HRARPES. A scheme of the reciprocal space of the $(3 \times 3)$ and $\sqrt{3}$ structures is shown in Fig. 15. HRARPES data along $\overline{\Gamma M_{(3 \times 3)}}$ direction [Fig. 16(b)] show that the upper surface state $S_2$ crosses the Fermi level $E_F$ in the metallic-(3 × 3) phase at 105 K, but at intermediate temperatures (data at 37 K are shown) the surface becomes insulating for the COI-(3 × 3) phase, a small band gap develops (shift of the leading edge of ~25 meV; see also Fig. 17), and $S_2$ shows a kink. A comparison with 12 K spectra indicates that the kink is indeed the starting point of the three band structure observed at this temperature, where the shift of the leading edge is ~75 meV. $S_2$ becomes rather flat near $E_F$, and we assign this band and the deeper lying band at -0.5 eV to the COI-(3 × 3) phase. The third flat band at -0.25 eV BE is assigned to the Mott-$\sqrt{3}$ phase, as it becomes increasingly intense below 20 K. Therefore, HRARPES data at 12 K show that the COI-(3 × 3) and the Mott phases coexist at low temperature.

<figure><img src="image_16.png" /><figcaption>FIG. 15. (Color online) Surface Brillouin zones of the $(1 \times 1)$, $(\sqrt{3} \times \sqrt{3})R30^{\circ}$, and $(3 \times 3)$ structures. The high symmetry points correspond to the $(3 \times 3)$ phase.</figcaption></figure>

This band assignment has been theoretically analyzed using the model described in the previous section [Eq. (2)]. The energy levels are shown in Fig. 16(c). In the Mott-$\sqrt{3}$ phase, the filled Mott-Hubbard band appears at $\epsilon_0$ ($\epsilon_0$ taken as the energy reference) and the empty one at $\epsilon_0 + U$. In the metallic-(3 × 3) phase the vertical distortion moves the doubly filled state by -2Δ/3, and the 1/4-filled levels by Δ/3 (Δ = 0 corresponds to a flat surface):39,41 an additional Hartree energy shift due to charge transfer occurs (exchange and correlation effects are negligible), equal to $U - 3V$ for the doubly occupied states and $U/4 + 3V/2$ for the partially filled ones. The final levels for this metallic-(3 × 3) phase are $\epsilon_0 + U - 3V - 2\Delta/3$ and $\epsilon_0 + U/4 + 3V/2 + \Delta/3$. Within the COI-(3 × 3), the doubly filled state feels the same mean electronic density and therefore its energy is again $\epsilon_0 + U - 3V - 2\Delta/3$. Due to correlation effects in the COI-(3 × 3), the partially filled states of the metallic-(3 × 3) phase split into an empty level, with energy $\epsilon_0 + 3V + \Delta/3$, and a singly occupied level, which again splits into $\epsilon_0 + \Delta/3$ and $\epsilon_0 + \Delta/3 + U$. As it has been previously mentioned, experimental observations are best described by taking $U = 0.56$ eV, $V = 0.13$ eV, and $\Delta = 0.54$ eV values. Bandwidths are incorporated by taking a hopping energy $t = 0.04$ eV,39,41 and calculated through the DMFT formalism that allows describing simultaneously metallic, insulating, and intermediate phases.47 The level structure persists and the bands [Fig. 16(d)] are in a one-to-one correspondence with the energy levels [Fig. 16(c)] and with the HRARPES data [Fig. 16(b)]. For the metallic phase [Fig. 16(d), left], we find a fully occupied band (labeled a) 0.5 eV below the Fermi level, and two bands (labeled b) around the Fermi level. This is similar to the results previously found within LDA calculations,38,39 although in our DMFT calculations bands are narrowed and the separation between bands a and b is larger. This is probably due to the simplified model of Eq. (2) used and the bath discretization introduced for the Lanczos solution of the DMFT equations. Such bath parametrization is also responsible for the peak structure instead of the continuous spectra defining the lower and upper Hubbard bands typical of the Mott-$\sqrt{3}$ phase [Fig. 16(d), right]. Regarding the COI-(3 × 3) phase, notice how the original $S_2$ surface band of the metallic-(3 × 3) opens a gap, as observed in HRARPES [Fig. 16(d), center]. In addition, spectral features observed
<figure><img src="image_17.png" /><figcaption>FIG. 16. (Color online) Electronic properties: experimental and theoretical results for the metallic-(3 × 3), the COI-(3 × 3), and the Mott-$\sqrt{3}$ phases. (a) Differential conductance spectra acquired at different sites of the surface unit cell. Note that the temperature of the COI-(3 × 3) spectra is 77 K (see text for details). (b) Second derivative HRARPES data along $\overline{\Gamma M_{(3\times3)}}$ ($h\nu = 23$ eV). $\overline{\Gamma}$ belongs to the second Brillouin zone. (c) Theoretical electronic level structure in the atomic limit. (d) DOS obtained from a DMFT approach. Electronic bands are referred to $E_F$. Arrows in the right panel denote the position of the center of gravity of the lower and upper Mott-Hubbard bands for the Mott-$\sqrt{3}$ phase, which appear as a set of $\delta$ peaks due to the bath discretization. An envelope of the $\delta$ peaks has been added to represent the actual Mott-Hubbard bands more clearly.</figcaption></figure>

by HRARPES when entering the Mott-$\sqrt{3}$ phase indicate the coexistence of COI-(3 × 3) and Mott-$\sqrt{3}$ phases, as a three-occupied level structure is observed. According to the theoretical analysis, these three levels correspond to the doubly occupied level of the COI-(3 × 3) ($a'$ in Fig. 16), the lowest occupied Mott-Hubbard level associated with the Mott-$\sqrt{3}$ ($a''$) and the Mott-Hubbard singly occupied level of the COI-(3 × 3) phase ($b'$). Finally, theory shows that the COI-(3 × 3) with charge ordering in the down sites occurs for $3V > W$ ($W$, bandwidth), the parameters of Sn/Ge(111) being within this range.

<figure><img src="image_18.png" /><figcaption>FIG. 17. (Color online) EDCs of the metallic-(3 × 3) ($T = 139$ K) and COI-(3 × 3) ($T = 27$ K) phases corresponding to the momentum at which the upper surface state $S_2$ crosses the Fermi level in the metallic-(3 × 3) phase equal to $0.233 \AA^{-1}$ measured from $\overline{\Gamma}$ of the corresponding Brillouin zone). We note that this reciprocal space point coincides with the point of closest approach at low temperature. The crossing point and point of closest approach were determined from a series of EDCs covering a broad range around the crossing point.<sup>45,46</sup> A zoom of the region closer to the Fermi level, highlighted by the rectangle in panel (a), is shown in panel (b).</figcaption></figure>

STS data for the three phases are displayed in Fig. 16(a). Spectra at 110 K corresponding to the metallic-(3 × 3) phase show one peak in the occupied and one in the unoccupied states.<sup>21</sup> The intensity of the spectra corresponding to up atoms is larger for the occupied part, and vice versa for down atoms. Spectroscopy on the Mott-$\sqrt{3}$ at 5 K shows a large decrease of the intensity at $E_F$ and a gap (~100 meV). COI-(3 × 3) regions are imaged at 77 K [note that this temperature is above the transition temperature to the COI-(3 × 3) phase determined from photoemission experiments<sup>48</sup>]. The differential conductance at $E_F$ is reduced by one order of magnitude in the COI-(3 × 3) regions with respect to the metallic phase, reflecting its insulating character and the opening of a small gap (~40 meV).

The theoretical analysis described above explains these local electronic states, both occupied and unoccupied. The DMFT-DOS for the COI-(3 × 3) predicts four main peaks in the -0.6 to 0.5 eV range [Figs. 16(c) and 16(d)], two occupied and two unoccupied. STS spectra show two main peaks at -0.5 and +0.5 eV, and two minor peaks at -0.1 and +0.1 eV. Theoretical predictions also indicate a doubling of the gap when going from the COI-(3 × 3), where it is $3V - 6t$ (~0.1 eV), to the Mott-$\sqrt{3}$ with a gap of $U - 9t$ (~0.2 eV). All these features match the STS data. However, a specific assignment of the peaks in Fig. 16(a) (center) is not possible. Although the main feature of the spectra (observation of two occupied and two unoccupied peaks) is explained by the theory, the interpretation of the different line shapes found for up, down, and middle atoms is not straightforward. Several factors may contribute to this uncertainty, including the existence of thermal fluctuations and the lateral resolution of the tip in spectroscopic mode.
#### IV. DISCUSSION AND CONCLUSIONS

The observation of three components ($C_1$, $C_2$, and $C_3$) in the Sn 4d CL reveals the existence of three different kinds of Sn atoms in the unit cell.¹⁵ The correlation between the components seen by CL photoemission and STM images deserves some comments. In previous work it was established that the Sn 4d CL line shape at RT (corresponding to a disordered $\sqrt{3}$ phase) and at 100 K [metallic-(3 × 3)] are very similar, besides a resolution improvement at LT.²⁷,²⁹ However, while CL photoemission reveals at least two components at RT, STM images show identical atoms in a $\sqrt{3}$ structure. This apparent discrepancy was explained from the different probing times of STM (ms) and photoemission (fs). As Sn atoms are fluctuating at RT,²⁷,³⁶ STM images show the average, with apparently identical atoms, but photoemission is able to probe a frozen atomic fluctuation. Interestingly, a similar situation is observed at lower temperatures, as CL photoemission reveals three components in the CL, typical of the COI-(3 × 3) phase, at temperatures well above the phase transition. We propose that the two down atoms fluctuate between positions at slightly different heights, which produces three components in the Sn 4d CL. The two different heights of the two down atoms are clearly distinguished in STM only when the up-down fluctuation of down atoms is totally frozen, i.e., when a COI-(3 × 3) phase is stabilized. However, the situation seems to be more complex here, as the experimental results show that both the three components in the Sn 4d CL and asymmetric down atoms in STM images are seen at temperatures above the electronic phase transition to the COI-(3 × 3) phase.¹⁵ There are two possible explanations. First, it could be an apparent delay due to the progressive freezing of the up-down vibration of down atoms. Second, there could be an actual decoupling of the structural and electronic phase transitions. We cannot make a definitive statement on the basis of our data, but tend to support the first possibility, which in other words assumes that the process of stabilization of the asymmetric down atoms is gradual and that an insulating COI-(3 × 3) phase is formed only when the process is completed.

The COI-(3 × 3) phase should be metallic on the basis of electron counting, but a gap opening stabilizes an insulating state. Our calculations demonstrate that electron correlation effects are behind it. Finally, all the changes observed are reversible with temperature. These results converge to support a charge reorganization in the Sn atoms within the (3 × 3) unit cell as temperature is lowered, before the final transformation into a flat Mott-$\sqrt{3}$ phase. The expected bad metallicity of this phase is in agreement with observations of Morikawa et al.²¹ In addition, and despite their interpretation, STS spectra of Colonna et al. also reflect that conductivity at $E_F$ is reduced by a factor of ~10 between 20 and 5 K.²⁰ The phase characterized in Refs. 20 and 21 is probably what we

have named COI-(3 × 3). It had been previously observed, although misinterpreted as a 2U1D-(3 × 3) structure.³¹ The 40 meV gap of the COI-(3 × 3) phase increases to 100 meV when entering in the Mott-$\sqrt{3}$ phase, also in good agreement with calculations. The identification of the COI-(3 × 3) phase and of the phase sequence leading to the Mott-$\sqrt{3}$ phase sheds light in the complex experimental phenomenology of Sn/Ge(111).

An important feature of these phase transitions is that all exhibit phase coexistence. The COI-(3 × 3) phase coexists with the metallic-(3 × 3) around 60 K and with the Mott-$\sqrt{3}$ around 20 K, as observed both by photoemission and STM. Coexistence is explained by DMFT calculations predicting that the Mott-$\sqrt{3}$ is the most stable phase, with a small energy difference of only (5 ± 7) meV below the COI-(3 × 3), which suggests phase competition. It is worth recalling that the CL deconvolution required all the components in the whole temperature range, i.e., both the $C_1$, $C_2$, and $C_3$ components associated to (3 × 3) phases and the $C_4$ component associated to a $\sqrt{3}$ Mott phase. This is due in part to phase coexistence, as shown by STM images (Fig. 7). However, two different evidences point towards important fluctuations in the system. First, a $C_4$ component (related to the Mott-$\sqrt{3}$ phase) is observed in the range of stability of the metallic-(3 × 3) phase. Second, (3 × 3) components are seen at the lowest temperature reached (12 K). While possibly the Mott transition is not complete at 12 K, which might explain in part the observation of (3 × 3) components, it is clear that the significant fraction of these unexpected components ($C_4$ at high temperatures and $C_1-C_3$ at low temperatures) does not correlate with STM images. As photoemission is much faster than STM, this suggests that the system explores dynamically different configurations.

In short, the sequence of phases [metallic-(3 × 3), COI-(3 × 3), and Mott-$\sqrt{3}$ phases] reported here ends up with the controversy concerning the ground state of α-Sn/Ge(111) and highlights the rich physical phenomenology of correlated surfaces. We obtained clear insight into the onset of Mott transitions and the delicate regime between metallic and insulating phases. This surface is a promising system for understanding collective phenomena in two dimensional layers, and in particular for the evolution from Mott insulator to 2D superconductor.

#### ACKNOWLEDGMENTS

This work was funded by MICINN (MAT2010-21156-C03-02, MAT2011-22491, and FIS2011-23230), SurMott ANR, and CNRS PICS. We are grateful to L. Patthey of the SIS beamline at the Swiss Light Source for his help during the photoemission experiments.

* Author to whom correspondence should be addressed: antonio.tejeda@synchrotron-soleil.fr

¹N. F. Mott, *Metal-Insulator Transitions* (Taylor and Francis, London, 1974).

²F. Gebhard, *The Mott Metal-Insulator Transition* (Springer, New York, 1997).

³F. Venturini, M. Opel, T. P. Devereaux, J. K. Freericks, I. Tütto, B. Revaz, E. Walker, H. Berger, L. Forró, and R. Hackl, *Phys. Rev. Lett.* **89**, 107003 (2002).

⁴L. Cano-Cortés, J. Merino, and S. Fratini, *Phys. Rev. Lett.* **105**, 036405 (2010).
⁵H.-Y. Yang, A. M. Laüchli, F. Mila, and K. P. Schmidt, Phys. Rev. Lett. **105**, 267204 (2010).

⁶P. A. Lee, N. Nagaosa, and X.-G. Wen, Rev. Mod. Phys. **78**, 17 (2006).

⁷K. M. F. Kagawa and K. Kanoda, Nature (London) **436**, 534 (2005).

⁸N. J. DiNardo, T. M. Wong, and E. W. Plummer, Phys. Rev. Lett. **65**, 2177 (1990).

⁹H. H. Weitering, X. Shi, P. D. Johnson, J. Chen, N. J. DiNardo, and K. Kempa, Phys. Rev. Lett. **78**, 1331 (1997).

¹⁰L. A. Cardenas, Y. Fagot-Revurat, L. Moreau, B. Kierren, and D. Malterre, Phys. Rev. Lett. **103**, 046804 (2009).

¹¹L. I. Johansson, F. Owman, and P. Mårtensson, Surf. Sci. **360**, L478 (1996).

¹²R. Cortés, A. Tejeda, J. Lobo, C. Didiot, B. Kierren, D. Malterre, E. G. Michel, and A. Mascaraque, Phys. Rev. Lett. **96**, 126103 (2006).

¹³J. M. Carpinelli, H. H. Weitering, M. Bartkowiak, R. Stumpf, and E. W. Plummer, Phys. Rev. Lett. **79**, 2859 (1997).

¹⁴A. Mascaraque, J. Avila, J. Alvarez, M. C. Asensio, S. Ferrer, and E. G. Michel, Phys. Rev. Lett. **82**, 2524 (1999).

¹⁵A. Tejeda, R. Cortés, J. Lobo-Checa, C. Didiot, B. Kierren, D. Malterre, E. G. Michel, and A. Mascaraque, Phys. Rev. Lett. **100**, 026103 (2008).

¹⁶S. Modesti, L. Petaccia, G. Ceballos, I. Vobornik, G. Panaccione, G. Rossi, L. Ottaviano, R. Larciprete, S. Lizzit, and A. Goldoni, Phys. Rev. Lett. **98**, 126401 (2007).

¹⁷T. Hirahara, T. Komorida, Y. Gu, F. Nakamura, H. Idzuchi, H. Morikawa, and S. Hasegawa, Phys. Rev. B **80**, 235419 (2009).

¹⁸G. Profeta and E. Tosatti, Phys. Rev. Lett. **98**, 086401 (2007).

¹⁹G. Li, P. Hönner, J. Schäfer, C. Blumenstein, S. Meyer, A. Bostwick, E. Rotenberg, R. Claessen, and W. Hanke, Nat. Commun. **4**, 1620 (2013).

²⁰S. Colonna, F. Ronci, A. Cricenti, and G. LeLay, Phys. Rev. Lett. **101**, 186102 (2008).

²¹H. Morikawa, S. Jeong, and H. W. Yeom, Phys. Rev. B **78**, 245307 (2008).

²²T. Shirasawa, H. Tochihara, K. Kubo, W. Voegeli, and T. Takahashi, Phys. Rev. B **81**, 081409(R) (2010).

²³S. Colonna, F. Ronci, A. Cricenti, and G. LeLay, Phys. Rev. Lett. **102**, 159602 (2009).

²⁴H. Morikawa and H. W. Yeom, Phys. Rev. Lett. **102**, 159601 (2009).

²⁵I. Horcas, R. Fernandez, J. M. Gomez-Rodriguez, J. Colchero, J. Gomez-Herrero, and A. M. Baro, Rev. Sci. Instrum. **78**, 013705 (2007).

²⁶A. Ino, C. Kim, M. Nakamura, T. Yoshida, T. Mizokawa, Z.-X. Shen, A. Fujimori, T. Kakeshita, H. Eisaki, and S. Uchida, Phys. Rev. B **62**, 4137 (2000).

²⁷J. Avila, A. Mascaraque, E. G. Michel, M. C. Asensio, G. LeLay, J. Ortega, R. Pérez, and F. Flores, Phys. Rev. Lett. **82**, 442 (1999).

²⁸R. I. G. Uhrberg, H. W. Zhang, and T. Balasubramanian, Phys. Rev. Lett. **85**, 1036 (2000).

²⁹R. I. G. Uhrberg and T. Balasubramanian, Phys. Rev. Lett. **81**, 2108 (1998).

³⁰A. Cano, A. P. Levanyuk, and E. G. Michel, Nanotechnology **16**, 325 (2005).

³¹A. V. Melechko, J. Braun, H. H. Weitering, and E. W. Plummer, Phys. Rev. B **61**, 2235 (2000).

³²A. Cano, A. P. Levanyuk, and E. G. Michel, Z. Kristallogr. **220**, 663 (2005).

³³R. M. Feenstra, S. Gaan, G. Meyer, and K. H. Rieder, Phys. Rev. B **71**, 125316 (2005).

³⁴D. M. Riffe and G. K. Wertheim, Phys. Rev. B **61**, 2302 (2000).

³⁵R. Pérez, J. Ortega, and F. Flores, Phys. Rev. Lett. **86**, 4891 (2001).

³⁶D. Farías, W. Kaminski, J. Lobo, J. Ortega, E. Hulpke, R. Pérez, F. Flores, and E. G. Michel, Phys. Rev. Lett. **91**, 016103 (2003).

³⁷E. Pehlke and M. Scheffler, Phys. Rev. Lett. **71**, 2338 (1993).

³⁸J. Ortega, R. Pérez, and F. Flores, J. Phys.: Condens. Matter **12**, L21 (2000).

³⁹S. de Gironcoli, S. Scandolo, G. Ballabio, G. Santoro, and E. Tosatti, Surf. Sci. **454-456**, 172 (2000).

⁴⁰O. Pulci, M. Marsili, P. Gori, M. Palummo, A. Cricenti, F. Bechstedt, and R. del Sole, Appl. Phys. A **85**, 361 (2006).

⁴¹F. Flores, J. Ortega, R. Pérez, A. Charrier, F. Thibaudau, J.-M. Deveber, and J.-M. Themlin, Prog. Surf. Sci. **67**, 299 (2001).

⁴²X. Y. Zhang, M. J. Rozenberg, and G. Kotliar, Phys. Rev. Lett. **70**, 1666 (1993).

⁴³P. Hohenberg and W. Kohn, Phys. Rev. **136**, B864 (1964).

⁴⁴L. Fritsche, Phys. Rev. B **33**, 3976 (1986).

⁴⁵Note that the quoted value is slightly larger than the value determined from Fermi surface analysis in Ref. 46, which is attributed to the different method used.

⁴⁶A. Tejeda, R. Cortés, J. Lobo, E. G. Michel, and A. Mascaraque, J. Phys.: Condens. Matter **19**, 355008 (2007).

⁴⁷A. Georges, G. Kotliar, W. Krauth, and M. J. Rozenberg, Rev. Mod. Phys. **68**, 13 (1996).

⁴⁸Upon cooling at 77 K, local COI-(3 × 3) regions are observed (above the transition temperature determined from an averaging technique like photoemission). STS from these areas is shown as it exhibits better defined features than at lower temperatures.