# In-situ observations of misfit dislocation motion in AlSb/GaSb dislocation filter layer structures

Karl Graser, Audrey Gilbert, Jean-Baptiste Rodriguez, Eric Tournié, Achim Trampert

Trampert

► To cite this version:

Karl Graser, Audrey Gilbert, Jean-Baptiste Rodriguez, Eric Tournié, Achim Trampert. In-situ observations of misfit dislocation motion in AlSb/GaSb dislocation filter layer structures. Acta Materialia, 2026, 326, pp.121946. 10.1016/j.actamat.2026.121946. hal-05470979

HAL Id: hal-05470979

https://hal.science/hal-05470979v1

Submitted on 22 Jan 2026

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

# *In-situ* observations of misfit dislocation motion in AlSb/GaSb dislocation filter layer structures

Karl Graser<sup>a, *, ID</sup>, Audrey Gilbert<sup>b, ID</sup>, Jean-Baptiste Rodriguez<sup>b</sup>, Eric Tournié<sup>b, ID</sup>, Achim Trampert<sup>a, ID</sup>

## ARTICLE INFO

### Keywords:

In situ transmission electron microscopy

Epitaxial growth

Semiconductors

Dislocation structures

Dislocation dynamics

## ABSTRACT

The dislocation dynamics in epitaxially strained GaSb/AlSb/GaSb heterostructures grown on a Si(001) substrate via molecular beam epitaxy have been studied by means of scanning transmission electron microscopy. The 100 nm-thick AlSb layer – intentionally inserted as dislocation filter – is partially relaxed by bending pre-existing threading dislocations of the GaSb buffer into the interface with AlSb, creating a complex network of misfit dislocations. In addition to the expected misfit dislocation lines along the orthogonal (110) directions, irregular curved lines are also detected. Plan-view lamellae of the as-grown heterostructure were annealed in-situ in the microscope at temperatures well above the growth temperature to observe the propagation and interaction of the dislocations in real time. It was found that the (thermally activated) motion of misfit dislocations occurs independently of their Burgers vectors within the interface plane, which is not one of the primary slip systems of zinc-blende crystal structure, and without affecting the threading dislocation configuration in the filter layer. This unusual dislocation motion at this later stage of plastic relaxation (already 50% of lattice mismatch is relieved in the as-grown heterostructure AlSb layer) can only be explained by a non-conservative dislocation climb process on the one hand, and a vacancy-assisted slip on the other hand as it was recently discovered using molecular dynamics modeling. The results provide deeper insights into the thermal activation of dislocations in epitaxially strained systems and reveal potential for re-examining the conditions of those epi-layers as dislocation filters.

## 1. Introduction

The epitaxial integration of III-V semiconductors onto silicon offers a pathway toward a broad range of CMOS-compatible photonic devices [1–3]. These devices provide an energy-efficient approach to increasing the speed and capacity of data transfer and storage, a crucial characteristic in today's world, and also facilitate the realization of novel functionalities and applications such as integrated sensors. Particularly III-Sb materials cover the technological important mid-infrared wavelength range. However, a remaining challenge of epitaxial growth of III-Sb on Si(001) is the high number of threading dislocations (TDs) arising from the large lattice mismatch between the materials - a common finding that also applies to other epitaxial material systems, such as nitrides or oxides (see e.g. [4,5]). The high TD densities lead to a strongly reduced performance and lifetime of photonic devices [6–8]. Therefore, the development of efficient methods for TD reduction is an important part of current research [9].

There are several approaches to TD minimization in high-mismatch epitaxy, targeting different stages of the growth process. For example, optimizing the interfacial MD array can reduce the number of nucleating TDs [10,11], while other methods such as thick buffer layers or complex short-period superlattice structures aim to reduce the number of remaining TDs at later stages of growth [12,13]. In this context also, thin lattice-mismatched dislocation filter layers represent an easy-to-implement and material-efficient approach for reducing TD densities in various epitaxial III-V material systems, as demonstrated e.g. for GaSb-on-Si, and GaSb-on-GaAs growth [14–16]. Here, the mechanism for reducing TDs is based on the annihilation of TDs, which is made possible by dislocation propagation due to the stress field of the lattice-mismatched layer. This propagation of TD arms is coupled to the formation of mismatch dislocations (MDs) in the interfaces of the filter layer, which reduce the epitaxial strain [17,18]. Furthermore, a combination of dislocation filter layers with post growth annealing and thermal cycle annealing (TCA) during the growth stage enables a further

<footnote><sup>a</sup> Paul-Drude-Institut für Festkörperelektronik, Leibniz-Institut im Forschungsverbund Berlin e.V., Hausvogteiplatz 5-7, 10117, Berlin, Germany</footnote>

<footnote><sup>b</sup> IES, University of Montpellier, CNRS, F34000, Montpellier, France</footnote>

<footnote>* Corresponding author.
E-mail address: graser@pdi-berlin.de (K. Graser).</footnote>
reduction of TD density in many cases [19–21]. The thermal treatment enhances dislocation mobility and can be the origin of additional stress between materials with different coefficients of thermal expansion.

In our recent studies we found AlSb dislocation filter layers to be particularly effective for GaSb-on-Si epitaxy [22,23]. Our studies demonstrate a clear correlation between TD reduction and epitaxial strain relief, which is evidenced by the measured evolution of the MD network. Furthermore, at a later stage of strain relaxation, a hardening effect becomes apparent, which impedes dislocation movement and thus reduces the dislocation annihilation efficiency. It implies that the nature of the MD network including dislocation pinning/blocking and dislocation reaction influences the plastic relaxation process being responsible for TD reduction. This experimental observation has so far been little considered in research on dislocation filter layers and their design optimization. It is remarkable that this coupling between TD reduction and plastic relaxation is inconsistent with our observation that TCA causes significant changes in the MD networks in the interface between AlSb layer and the underlying GaSb buffer resulting in complex arrangement with a high number of dislocation reactions and MD lines deviating from the common (110) directions, and simultaneously, revealing no definite TD reduction effect associated with the annealing [22].

Beanland et al. [24] observed the formation of a similar complex MD network after post growth sample annealing in the (InGa)As/GaAs(001) system. They attributed the formation of the MD network to a rotation of MDs with Burgers vector $\mathbf{b} = a/2 \langle 101 \rangle$ exhibiting a component outside the (001) interface plane by dislocation climb during the annealing, leading to more efficient plastic relaxation. But, although a cross network of edge-type $\mathbf{b} = a/2 \langle 110 \rangle$ dislocations with their Burgers vectors inside the (001) plane would be most efficient for strain relaxation, the formation of such a MD network during the annealing was not observed. The authors concluded that because of the random distribution of MDs during plastic relaxation, high energy would be required to redistribute the MDs in the interface to form this cross array. In the AlSb/GaSb interface, rotated mixed-type MDs and occasional reaction to edge-type MDs can be observed already in the as-grown, i.e. not annealed, sample [25]. In contrast to the (InGa)As/GaAs system, TCA during AlSb/GaSb sample growth even leads to the local formation of edge-type MD arrays [22]. Overall, these previous results highlight the need to clarify the mechanisms activated by the thermal annealing of the MD network and to investigate why no further TD reduction was observed in some systems as in the present case of group III-antimonides.

In this study, GaSb/AlSb/GaSb dislocation filter layer structures are investigated by in-situ heating experiments within a scanning transmission electron microscope (STEM). The temperature-induced changes

in the dislocation structures are directly observed in real time and will help to explain the formation processes of the uncommon interfacial MD network. The observed MD propagation mechanisms are analyzed in detail and may contribute to discuss their effects on the TD reduction process in the filter layer system.

## 2. Experimental

### 2.1. Molecular beam epitaxy

A detailed description of the growth process of the GaSb/AlSb/GaSb dislocation filter layer structures grown by solid-source molecular beam epitaxy can be found in [23]. In this study, two specific samples are examined. Both exhibit the same layer structure depicted in Fig. 1(a). They consist of a 1 µm-thick GaSb buffer layer grown on Si(001) substrates with 0.5° cut-off along the [110] direction, followed by the deposition of a 100 nm AlSb filter layer and a 20 nm GaSb cap layer. While the GaSb has a large lattice mismatch of >12 % to Si, the lattice constant difference of 0.65 % between AlSb and GaSb is much smaller. All layers were grown at 550 °C growth temperature (thermocouple). The deposition of the GaSb cap layer is needed to prevent oxidation of the AlSb layer. A 20 nm GaSb cap layer was found to uniformly cover the AlSb while being sufficiently thin to remain coherently strained, preventing the formation of a MD network between the cap and the AlSb layer. For one of the two samples, a three-cycle thermal annealing was performed immediately after the growth of the AlSb layer. Each cycle consisted of heating the sample to 750 °C, maintaining this temperature for 5 min, followed by cooling down to 100 °C, where the temperature was again held constant for 5 min. Following the last cool-down step, the temperature was increased back to 550 °C to resume the growth. On the other hand, no annealing at the growth stage was applied for the second sample.

### 2.2. Sample preparation for in-situ heating study

As indicated by the red marked area in Fig. 1(a), plan-view specimens of 200 nm to 300 nm thickness were prepared from the near-surface region to image both, the MD network in the AlSb/GaSb interface and the TDs in the layer above. The various preparation steps for STEM in-situ heating sample are shown in Fig. 1(b) – (d). The specimens were first thinned in a conventional way including mechanical grinding, dimpling and Ar-ion sputtering to achieve electron transparency for STEM observations. Subsequently, they were coated with a thin carbon layer from both sides in a carbon coater (Quorum Technologies CC7656) to protect the GaSb-surfaces of the plan-view specimens. Next, a suitable

<figure><img src="image_3.png" /><figcaption>**Fig. 1.** Sample structure and sample preparation process. (a): Schematic of the layer structure. (b): STEM image inside the FIB showing the selected area of the conventional prepared plan-view specimen. (c) and (d): SEM images showing the lift-off of a lamella and its transfer to the heating chip.</figcaption></figure>
area of interest was selected by using the build-in STEM detector of a
Thermo Fisher Helios 5 UX focused ion beam system (FIB) (b). Finally, a
lamella was cut out of the selected area by the FIB (c) and transferred to
the heating chip by using a nano-manipulator (d). During the FIB cut out
it was ensured that only the sample edges were exposed to the Ga ion
beam, so that no additional Ga is incorporated in the center area.

## 2.3. In-situ heating experiment

In-situ heating studies were conducted using a JEOL 2100F field-emission STEM equipped with a DENS solutions Lightning D9+ heating holder and Wildfire through hole heating chips. Predefined temperature profiles were applied. Starting from room temperature the sample was heated to the targeted annealing temperatures of 580 °C and 590 °C, respectively, within 1 min and then held at that temperature for times up to 7 min before cooled down again to room temperature within 30 s. During the heating experiment the Frame Grabber script by D. R. G. Mitchell for Gatan Digital Micrograph was used to continuously capture STEM images. Scanning times between 4 and 7 s per frame were chosen to ensure sufficient image contrast while still allowing the observation of dislocation movements, which were found to happen at time scales of several seconds. Video files have been produced out of the image series by using the software ImageJ [26].

In initial in-situ heating experiments the sample was slowly heated up to the nominal growth temperature of 550 °C. However, no significant dislocation propagation was observed for these temperatures, even for heating times of up to 10 min. Only very rarely observed dislocation movements were limited to TD segments very close to the specimen surface and did not affect the MD networks. Above 550 °C, continuous MD motion and reactions occurred throughout the MD network at the AlSb/GaSb interface. For heating temperatures above 600 °C, however, despite the carbon coating of the specimen, strong sample deterioration was observed, including the formation of Ga droplets and holes at the specimen surface. Therefore, the temperature range between 560 °C and 600 °C was identified as optimal for observing temperature-induced effects on the dislocation structures.

# 3. Results and discussion

## 3.1. Dislocation contrast in STEM

A typical (001) zone-axis plan-view bright-field STEM image of the annealed sample is shown in Fig. 2(a) to clarify the dislocation contrast. The STEM micrograph reveals the MD network in the AlSb/GaSb interface. Two main types of MDs can be distinguished based on their differing contrast features in STEM. One type of MD line exhibits a single black contrast, while the other shows a black-white-black contrast in the STEM images. Intensity line scans perpendicular to the MD lines confirm the obviously different contrast of the two MD types. Fig. 2(b) and (c) show plan-view STEM images of the same sample area but tilted in different diffraction conditions. The sample is tilted about 10° along the

[110] and [−110] axis, respectively, corresponding to $g = (220)$ and $g = (−220)$ two-beam conditions in conventional TEM, respectively. In these conditions, the STEM diffraction contrast is dominated by the diffraction on the (220) and (−220) atomic planes. The MD lines with the black-white-black contrast vanish in (b) and (c) if the g-vector is orientated along the MD line direction. The same dislocation contrast observation was verified by conventional two-beam and weak-beam TEM. According to the $g \bullet b$ invisibility criterion these MDs are 90°-type dislocations with a Burgers vector $b = a/2 \langle 110 \rangle$ inside the interface and perpendicular to their line direction. In contrast the MD lines with the single black contrast forming a zig-zag structure are visible in all different tilt conditions. They can be attributed to dislocations with a Burgers vector $b = a/2 \langle 101 \rangle$ exhibiting a component out of the interface plane. These misfit segments form during the plastic relaxation process by glide of pre-existing TDs on their {111}-glide planes. They typically form along the $\langle 110 \rangle$ intersections of the {111} glide planes with the (001) interface and are therefore referred to as 60°-type MDs (60° angle between Burgers vector and line direction). Two matching 60°-type dislocations can react to form one 90°-type MD. As visible in the STEM micrographs, in many cases both the 60°-type and 90°-type MD segments are not perfectly orientated along the $\langle 110 \rangle$ directions. Therefore, the terms '90°-type' and '60°-type' MDs, defining the angle between Burgers vector and line direction, are not strictly accurate. Nevertheless, this terminology is used in this paper to distinguish between these two different types of dislocations, which are characterized by their Burgers vector either located in the interface plane (a/2 <110>) or inclined towards it (a/2 <101>).

## 3.2. Effect of TCA during sample growth

Zone-axis plan-view images of both the unannealed and the annealed sample are compared in Fig. 3(a) and (b) respectively. The contrast of 60°-type and 90°-type MDs can be clearly distinguished as highlighted in the two magnified sections on the right of each micrograph.

A strong effect of TCA on the MD network can be observed. Both networks shown in Fig. 3 contain a mixture between 60°-type and 90°-type MDs. However, the proportion of 90°-MDs is small in the not annealed sample and long segments of 60° MDs along the orthogonal [110] and [1-10] directions can be observed. In a few parts of the network the 60°-type MDs react to form 90°-type MDs. The interaction of the two types of MDs results in dislocation segments also in other directions than the typically observed (110) MD line directions [25,27]. At this later stage of plastic relaxation, where about 50 % of the lattice mismatch is relieved in the AlSb, the plastic relaxation process is mainly determined by dislocation interactions. On the other hand, in the annealed sample the proportion of 90° MDs is strongly increased and the network is narrower. This goes along with an increased degree of plastic relaxation of the AlSb layer and a different arrangement of the MD network. It exhibits a mixture between cross-networks of 90°-type MDs and parts with hexagonally shaped MD structures resulting from the interaction between crossing 90°-type and 60°-type MDs [25,28]. Such a

<figure><img src="image_4.png" /><figcaption>**Fig. 2.** Plan-view STEM images of the TCA sample in different tilt conditions. (a) (001) zone axis orientation, the white arrows indicate the two orthogonal (110) directions. Two intensity line scans demonstrate different STEM contrasts of two types of MDs. (b) and (c) tilted about 10° away from zone axis in perpendicular (110) directions. The direction of the diffraction vector *g* is indicated by arrows.</figcaption></figure>
<figure><img src="image_5.png" /><figcaption>**Fig. 3.** STEM images of about 200 nm - 300 nm thick plan view specimen prepared from the surface of a sample without TCA (a) and with TCA after the AlSb growth step (b). Enlarged areas of the visible MD networks are shown on the right of the two micrographs including an indication of the different STEM contrast of 90°-type and 60°-type MDs in (a).</figcaption></figure>

MD network is highly uncommon for the (001) epitaxy of cubic zinc-blende materials and cannot form by simple dislocation glide processes on the {111} planes.

On the other hand, our previous studies on TCA of AlSb dislocation filter layer structures have shown, that the annealing has no significant effect on the TD density at sample surface [22]. On the first glance this appears to be in conflict with the concept of the TD reduction process in such filter layer structures, which relies on the coupling between MD formation and a movement of TD arms that results in TD annihilations. In the following the effect of high-temperature annealing on the rearrangement of the MD network in the AlSb/GaSb interface by in-situ STEM will be investigated to better understand the formation process of the uncommon complex MD network and the coupling/non-coupling to TD arm propagation.

### 3.3. *In-situ* heating observations

Fig. 4 displays a series of zone-axis bright-field STEM micrographs taken at a time interval of 360 s during the in-situ experiment performed at 590 °C. The series of images is part of the video that can be seen as Video_1 in the supplement. The snapshots are excerpts from a larger recorded area. The figures show a region of the MD network consisting of 90°-type and 60°-type MDs with an irregular arrangement. A

schematic depiction is shown in (b), indicating 90°-type and 60°-type segments as blue and orange lines. The MD structure can be described by 60°-type MDs with two different Burgers vectors $b_1$ and $b_2$ that combine to 90°-type MDs at different positions of the structure (e.g. by the possible reaction $a/2 [101] + a/2 [01\bar{1}-1] = a/2 [110]$). The successive individual images illustrate how a 60°-type MD loop formed by these reactions propagates toward the long 90°-type dislocation at the left and ultimately react with it, thereby extending the line segment of the 90°-type dislocation. This 90°-type MD does not run exactly along the ⟨110⟩ direction, nevertheless, the dislocation movements and reactions lead to more efficient epitaxial strain relief and lower total line energy

Since there is no threading dislocation involved, all dislocation motion takes place in the (001) interface. Because this plane does not form a slip plane for 60°-type dislocations, the MD motion cannot be explained by conventional dislocation glide. It must be assumed that climbing processes occur, enabling a movement of the MDs out of their slip plane [29,30]. Climb is a non-conservative motion that involves the creation and migration of point defects towards the jog, a short section of the dislocation line that lies out of the slip plane [31]. This climb process of MDs in the (001) interface has been observed in various semiconductor heterostructures both during non-equilibrium growth (e.g. [32]) and due to thermal treatment (e.g. [33]). In principle, the mobility of dislocations by glide and by climb under stress at a given temperature can

<figure><img src="image_6.png" /><figcaption>**Fig. 4.** (a) STEM image sequence showing the elongation of a 90° MD segment during an in-situ annealing at 590 °C. (b) a depiction of the MD structure and its evolution during the heating is shown together with the Burgers vectors of the various segments.</figcaption></figure>
be formulated in an analogous manner as [34,35]:

$$ \eta_g = C_g \exp\left(-\frac{E_g}{kT}\right) \quad \text{and} \quad \eta_c = C_c \exp\left(-\frac{E_c}{kT}\right), \qquad (1) $$

where k and T are the Boltzmann constant and the temperature, respectively, and C_g and C_c are kinetic pre-factors determining the relative importance of glide and climb processes at the given temperature. E_g and E_c are activation energies for dislocation glide and climb, respectively, material parameters, which are not well known for the GaSb-AlSb material system. The condition governing the nature of the dislocation propagation depends on the comparison of the magnitudes of η_g and η_c. Generally, if η_g > η_c, dislocation glide will be preferred, as it is the case at lower temperatures. At high temperatures, C_c will increase significantly compared to C_g, as the density of the mobile point defects that control climbing will increase considerably. In fact, the actual in-situ observation temperature is about 80 % of the melting temperature of GaSb (the melting temperature T_m of GaSb is 712 °C, the lowest of all group-III-Vs), so we assume that there are sufficient point defects for this type of MD motion and dislocation climb prevails over slip movements. It can be further assumed that during MBE growth, point defects are incorporated in addition to the thermodynamic equilibrium and that one type of point defect (either Ga or Sb) is adequate to activate dislocation climb in III-V compounds [36].

Fig. 5 presents another example of the unusual lateral movement of MDs in the AlSb/GaSb interface. The bright-field STEM images in the figure also stems from the video but depicts a different interface area. In the center of this image series, the movement of a long, slightly curved 90°-type MD can be unambiguously observed based on its relative position to a reference line defined by a fixed surface contamination point (marked by an orange tip). Although this type of misfit dislocation with Burgers vector of a/2 <110> or a/2 <1-1 0> is often described as “sessile”, the lateral movement along the (001) interface is clearly demonstrated in the image sequence. Within the time interval of 405 s, a continuous back and forth movement of the MD can be observed (see Video_1) resulting in a total movement of the upper part of the MD to the right by about 80 nm. As the (001) glide plane does not belong to the primary slip systems in zinc-blende structures, the glide of these 90°-type MD should be significantly more difficult compared to regular 60°-type dislocation gliding in {111} planes and should only be activated at higher stresses – in contrast to the present case of low lattice misfit. Very recently, L. Barbisan et al. used molecular dynamics simulations on dislocation structures and movements in the Ge/Si(001) interface in order to demonstrate that 90°-type MDs are indeed capable of gliding in (001) planes [37]. Their simulations revealed that the presence of even a single vacancy is sufficient to greatly accelerate the (001) gliding process and thereby achieve homogenization and thus higher ordering of the 90°-type MD network. This process is driven solely by the tendency to equalize MD spacings, rather than by the initial epitaxial mismatch in the Ge/Si system. This case of vacancy-assisted gliding may can also be transferred to the present AlSb/GaSb interface with its 90°-type MDs. As

already seen from the observed climb movements of 60°-type dislocations, at the given growth temperatures, there are sufficient vacancy sites in both group III and group V sublattices, so that the process can proceed even at low stress levels. Also the fact that the onset of 90°-type MD movement was observed at a well-defined temperature across all specimen areas, independent of the local network density suggests that these propagations are a thermally activated effect, that could be directly coupled to point defect mobility. Video_1 shows that, in many cases, MD propagation is not strictly directional. Instead, it is accompanied by non-periodic oscillatory motions of individual MD segments, which appear to occur only for 90°-type MDs and are more pronounced for longer segment lengths. Some 60°-type MDs follow these oscillatory motions, but only when they are connected by a reaction point to 90°-type MDs. This highlights the fundamentally different motion mechanisms of the two MD types. Although both appear to be activated by point defects, the 90°-type motion is glide-based, whereas the 60°-type motion occurs by climb. Importantly, these back-and-forth motions of 90°-type MDs are neither periodic in time nor correlated with the STEM acquisition rate. The MD positions can remain stable over several STEM frames before an abrupt forward or backward movement of the MDs occurs. These non-directional movements may be attributed to the response of the dislocations to the changing interaction forces with neighboring MDs in the complex rearranging network.

In the image sequence in Fig. 5 another interesting dislocation motion and reaction can be observed on the left side: the joint propagation of 60°-type and 90°-type MDs inside the interface by climb and vacancy-assisted glide, respectively, which finally results in hexagonal MD structures similar to those observed for the TCA sample (cf. Fig. 3(b)). As demonstrated by Beanland et al., such hexagonal shaped MD networks with 60°-type MDs orientated close to the (100) directions represent a local energy minimum of the network [24]. The observations clearly demonstrate that the formation of these structures results from reactions between 60°-type and 90°-type MDs, which are facilitated by the annealing.

Fig. 6(a) shows a typical cross-sectional TEM image of the TCA sample. Three MDs are observed end-on, all positioned at the AlSb/GaSb interface. The dislocation core of the MDs tends to be located slightly below the interface within the GaSb, which is most clearly visible for the rightmost MD in the figure. This is further illustrated in Fig. 6(b), which shows an atomically resolved high-angle annular dark-field image of an MD dislocation core, artificially colored using a temperature color code. The AlSb and GaSb can be clearly distinguished by the contrast difference in the atomic dumbbells of the two materials. Due to the higher atomic mass of the Ga compared to the Al, the dumbbells in the GaSb region appear more pronounced. A Burgers circuit analysis, indicated by dotted white lines, confirms the 90° character of the MD. The dislocation core, highlighted by the two ending half planes (see black dashed lines), is positioned closely below the interface. This shows that the observed MD movements during heating can indeed be attributed to dislocation propagation within the (001) plane, predominantly occurring in the GaSb layer, which has the lower melting point compared to the AlSb and

<figure><img src="image_7.png" /><figcaption>Fig. 5. STEM image sequence captured during the heating to 590 °C. Orange line and orange arrow mark stationary positions in the image sequence to visualize the movement of a long 90°-type MD segment.</figcaption></figure>
<figure><img src="image_8.png" /><figcaption>Fig. 6. (a) Cross-sectional TEM image showing MDs in the AlSb/GaSb interface of the TCA sample. (b) High-angle annular dark-field STEM micrograph of the core of a 90° MD in the AlSb interface. The image is artificially colored in a temperature color scale to visualize the transition between GaSb and AlSb. A Burgers circuit around the dislocation core is shown in white.</figcaption></figure>

therefore provides a higher mobility of point defects.

Fig. 7 shows the results of a further in-situ heating experiment for a lamella annealed at 580 °C. Here, the speckle contrast due to the surface contamination is pronounced, but the dislocation lines are still observable. The image sequence in (a) proves the lateral rotation of a straight 90°-type MD toward an orientation of the dislocation line along the crystallographic (110) direction, and, as a result, toward the formation of a higher order of the dislocation network. The dislocation is initially located between two dislocation nodes highlighted by yellow arrows. At the beginning of the motion, the dislocation detaches from the lower node (marked blue in Fig. 7) and propagates leftwards along the bordering 60° MD within 20 s to its final position.

The dislocation movement with the heating time is plotted in Fig. 7 (b). The distance of the TD from its starting position $s_{start}$ at time $t_0$, where the MD detached from the lower pinning point was measured in every frame of the recorded STEM image sequence. Initially, when the dislocation detaches from the pinning point, it moves 15 nm/s. This velocity decreases as the segment approaches alignment along the (110) direction until it finally enters a quasi-static state.

### 3.4. Discussion

The results presented here reveal that during in-situ heating of the partially relaxed 100 nm-thick AlSb filter layer on GaSb-buffer, the motion of dislocations is dominated by the propagation and rearrangement of MDs inside the (001) interface. This is surprising insofar as the (001) crystal plane is not one of the primary slip systems in zinc-blende lattices and no shear stress acts on it due to the epitaxial lattice mismatch. The observed dislocation propagation must thus be explained by thermally activated non-conservative climb processes of the 60°-type MD (Burgers vector inclined to interface) and by vacancy-induced glide of the 90° type MD (Burgers vector parallel to interface). The driving force behind these propagations is the formation of energetically more efficient MD configurations accompanied by local changes of the strain

field close to the interface. These results therefore provide important insights into MD propagation. They demonstrate that MD networks can substantially rearrange under suitable conditions, challenging the common assumption that 90° interfacial misfit arrays are sessile and fixed. The ability for rearrangement of the MD networks may strongly depend on the initial density of the network and the magnitude of the oscillatory movement that was observed for 90°-type MDs.

The total epitaxial compressive stress of the AlSb layer compared to the bulk-like GaSb buffer appears to play a minor role in the present case of in-situ plan-view observations. Due to the removal of the Si substrate and most of the GaSb buffer, it can no longer be assumed that the substrate is mechanically rigid, and the stress conditions between the AlSb layer and the thin GaSb residual layer have changed drastically, i.e., the original residual stress in the AlSb layer can now be partially transferred to the thin GaSb and the shear forces acting on dislocation due to the epitaxial strain are drastically reduced. In general, due to the many factors influencing dislocation mobility, the in-situ heating experiment is difficult to compare directly with the TCA during growth. In addition to differences in strain state and applied temperature, this also includes variations in the density and dynamics of point defects, which may be changed by the specimen geometry and the sample preparation process.

Nevertheless, the similarity of the MD configurations caused by in-situ annealing to those of the samples that experienced TCA during growth indicates that, also in the case of TCA, the here observed propagations and interactions of MDs within the interface play a crucial role in explaining the resulting MD networks, independent from the exact point defect dynamics that may differ between the two experiments.

The observed rearrangement of MDs is not accompanied by TD propagation. Together with the previous finding that TCA changes the MD network at the AlSb/GaSb interface but does not affect the TD density at the sample surface, this leads to the conclusion that, in the GaSb/AlSb/GaSb system, once a certain degree of plastic relaxation is reached, and particularly under thermal annealing, no additional TD mobility occurs. Consequently, rearrangements of the MD network are

<figure><img src="image_9.png" /><figcaption>Fig. 7. (a) STEM image sequence of a moving 90°-MD segment for in-situ annealing at 580 °C. (b) The plot shows a measurement of the movement of the lower part of segment with heating time.</figcaption></figure>
no longer linked to a reduction in TD density, which is based on TD arm motion and subsequent annihilation, as it was observed for plastic relaxation at smaller layer thicknesses of the GaSb/AlSbGaSb system. This is an important finding for understanding the coupling between MD formation and TD reduction in filter layers, and thus the efficiency of the filtering process at higher temperatures and at different stages of plastic relaxation. Since similar complex networks have also been observed in other material systems, these processes do not seem to be limited to AlSb filter layers. We believe that MD climb or defect-assisted glide processes predominantly occur at later stages of plastic relaxation, where dense MD networks are already present at the interface and TD arms become pinned or blocked within these networks. If additionally sufficient point-defect mobility is available, the MD networks can reorganize without requiring motion of the immobilized TDs. Therefore, such regimes should be avoided for effective TD filtering. This may lead to the conclusion that exploiting only the early stages of plastic relaxation, when the MD formation is still nucleation driven, is more beneficial than growing thick layers to achieve high degrees of plastic relaxation.

## 4. Conclusions

Real-time in-situ heating STEM observations of misfit dislocation dynamics in the AlSb/GaSb interface were performed to gain deeper insights into the processes occurring during growth stage thermal cycle annealing in GaSb/AlSb/GaSb dislocation filter layer structures. The increased dislocation mobility at temperatures above the growth temperature enabled a variety of MD movements and dislocation reactions, which depend on the initial relaxation stage in the 100 nm-thick AlSb layer prior to in-situ heating. These processes led to the formation of MD structures similar to those observed in samples subjected to TCA during growth. The in-situ heating investigations revealed that the changes in the MD network involve dislocation climb of 60°-type MDs and an uncommon propagation of 90°-type MD segments within the interface resulting in a lower-energy configuration. This propagation could be attributed to a point defect assisted glide of the 90°-type MDs, that was proposed earlier by molecular dynamic simulations. It allows for extensive redistribution of MDs in the interfacial network. These results demonstrate that, particularly during thermal annealing, rearrangement of MD networks can occur without the involvement of TD motion. In such cases, local changes in plastic relaxation are no longer coupled to increased TD mobility and a reduction in TD density.

## Acknowledgments

We thank M. Hanke and P. John for critical reading of the manuscript and valuable comments and C. Matzeck for TEM sample preparation. This work was partially supported by the France 2030 program (EquipEx EXTRA, ANR-11-EQPX-0016, and EquipEx+ HYBAT, ANR-21-ESRE-0026) and by the Deutsche Forschungsgemeinschaft and the French ANR within the DFG-ANR Project FILTER (ID No 446,180,337/ANR-20CE92-0045). The authors acknowledge the financial support provided by the European COST Action OPERA (CA-20,116) through the short-term scientific mission program and the use of the facilities of the Application Laboratory Electron Tomography (BALET) of the Paul-Drude-Institute (ERDF project No 2016,011,843)

## CRediT authorship contribution statement

**Karl Graser:** Writing – original draft, Visualization, Methodology, Investigation, Formal analysis, Conceptualization. **Audrey Gilbert:** Writing – review & editing, Resources, Investigation, Conceptualization. **Jean-Baptiste Rodriguez:** Writing – review & editing, Supervision, Project administration, Funding acquisition. **Eric Tournié:** Writing – review & editing, Supervision, Project administration, Funding acquisition. **Achim Trampert:** Writing – review & editing, Supervision, Project administration, Funding acquisition, Conceptualization.

## Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

## Supplementary materials

Supplementary material associated with this article can be found, in the online version, at doi:10.1016/j.actamat.2026.121946.

## References

[1] R. Soref, Mid-infrared photonics in silicon and germanium, Nat. Photonics, 4 (2010) 495-497.

[2] D.A. Miller, Device requirements for optical interconnects to silicon chips, Proc. IEEE 97 (2009) 1166-1185.

[3] E. Tournie, Molecular-beam epitaxy of antimonides for optoelectronic devices, Mol. Beam Epitaxy Mater. Appl. Electron. Optoelectron. (2019) 233-246.

[4] H. Takane, S. Konishi, Y. Hayasaka, R. Ota, T. Wakamatsu, Y. Isobe, K. Kaneko, K. Tanaka, Structural characterization of threading dislocation in α-Ga2O3 thin films on c-and m-plane sapphire substrates, J. Appl. Phys. 136 (2024).

[5] M. Moram, C. Ghedia, D. Rao, J. Barnard, Y. Zhang, M. Kappers, C. Humphreys, On the origin of threading dislocations in GaN films, J. Appl. Phys. 106 (2009).

[6] S. Mahajan, Defects in semiconductors and their effects on devices, Acta Mater. 48 (2000) 137-149.

[7] P. Hutchinson, P. Dobson, Defect structure of degraded GaAlAs-GaAs double heterojunction lasers, Philos. Mag. 32 (1975) 745-754.

[8] A. Remis, L. Monge-Bartolomé, G. Boissier, M. Waguaf, J.-B. Rodriguez, L. Cerutti, E. Tournié, Effect of dislocations on the performance of GaSb-based diode lasers grown on silicon, J. Appl. Phys. 133 (2023) 093103.

[9] B. Kunert, Y. Mols, M. Baryshniskova, N. Waldron, A. Schulze, R. Langer, How to control defect formation in monolithic III/V hetero-epitaxy on (100) Si? A critical review on current approaches, Semicond. Sci. Technol. 33 (2018) 093002.

[10] S. Huang, G. Balakrishnan, A. Khoshakhlagh, L. Dawson, D. Huffaker, Simultaneous interfacial misfit array formation and antiphase domain suppression on miscut silicon substrate, Appl. Phys. Lett. 93 (2008).

[11] S. Huang, G. Balakrishnan, D.L. Huffaker, Interfacial misfit array formation for GaSb growth on GaAs, J. Appl. Phys. 105 (2009) 103104.

[12] R. Hao, S. Deng, L. Shen, P. Yang, J. Tu, H. Liao, Y. Xu, Z. Niu, Molecular beam epitaxy of GaSb on GaAs substrates with AlSb/GaSb compound buffer layers, Thin. Solid. Films. 519 (2010) 228-230.

[13] D. Han, W.-Q. Wei, M. Ming, Z. Wang, T. Wang, J.-J. Zhang, Epitaxial GaSb films directly grown on on-axis Si (001) with low defect density by MBE, Appl. Phys. Lett. 122 (2023).

[14] I. George, F. Becagli, H. Liu, J. Wu, M. Tang, R. Beanland, Dislocation filters in GaAs on Si, Semicond. Sci. Technol. 30 (2015) 114004.

[15] T. Ward, A.M. Sánchez, M. Tang, J. Wu, H. Liu, D.J. Dunstan, R. Beanland, Design rules for dislocation filters, J. Appl. Phys. 116 (2014) 063508.

[16] A. Mansoori, S. Addamane, E. Renteria, D. Shima, M. Behzadirad, E. Vadiee, C. Honsberg, G. Balakrishnan, Reducing threading dislocation density in GaSb photovoltaic devices on GaAs by using AlSb dislocation filtering layers, Sol. Energy Mater. Sol. Cells 185 (2018) 21-27.

[17] J. Matthews, A. Blakeslee, S. Mader, Use of misfit strain to remove dislocations from epitaxial thin films, Thin. Solid. Films. 33 (1976) 253-266.

[18] A. Romanov, W. Pompe, S. Mathis, G. Beltz, J. Speck, Threading dislocation reduction in strained layers, J. Appl. Phys. 85 (1999) 182-192.

[19] W. Li, S. Chen, M. Tang, J. Wu, R. Hogg, A. Seeds, H. Liu, J. Ross, Effect of rapid thermal annealing on threading dislocation density in III-V epilayers monolithically grown on silicon, J. Appl. Phys. 123 (2018) 215303.

[20] Y. Takano, M. Hisaka, N. Fujii, K. Suzuki, K. Kuwahara, S. Fuke, Reduction of threading dislocations by InGaAs interlayer in GaAs layers grown on Si substrates, Appl. Phys. Lett. 73 (1998) 2917-2919.

[21] B. Ar papay, Y.E. Suyolcu, G. Çorapçıoğlu, P.A. van Aken, M.A. Gülgün, U. Serincan, A comparative study on GaSb epilayers grown on nominal and vicinal Si (100) substrates by molecular beam epitaxy, Semicond. Sci. Technol. 36 (2020) 025011.

[22] A. Gilbert, K. Graser, M. Ramonda, A. Trampert, J.-B. Rodriguez, E. Tournié, Reduction of the threading dislocation density in GaSb layers grown on Si (001) by molecular beam epitaxy, Adv. Phys. Res. (2024) 2400126.

[23] K. Graser, A. Gilbert, J.-B. Rodriguez, E. Tournié, A. Trampert, Strain relief and threading dislocation reduction in GaSb/AlSb/GaSb heterostructures grown on Si (001) substrate, J. Appl. Phys. 137 (2025).

[24] R. Beanland, M. Lourenco, K. Homewood, Relaxation of strained epitaxial layers by dislocation rotation, reaction and generation during annealing, Microsc. Semicond. Mater. 1997 (1997) 145-148.

[25] S.J. Addamane, D.M. Shima, A. Mansoori, G. Balakrishnan, A transmission electron microscopy study of dislocation propagation and filtering in highly mismatched GaSb/GaAs heteroepitaxy, J. Appl. Phys. 128 (2020) 225301.

[26] M.D. Abrámoff, P.J. Magalhães, S.J. Ram, Image processing with ImageJ, Biophotonics Int. 11 (2004) 36-42.
[27] J. Zhu, C. Palmstrøm, C. Carter, Misfit dislocations at ErAs/GaAs heterojunctions, Acta Metall. Mater. 43 (1995) 4171–4177.

[28] J.G. Zhu, C.B. Carter, 60° dislocations in (001) GaAs/Si interfaces, Philos. Mag. A 62 (1990) 319–328.

[29] D. Seidman, R. Balluffi, On the efficiency of dislocation climb in gold, Phys. Status Solidi B 17 (1966) 531–541.

[30] D. Hull, D.J. Bacon, Introduction to Dislocations, Butterworth-Heinemann, 2001.

[31] L. Hirth, Theory of Dislocations, John Wiley and sons, 1982.

[32] A. Trampert, K.H. Ploog, E. Tournié, Anisotropic misfit dislocation nucleation in two-dimensional grown InAs/GaAs (001) heterostructures, Appl. Phys. Lett. 73 (1998) 1074–1076.

[33] X. Liu, A. Hopgood, B.F. Usher, H. Wang, N. Braithwaite, Edge-type misfit dislocations produced by thermal processing of pre-relaxed InₓGa₁₋ₓAs/GaAs heterostructures, J. Appl. Phys. 88 (2000) 5975–5980.

[34] B.W. Dodson, J.Y. Tsao, Stress dependence of dislocation glide activation energy in single-crystal silicon-geranium alloys up to 2.6 GPa, Phys. Rev. B 38 (1988) 12383.

[35] J. Zou, C. Chou, D. Cockayne, A. Sikorski, M. Vaughan, Misfit dislocations lying along <100>-in [001]GaAs/In0.25Ga0.75As/GaAs quantum well heterostructures, Appl. Phys. Lett. 65 (1994) 1647–1649.

[36] P. Petroff, L. Kimerling, Dislocation climb model in compound semiconductors with zinc blende structure, Appl. Phys. Lett. 29 (1976) 461–463.

[37] L. Barbisan, A. Marzegalli, F. Montalenti, Atomic-scale insights on the formation of ordered arrays of edge dislocations in Ge/Si (001) films via molecular dynamics simulations, Sci. Rep. 12 (2022) 3235.