# Gauge origami and quiver W-algebras

Taro Kimura, Go Noshita

► To cite this version:

Taro Kimura, Go Noshita. Gauge origami and quiver W-algebras. Journal of High Energy Physics, 2024, 5, pp.208. 10.1007/jhep05(2024)208. hal-04240107

HAL Id: hal-04240107

https://hal.science/hal-04240107v1

Submitted on 13 Sep 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
RECEIVED: *October 24, 2023*

ACCEPTED: *April 2, 2024*

PUBLISHED: *May 16, 2024*

# Gauge origami and quiver W-algebras

Taro Kimura$^{a}$ and Go Noshita$^{b}$

E-mail: taro.kimura@u-bourgogne.fr, noshita-go969@g.ecc.u-tokyo.ac.jp

**ABSTRACT:** We explore the quantum algebraic formalism of the gauge origami system in $C^4$, where D2/D4/D6/D8-branes are present. We demonstrate that the contour integral formulas have free field interpretations, leading to the operator formalism of $qq$-characters associated with each D-brane. The $qq$-characters of D2 and D4-branes correspond to screening charges and generators of the affine quiver W-algebra, respectively. On the other hand, the $qq$-characters of D6 and D8-branes represent novel types of $qq$-characters, where monomial terms are characterized by plane partitions and solid partitions. The composition of these $qq$-characters yields the instanton partition functions of the gauge origami system, eventually establishing the BPS/CFT correspondence.

Additionally, we demonstrate that the fusion of $qq$-characters of D-branes in lower dimensions results in higher-dimensional D-brane $qq$-characters. We also investigate quadratic relations among these $qq$-characters. Furthermore, we explore the relationship with the representations, $q$-characters, and the Bethe ansatz equations of the quantum toroidal $gl_1$. This connection provides insights into the Bethe/Gauge correspondence of the gauge origami system from both gauge-theoretic and quantum-algebraic perspectives.

We finally present conjectures regarding generalizations to general toric Calabi-Yau four-folds. These generalizations imply the existence of an extensive class of $qq$-characters, which we call BPS $qq$-characters. These BPS $qq$-characters offer a new systematic approach to derive a broader range of BPS/CFT correspondence and Bethe/Gauge correspondence.

**KEYWORDS:** D-Branes, Quantum Groups, Supersymmetric Gauge Theory, Supersymmetry and Duality

ARXIV EPRINT: 2310.08545

<footnote>$^{a}$Institut de Mathématiques de Bourgogne, Université de Bourgogne,
CNRS, Dijon, France</footnote>

<footnote>$^{b}$Department of Physics, The University of Tokyo,
Tokyo, Japan</footnote>
# Contents

<table><tbody><tr><td>1</td><td>Introduction and summary</td><td>3</td></tr><tr><td>2</td><td>Multi-dimensional partitions</td><td>9</td></tr><tr><td>2.1</td><td>Partitions</td><td>9</td></tr><tr><td>2.2</td><td>Coordinates and q-contents</td><td>14</td></tr><tr><td>3</td><td>Instanton partition functions of the gauge origami system</td><td>17</td></tr><tr><td>3.1</td><td>Physical setup</td><td>17</td></tr><tr><td>3.2</td><td>Equivariant index formalism</td><td>25</td></tr><tr><td>3.3</td><td>Magnificent four</td><td>32</td></tr><tr><td>3.4</td><td>Tetrahedron instanton</td><td>35</td></tr><tr><td>3.5</td><td>Spiked instanton</td><td>38</td></tr><tr><td>3.6</td><td>Coupled vortex system</td><td>41</td></tr><tr><td>3.7</td><td>Decomposition of partition functions</td><td>45</td></tr><tr><td>3.8</td><td>qq-characters from instanton partition functions</td><td>48</td></tr><tr><td>4</td><td>Free field realizations of contour integral formulas</td><td>57</td></tr><tr><td>4.1</td><td>Magnificent four</td><td>58</td></tr><tr><td>4.2</td><td>Tetrahedron instanton</td><td>60</td></tr><tr><td>4.3</td><td>Spiked instanton</td><td>62</td></tr><tr><td>4.4</td><td>Coupled vortex system</td><td>64</td></tr><tr><td>4.5</td><td>Zero modes conditions</td><td>65</td></tr><tr><td>4.6</td><td>Quiver structure and generalizations</td><td>68</td></tr><tr><td>5</td><td>D2-brane qq-characters</td><td>81</td></tr><tr><td>5.1</td><td>D2 qq-characters and screening charges</td><td>82</td></tr><tr><td>5.2</td><td>Coupled vortex system and D2 qq-characters</td><td>85</td></tr><tr><td>6</td><td>D4-brane qq-characters</td><td>87</td></tr><tr><td>6.1</td><td>D4 qq-characters and affine quiver W-algebra</td><td>87</td></tr><tr><td>6.2</td><td>Spiked instantons and D4 qq-characters</td><td>90</td></tr><tr><td>6.3</td><td>Spiked instantons and screening currents</td><td>91</td></tr><tr><td>6.4</td><td>General D4 qq-characters</td><td>92</td></tr><tr><td>6.5</td><td>Quadratic relations of crossed instantons</td><td>95</td></tr><tr><td>7</td><td>D6-brane qq-characters</td><td>96</td></tr><tr><td>7.1</td><td>D6 qq-characters and plane partition</td><td>97</td></tr><tr><td>7.2</td><td>Tetrahedron instantons and D6 qq-characters</td><td>101</td></tr><tr><td>7.3</td><td>Fusion of D4 qq-characters to D6 qq-characters</td><td>102</td></tr><tr><td>7.4</td><td>Description in lower dimensional qq-characters</td><td>104</td></tr><tr><td>7.5</td><td>General D6 qq-characters</td><td>105</td></tr></tbody></table>
<table>
  <tbody>
    <tr>
      <td>8</td>
      <td>Towards D8-brane <i>qq</i>-characters</td>
      <td>112</td>
    </tr>
    <tr>
      <td>8.1</td>
      <td>Fusion of D6 <i>qq</i>-characters to D8 <i>qq</i>-characters</td>
      <td>112</td>
    </tr>
    <tr>
      <td>8.2</td>
      <td>Higher rank magnificent four and D8 <i>qq</i>-characters</td>
      <td>115</td>
    </tr>
    <tr>
      <td>9</td>
      <td>Quantum toroidal algebras and BPS <i>qq</i>-characters</td>
      <td>116</td>
    </tr>
    <tr>
      <td>9.1</td>
      <td>Quantum toroidal <i>gl</i><sub>1</sub></td>
      <td>116</td>
    </tr>
    <tr>
      <td>9.2</td>
      <td>Vector representation and D2 <i>qq</i>-character</td>
      <td>118</td>
    </tr>
    <tr>
      <td>9.3</td>
      <td>Fock representation and D4 <i>qq</i>-character</td>
      <td>119</td>
    </tr>
    <tr>
      <td>9.4</td>
      <td>MacMahon representation and D6 <i>qq</i>-character</td>
      <td>121</td>
    </tr>
    <tr>
      <td>9.5</td>
      <td>BPS <i>qq</i>-characters and quiver quantum toroidal algebras</td>
      <td>123</td>
    </tr>
    <tr>
      <td>10</td>
      <td>Semi-classical analysis and Bethe ansatz equation</td>
      <td>126</td>
    </tr>
    <tr>
      <td>10.1</td>
      <td>D4 system</td>
      <td>128</td>
    </tr>
    <tr>
      <td>10.2</td>
      <td>D6 system</td>
      <td>129</td>
    </tr>
    <tr>
      <td>10.3</td>
      <td>D2 system</td>
      <td>130</td>
    </tr>
    <tr>
      <td>10.4</td>
      <td>General case</td>
      <td>132</td>
    </tr>
    <tr>
      <td>10.5</td>
      <td>Quantum toroidal <i>gl</i><sub>1</sub> and <i>q</i>-characters</td>
      <td>133</td>
    </tr>
    <tr>
      <td>11</td>
      <td>Geometric realization of <i>qq</i>-character</td>
      <td>137</td>
    </tr>
    <tr>
      <td>11.1</td>
      <td>Quantum affine algebra</td>
      <td>137</td>
    </tr>
    <tr>
      <td>11.2</td>
      <td>Quantum toroidal <i>gl</i><sub>1</sub></td>
      <td>138</td>
    </tr>
    <tr>
      <td>11.3</td>
      <td>Quantum toroidal <i>gl</i><sub>n</sub></td>
      <td>139</td>
    </tr>
    <tr>
      <td>12</td>
      <td>Quiver elliptic W-algebra</td>
      <td>139</td>
    </tr>
    <tr>
      <td>13</td>
      <td>Conclusion and discussions</td>
      <td>141</td>
    </tr>
    <tr>
      <td>A</td>
      <td>Special functions</td>
      <td>144</td>
    </tr>
    <tr>
      <td>A.1</td>
      <td><i>q</i>-functions</td>
      <td>144</td>
    </tr>
    <tr>
      <td>A.2</td>
      <td>Elliptic formulas</td>
      <td>146</td>
    </tr>
    <tr>
      <td>B</td>
      <td>U(1) partition functions and structure functions</td>
      <td>146</td>
    </tr>
    <tr>
      <td>B.1</td>
      <td>Structure functions</td>
      <td>146</td>
    </tr>
    <tr>
      <td>B.2</td>
      <td>Examples of structure functions</td>
      <td>147</td>
    </tr>
    <tr>
      <td>B.3</td>
      <td>D6 partition functions</td>
      <td>149</td>
    </tr>
    <tr>
      <td>B.4</td>
      <td>D4 partition functions</td>
      <td>153</td>
    </tr>
    <tr>
      <td>B.5</td>
      <td>D2 partition functions</td>
      <td>156</td>
    </tr>
    <tr>
      <td>C</td>
      <td>Zero modes of vertex operators</td>
      <td>158</td>
    </tr>
    <tr>
      <td>D</td>
      <td>Supergroup gauge theory</td>
      <td>160</td>
    </tr>
    <tr>
      <td>D.1</td>
      <td>Contour integral formula and partition functions</td>
      <td>160</td>
    </tr>
    <tr>
      <td>D.2</td>
      <td>Supergroup analog of affine quiver W-algebra</td>
      <td>162</td>
    </tr>
  </tbody>
</table>
# 1 Introduction and summary

**Rise of BPS/CFT correspondence.** Towards a non-perturbative understanding of string theory and quantum field theory, exact computations of physical observables have played significant roles. Among these observables, the instanton partition functions of supersymmetric gauge theories have emerged as significant quantities that can be precisely computed through localization techniques [1–7]. These partition functions exhibit remarkable characteristics, particularly in the realm of rich geometric and algebraic properties. Our specific interest lies in the algebraic structures of these partition functions, which have, in turn, led to the revelation of a novel duality known as the BPS/CFT correspondence [8–16]. The BPS/CFT correspondence claims that correlation functions of BPS observables in supersymmetric gauge theories are dual to correlation functions of infinite-dimensional algebras. A well-known example of this duality is the Alday-Gaiotto-Tachikawa (AGT) duality [17–19] (see also [20] for a review), which established a connection between instanton partition functions of 4d $\mathcal{N}=2$ theories and conformal blocks of 2d conformal field theories (CFT), where both theories originate from a 6d $\mathcal{N}=(2,0)$ theory compactified on a Riemann surface. A 5d lift-up of the AGT duality called the 5d AGT duality [21–23], was subsequently discovered and the partition functions are dual to correlation functions of quantum algebras. Over the last decade, efforts have been dedicated to generalizing both the gauge theory side and the infinite-dimensional algebra side to achieve a more comprehensive understanding of the BPS/CFT correspondence [24–34].

**Generalized gauge theory.** The usual instantons appear as topologically nontrivial field configurations minimizing the action of the Yang-Mills theory on $\mathbb{R}^4$ and the moduli space comes from the ADHM construction [35, 36]. After introducing a nontrivial background flux called $\Omega$-background, Nekrasov showed that the instanton partition function takes the form of $\mathcal{Z} = \sum_{\lambda} q^{|\lambda|} \mathcal{Z}[\lambda]$ where the terms are a summation of two-dimensional partitions [1, 2, 37, 38]. Over the last few years, researches on generalizations of the Yang-Mills instantons have been conducted and we now know that there are higher dimensional generalizations of instantons having ADHM-like constructions [39–43]. The instanton partition functions are given as a statistical sum of random partitions as $\mathcal{Z} = \sum_{\Lambda} q^{|\Lambda|} \mathcal{Z}[\Lambda]$, where $\Lambda$ is a random partition. Considering 6d and 8d theories, the random partitions $\Lambda$ will be plane partitions [41, 44–46] and solid partitions [15, 16], respectively (see also [47]). In type IIB string theory on $\mathbb{R}^{10}$, such kind of setup appears as the low energy limit of the worldvolume theory of the D1-branes probing the D$(2p+1)$-branes ($p=2,3,4$) [48–51]: 2d ($p=2$), 3d ($p=3$), 4d ($p=4$) partitions. Mathematically, the $p=3$ case gives the equivariant Donaldson-Thomas invariants of $\mathbb{C}^3$, while the $p=4$ case is called the magnificent four model and gives the equivariant Donaldson-Thomas invariants of $\mathbb{C}^4$.

Another direction of generalizations of the Yang-Mills instantons is the *generalized gauge theory* [10–12], which is a theory defined on several, generally, intersecting components as $S = \bigcup_i S_i$ where on each space-time component $S_i$ there is an original field theory. On each space-time component $S_i$, we have a gauge group $G_i$ and at the intersection $S_i \cap S_j$ we have bifundamental fields transforming under $G_i \times G_j$, and thus it could be understood as a generalized quiver gauge theory. From the viewpoint of each field theory on $S_i$, the
intersection of other components plays the role of defects. The first example of such general gauge theory is the so-called spiked instanton system, which was introduced in [10, 11]. The spiked instantons arise from the low energy limit of D1-branes probing intersecting D5 (and anti-D5)-branes. Later, it was generalized to D1-branes probing intersecting D7-branes in [52] (see also [53, 54]), and the arising instantons are called the tetrahedron instantons. Note that these instantons are generalizations of the higher dimensional instantons introduced in the previous paragraph.

**Gauge origami.** The setups discussed previously are collectively called the gauge origami and the arising partition function is called the gauge origami partition function [12]. Consider a type IIB theory on $Z \times C$ where $Z$ is a toric Calabi-Yau four-fold and $C = \mathbb{C}, \mathbb{C}^\times, \mathbb{T}^2$ and the low energy limit of the D1-branes probing $D(2p+1)$-branes. The D1-branes wrap $C$ while the $D(2p+1)$-branes wrap the product of $C$ and non-compact toric submanifolds of $Z$ in a way preserving a suitable number of supersymmetries. Depending on $C$, the arising partition function becomes rational, trigonometric, and elliptic, respectively. The gauge origami partition function generally takes the form as

$$ \mathcal{Z}_{\text{origami}} = \sum_{\{\Lambda_i^{(\alpha)}\}} q^{|\Lambda|} \prod_{(i,\alpha)} \mathcal{Z}[\Lambda_i^{(\alpha)}] \prod_{(i,\alpha) \neq (j,\beta)} \mathcal{Z}(\Lambda_i^{(\alpha)} | \Lambda_j^{(\beta)}) \quad (1.1) $$

where $i$ labels the possible types of toric submanifolds and $\alpha$ labels the number of D-branes wrapping them. The partition function is a summation of random BPS crystals, which are generalizations of the partitions, and they are denoted $\Lambda_i^{(\alpha)}$. These crystals are expected to be, generally, truncations of four-dimensional BPS crystals which are generalizations of the three-dimensional BPS crystals [55–58]. The $\mathcal{Z}[\Lambda_i^{(\alpha)}]$ part comes from the contribution of each D(2p+1)-branes while the $\mathcal{Z}(\Lambda_i^{(\alpha)} | \Lambda_j^{(\beta)})$ part comes from the bifundamental contributions at the junctions. In this paper, we mainly focus on the case when $Z = \mathbb{C}^4$ and $C = \mathbb{C}^\times \simeq \mathbb{R} \times \mathbb{S}^1$ which gives the K-theoretic magnificent four [15, 16], tetrahedron instanton [52], and spiked instanton [59] setups. From the string theory viewpoint, we take the T-duality of the D1-D(2p+1) system and consider a D0-D(2p) system, where each D(2p)-brane gives a (2p+1)-dimensional gauge theory and the D0-branes play the roles of instantons.

**Quantum algebra of gauge origami.** An interesting property of the gauge origami partition function is the existence of an infinite set of non-perturbative Dyson-Schwinger equations related to the symmetries of adding and removing instantons [10]. The $qq$-characters are physical observables characterizing them, and interestingly, there is an operator formalism of them called the quiver W-algebra [60–63] (see [64] for a review). From the gauge theoretic viewpoint, such algebras are associated with the Dynkin diagram corresponding to the quiver structure of the gauge theory. In the gauge origami formalism, the $qq$-characters and quiver W-algebras appear from the gauge origami system in $Z = \mathbb{C}^2 \times \mathbb{C}^2/\Gamma$, where $\Gamma$ denotes the finite subgroup of SU(2) associated with the quiver structure through the McKay correspondence. Placing D4-branes wrapping the $\mathbb{C}^2$ part gives the 5d $\mathcal{N} = 1$ (affine) quiver gauge theories and the D4-branes wrapping the $\mathbb{C}^2/\Gamma$ give the $qq$-characters or the quiver W-algebras of the theory. Physically, they are codimension four defects of the quiver gauge theory [65].

One of the reasons why such algebras are considered important is because they give the BPS/CFT correspondence for the present case. One can construct screening currents
from the quiver structure and the vacuum expectation value of them gives the instanton partition function of the quiver gauge theory. Moreover, after defining the highest weight, the commutativity with the screening charges determines the generator of the quiver W-algebra uniquely and they are the operator version of the $qq$-characters. These quiver W-algebras include the well-known $q$-Virasoro [66], $q$-W$_N$ [67, 68], and Frenkel-Reshetkhin's deformed W-algebras [69, 70].

Although the quiver W-algebras indeed gave a way to discuss the BPS/CFT correspondence from the quantum algebraic viewpoint, the applicable theory is still limited and needs to be extended. For example, while the $qq$-characters associated with $Z = \mathbb{C}^2/\Upsilon \times \mathbb{C}^2/\Gamma$ were studied in terms of partition functions in [9, 10, 71-73], it seems that the complete operator formalism of such cases is still missing in the literature.<fnref n="1" /> Moreover, the quiver W-algebra is only applicable to discuss two stacks of D-branes in transverse directions while we have multiple intersecting D-branes in the gauge origami system. Based on recent studies such as the tetrahedron instanton system we also have D-branes wrapping not only complex two-dimensional surfaces but also complex three-dimensional manifolds. Most importantly, we need to generalize the operator formalism to describe gauge origami systems associated with general toric Calabi-Yau four-folds $Z$. See [76-80] for discussions along this direction.

The goal of this paper is to fill in this gap by generalizing the concept of quiver W-algebras and showing the BPS/CFT correspondence of the gauge origami system associated with $\mathbb{C}^4$. We will only give some conjectures for generalizations to toric Calabi-Yau four-folds and details are postponed for future work [74].

## Summary of the results

Let us summarize the main results of this paper.

**D2/D4/D6 $qq$-characters.** We introduce a D2 gauge origami system which we call the coupled vortex system (section 3.6). This theory comes from the low energy limit of the D0-branes probing the D2-branes spanning $C_a \subset \mathbb{C}^4$ ($a = 1, 2, 3, 4$; see section 3.2 for notations), which is obtained through a dimensional reduction of the D4-brane gauge origami system. This coupled vortex system also plays an important role similar to higher-dimensional branes in the gauge origami systems.

Considering a stack of D2-branes, D4-branes and D6-branes spanning transversely in $\mathbb{C}^4$, we derive the D2/D4/D6 $qq$-characters in terms of partition functions in section 3.8. The D4 $qq$-character is known to be characterized by 2d partitions. The D2 $qq$-characters are $qq$-characters where the monomial terms are 1d partitions, while the D6 $qq$-characters are $qq$-characters whose monomial terms are 3d partitions (plane partitions).

**D-brane vertex operators.** We introduce vertex operators $A(x), S_a(x), X_A(x), W_{\tilde{a}}(x), Z(x)$ corresponding to D0, D2, D4, D6, and D8-branes wrapping the possible subspaces pt, $C_a, \mathbb{C}_A^2, \mathbb{C}_a^3, \mathbb{C}_a^4$ ($a \in \mathbf{4}, A \in \mathbf{6}$), respectively. The main result of section 4 is as follows.

<footnote n="1">¹We hope to report it in a future work [74] (see also [75]).</footnote>
**Theorem 1.1** (Theorem 4.1). For each D-brane (D0, D2, D4, D6, D8), we define the corresponding vertex operators as

<table><thead><tr><th>D-brane</th><th>space-time</th><th>vertex operator</th></tr></thead><tbody><tr><td>D0-brane</td><td>pt × S<sup>1</sup></td><td>A(x)</td></tr><tr><td>D2-brane</td><td>C<sub>a</sub> × S<sup>1</sup> (a ∈ 4)</td><td>S<sub>a</sub>(x)</td></tr><tr><td>D4-brane</td><td>C<sup>2</sup><sub>A</sub> × S<sup>1</sup> (A ∈ 6)</td><td>X<sub>A</sub>(x)</td></tr><tr><td>D6-brane</td><td>C<sup>3</sup><sub>a</sub> × S<sup>1</sup> (a ∈ 4)</td><td>W<sub>a</sub>(x)</td></tr><tr><td>D8-brane</td><td>C<sup>4</sup> × S<sup>1</sup></td><td>Z(x)</td></tr></tbody></table>

We have multiple copies of vertex operators if there are multiple ways that the D-branes can wrap. Then, the contour integral formula of the k-instanton partition function takes the form as

$$Z_k = \oint \prod_{I=1}^{k} \frac{dx_I}{2\pi i x_I} \langle 0 | \prod_{I=1}^{k} A(x_I)^{-1} : \prod_{I} V_i(v_I) : | 0 \rangle, \quad (1.3)$$

where $V_i(x)$ is an operator written from {$S_a(x), X_A(x), W_{\bar{a}}(x), Z(x)$}.

The vertex operators have a q-Cartan matrix understanding associated with quivers and thus are generalizations of the conventional quiver W-algebras (see section 4.6).

**Operator formalism of $\overline{qq}$-characters.** We introduce the operator formalism of the $\overline{qq}$-characters of the gauge origami system and show the BPS/CFT correspondence. For each D(2p)-brane, we can associate a $\overline{qq}$-character. The D2 $\overline{qq}$-characters $\mathcal{Q}_a(x)$ ($a \in \underline{4}$) are four copies of the screening charge of the $\tilde{A}_0$ quiver W-algebra [60] and the D4 $\overline{qq}$-characters $\mathcal{T}_A(x)$ ($A \in \underline{6}$) are six copies of the generator of the $\tilde{A}_0$ quiver W-algebra. The D6 $\overline{qq}$-characters $\mathcal{T}_{\bar{a}}(x)$ ($a \in \underline{4}$) are the new $\overline{qq}$-characters where the monomial terms are labeled by plane partitions. These $\overline{qq}$-characters represent the quantum algebras associated with complex 1, 2, 3-dimensional submanifolds. We will see that their compositions indeed give the gauge origami partition function which shows the BPS/CFT correspondence.

**Theorem 1.2** (Theorems 5.6, 6.6, 7.10). The gauge origami partition function is in general given as a correlation function of the $\overline{qq}$-characters,

$$Z_{\text{origami}} = \langle 0 | \prod_{(i, \alpha)} \mathcal{T}_i(x_{i, \alpha}) | 0 \rangle, \quad i \in \underline{4} \oplus \underline{6} \oplus \underline{4}^{\vee}. \qquad (1.4)$$

For the case $\mathcal{C} = \mathbb{T}^2$, the corresponding partition function is given by a torus correlator of the $\overline{qq}$-characters instead of the vacuum expectation value. We can also use the elliptic version of the vertex operators discussed in section 12 to discuss the case $\mathcal{C} = \mathbb{T}^2$.

In the context of quiver W-algebra and also the quantum integrable system, the commutation relation between the vertex operators plays a fundamental role. We obtain an interesting commutativity among $\overline{qq}$-characters: the D2, D4, D6 $\overline{qq}$-characters associated with subspaces of $\mathbb{C}^4$ that are transverse with each other commutes.
**Theorem 1.3** (Theorem 7.5). The *qq*-characters associated with the elements $i, j \in \mathfrak{4} \oplus \mathfrak{6} \oplus \mathfrak{4}^\vee$ commute with each other up to trivial zero modes (see (6.55)) when $i$ and $j$ are transverse with each other:

$$ T_i(x)T_j(x') - f_{ij}(x, x')T_j(x')T_i(x) = 0 \iff i \cap j = \emptyset, \qquad (1.5) $$

where $f_{ij}(x, x')$ are zero mode factors.

Moreover, infinite fusion of D2 (D4) *qq*-characters give D4 (D6) *qq*-characters (Thm. 6.9, 7.12). Using the fusion process and fusing an infinite number of D6 *qq*-characters, we define the D8 *qq*-characters (section 8.1). We also show the BPS/CFT correspondence of the magnificent four model up to sign factors.

**Theorem 1.4** (Theorem 8.5). The composition of the D8 *qq*-characters gives the partition function of higher rank magnificent four system up to sign factors, which establishes the BPS/CFT correspondence for the magnificent four:

$$ \langle 0 | T_{\mathfrak{4};a_N}(x_N) \cdots T_{\mathfrak{4};a_1}(x_1) | 0 \rangle = \sum_{\rho^{(1)}, \cdots, \rho^{(N)}} q^{\lvert\rho\rvert} \prod_{i=1}^{N} \mathcal{Z}_{\mathfrak{4};a_i}^{\mathrm{D}8}[\rho^{(i)}, K_i] \prod_{j>i} \mathcal{Z}_{\mathrm{1-loop}}^{\mathrm{D}8-\mathrm{D}8}(x_i, K_i | x_j, K_j) \\ \times \prod_{j>i} \mathcal{Z}_{K_i|K_j}^{\mathrm{D}8-\mathrm{D}8}(x_i, \rho^{(i)} | x_j, \rho^{(j)}). \qquad (1.6) $$

We then give conjectures related to generalizations to toric Calabi-Yau four-folds (section 4.6.2, 4.6.3, 4.6.4, 7.5, Conj. 4.9, 4.7, 7.17). The *qq*-characters appearing in such generalizations are called BPS *qq*-characters.

**Quantum toroidal algebra, Bethe ansatz.** We also show that the *qq*-characters associated with $\mathbb{C}^4$ geometry have a correspondence with quantum toroidal algebras (section 9.1). In particular, we show that the D2/D4/D6 system corresponds to the vector/Fock/MacMahon representation of quantum toroidal $gl_1$. We also consider the quiver quantum toroidal algebras associated with toric Calabi-Yau three-folds and construct generic *qq*-characters, that we call the BPS *qq*-characters (section 9.5). In section 10, through the semi-classical analysis of the gauge origami system, we obtain the universal form of the Bethe ansatz equations (BAEs) for the gauge origami system on $\mathbb{C}^4$.

**Theorem 1.5** (Theorem 10.1). The BAE obtained as the saddle point equation of the D2/D4/D6 system partition function is generally written as follows,

$$ 1 = -q \frac{\mathrm{Q}(q_1^{-1}x)\mathrm{Q}(q_2^{-1}x)\mathrm{Q}(q_3^{-1}x)}{\mathrm{Q}(q_1x)\mathrm{Q}(q_2x)\mathrm{Q}(q_3x)}, \qquad (1.7) $$

where the three parameters obey $q_1q_2q_3 = 1$ and the corresponding $Q$-functions for the D2/D4/D6 system are given by (10.24), (10.8), (10.16), (10.40), (10.43).

In fact, it turns out that this BAE is associated with quantum toroidal $gl_1$. We can further discuss the representation dependence by imposing the flavor contribution.
**Theorem 1.6** (Theorem 10.2). Let $Q(x)$, $a(x)$, $d(x)$ be polynomials in $x \in \mathbb{C}^\times$ with the parameters obeying $q_1q_2q_3 = 1$. The saddle point equation of the D2 gauge origami system with flavor $D8-\overline{D8}$ branes gives the BAE involving the additional polynomials $a(x)$ and $d(x)$ specifying the representation of quantum toroidal $gl_1$,

$$1 = -q \frac{a(x) Q(q_1^{-1}x) Q(q_2^{-1}x) Q(q_3^{-1}x)}{d(x) Q(q_1 x) Q(q_2 x) Q(q_3 x)} \qquad (1.8)$$

This structure is analogous to Yangian and quantum affine algebra, where the BAE has a universal form, which does not depend on the representation, while the (highest-weight) representation data appear only in the $a$ and $d$ polynomials.

In addition to the saddle point equation, that gives rise to the BEA, we also study the $q$-character, the semi-classical reduction of the $qq$-character. For generic $q$-characters, we have the following relation.

**Theorem 1.7** (Theorem 10.4). For any $q$-characters obtained in the semi-classical limit, $\mathcal{T}(x), \mathcal{T}'(x) \in \{\hat{Q}_{1,2,3}(x), \hat{T}_{12,23,13}(x), \hat{T}_{123}^K(x)\}$, we have

$$[\mathcal{T}(x), \mathcal{T}'(x')] = 0. \qquad (1.9)$$

From this point of view, one may obtain a wide class of commuting operators from the $q$-character, that would be identified with the commuting Hamiltonians of the corresponding quantum integrable system. We also mention generalizations of these arguments to other Calabi-Yau geometries and discuss the corresponding BAE.

**Geometric realization of $qq$-characters.** Although we have focused on the algebraic aspects so far, the gauge origami construction also provides geometric insights on $qq$-character and the underlying quantum algebraic structures. For example, for the Fock module of quantum toroidal $gl_1$, the $qq$-character is given by the equivariant integral over the Hilbert scheme of points on $\mathbb{C}^2$, which is given as a quiver variety associated with $\hat{A}_0$ quiver (Prop. 11.1). Considering the higher rank framing space, the quiver variety gives rise to the Quot scheme, and one can then obtain the tensor product module. Generalizing this construction, we have the geometric realization of the $qq$-character of the MacMahon module of quantum toroidal $gl_1$.

**Theorem 1.8** (Theorem 11.4). Let $x \in \mathbb{C}^\times$. The $qq$-character of the MacMahon module of quantum toroidal $gl_1$ is given as follows,

$$T_{1;x}[\mathbf{Y}] = \sum_{v \ge 0} T_{1,v;x}[\mathbf{Y}], \qquad T_{1,v;x}[\mathbf{Y}] = \int_{[\mathfrak{M}_{1,v}]^{\text{vir}}} \mathrm{ch} \wedge \bullet \mathbf{Y}_{1,v}^{\vee} \mathbf{Y} \mathrm{td}(T\mathfrak{M}_{1,v}), \qquad (1.10)$$

where the integral is equivariantly taken over the virtual fundamental cycle of the moduli space of $v$ D0 branes on a single D6 brane, isomorphic to the Hilbert scheme of points on $\mathbb{C}^3$, $\mathfrak{M}_{1,v} \cong \mathrm{Hilb}^v(\mathbb{C}^3)$. We denote the observable bundle and the formal bundle over $\mathfrak{M}_{1,v}$ by $\mathbf{Y}_{1,v}$ and $\mathbf{Y}$.

The $qq$-character of the tensor product of the MacMahon module can be similarly given by the equivariant integral over the moduli space of the higher rank D6-brane system, which is isomorphic to the Quot scheme, $\mathfrak{M}_{w,v} \cong \mathrm{Quot}_{\mathbb{C}^3}^v(\mathcal{O}^{\oplus w})$. Physically, this integral computes the
codimension six defect partition function of the tetrahedron instanton system. This formalism should be straightforwardly extended to generic representations of quantum toroidal $gl_1$ by replacing the moduli space with the corresponding one.

## Organization of this paper

The paper is organized as follows. In section 2, we introduce the notations related to 1d, 2d, 3d, and 4d partitions, highlighting their relationships with lower dimensional partitions and defining $q$-coordinates for boxes of the partitions. In section 3, we delve into explicit formulas and properties of the instanton partition functions of the gauge origami system. This section covers various aspects, including the physical setup, contour integral formulas, partition functions, their decompositions, and the concept of $qq$-characters. Free field realizations of the contour integral formulas for each system are given in section 4. Vertex operators associated with the D0, D2, D4, D6, D8-branes are introduced. We also discuss the quiver structure and generalizations to toric Calabi-Yau four-folds. Sections 5, 6, 7, and 8 are dedicated to the study of D2, D4, D6, and D8-brane $qq$-characters, examining their relationships with various gauge origami systems. We discuss the fusion process, generalizations, quadratic relations, and each system's BPS/CFT correspondence. In section 9, we delve into quantum toroidal algebras and discuss the relation with BPS $qq$-characters. Section 10 explores applications of the D-brane $qq$-characters, including their role in semi-classical analysis and the Bethe ansatz equations of the gauge origami system. Geometric realization of the $qq$-characters and elliptic generalizations are discussed in section 11 and section 12, respectively. In section 13, we finally conclude our findings and engage in discussions about the implications of our work. Additionally, we provide some supplementary formulas used in the main sections of the paper in the appendix.

# 2 Multi-dimensional partitions

Let us summarize the notation we use for multi-dimensional partitions. We follow the convention and the terminology of [15, 16, 80].

## 2.1 Partitions

**One-dimensional partition.** A one-dimensional partition is a row of boxes stacked only in one direction as

$$ \begin{tikzpicture}[baseline=(current bounding box.center), scale=0.5] \node[draw, rectangle, minimum width=3mm, minimum height=1mm] (top) at (0,0.5) {}; \node[draw, rectangle, minimum width=3mm, minimum height=1mm] (bottom) at (0,-0.5) {}; \draw (top) -- (bottom); \end{tikzpicture} \qquad (2.1) $$

Such a kind of partition is specified by a non-negative integer $k \in \mathbb{Z}_{\ge 0}$.

We can also introduce one-dimensional partitions where the boxes are extended semi-infinite to the left of a border as

$$ \begin{tikzpicture}[baseline=(current bounding box.center), scale=0.5] \node[draw, rectangle, minimum width=3mm, minimum height=1mm] (top) at (0,0.5) {}; \node[draw, rectangle, minimum width=3mm, minimum height=1mm] (bottom) at (0,-0.5) {}; \draw (top) -- (bottom); \node[below=2pt of top] (top label) {$\cdots$}; \node[below=2pt of bottom] (bottom label) {}; \draw (top) -- (bottom); \end{tikzpicture} \qquad (2.2) $$
For these kinds of partitions, an integer $k \in \mathbb{Z}$ specifies it. The integer $k$ denotes the number of boxes counted from the border to the right. If no boxes are on the right and the rightmost box is left to the border, then $k$ will be negative.

**Two-dimensional partition (Young diagram).** A two-dimensional partition is a non-decreasing sequence of non-negative integers

$$ \lambda = (\lambda_1 \ge \lambda_2 \ge \dots \ge \lambda_{\ell(\lambda)} \ge 0), \quad |\lambda| = \lambda_1 + \dots + \lambda_{\ell(\lambda)} \qquad (2.3) $$

where $|\lambda|$ is the size, and $\ell(\lambda)$ is the length of the partition. A partition will be drawn as a collection of square boxes $\square$, and this collection is called the Young diagram.<fnref n="2" /> For example, a partition $\lambda = (5, 3, 2, 1)$ will look like

<figure><img src="image_2.png" /></figure>

A box $\square$ positioned in the $i$'th row counted from the bottom and $j$'th column counted from the left in the Young diagram is assigned a coordinate $(i, j)$, $1 \le i \le \ell(\lambda)$, $1 \le j \le \lambda_i$. We denote the set of all possible 2d partitions/Young diagrams as $\mathcal{P}$. The transpose is denoted as $\lambda^T$ in the usual sense. We call this type of description the (1,1)-type description where we followed the terminology in [15]. This is because we are assigning a one-dimensional partition with only non-negative integers $\lambda_i \ge 0$ to each $i = 1, \dots, \ell(\lambda)$. Note that there are two (1,1)-type descriptions of the 2d partition, depending on whether we are using $\lambda$ or the transpose $\lambda^T$.

**Three-dimensional partition (plane partition).** The plane partition is a stack of cubes that obeys a generalization of the condition (2.3). We denote the set of all possible plane partitions as $\mathcal{PP}$. There are two descriptions of the plane partition which we call (2,1)-type and (1,2)-type.

The (2,1)-type description understands the plane partition $\pi$ as a 2d partition $\lambda_\pi$ where there is a map mapping each box $\square = (i, j) \in \lambda_\pi$ a number $\pi_{i,j}$ obeying the following condition (see figure 1 and 2)

$$ \pi_{i,j} \ge \pi_{i+1,j}, \quad \pi_{i,j} \ge \pi_{i,j+1}. \qquad (2.5) $$

The size of the plane partition is defined as the number of cubes included in the configuration

$$ |\pi| = \sum_{i,j} \pi_{i,j}. \qquad (2.6) $$

We have three descriptions depending on which two-dimensional plane we project the plane partition to get the Young diagram. Given three axes 1, 2, 3, we use the symbol $\pi$ when

<footnote n="2">²In this paper, Young diagrams and 2d partitions are not distinguished. Sometimes, we will also omit the 2d and just say partitions for 2d partitions when it is obvious. Similarly, plane partitions and 3d partitions, solid partitions, and 4d partitions are not distinguished.</footnote>
<figure><img src="image_3.png" /><figcaption>Figure 1. Plane partition.</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Figure 2. (2,1)-type and (1,2)-type description of the plane partition.</figcaption></figure>

the plane partition is projected to a Young diagram in the 12-plane, $\tilde{\pi}$ when the Young diagram is projected onto the 13-plane, and $\tilde{\tilde{\pi}}$ when it is projected onto the 23-plane. This is the 3d analog of the transpose of the Young diagram. Like the Young diagram case, a cube ▷ in the plane partition $\pi$ is assigned a coordinate $(i, j, k)$ $(i, j, k \ge 1)$ as figure 1. In this description, we have

$$ (i, j, k) \in \pi \Leftrightarrow 1 \le k \le \pi_{i,j}. \qquad (2.7) $$

The other (1,2)-type description understands the plane partition as a non-increasing sequence of Young diagrams (see figure 2):

$$ \begin{gathered} \pi = (\Lambda^{(1)}, \Lambda^{(2)}, \dots, \Lambda^{(h(\pi))}), \quad \Lambda^{(k)} = (\Lambda_1^{(k)}, \dots, \Lambda_i^{(k)} \dots) \qquad (2.8) \\ \Lambda^{(k)} \supseteq \Lambda^{(k+1)}, \forall k \end{gathered} $$
<figure><img src="image_5.png" /><figcaption>Figure 3. 4-cube.</figcaption></figure>

where $\Lambda^{(k)} \supseteq \Lambda^{(k+1)}$ means $\forall (i, j) \in \Lambda^{(k+1)} \Rightarrow (i, j) \in \Lambda^{(k)}$ and $h(\pi)$ is the height of the plane partition which is defined as

$$h(\pi) = \min\{k \ge 0 | (1, 1, k+1) \notin \pi\}. \qquad (2.9)$$

The size is defined as

$$|\pi| = \sum_k |\Lambda^{(k)}| \qquad (2.10)$$

in this description. Similarly, the $\square = (i, j, k)$ in the plane partition obeys the condition

$$(i, j, k) \in \pi \Leftrightarrow 1 \le j \le \Lambda_i^{(k)} \qquad (2.11)$$

Note that we also have three possible descriptions of this type depending on which axis we define the height of the plane partition.

**Four-dimensional partition (solid partition).** A solid partition is a four-dimensional analog of the Young diagram and plane partition. It is a stack of 4-cubes (see figure 3) obeying similar conditions to (2.3) and (2.5). We denote the set of all possible solid partitions as $SP$. We have three ways to describe the solid partition: (3, 1), (2, 2), and (1, 3)-types.

1. (3, 1)-type: this description is similar to the plane partition's (2, 1)-type description. We project the solid partition to a plane partition $\pi_{\rho}$ and for each cube $(i, j, k) \in \pi_{\rho}$, a height function $\rho_{i,j,k}$ is defined. The height function obeys the condition

$$\rho_{i,j,k} \ge \rho_{i+1,j,k}, \quad \rho_{i,j,k} \ge \rho_{i,j+1,k}, \quad \rho_{i,j,k} \ge \rho_{i,j,k+1}. \qquad (2.12)$$

The size is defined as

$$|\rho| = \sum_{(i,j,k) \in \pi_{\rho}} \rho_{i,j,k}. \qquad (2.13)$$

4-cubes $\square$ in the solid partition are assigned coordinates in a natural way $(i, j, k, l)$ and obey

$$(i, j, k, l) \in \rho \Leftrightarrow 1 \le l \le \rho_{i,j,k}. \qquad (2.14)$$

Depending on which three axes we project the solid partition to obtain a plane partition, we have four choices for this description.
<figure><img src="image_6.png" /><figcaption>Figure 4. (2, 2)-type description of solid partition.</figcaption></figure>

2. (2,2)-type: another way to describe the solid partition is to understand it as a Young diagram $\lambda_\rho$ where on each $(i, j)$ there is another Young diagram $\mu_{(i,j)}^\rho$ obeying the conditions (see figure 4):

$$ \mu_{(i,j)}^{\rho} \succeq \mu_{(i,j+1)}^{\rho}, \quad \mu_{(i,j)}^{\rho} \succeq \mu_{(i+1,j)}^{\rho}, \qquad (2.15) $$

where $\mu_{(i,j)}^{\rho} \succeq \mu_{(i',j')}^{\rho}$ means

$$ (k,l) \in \mu_{(i',j')}^{\rho} \Rightarrow (k,l) \in \mu_{(i,j)}^{\rho}. \qquad (2.16) $$

Under this description, the size is defined as

$$ |\rho| = \sum_{(i,j) \in \lambda_{\rho}} |\mu^{\rho}| \qquad (2.17) $$

and

$$ (i, j, k, l) \in \rho \Leftrightarrow (k, l) \in \mu_{(i,j)}^{\rho}. \qquad (2.18) $$

We have six possible descriptions of this type depending on the choice of the two axes.

3. (1,3)-type: This description resembles the (1,2)-type description of plane partition. The solid partition is understood as non-increasing sequences of plane partitions (see figure 5):

$$ \rho = (\Pi^{(1)}, \Pi^{(2)}, \dots), \quad \Pi^{(l)} \succeq \Pi^{(l+1)} \qquad (2.19) $$

where $\Pi^{(l)} \succeq \Pi^{(l+1)}$ means

$$ (i, j, k) \in \Pi^{(l+1)} \Rightarrow (i, j, k) \in \Pi^{(l)}. \qquad (2.20) $$

Under this description, the size is defined as

$$ |\rho| = \sum_{l} |\Pi^{(l)}| \qquad (2.21) $$

and obviously,

$$ (i, j, k, l) \in \rho \Leftrightarrow (i, j, k) \in \Pi^{(l)}. \qquad (2.22) $$

We again have four possible descriptions for this type.
• 2d partitions:

<figure><img src="image_7.png" /></figure>

• 3d partitions:

<figure><img src="image_8.png" /></figure>

We also define the multiplicative coordinates of the boxes called q-contents or q-coordinates by taking the exponential:

$$ q(\square) = e^{c(\square)} = \begin{cases} uq_a^{i-1} & (i \in \mathbb{Z}_{\ge 1} \text{ or } i \in \mathbb{Z}) & \text{1d partition} \\ uq_a^{i-1}q_b^{j-1} & (i, j \in \mathbb{Z}_{\ge 1}) & \text{2d partition} \\ uq_a^{i-1}q_b^{j-1}q_c^{k-1} & (i, j, k \in \mathbb{Z}_{\ge 1}) & \text{3d partition} \\ uq_1^{i-1}q_2^{j-1}q_3^{k-1}q_4^{l-1} & (i, j, k, l \in \mathbb{Z}_{\ge 1}) & \text{4d partition} \end{cases} \quad (2.27) $$

where $q_a = e^{\epsilon_a}$ ($a = 1, 2, 3, 4$) and $u = e^a$.

**Other types of description.** Looking at the q-coordinates, we can also see how the higher-dimensional partitions can be decomposed into multiple lower-dimensional partitions.

• 1d partitions: the 1d partitions can be understood as a collection of the q-coordinates as $\{uq_a^{i-1} | 1 \le i \le k, k \in \mathbb{Z}_{\ge 0}\}$ or $\{uq_a^{i-1} | i \le k, k \in \mathbb{Z}\}$.
• 2d partition: the boxes included in the Young diagram are labeled by $(i, j)$, $(1 \le j \le \lambda_i, 1 \le i \le \ell(\lambda))$ with coordinates $\{uq_a^{i-1}q_b^{j-1}\}$. We can decompose this set as

$$ \{uq_a^{i-1}q_b^{j-1} \mid \substack{1 \le j \le \lambda_i \\ 1 \le i \le \ell(\lambda)}\} = \bigcup_{i=1}^{\ell(\lambda)} \{(uq_a^{i-1}q_b^{j-1}) \mid 1 \le j \le \lambda_i\}. \quad (2.28) $$

This means the 2d partition is a collection of 1d partitions where the origins of the 1d partitions are shifted as $uq_a^{i-1}$. This gives the (1,1)-type decomposition of the 2d partitions.

• 3d partition: it is understood as a collection of q-coordinates $\{uq_a^{i-1}q_b^{j-1}q_c^{k-1}\}$ where $(i, j, k) \in \pi$. The (2,1)-type description comes from the decomposition

$$ \{uq_a^{i-1}q_b^{j-1}q_c^{k-1} \mid (i,j,k) \in \pi\} = \bigcup_{(i,j) \in \lambda_{\pi}} \{(uq_a^{i-1}q_b^{j-1})q_c^{k-1} \mid 1 \le k \le \pi_{ij}\}, \quad (2.29) $$

which means the plane partition is decomposed into 1d partitions with length $\pi_{ij}$ and coordinates of the origins as $uq_a^{i-1}q_b^{j-1}$. The (1,2)-type description comes from

$$ \{uq_a^{i-1}q_b^{j-1}q_c^{k-1} \mid (i,j,k) \in \pi\} = \bigcup_k \{(uq_c^{k-1})q_a^{i-1}q_b^{j-1} \mid (i,j) \in \Lambda^{(k)}\}. \quad (2.30) $$

The plane partition is then decomposed into multiple 2d partitions $\Lambda^{(k)}$ whose coordinates of the origins are $uq_c^{k-1}$.

• 4d partition: the solid partition is a collection of q-coordinates $\{uq_a^{i-1}q_b^{j-1}q_c^{k-1}q_d^{l-1}\}$ where $(i, j, k, l) \in \rho$.

1. (3,1)-type: it is decomposed into multiple 1d partitions with length $\rho_{i,j,k}$ and origins at $uq_a^{i-1}q_b^{j-1}q_c^{k-1}$.

$$ \{uq_a^{i-1}q_b^{j-1}q_c^{k-1}q_d^{l-1} \mid (i,j,k,l) \in \rho\} = \bigcup_{(i,j,k) \in \lambda_{\rho}} \{(uq_a^{i-1}q_b^{j-1}q_c^{k-1})q_d^{l-1} \mid 1 \le l \le \rho_{i,j,k}\} \quad (2.31) $$

2. (2,2)-type: it is decomposed into multiple 2d partitions $\mu_{(i,j)}^{\rho}$ with origins with origins at $uq_a^{i-1}q_b^{j-1}$.

$$ \{uq_a^{i-1}q_b^{j-1}q_c^{k-1}q_d^{l-1} \mid (i,j,k,l) \in \rho\} = \bigcup_{(i,j) \in \lambda_{\rho}} \{(uq_a^{i-1}q_b^{j-1})q_c^{k-1}q_d^{l-1} \mid (k,l) \in \mu_{(i,j)}^{\rho}\} \quad (2.32) $$

3. (1,3)-type: it is decomposed into multiple 3d partitions $\Pi^{(l)}$ with origins at $uq_d^{l-1}$.

$$ \{uq_a^{i-1}q_b^{j-1}q_c^{k-1}q_d^{l-1} \mid (i,j,k,l) \in \rho\} = \bigcup_l \{(uq_d^{l-1})q_a^{i-1}q_b^{j-1}q_c^{k-1} \mid (i,j,k) \in \Pi^{(l)}\} \quad (2.33) $$
### 3 Instanton partition functions of the gauge origami system

In this section, we review the physical setup (section 3.1), the equivariant index formalism (section 3.2), contour integrals and explicit instanton partition functions (section 3.3, 3.4, 3.5, 3.6) of the gauge origami system introduced first by Nekrasov in the context of BPS/CFT correspondence [10–16]. We are, in particular, interested in the following three setups: spiked instantons [11, 12], tetrahedron instantons [52, 53, 81], and the magnificent four [15, 16, 80]. We additionally introduce a system that we call a *coupled vortex system* whose characteristics are yet to be studied.

We also study the decompositions of the partition functions and show that partition functions of higher dimensional theory are obtained by infinite products of the partition functions of lower dimensional theories in section 3.7. The decomposition depends on how the multi-dimensional partitions are described as discussed in detail in section 2. We finally introduce the *qq*-characters in terms of partition functions in section 3.8.

#### 3.1 Physical setup

In this section, we review the gauge origami construction which is a generalization of the ADHM construction of instantons of supersymmetric gauge theories. Roughly speaking, the gauge origami system is a *generalized gauge theory* whose space-time $S$ contains several intersecting components as

$$S = \bigcup_{i} S_{i}. \qquad (3.1)$$

We have a gauge group $G_i$ for each component $S_i$ and matter fields in the intersection $S_i \cap S_j$ transforming under the gauge group $G_i \times G_j$ and thus are bifundamental multiplets. In this sense, we can understand such theories as generalized quiver gauge theories.

The gauge origami system is described as a low-energy-limit of the world volume theory on multiple intersecting D-branes in type II string theory. Let $Z \times \mathbb{R}^2$ be a ten-dimensional space-time where $Z = \mathbb{C}^4$. Generally, $Z$ can be a toric Calabi-Yau four-fold. We denote the four complex coordinates of $\mathbb{C}^4$ as $\{z_a\}_{a \in \mathbf{4}}$ where $\mathbf{4} = \{1, 2, 3, 4\}$ (see section 3.2 for the notations). We have four types of subspaces: $\mathbb{C}$, $\mathbb{C}^2$, $\mathbb{C}^3$, $\mathbb{C}^4$. There are four possible $\mathbb{C}$ and $\mathbb{C}^3$ subspaces:

$$\mathbb{C}_a, \quad \mathbb{C}_{\bar{a}}^3 \quad (a \in \mathbf{4}), \qquad (3.2)$$

where $\bar{a}$ is the complement of $a \in \mathbf{4}$: $\bar{a} \in \{123, 124, 134, 234\}$ (see section 3.2 for details of the notations). For $\mathbb{C}^2$, there are six possible subspaces:

$$\mathbb{C}_A^2 \subset \mathbb{C}^4, \quad A \in \mathbf{6} = \{12, 13, 14, 23, 24, 34\}. \qquad (3.3)$$

Depending on which type of subspaces the D-branes wrap different gauge origami systems appear. In this paper, we are interested in the following four setups: spiked instantons, tetrahedron instantons, magnificent four, and the coupled vortex system. The spiked instanton setup is a gauge origami system whose gauge theory is described by multiple D-branes wrapping the six possible subspaces $\mathbb{C}_A^2$ ($A \in \mathbf{6}$). On the other hand, the tetrahedron instanton setup is a setup appearing on D-branes wrapping the four possible subspaces $\mathbb{C}_{\bar{a}}^3$ ($a \in \mathbf{4}$). We can also consider a system where D-branes wrap the entire $\mathbb{C}^4$ and this is the magnificent
four system. These three setups were previously studied, but for future use, we introduce a system we call the coupled vortex system. We expect such theories to be described by D-branes wrapping the four possible subspaces $\mathcal{C}_a$ ($a \in \mathbf{4}$) but a detailed analysis of this system is postponed for future work.

Let us review each system in more detail.

## Spiked instanton

The spiked instanton system comes from intersecting D-brane configurations<fnref n="4" /> [10–12, 59]:

<table><tr><td>Type IIB D(-1)-D3-<span style="text-decoration: overline">D3</span></td><td>$\mathbb{C}^4 \times \mathbb{R}^2$</td><td>rational</td></tr><tr><td>Type IIA D0-D4-<span style="text-decoration: overline">D4</span></td><td>$\mathbb{C}^4 \times \mathbb{R} \times \mathbb{S}^1$</td><td>trigonometric</td></tr><tr><td>Type IIB D1-D5-<span style="text-decoration: overline">D5</span></td><td>$\mathbb{C}^4 \times \mathbb{T}^2$</td><td>elliptic</td></tr></table>

Depending on the total space-time $\mathbb{C}^4 \times \mathcal{C}$ ($\mathcal{C} = \mathbb{R}^2, \mathbb{R} \times \mathbb{S}^1, \mathbb{T}^2$), the generalized gauge theory will be a combination of 4d ($\mathcal{C} = \mathbb{R}^2$), 5d ($\mathcal{C} = \mathbb{R} \times \mathbb{S}^1$), 6d ($\mathcal{C} = \mathbb{T}^2$) theories and the arising instanton partition function will be rational, trigonometric, and elliptic.

We focus on the type IIA setup as in table 1. To do the supersymmetric localization, the $\mathbb{C}^4$ part of the system is placed under an $\Omega$-background. We label each stack of $n_A$ D4 ($\overline{D4}$)-branes by $D4_A$ ($\overline{D4}_A$) where the subindex indicates which $\mathbb{C}_A^2$ subspace they wrap. For each stack of D4 ($\overline{D4}$)-branes wrapping $\mathbb{C}_A^2 \times \mathbb{S}^1$, we have a 5d $\mathcal{N} = 1^*$ U($n_A$) gauge theory. On the junctions of $D4_A$ and $D4_B$ defined on $\mathcal{C}_A \cap \mathcal{C}_B$ ($A \neq B \in \mathbf{6}$), the open strings connecting them give bifundamental contributions of U($n_A$) $\times$ U($n_B$). Note that for the intersecting brane configuration to preserve two supercharges, we need to include two stacks of anti D-branes in the system [11, 59]. In later sections, we will not distinguish the D4-branes and anti D4-branes.

The D0-branes in the system play the roles of instantons. There are three types of instantons in the spiked instanton system. The first type is the instantons in the $\hat{A}_0$ quiver gauge theory coming from each stack of $D4_A$ ($A \in \mathbf{6}$)-branes. The instanton contribution coming from the bifundamental multiples connecting 5d theories such as $D4_{ab}$ and $D4_{ac}$ are called folded instantons. The last type is the crossed instantons coming from two orthogonal stacks of D-branes such as $D4_{12}$ and $D4_{34}$. Namely, we have

$$ \text{spiked instantons} = \begin{cases} \text{instantons of } \hat{A}_0 \text{ quiver theory,} \\ \text{folded instantons,} \\ \text{crossed instantons.} \end{cases} \qquad (3.5) $$

Let us briefly review the generalized ADHM construction of the spiked instantons. For each gauge group we associate a vector space $\mathbf{N}_A = \mathbb{C}^{n_A}$ ($A \in \mathbf{6}$). Another vector space $\mathbf{K} = \mathbb{C}^k$ is associated to the $k$ D0-branes corresponding to the instantons. Similar to the ADHM construction, there are maps acting on $\{\mathbf{N}_A\}_{A \in \mathbf{6}}$ and $\mathbf{K}$:

$$ I_A : \mathbf{N}_A \to \mathbf{K}, \quad J_A : \mathbf{K} \to \mathbf{N}_A, \quad B_a : \mathbf{K} \to \mathbf{K} \qquad (3.6) $$

<footnote n="4">⁴Note that this is a system obtained by taking T-duality of the setup where D1-branes probe D(2p+1)-branes as explained in section 1.</footnote>
<table><caption>**Table 1.** Brane configuration of gauge origami of spiked instanton. Point-like directions of the D-branes are denoted •, while the directions where the D-branes are extending are denoted —.</caption><thead><tr><th rowspan="2"></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R × S<sup>1</sup></th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><td>D0</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><td>D4<sub>12</sub></td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><td>⟨D4<sub>13</sub></td><td>—</td><td>—</td><td>•</td><td>•</td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><td>D4<sub>14</sub></td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td><td>—</td><td>•</td><td>—</td></tr><tr><td>D4<sub>23</sub></td><td>•</td><td>•</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><td>⟨D4<sub>24</sub></td><td>•</td><td>•</td><td>—</td><td>—</td><td>•</td><td>•</td><td>—</td><td>—</td><td>•</td><td>—</td></tr><tr><td>D4<sub>34</sub></td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr></tbody></table>

where $A \in \mathbf{6}$, $a \in \mathbf{4}$. From the world-volume theory of the instantons, the maps $I_A, J_A$ are understood as open strings connecting the D4$_A$ and D0-branes. The maps $B_a$ correspond to the four transverse directions of the D0 theory which are the four complex coordinates of $\mathbb{C}^4$. We then introduce the following moment maps:

$$ 
\begin{align} 
\mu_A &= [B_a, B_b] + I_A J_A, \quad A = ab, \quad (a < b), \notag \\ 
s_A &= \mu_A + \epsilon_{A\bar{A}} \mu_{\bar{A}}^{\dagger}, \quad A \in \mathbf{6}, \tag{3.7} \\ 
\mu_R &= \sum_{a \in \mathbf{4}} [B_a, B_a^{\dagger}] + \sum_{A \in \mathbf{6}} (I_A I_A^{\dagger} - J_A^{\dagger} J_A) \notag 
\end{align} 
$$

where $\epsilon_{A\bar{A}} = \epsilon_{abcd}$ ($A = ab, \bar{A} = cd$) for $a < b, c < d$ is the total antisymmetric tensor with $\epsilon_{1234} = 1$. We also have the property $s_A^{\dagger} = \epsilon_{A\bar{A}} s_{\bar{A}}$ which implies $s_A$ is a real map giving six real conditions. Note also that $\mu_A, s_A, \mu_R \in \mathrm{Hom}(\mathbf{K}, \mathbf{K})$. The moment map equations are then given as

$$ \{s_A = 0\}/\mathrm{U}(k), \quad (A \in \mathbf{6}), \quad \{\mu_R = \zeta \cdot 1_k\}/\mathrm{U}(k), \quad (\zeta > 0) \quad (3.8) $$

where we turned on the FI parameter. We additionally have the following condition

$$ \{\sigma_{aA} = 0\}/\mathrm{U}(k), \quad \sigma_{aA} = B_a I_A + \epsilon_{abA} B_b^{\dagger} J_A^{\dagger} \quad (3.9) $$

where $a, b \in \bar{A}$, ($a \neq b$) and $\sigma_{aA} \in \mathrm{Hom}(\mathbf{N}_A, \mathbf{K})$. Note that these conditions do not appear in the original ADHM construction. They are conditions appearing only when we consider D0-branes and intersecting D4-branes. We also impose the condition

$$ \{\Upsilon_A = 0\}/\mathrm{U}(k), \quad \Upsilon_A = J_{\bar{A}} I_A - I_{\bar{A}}^{\dagger} J_A^{\dagger} : \mathbf{N}_A \to \mathbf{N}_{\bar{A}} \quad (3.10) $$

obeying $\Upsilon_A = -\Upsilon_{\bar{A}}$. This condition comes from the open strings connecting the D4$_A$ and D4$_{\bar{A}}$-branes. The instanton moduli space is then defined as$^5$

$$ \mathfrak{M}_{\vec{n},k} = \{(\vec{B}, \vec{I}, \vec{J}) \mid s_A = 0, \mu_R = \zeta \cdot 1_k, \sigma_{aA} = 0, \Upsilon_A = 0\} / \mathrm{U}(k). \quad (3.11) $$

<footnote>$^5$In order to apply the equivariant localization formalism, the real moment map equation would be replaced with the stability condition, and the quotient is then accordingly replaced with the geometric invariant theory (GIT) quotient with the automorphism group $\mathrm{GL}(\mathbf{K})$, that is a complexification of the unitary group $\mathrm{U}(k)$.</footnote>
Using the following identity [11, eq. (54)]:

$$ \sum_{A \in \mathbf{6}} \mathrm{Tr} s_A s_A^\dagger + \sum_{A \in \mathbf{6}, a \in \bar{A}} \mathrm{Tr} \sigma_{aA} \sigma_{aA}^\dagger + \sum_{A \in \mathbf{6}} \mathrm{Tr} \Upsilon_A \Upsilon_A^\dagger \\ = 2 \sum_{A \in \mathbf{6}} (\|\mu_A\|^2 + \|J_{\bar{A}} I_A\|^2) + \sum_{A \in \mathbf{6}, a \in \bar{A}} (\|B_a I_A\|^2 + \|J_A B_a\|^2) \quad (3.12) $$

the conditions (3.8), (3.9), (3.10), we have

$$ \begin{aligned} s_A &= 0 \rightarrow \mu_A = 0 \quad (A \in \mathbf{6}), \\ \sigma_{aA} &= 0 \rightarrow B_a I_A = 0, \quad J_A B_a = 0 \quad (A \in \mathbf{6}, a \in \bar{A}), \\ \Upsilon_A &= 0 \rightarrow J_{\bar{A}} I_A = 0 \quad (A \in \mathbf{6}). \end{aligned} \quad (3.13) $$

From the world-volume theory of the instantons viewpoint, the condition $\mu_R = 0$ is equivalent to the D-term condition and the conditions are equivalent to the J-term and E-term conditions.

When there is only one stack of D4-branes, say D412-branes, we have $N_A = 0$ ($A \neq 12$) and we can simply set $I_A = J_A = 0$ ($A \neq 12$). From (3.13), we have $B_3, B_4 = 0$ on the solutions, and then the moduli space indeed reduces to the standard one [11]:

$$ \left\{ (B_{12}, I_{12}, J_{12}) \left| \begin{array}{l} [B_1, B_2] + I_{12}J_{12} = 0, \\ \sum_{a=1,2} [B_a, B_a^\dagger] + (I_{12}I_{12}^\dagger - J_{12}^\dagger J_{12}) = \zeta \cdot 1_k \end{array} \right. \right\} / \mathrm{U}(k) \quad (3.14) $$

Let us discuss the symmetries of the ADHM variables $(\vec{B}, \vec{I}, \vec{J})$. We first have the $U(k)$ symmetry coming from the gauge symmetries of the D0-branes:

$$ (B_a, I_A, J_A)_{a \in \mathbf{4}, A \in \mathbf{6}} \mapsto (g^{-1}B_a g, g^{-1}I_A, J_A g), \quad g \in \mathrm{U}(k). \quad (3.15) $$

We also have the $\prod_{A \in \mathbf{6}} U(n_A)$ symmetry acting as

$$ (B_a, I_A, J_A) \mapsto (B_a, I_A h_A, h_A^{-1} J_A), \quad \underline{h} = (h_A)_{A \in \mathbf{6}} \in \prod_{A \in \mathbf{6}} U(n_A). \quad (3.16) $$

This symmetry is called the framing rotation in [11] because they are flavor symmetries from the world-volume theory of the D0-branes viewpoint. We finally have a symmetry corresponding to the spatial rotations of $\mathbb{C}^4$:

$$ (B_a, I_A, J_A) \mapsto (q_a B_a, I_A, q_A J_A) \quad (3.17) $$

where $q_a$ ($a = 1, 2, 3, 4$) and $q_A = q_a q_b$. Obviously, this symmetry preserves the conditions coming from $\mu_R$. For the conditions coming from $s_A, \sigma_{aA}, \Upsilon_A$ to have this symmetry we need

$$ q_1 q_2 q_3 q_4 = 1. \quad (3.18) $$

For example, we have $\mu_A \rightarrow q_A \mu_A$ and for $A = 12$, $s_{12} \rightarrow q_{12} \mu_{12} + q_{34}^{-1} \mu_{34}^\dagger$. Thus, with the condition (3.18), we have $s_{12} \rightarrow q_{12} s_{12}$. Other conditions can be checked similarly and the symmetry transformation of the generalized ADHM constraints are

$$ s_A \mapsto q_A g^{-1} s_A g, \quad \sigma_{aA} \mapsto q_a g^{-1} \sigma_{aA} h_A, \quad \Upsilon_A \mapsto q_A^{-1} h_A^{-1} \Upsilon_A h_A, \quad (3.19) $$

where $g \in U(k)$, $h_A \in U(n_A)$. The reason why it is called the spatial rotation is because $\{B_a\}_{a \in \mathbf{4}}$ correspond to the four complex coordinates of $\mathbb{C}^4$ and $\{q_a\}_{a \in \mathbf{4}}$ act as the maximal torus $U(1)^3$ of the group $SU(4) \subset \mathrm{Spin}(8)$ rotating the $\mathbb{C}^4$.
<table><caption>**Table 2.** Brane configuration of gauge origami of tetrahedron instanton.</caption><thead><tr><th rowspan="2"></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R × S<sup>1</sup></th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><th>D0</th><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><th>D6<sub>123</sub></th><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><th>D6<sub>124</sub></th><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>•</td><td>—</td><td>—</td><td>•</td><td>—</td></tr><tr><th>D6<sub>134</sub></th><td>—</td><td>—</td><td>•</td><td>•</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr><tr><th>D6<sub>234</sub></th><td>•</td><td>•</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr></tbody></table>

## Tetrahedron instanton

The spiked instanton system is a system where D-branes wrapping $\mathbb{C}^2 \subset \mathbb{C}^4$ appeared. Recently, the gauge origami system was generalized to stacks of D-branes wrapping $\mathbb{C}^3 \subset \mathbb{C}^4$ and is called the tetrahedron instanton system [52, 53, 81]. The intersecting D-brane configuration is given by

$$ \begin{array}{l|l|l} \text{Type IIB D(-1)-D5} & \mathbb{C}^4 \times \mathbb{R}^2 & \text{rational} \\ \text{Type IIA D0-D6} & \mathbb{C}^4 \times \mathbb{R} \times \mathbb{S}^1 & \text{trigonometric} \\ \text{Type IIB D1-D7} & \mathbb{C}^4 \times \mathbb{T}^2 & \text{elliptic} \end{array} \qquad (3.20) $$

We focus on the type IIA theory as in table 2 with the $\Omega$-background in $\mathbb{C}^4$. Similar to the spiked instanton case, we label each stack of $n_{\bar{a}}$ D6-branes as $D6_{\bar{a}}(a \in \mathbf{4})$ depending on which $\mathbb{C}^3_{\bar{a}}$ subspace they wrap. For each $D6_{\bar{a}}$-brane, we have a 7d $\mathcal{N} = 1$ U($n_{\bar{a}}$) theory. We also have bifundamental contributions coming from the junctions of $D6_{\bar{a}}$ and $D6_{\bar{b}}$, for $a \neq b$. The D0-branes play the roles of instantons and are called the tetrahedron instantons. In this case, there are only two types of instantons. The first type of instantons are the instantons on $\mathbb{C}^3$ coming from the $D6_{\bar{a}}$ theory. The second type is a folded instanton contribution coming from $D6_{\bar{a}}$ and $D6_{\bar{b}}$, where $a \neq b$. In this case, we only have this type because any two stacks of D6-branes spanning different subspaces will share 4+1 dimensions. Therefore, focusing on one of the stacks of D6-branes, other stacks of D6-branes are codimension-two defects.

Let us review the instanton moduli space of the tetrahedron instanton system discussed in [52] (see also [53, 81]). For each gauge group U($n_{\bar{a}}$) ($a \in \mathbf{4}$) coming from the $D6_{\bar{a}}$-branes, we associate a vector space $\mathbf{N}_{\bar{a}} = \mathbb{C}^{n_{\bar{a}}}$ ($a \in \mathbf{4}$). We also associate the k D0-branes with a vector space $\mathbf{K} = \mathbb{C}^k$. The open strings connecting the D0-branes give $B_a \in \text{Hom}(\mathbf{K}, \mathbf{K})$ which is the same with the spiked instanton setup. Maps $I_{\bar{a}} \in \text{Hom}(\mathbf{N}_{\bar{a}}, \mathbf{K})$ correspond to the open strings connecting the D0 and D6-branes. We then introduce the following moment maps:

$$ \begin{align} \mu_A &= [B_a, B_b], \quad A = ab \ (a < b) \notag \\ s_A &= \mu_A + \epsilon_{A\bar{A}} \mu_{\bar{A}}^{\dagger} = [B_a, B_b] + \frac{1}{2} \epsilon_{abcd} [B_c^{\dagger}, B_d^{\dagger}], \quad A = ab \tag{3.21} \\ \mu_{\mathbb{R}} &= \sum_{a \in \mathbf{4}} [B_a, B_a^{\dagger}] + \sum_{a \in \mathbf{4}} I_{\bar{a}} I_{\bar{a}}^{\dagger} \notag \end{align} $$

where all of them belong to $\text{Hom}(\mathbf{K}, \mathbf{K})$. The moment map equations are given

$$ \{\mu_{\mathbb{R}} = \zeta \cdot 1_k\}/\text{U}(k) \ (\zeta > 0), \quad \{s_A = 0\}/\text{U}(k). \qquad (3.22) $$
We also have the contributions coming from the open strings connecting the D0 and D6-branes:

$$ \{\sigma_{\bar{a}} = 0\}_{a \in \mathcal{D}} / U(k), \quad \sigma_{\bar{a}} = B_a I_{\bar{a}} \in \mathrm{Hom}(\mathbf{N}_{\bar{a}}, \mathbf{K}). \qquad (3.23) $$

The instanton moduli space of the tetrahedron instanton system is defined as

$$ \mathfrak{M}_{\vec{n},k} = \left\{ (\vec{B}, \vec{I}) \mid \mu_{\mathbb{R}} - \zeta \cdot 1_k = s_A = \sigma_{\bar{a}} = 0 \right\} / U(k). \qquad (3.24) $$

Note that similar to the spiked instanton case, using

$$ \sum_{a<b} \mathrm{Tr} [B_a, B_b][B_a, B_b]^\dagger = \frac{1}{2} \sum_{a<b} \mathrm{Tr} s_{ab} s_{ab}^\dagger, \qquad (3.25) $$

the condition $s_A = 0$ is replaced with a stronger condition $\mu_A = 0$, which coincides with the F-term condition of the world-volume theory of the D0-branes. When there is only one stack of D6-branes, say D6$_{123}$, we can set $I_{\bar{a}} = 0$ ($a \neq 4$) and indeed obtain the standard moduli space of the D0-D6 theory [41, 44, 45, 47].

We also have similar gauge symmetries of D0 and flavor symmetries coming from $D6_{\bar{a}}$. The gauge symmetry is

$$ (B_a, I_{\bar{b}})_{a,b \in \mathcal{D}} \mapsto (g^{-1} B_a g, g^{-1} I_{\bar{b}}), \quad g \in U(k), \qquad (3.26) $$

while the flavor symmetries are given

$$ (B_a, I_{\bar{b}}) \mapsto (B_a, I_{\bar{b}} h_{\bar{b}}), \quad \underline{h} = (h_{\bar{a}})_{a \in \mathcal{D}} \in \prod_{a \in \mathcal{D}} U(n_{\bar{a}}). \qquad (3.27) $$

The rotational symmetry is given as

$$ (B_a, I_{\bar{b}}) \mapsto (q_a B_a, I_{\bar{b}}). \qquad (3.28) $$

Note that the condition (3.18) comes from the invariance of $s_A = 0$. The symmetry transformation of the generalized ADHM constraints are

$$ s_A \mapsto q_A g^{-1} s_A g, \quad \sigma_{\bar{a}} \mapsto q_a g^{-1} \sigma_{\bar{a}} h_{\bar{a}} \qquad (3.29) $$

where $g \in U(k)$, $h_{\bar{a}} \in U(n_{\bar{a}})$.

## Magnificent four

Instead of considering D-branes wrapping subspaces of $\mathbb{C}^4$, we can consider D-branes wrapping the entire $\mathbb{C}^4$ [15, 16, 80]. It is called the magnificent four system. The brane configuration is given

<table><tr><td>Type IIB D(-1)-D7-D7&#773;</td><td>&#8477;<sup>4</sup> &times; &#8477;<sup>2</sup></td><td>rational</td></tr><tr><td>Type IIA D0-D8-D8&#773;</td><td>&#8477;<sup>4</sup> &times; &#8477; &times; &#8477;<sup>1</sup></td><td>trigonometric</td></tr><tr><td>Type IIB D1-D9-D9&#773;</td><td>&#8477;<sup>4</sup> &times; &#8477;<sup>2</sup></td><td>elliptic</td></tr></table>

Again, we focus on the type IIA case. The D8 and D8&#773; branes wrap the $\mathbb{C}^4 \times \mathbb{S}^1$. In order to stabilize the configuration, the D8-branes need to appear in pair with D8&#773;-branes. Considering n D8-D8&#773; branes, we get a $U(n|n)$ supergroup gauge theory. Similar to other cases, the D0-branes play the roles of $\mathbb{C}^4$-instantons.
<table><caption>**Table 3.** Brane configuration of magnificent four.</caption><thead><tr><th></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R × S<sup>1</sup></th></tr><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><th>D0</th><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><th>D8</th><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr><tr><th>D8&#773;</th><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr></tbody></table>

We denote the vector space coming from the D8-branes as $\mathbf{n} = \mathbb{C}^n$ and the vector space coming from the instantons as $\mathbf{K} = \mathbb{C}^k$. We have maps $B_a \in \mathrm{Hom}(\mathbf{K}, \mathbf{K})$ and $I \in \mathrm{Hom}(\mathbf{n}, \mathbf{K})$ corresponding to the D0-D0 and D0-D8 strings. We define the moment maps as

$$ \begin{aligned} \mu_{\mathbb{R}} &= \sum_{a \in \mathfrak{A}} [B_a, B_a^{\dagger}] + II^{\dagger}, \\ \mu_A &= [B_a, B_b], \quad A = ab \ (a < b), \\ s_{ab} &= [B_a, B_b] + \frac{1}{2} \epsilon_{abcd} [B_c^{\dagger}, B_d^{\dagger}] \end{aligned} \qquad (3.31) $$

and the instanton moduli space is defined as

$$ \mathfrak{M}_{n,k} = \{ (\vec{B}, I) \mid \mu_{\mathbb{R}} - \zeta \cdot 1_k = s_A = 0 \} / U(k). \qquad (3.32) $$

Similarly, using (3.25), the condition $s_A = 0$ is replaced with $\mu_A = 0$. The difference with the tetrahedron system is that there are no conditions coming from $\sigma_{\bar{a}} = 0$.

The symmetries of the ADHM variables and ADHM constraints are similar to the tetrahedron case:

$$ \begin{aligned} (B_a, I)_{a \in \mathfrak{A}} &\mapsto (g^{-1}B_a g, g^{-1}I), \quad g \in U(k), \\ (B_a, I) &\mapsto (B_a, Ih), \quad h \in U(n), \\ (B_a, I) &\mapsto (q_a B_a, I) \end{aligned} \qquad (3.33) $$

and

$$ s_A \mapsto q_A g^{-1} s_A g, \quad g \in U(k). \qquad (3.34) $$

## Coupled vortex system

Analogous to the setup introduced before, one would like to consider a system where D-branes wrapping $\mathbb{C} \subset \mathbb{C}^4$ appear. For the moment, we do not know what kind of system it would be but based on the previous discussions, we expect it will be a theory of intersecting 2d/3d/4d gauge theories coming from the following:

$$ \begin{array}{c|c|c} \text{Type IIB D(-1)-D1} & \mathbb{C}^4 \times \mathbb{R}^2 & \text{rational} \\ \text{Type IIA D0-D2} & \mathbb{C}^4 \times \mathbb{R} \times \mathbb{S}^1 & \text{trigonometric} \\ \text{Type IIB D1-D3} & \mathbb{C}^4 \times \mathbb{T}^2 & \text{elliptic} \end{array} \qquad (3.35) $$

We expect that vortex-like objects appear and we call it the coupled vortex system. Focusing on the D0-D2 setup, the brane set-up should look like table 4. We will have a 3d $U(n_a)$ theory on each $\mathbb{C}_a \times \mathbb{S}^1$ and there should be bifundamentals connecting such 3d theories. Note that in this case, any two stack of D2-branes $D2_a$ and $D2_b$, where $a \neq b$ will share a one-dimensional space $\mathbb{S}^1$.
For the moment, we do not know how to define the moduli space of this system. The discussion here is still a conjecture and a detailed analysis is postponed for future work (see [82, eq. (60)], [41, 83] for related works).

We denote the vector space coming from the D2-branes as $N_a = \mathbb{C}^{n_a}$ and the vector space coming from the instantons as $K = \mathbb{C}^k$. The contributions coming from the D0-D0 and D0-D2 strings give $B_a \in \text{Hom}(K, K)$ and $I_a \in \text{Hom}(N_a, K)$. We also have $J_a^{(b)} \in \text{Hom}(K, N_a)$ where $b \neq a$ corresponding to open strings in the opposite direction.

We conjecture that the D-term condition for this system is<fnref n="6" />

$$ \mu_{\mathbb{R}} = \sum_{a \in \mathbf{4}} [B_a, B_a^{\dagger}] + \sum_{a \in \mathbf{4}} I_a I_a^{\dagger} - \sum_{a \in \mathbf{4}} \sum_{b \neq a} J_a^{(b)\dagger} J_a^{(b)} \in \text{Hom}(K, K) \quad (3.36) $$

and

$$ \{\mu_{\mathbb{R}} = \zeta \cdot 1_k\}/\text{U}(k) \quad (\zeta > 0). \quad (3.37) $$

Additional contributions coming from the F-term are necessary. We conjecture that there is a condition generalizing (3.7) and (3.8):

$$ \{s_A = 0\}/\text{U}(k), \quad (3.38) $$

where

$$ s_A = \mu_A + \epsilon_{A\bar{A}} \mu_{\bar{A}}^{\dagger} \in \text{Hom}(K, K), \quad \mu_A = [B_a, B_b] + \dots. \quad (3.39) $$

The explicit form of the right-hand side of $\mu_A$ is unknown. Moreover, we expect that we have two more sets of conditions

$$ \{\sigma_{b;a} = 0\}/\text{U}(k) \quad (b \neq a), \quad \{\tilde{\sigma}_a = 0\}/\text{U}(k) \quad (3.40) $$

where $\sigma_{b;a}, \tilde{\sigma}_a \in \text{Hom}(N_a, K)$ are generalizations of (3.9). The conditions above should also be invariant under the following symmetries

$$ \begin{align} (B_a, I_a, J_a^{(b)})_{a \in \mathbf{4}, b \in \bar{a}} &\mapsto (g^{-1}B_a g, g^{-1}I_a, J_a^{(b)}g), \quad g \in \text{U}(k), \notag \\ (B_a, I_a, J_a^{(b)}) &\mapsto (B_a, I_a h_a, h_a^{-1}J_a^{(b)}), \quad \underline{h} = (h_a)_{a \in \mathbf{4}} \in \prod_{a \in \mathbf{4}} \text{U}(n_a), \tag{3.41} \\ (B_a, I_a, J_a^{(b)}) &\mapsto (q_a B_a, I_a, q_{ab} J_a^{(b)}), \notag \end{align} $$

where the $\text{U}(1)^3$ transformation of $J_a^{(b)}$ is still a conjecture. The transformation of the generalized ADHM constraints is expected to be<fnref n="7" />

$$ s_A \mapsto q_A g^{-1} s_A g, \quad \sigma_{b;a} \mapsto q_b g^{-1} \sigma_{b;a} h_a, \quad \tilde{\sigma}_a \mapsto q_a^{-1} g^{-1} \tilde{\sigma}_a h_a, \quad (3.42) $$

where $g \in \text{U}(k), h_a \in \text{U}(n_a)$.

$$ \mathcal{Z}_k^{\text{D2}} \propto \oint \prod_{I=1}^{k} \frac{dx_I}{2\pi i x_I} \prod_{a \in \mathbf{4}} \prod_{\alpha=1}^{n_a} \prod_{I=1}^{k} g_{\bar{a}\alpha} \left( \frac{v_{a,\alpha}}{x_I} \right) \prod_{I,J} A_{\mathbb{C}^4} \left( \frac{x_I}{x_J} \right)^{-1}. $$

The ADHM variables $(B_a, I_a, J_a^{(b)})$ ($b \neq a$) give the denominators

$$ B_a : \prod_{a \in \mathbf{4}} \prod_{I \neq J} (1 - q_a x_I / x_J)^{-1}, \quad I_a : \prod_{a \in \mathbf{4}} \prod_{\alpha=1}^{n_a} \prod_{I=1}^{k} (1 - v_{a,\alpha} / x_I)^{-1}, \quad J_a^{(b)} : \prod_{a \in \mathbf{4}} \prod_{b \neq a} \prod_{\alpha=1}^{n_a} \prod_{I=1}^{k} (1 - q_{ab} x_I / v_{a,\alpha})^{-1} $$

<footnote n="6">⁶Like other cases, $\mu_{\mathbb{R}}$ is a sum of the D-term conditions of the vortex moduli space after setting $J_a^{(b)} = 0$ on the moduli space. Recall that the standard vortex theory comes from the moduli space $\{(B, I) | [B, B^{\dagger}] + II^{\dagger} = \zeta \cdot 1_k\}/\text{U}(k)$ (see [82, eq. (60)]).</footnote>

<footnote n="7">⁷The charges were assigned so that we can reproduce the contour integral formula that will be proposed in section 3.6. The contour integral formula in (3.155) is rewritten as</footnote>
<table><caption>**Table 4.** Brane configuration of coupled vortex system.</caption><thead><tr><th></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R &times; S<sup>1</sup></th></tr><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><th>D0</th><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td></tr><tr><th>D2<sub>1</sub></th><td>&mdash;</td><td>&mdash;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td></tr><tr><th>D2<sub>2</sub></th><td>&bull;</td><td>&bull;</td><td>&mdash;</td><td>&mdash;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td></tr><tr><th>D2<sub>3</sub></th><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td><td>&mdash;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td></tr><tr><th>D2<sub>4</sub></th><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&mdash;</td><td>&mdash;</td><td>&bull;</td><td>&mdash;</td></tr></tbody></table>

## 3.2 Equivariant index formalism

In this section, we briefly review the equivariant index formalism and introduce the notation we use in this paper. Some of the notations were already used in the previous section. We basically follow the notations of [12]. Explicit applications to the gauge origami system introduced in the previous section will be discussed in later sections.

**Index functor.** For a vector bundle with the virtual character

$$ \mathrm{ch} \mathbf{X} = \sum_i n_i e^{x_i}, \qquad (3.43) $$

where $n_i \in \mathbb{Z}$ here is the multiplicity and $x_i$'s are the Chern roots, the index functor to convert the additive components to multiplicative components is defined as

$$ \mathbb{I}[\mathbf{X}] = \prod_i [[x_i]]^{n_i}, \qquad [[x]] = \begin{cases} x, & (4\mathrm{d}) \\ 1 - e^{-x}, & (5\mathrm{d}) \\ \theta(e^{-x}; p), & (6\mathrm{d}) \end{cases} \qquad (3.44) $$

The theta function here is defined in appendix A.2. The hierarchical structure between rational, trigonometric, and elliptic functions appears here by taking the limit as

$$ \theta(e^{-x}; p) \xrightarrow{p \to 0} 1 - e^{-x} = x + \mathcal{O}(x^2). \qquad (3.45) $$

Depending on the type $Z \times C$ where $Z$ is a toric Calabi-Yau four-fold and $C = \mathbb{C}, \mathbb{C}^{\times}, \mathbb{T}^2$, the index is obtained from (3.44).

while the ADHM constraints $s_A = 0, \sigma_{b;a} = 0, \tilde{\sigma}_a = 0$ give the numerators

$$ s_A = 0: \prod_{I<J} \prod_{A \in \mathcal{B}} (1 - q_A x_I / x_J), \quad \sigma_{b;a} = 0 (b \neq a): \prod_{I=1}^{k} \prod_{a \in \mathcal{A}} \prod_{\alpha=1}^{n_a} (1 - q_b v_{a,\alpha} / x_I), \quad \tilde{\sigma}_a = \prod_{a \in \mathcal{A}} \prod_{I=1}^{k} \prod_{\alpha=1}^{n_a} (1 - q_a^{-1} v_{a,\alpha} / x_I). $$

Including the Haar measure contribution

$$ U(k) : \prod_{I \neq J} (1 - x_I / x_J), $$

we obtain the contour integral formula. See [47, 64] for example for a review on how to relate the ADHM variables and constraints with the contour integral formula.
Most of the computations in this paper will be done explicitly using the trigonometric notation, so when not mentioned we are using the following convention:

$$ \mathbb{I}[x] = (1 - x^{-1}) = \exp \left( - \sum_{n=1}^{\infty} \frac{1}{n} x^{-n} \right) \qquad (3.46) $$

but using the formula in (3.44) one can convert the results to rational and elliptic ones. For the elliptic case, we distinguish the index as $\mathbb{I}_p[x]$ and a brief computation will be done in section 12.

Note that the character of the dual of bundle $\mathbf{X}$ is defined as

$$ \mathrm{ch} \mathbf{X}^{\vee} = \sum_{i} n_{i} e^{-x_{i}} \qquad (3.47) $$

and we have the reflection property

$$ \mathbb{I}[\mathbf{X}^{\vee}] = (-1)^{\mathrm{rk}\mathbf{X}} \det \mathbf{X} \mathbb{I}[\mathbf{X}] \qquad (3.48) $$

where $\mathrm{rk}\mathbf{X} = \sum_i n_i$ and $\det \mathbf{X} = \prod_i e^{n_i x_i}$. The index in the 5d notation can be written using the character of the anti-symmetrization,

$$ \wedge^{\bullet} \mathbf{X} = \sum_{n=0}^{\infty} (-1)^n \wedge^n \mathbf{X}, \quad \mathbb{I}[\mathbf{X}] = \mathrm{ch} \wedge^{\bullet} \mathbf{X}^{\vee}. \qquad (3.49) $$

Since $\wedge^n \mathbf{X} = 0$ for $n > \mathrm{rk}\mathbf{X}$, the summation is finite if $\mathrm{rk}\mathbf{X} < \infty$. Then, the twisted index gives rise to the characteristic polynomial

$$ \mathbb{I}[\mathbf{X}y^{\vee}] = \prod_{i} (1 - y/y_i) = \sum_{k=0}^{\infty} (-y)^k \mathrm{ch} \wedge^k \mathbf{X}^{\vee} =: \mathrm{ch} \wedge^{\bullet} \mathbf{X}, \quad \mathrm{ch} \mathbf{X} = \sum_{i} y_i. \qquad (3.50) $$

We also have the $p$-th Adams operation on $\mathbf{X}$ defined as

$$ \mathrm{ch} \mathbf{X}^{[p]} = \sum_{i} n_{i} e^{p x_{i}}. \qquad (3.51) $$

From now on, the characters and the bundles will be identified and we omit the $\mathrm{ch}$ and simply write it as

$$ \mathbf{X} = \sum_{i} n_{i} e^{x_{i}}. \qquad (3.52) $$

**Four and six.** The set of non-negative integers are denoted as $[n] = \{1, 2, \dots, n\}$ where $n \in \mathbb{N}$. Let $\underline{4}$ denote the set [4], $\underline{6}$ denote the set of 2-element subsets of $\underline{4}$, and $\underline{4}^{\vee}$ denote the set of 3-element subsets of $\underline{4}$ as

$$ \underline{4} = \{1, 2, 3, 4\}, \quad \underline{6} = \{12, 13, 14, 23, 24, 34\}, \quad \underline{4}^{\vee} = \{123, 124, 134, 234\}. \qquad (3.53) $$

The order is defined in the lexicographic order as $12 < 13 < 14 < 23 < 24 < 34$. The complement $\bar{A}$ of $A \in \underline{6}$ is defined for example as

$$ A = 12, \quad \bar{A} = 34. \qquad (3.54) $$
<figure><img src="image_9.png" /><figcaption>**Figure 6.** The four vertices of the tetrahedron correspond to the $C_a$ ($a \in \underline{\mathbf{4}}$), the six edges connecting two vertices of the tetrahedron correspond to the $C_A^2$ ($A \in \underline{\mathbf{6}}$), the four faces surrounded by three vertices and the three edges connecting them correspond to the complex 3d subspace $C_{\bar{a}}^3$ ($a \in \underline{\mathbf{4}}$), and the whole tetrahedron correspond to the $C_{\underline{\mathbf{4}}}^4$.</figcaption></figure>

Note that $\underline{\mathbf{4}} \simeq \underline{\mathbf{4}}^{\vee}$ under the map $a \in \underline{\mathbf{4}} \leftrightarrow \bar{a} \in \underline{\mathbf{4}}^{\vee}$. We introduce the set $\underline{\mathbf{3}}$ as the quotient $\underline{\mathbf{6}}/\sim$ where $A \sim \bar{A}$ is

$$ (12) \sim (34), \quad (13) \sim (24), \quad (23) \sim (14) \qquad (3.55) $$

and choose the representative as $A = a4$, $a \in [3]$. We also use

$$ A = (ab), \quad \sup(A) = b, \quad \inf(A) = a \qquad (3.56) $$

for $a < b$.

From the geometric viewpoint, the set $\underline{\mathbf{4}}$ denotes the complex dimension 1 and 3 submanifolds of the Calabi-Yau four-fold, while the set $\underline{\mathbf{6}}$ denotes the complex dimension 2 submanifolds. We can summarize the data of this in a tetrahedron (see figure 6).

**Data.** The coordinates of $C^4$ are denoted $z_i$ ($i = 1, 2, 3, 4$). The $\Omega$-deformation parameters are

$$ q_a = e^{\epsilon_a}, \quad \sum_{a \in \underline{\mathbf{4}}} \epsilon_a = 0, \qquad (3.57) $$

where we simply omit the $\mathbb{S}^1$ radius. This is the Cartan torus of SU(4), which acts on the four complex coordinates as

$$ (z_1, z_2, z_3, z_4) \mapsto (q_1 z_1, q_2 z_2, q_3 z_3, q_4 z_4) \qquad (3.58) $$

with the condition $q_1 q_2 q_3 q_4 = 1$.

We also use the notations:

$$ \mathbf{Q}_a = q_a = e^{\epsilon_a}, \quad \mathbf{P}_a = \wedge \cdot \mathbf{Q}_a = 1 - q_a, \quad \mathbf{P}_a^{\vee} = 1 - q_a^{-1}, \quad a \in \underline{\mathbf{4}} \qquad (3.59) $$

and for any subset $S \subseteq \underline{\mathbf{4}}$

$$ \mathbf{Q}_S = \prod_{a \in S} \mathbf{Q}_a, \quad \mathbf{P}_S = \prod_{a \in S} \mathbf{P}_a. \qquad (3.60) $$
For example, we have

$$ 
\begin{aligned}
\mathbf{P}_A &= (1 - q_1)(1 - q_2), \quad A = 12, \\
\mathbf{P}_{abc} &= (1 - q_a)(1 - q_b)(1 - q_c), \quad a, b, c \in \mathbf{4}.
\end{aligned} 
\qquad (3.61) 
$$

Some properties of the index are

$$ q_{\mathbf{4}} = q_0 = 1, \quad \mathbf{P}_{\mathbf{4}} = \mathbf{P}_1\mathbf{P}_2\mathbf{P}_3\mathbf{P}_4, \quad q_{\bar{S}} = q_{\bar{S}}^{-1}, \quad \mathbf{P}_{\bar{S}}^{\vee} = (-1)^{|S|} q_{\bar{S}}^{-1} \mathbf{P}_S, \qquad (3.62) $$

for any subset $S \subseteq \mathbf{4}$. Note here, we denote $\bar{S}$ as the complement of the subset $S$. For later use, we also define $\mathbf{P}_{\bar{S}} = \prod_{a \in \bar{S}} \mathbf{P}_a$.

For example, we have

$$ \mathbf{P}_{\bar{4}} = \mathbf{P}_{123} = (1 - q_1)(1 - q_2)(1 - q_3). \qquad (3.63) $$

For the subsets $S \subset \mathbf{4}$, we use the following notation

$$ S = \begin{cases} a, & a = 1, 2, 3, 4, \\ A, & A \in \mathbf{6}, \\ \bar{a}, & \bar{a} \in \{123, 124, 134, 234\} \end{cases} \qquad (3.64) $$

We can also decompose $\mathbf{P}_{\mathbf{4}}$ as

$$ \mathbf{P}_{\mathbf{4}} = \mathbf{P}_{\mathbf{3}} + \mathbf{P}_{\mathbf{3}}^{\vee}, \quad \mathbf{P}_{\mathbf{4}} = \mathbf{P}_{\mathbf{4}}^{\vee}. \qquad (3.65) $$

Using the reflection property of the index in (3.48), we obtain the reflection properties of the index of the bundles $\mathbf{P}_a$ as

$$ \mathbb{I}[\mathbf{P}_a x] = q_a^{-1} \mathbb{I}[\mathbf{P}_a^{\vee} x^{\vee}], \quad \mathbb{I}[\mathbf{P}_{ab} x] = \mathbb{I}[\mathbf{P}_{ab}^{\vee} x^{\vee}], \quad \mathbb{I}[\mathbf{P}_{abc} x] = \mathbb{I}[\mathbf{P}_{abc}^{\vee} x^{\vee}], \quad \mathbb{I}[\mathbf{P}_{\mathbf{4}} x] = \mathbb{I}[\mathbf{P}_{\mathbf{4}}^{\vee} x^{\vee}] \qquad (3.66) $$

where $a, b, c \in \mathbf{4}$ and $a \neq b, b \neq c, a \neq c$.

**Structure functions.** Let us introduce special functions which we call structure functions that will be used frequently later:<fnref n="8" />

$$ 
\begin{aligned}
\mathcal{V}_a(x) &= \mathbb{I}[-\mathbf{P}_a^{\vee} x^{\vee}] = \frac{1 - q_a x}{1 - x}, \quad a \in \mathbf{4}, \\
\mathcal{S}_{ab}(x) &= \mathbb{I}[-\mathbf{P}_{ab}^{\vee} x^{\vee}] = \frac{(1 - q_a x)(1 - q_b x)}{(1 - x)(1 - q_a q_b x)}, \quad ab \in \mathbf{6}, \\
g_{\bar{a}}(x) &= \mathbb{I}[-\mathbf{P}_{\bar{a}}^{\vee} x^{\vee}] = \frac{\prod_{i \neq a} (1 - q_i x)(1 - q_a^{-1} x)}{(1 - x) \prod_{i \neq a} (1 - q_a^{-1} q_i^{-1} x)}, \quad a \in \mathbf{4}, \\
\mathcal{A}_{\mathbb{C}^4}(x) &= \mathbb{I}[-\mathbf{P}_{\mathbf{4}}^{\vee} x^{\vee}] = \frac{\prod_{i=1}^4 (1 - q_i x) \prod_{i=1}^4 (1 - q_i^{-1} x)}{(1 - x)^2 \prod_{i \neq j} (1 - q_i q_j x)}.
\end{aligned} 
\qquad (3.67) 
$$

$$ g_{123}(x) = \frac{(1 - q_1 x)(1 - q_2 x)(1 - q_3 x)(1 - q_4^{-1} x)}{(1 - x)(1 - q_4^{-1} q_1^{-1} x)(q_4^{-1} q_2^{-1} x)(1 - q_4^{-1} q_3^{-1} x)} \xrightarrow{q_4 \to 1} \frac{(1 - q_1 x)(1 - q_2 x)(1 - q_3 x)}{(1 - q_1^{-1} x)(1 - q_2^{-1} x)(1 - q_3^{-1} x)} $$

<footnote n="8">⁸The terminology structure functions comes from an observation that if we take $\bar{a} = \bar{4} = 123$, and take the limit $q_4 \to 1$, the function $g_{\bar{a}}(x)$ will be</footnote>

<footnote>where $q_{1,2,3} \to q_{1,2,3}$ and $q_1 q_2 q_3 = 1$. It resembles the structure function of the quantum toroidal $\mathfrak{gl}_1$ (see section 9.1).</footnote>
Obviously, we have the following properties:

$$ 
\begin{aligned}
g_{abc}(x) &= \frac{S_{ab}(x)}{S_{ab}(q_c x)} = \frac{S_{bc}(x)}{S_{bc}(q_a x)} = \frac{S_{ac}(x)}{S_{ac}(q_b x)}, \\
S_{ab}(x) &= \frac{\mathcal{V}_a(x)}{\mathcal{V}_a(q_b x)} = \frac{\mathcal{V}_b(x)}{\mathcal{V}_b(q_a x)}, \quad \mathcal{A}_{\mathbb{C}^4}(x) = \frac{g_{\bar{a}}(x)}{g_{\bar{a}}(q_a x)}, \quad a, b, c \in \underline{\mathbb{Z}}.
\end{aligned} 
\quad (3.68) $$

Using (3.48), (3.62), and (3.66), we also have the following reflection formulas:

$$ 
\begin{aligned}
\mathcal{V}_a(x) &= \mathbb{I}[-\mathbf{P}_a^\vee x^\vee] = q_a \mathbb{I}[-\mathbf{P}_a x] = q_a \mathcal{V}_a(q_a^{-1}x^{-1})^{-1}, \\
S_{ab}(x) &= \mathbb{I}[-\mathbf{P}_{ab}^\vee x^\vee] = \mathbb{I}[-\mathbf{P}_{ab} x] = S_{ab}(q_a^{-1}q_b^{-1}x^{-1}) \\
g_{\bar{a}}(x) &= \mathbb{I}[-\mathbf{P}_{\bar{a}}^\vee x^\vee] = \mathbb{I}[-\mathbf{P}_{\bar{a}} x] = g_{\bar{a}}(q_a x^{-1})^{-1}, \\
\mathcal{A}_{\mathbb{C}^4}(x) &= \mathbb{I}[-\mathbf{P}_{\underline{4}}^\vee x^\vee] = \mathbb{I}[-\mathbf{P}_{\underline{4}} x] = \mathcal{A}_{\mathbb{C}^4}(x^{-1}).
\end{aligned} 
\quad (3.69) $$

**Example: $\hat{A}_0$ quiver gauge theory on $\mathbb{C}_{12}^2 \times \mathbb{S}^1$ (5d $\mathcal{N}=1^*$ theory).**

The k-instanton moduli space of U(n) gauge theory on $\mathbb{C}_{12}^2$ denoted by $\mathfrak{M}_{n,k}$ is parametrized by the topological data $(n, k)$ where k is the instanton number. The framing space and instanton space are

$$ \mathbf{N} = \mathbb{C}^n, \quad \mathbf{K} = \mathbb{C}^k \quad (3.70) $$

and the automorphism groups GL(N), GL(K) give the characters

$$ \mathbf{N} = \sum_{\alpha=1}^{n} e^{a_{\alpha}} = \sum_{\alpha=1}^{n} v_{\alpha}, \quad \mathbf{K} = \sum_{I=1}^{k} e^{\phi_I} = \sum_{I=1}^{k} x_I. \quad (3.71) $$

The observable sheaf, which is obtained from the universal sheaf via localization, is defined as

$$ \mathbf{Y} \equiv \mathbf{N} - \mathbf{P}_{12}\mathbf{K} \quad (3.72) $$

and the vector multiplet contribution, which is obtained by the tangent bundle over the instanton moduli space $T\mathfrak{M}_{n,k}$, is given as

$$ \mathbf{v}_{\text{vect.}} = \frac{\mathbf{Y}^{\vee}\mathbf{Y}}{\mathbf{P}_{12}} = \frac{\mathbf{N}^{\vee}\mathbf{N}}{\mathbf{P}_{12}} - \mathbf{N}^{\vee}\mathbf{K} - \mathbf{Q}_{12}^{\vee}\mathbf{K}^{\vee}\mathbf{N} + \mathbf{P}_{12}^{\vee}\mathbf{K}^{\vee}\mathbf{K}. \quad (3.73) $$

Without adding any other multiplets, the index of this contribution computes the partition function of the pure super Yang-Mills theory. To add an adjoint hypermultiplet with mass m, we need to add the following term to the vector multiplet contribution

$$ \mathbf{H}_{\text{adj}} = -\mathbf{M}_{\text{adj}} \frac{\mathbf{Y}^{\vee}\mathbf{Y}}{\mathbf{P}_{12}}, \quad \mathbf{M}_{\text{adj}} = e^m := \mu. \quad (3.74) $$

Then, the 5d $\mathcal{N}=1^*$ theory partition function will be given by the index of

$$ \mathbf{v} = \frac{\mathbf{Y}^{\vee}(1 - \mathbf{M}_{\text{adj}})\mathbf{Y}}{\mathbf{P}_{12}}. \quad (3.75) $$
Using the convention $q_3 = \mu^{-1}$, $q_4 = \mu q_{12}^{-1}$ obeying $q_1 q_2 q_3 q_4 = 1$, we have

$$ 
\begin{aligned}
\mathbf{v} &= \frac{\mathbf{Y}^{\vee} \mathbf{P}_3^{\vee} \mathbf{Y}}{\mathbf{P}_{12}} = \frac{\mathbf{N}^{\vee} \mathbf{P}_3^{\vee} \mathbf{N}}{\mathbf{P}_{12}} - \mathbf{P}_3^{\vee} \mathbf{N}^{\vee} \mathbf{K} - \mathbf{Q}_{12}^{\vee} \mathbf{P}_3^{\vee} \mathbf{K}^{\vee} \mathbf{N} + \mathbf{P}_{123}^{\vee} \mathbf{K}^{\vee} \mathbf{K} = \mathbf{\hat{v}} + \mathbf{v}_{\text{inst}}, \\
\mathbf{\hat{v}} &= \frac{\mathbf{N}^{\vee} \mathbf{P}_3^{\vee} \mathbf{N}}{\mathbf{P}_{12}}, \quad \mathbf{v}_{\text{inst}} = -\mathbf{P}_3^{\vee} \mathbf{N}^{\vee} \mathbf{K} - \mathbf{Q}_{12}^{\vee} \mathbf{P}_3^{\vee} \mathbf{K}^{\vee} \mathbf{N} + \mathbf{P}_{123}^{\vee} \mathbf{K}^{\vee} \mathbf{K}.
\end{aligned}
\quad (3.76) $$

The instanton partition function will be $Z_{(n,k)} = \mathbb{I}[\mathbf{v}_{\text{inst}}]$ which gives the contour integral formula.

**Proposition 3.1** (Losev-Moore-Nekrasov-Shatashvili (LMNS) formula [37, 38, 84]). The $k$-instanton partition function of 5d $\mathcal{N} = 1^*$ U(n) gauge theory defined on $\mathbb{C}^2 \times \mathbb{S}^1$ is given by the following contour integral,

$$ Z_{(n,k)} = \frac{1}{k!} \left( \frac{(1-q_{12})}{(1-q_1)(1-q_2)} S_{12}(q_3) \right)^k \times \oint \prod_{I=1}^{k} \frac{dx_I}{2\pi i x_I} \prod_{I=1}^{k} \frac{P(q_3^{-1} x_I) \tilde{P}(q_4^{-1} x_I)}{P(x_I) \tilde{P}(q_{34}^{-1} x_I)} \prod_{I \neq J} \frac{S_{12}(q_3 x_I / x_J)}{S_{12}(x_I / x_J)} \quad (3.77) $$

where $\iota = \sqrt{-1}$ and the gauge polynomials

$$ P(x) = \mathbb{I}[\mathbf{N}^{\vee} x] = \prod_{\alpha=1}^{n} \left(1 - \frac{v_{\alpha}}{x}\right), \quad \tilde{P}(x) = \mathbb{I}[x^{-1} \mathbf{N}] = \prod_{\alpha=1}^{n} \left(1 - \frac{x}{v_{\alpha}}\right). \quad (3.78) $$

This integral is taken over the Cartan torus of GL(K), which is interpreted as the Jeffrey-Kirwan residue [85] of the integrand.

The total instanton partition function is then given as

$$ Z_{\text{inst.}} = \sum_{k=0}^{\infty} q^k Z_{(n,k)}, \quad (3.79) $$

where $q^k = \exp(2k\pi i\tau)$ is the topological term with the complexified coupling constant, $\tau = \frac{\theta}{2\pi} + \frac{4\pi i}{g^2}$, with $\theta$ the theta angle and $g^2$ the gauge coupling.

The contour integral will actually localize on the fixed points classified by $n$-tuples of Young diagrams [1, 2, 8]:

$$ 
\begin{gathered}
\vec{v} = (v_{\alpha})_{\alpha=1,...,n}, \quad \vec{\lambda} = (\lambda^{(\alpha)})_{\alpha=1,...,n}, \quad |\vec{\lambda}| = \sum_{\alpha=1}^{n} |\lambda^{(\alpha)}| = k, \\
\{x_I\}_{I=1,...,k} \rightarrow \{\chi_{12,v_{\alpha}}(\square) = v_{\alpha} q_1^{i_1} q_2^{j_1}\}_{\alpha=1,...,n, \square=(i,j) \in \lambda^{(\alpha)}}.
\end{gathered}
\quad (3.80) $$

The instanton partition function is then obtained by summing the residues coming from each pole. The character of the instanton bundle K at each fixed point $\vec{\lambda}$ is given by

$$ K|_{\vec{\lambda}} = \sum_{\alpha=1}^{n} \sum_{\square \in \lambda^{(\alpha)}} \chi_{12, v_{\alpha}}(\square). \quad (3.81) $$
Inserting this in (3.76) and taking the index, we obtain the contribution to the instanton partition function of 5d $\mathcal{N} = 1^*$ theory from each fixed point $\vec{\lambda}$ as

$$Z_{12}^{\text{D4}}[\vec{v}, \vec{\lambda}; q_3] = \prod_{\alpha, \beta=1}^{n} \frac{\text{N}_{12}(q_3 v_{\alpha}, \lambda^{(\alpha)} | v_{\beta}, \lambda^{(\beta)})}{\text{N}_{12}(v_{\alpha}, \lambda^{(\alpha)} | v_{\beta}, \lambda^{(\beta)})} \quad (3.82)$$

where we defined the Nekrasov factor as

$$\text{N}_{12}(v_1, \lambda^{(1)} | v_2, \lambda^{(2)}) = \prod_{\square \in \lambda^{(1)}} \left(1 - \frac{q_{12} \chi_{12, v_1}(\square)}{v_2}\right) \prod_{\square \in \lambda^{(2)}} \left(1 - \frac{v_1}{\chi_{12, v_2}(\square)}\right) \prod_{\substack{\square \in \lambda^{(1)} \\ \bullet \in \lambda^{(2)}}} \mathcal{S}_{12} \left( \frac{\chi_{12, v_1}(\square)}{\chi_{12, v_2}(\square)} \right). \quad (3.83)$$

The instanton partition function will then be given as follows.

**Proposition 3.2** ([1, 2]). The instanton partition function of 5d $\mathcal{N} = 1^*$ U(n) gauge theory is given by summation over n-tuple partitions,

$$Z_{\text{inst.}} = \sum_{\vec{\lambda}} q^{|\vec{\lambda}|} Z_{12}^{\text{D4}}[\vec{v}, \vec{\lambda}; q_3]. \quad (3.84)$$

The index D4 comes from the fact that this theory comes from n D4-branes in the $\mathbb{C}^4$ set-up with $\Omega$-background.

**Nekrasov factors for 5d theory.** Using the quadrality in $q_a$ ($a \in \underline{4}$), we define similar Nekrasov factors<fnref n="9" /> which can be used to discuss gauge theories on $\mathbb{C}_A^2 \times \mathbb{S}^1$ ($A \in \underline{6}$):

$$\text{N}_A(v_1, \lambda^{(1)} | v_2, \lambda^{(2)}) = \prod_{\square \in \lambda^{(1)}} \left(1 - \frac{q_A \chi_{A, v_1}(\square)}{v_2}\right) \prod_{\square \in \lambda^{(2)}} \left(1 - \frac{v_1}{\chi_{A, v_2}(\square)}\right) \prod_{\substack{\square \in \lambda^{(1)} \\ \bullet \in \lambda^{(2)}}} \mathcal{S}_A \left( \frac{\chi_{A, v_1}(\square)}{\chi_{A, v_2}(\square)} \right), \quad (3.86)$$

where we defined the box content as

$$\chi_{ab,v}(\square) = v q_a^{i-1} q_b^{j-1}, \quad A = ab \in \underline{6}, \quad \square = (i,j) \in \lambda. \quad (3.87)$$

**Lemma 3.3.** The recursion formulas of the 5d Nekrasov factors are given as follows,

$$\begin{aligned} \frac{\text{N}_A(v_1, \lambda^{(1)} + \square | v_2, \lambda^{(2)})}{\text{N}_A(v_1, \lambda^{(1)} | v_2, \lambda^{(2)})} &= y_{\lambda^{(2)}, v_2}^{A \vee} (q_A \chi_{A, v_1}(\square)) \\ \frac{\text{N}_A(v_1, \lambda^{(1)} | v_2, \lambda^{(2)} + \bullet)}{\text{N}_A(v_1, \lambda^{(1)} | v_2, \lambda^{(2)})} &= y_{\lambda^{(1)}, v_1}^{A} (\chi_{A, v_2}(\square)), \end{aligned} \quad (3.88)$$

$$\text{N}_{ab}(v_1, \lambda | v_2, \nu) = \prod_{\square \in \lambda} (1 - Q q_a^{\ell_{\lambda}(\square)} q_b^{-a_{\lambda}(\square)}) \prod_{\square \in \nu} (1 - Q q_a^{-\ell_{\nu}(\square)} q_b^{a_{\lambda}(\square)+1}), \quad Q = v_1/v_2. \quad (3.85)$$

From the algebraic viewpoint, using the product form in (3.86) is much useful so we will use it rather than the simplified form in (3.85).

<footnote n="9">⁹The Nekrasov factors can be written using the arm and leg length of each box in the Young diagrams, $a_{\lambda}(\square) = \lambda_i - j$ and $\ell_{\lambda}(\square) = \lambda_j^T - i$ for $\square = (i, j)$, such as</footnote>
where we define the $\mathcal{Y}$-functions,

$$ 
\begin{aligned}
\mathcal{Y}_{\lambda,v}^{A}(x) &= \left(1 - \frac{v}{x}\right) \prod_{\square \in \lambda} S_A \left( \frac{\chi_{A,v}(\square)}{x} \right) = \frac{\prod_{\square \in A(\lambda)} (1 - \chi_{A,v}(\square)/x)}{\prod_{\square \in R(\lambda)} (1 - q_A \chi_{A,v}(\square)/x)}, \\
\mathcal{Y}_{\lambda,v}^{A \vee}(x) &= \left(1 - \frac{x}{v}\right) \prod_{\square \in \lambda} S_A \left( q_A^{-1} \frac{x}{\chi_{A,v}(\square)} \right) = \frac{\prod_{\square \in A(\lambda)} (1 - x/\chi_{A,v}(\square))}{\prod_{\square \in R(\lambda)} (1 - q_A^{-1} x/\chi_{A,v}(\square))}
\end{aligned}
\quad (3.89) $$

This $\mathcal{Y}$-functions have been introduced to describe the Seiberg-Witten curve of generic quiver gauge theory [71]. Physically, it can be understood as the contribution of the codimension four defect operator (Wilson loop along $\mathbb{S}^1$ in the current setup) under the instanton background [65]. See also section 3.8.

### 3.3 Magnificent four

Let us first consider the magnificent four setup introduced and discussed in [15, 16] (see also section 3.1). Mathematically, the partition function is interpreted as the generating function the Donaldson-Thomas invariants of the Calabi-Yau four-fold $\mathbb{C}^4$ [76, 86] and physically, it corresponds to the setup discussed in [87–90]. Let us review how to derive the instanton partition function.

The associated cotangent bundle of $\mathbb{C}^4$ is decomposed into

$$ \mathbf{Q} = \bigoplus_{i=1}^{4} \mathbf{Q}_i, \quad \mathrm{ch} \, \mathbf{Q}_i = q_i \quad (3.90) $$

where the Calabi-Yau condition is imposed $q_1q_2q_3q_4 = 1$. The framing and instanton bundles $\mathbf{N} = \mathbf{n} - \bar{\mathbf{n}}$, $\mathbf{K}$ with the characters are given by

$$ \mathbf{N} = \mathbf{n} - \bar{\mathbf{n}} = \sum_{\alpha=1}^{n} (e^{a_{\alpha}} - e^{b_{\alpha}}) = \sum_{\alpha=1}^{n} (v_{\alpha} - \bar{v}_{\alpha}), \quad \mathbf{K} = \sum_{I=1}^{k} e^{\phi_I} = \sum_{I=1}^{k} x_I. \quad (3.91) $$

Note that the contribution of $\bar{\mathbf{n}}$ in $\mathbf{N}$ corresponds to $\overline{\mathcal{D}8}$-branes, that are indispensable to stabilize the brane configuration. The observable sheaf is then defined as

$$ \mathbf{Y} = \mathbf{N} - \mathbf{P}_4\mathbf{K} \quad (3.92) $$

and the vector multiplet contribution is given as

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee}\mathbf{Y}}{\mathbf{P}_4} = \mathring{\mathbf{V}} + \mathbf{V}_{\text{inst}}, \quad \mathring{\mathbf{V}} = \frac{\mathbf{N}^{\vee}\mathbf{N}}{\mathbf{P}_4}, \quad \mathbf{V}_{\text{inst}} = -\mathbf{N}^{\vee}\mathbf{K} - \mathbf{K}^{\vee}\mathbf{N} + \mathbf{P}_{\mathbf{4}}^{\vee}\mathbf{K}^{\vee}\mathbf{K}. \quad (3.93) $$

Due to the lack of a perfect obstruction theory for the CY four-folds, we need to take the “square root” part of the total bundle $\mathbf{V}$ (see e.g., [91, 92]), which we choose

$$ \mathbf{V}_{\text{inst}} = \mathbf{v}_{\text{inst}} + \mathbf{v}_{\text{inst}}^{\vee}, \quad \mathbf{v}_{\text{inst}} = -\mathbf{N}^{\vee}\mathbf{K} + \mathbf{P}_{123}^{\vee}\mathbf{K}^{\vee}\mathbf{K} \quad (3.94) $$

where we used

$$ \mathbf{P}_4 = \mathbf{P}_{123} + \mathbf{P}_{123}^{\vee}. \quad (3.95) $$
We note that other decompositions using $\mathbf{P}_{234}$, $\mathbf{P}_{134}$, and $\mathbf{P}_{124}$ gives the same result up to sign factors [15, 16].

Following the procedure in the previous section, we have the following expression for the instanton partition function.

**Proposition 3.4** ([15, 16]). The total partition function of the magnificent four system is given by

$$Z_{\text{inst.}}^{\text{D8}} = \sum_{k=0}^{\infty} q^k Z_k^{\text{D8}}. \qquad (3.96)$$

Each contribution is given by the contour integral over the Cartan torus of $\text{GL}(\mathbf{K})$, which is interpreted as the Jeffrey-Kirwan residue of the integrand,

$$Z_k^{\text{D8}} = \mathbb{I}[\mathbf{v}_{\text{inst}}] = \frac{\mathcal{G}_4^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{I=1}^k \frac{\bar{P}(x_I)}{P(x_I)} \prod_{I \neq J} g_{\bar{I}} \left(\frac{x_I}{x_J}\right)^{-1} \qquad (3.97)$$

where we define

$$\mathcal{G}_{\bar{a}} = \frac{\prod_{i \neq a} (1 - q_a^{-1} q_i^{-1})}{(1 - q_a^{-1}) \prod_{i \neq a} (1 - q_i)}, \quad a \in \mathbf{4} \qquad (3.98)$$
$$P(x) = \prod_{\alpha=1}^{n} \left(1 - \frac{v_{\alpha}}{x}\right), \quad \bar{P}(x) = \prod_{\alpha=1}^{n} \left(1 - \frac{\bar{v}_{\alpha}}{x}\right).$$

Note that this contour integral formula is compatible with the symmetries of the moduli space given in (3.33), (3.34) (see also the discussion in footnote 7).

Actually, the poles (the equivariant fixed point in the moduli space) of the multi-integral are classified by n-tuples of solid partitions and we can rewrite the instanton partition function in a sum of solid partitions:

$$\begin{gathered} \vec{v} = (v_{\alpha})_{\alpha=1,...,n}, \quad \vec{\rho} = (\rho^{(\alpha)})_{\alpha=1,...,n}, \quad |\rho| = \sum_{\alpha=1}^{n} |\rho^{(\alpha)}| = k, \qquad (3.99) \\ \{x_I\}_{I=1,...,k} \rightarrow \{\chi_{\mathbf{4},v_{\alpha}}(\mathbb{Q}) = v_{\alpha} q_1^{j_1-1} q_2^{j_2-1} q_3^{k_3-1} q_4^{l_4-1}\}_{\alpha=1,...,n, \mathbb{Q}=(i,j,k,l)}. \end{gathered}$$

Note that the hyper cubes in the origin of the solid partitions have coordinates $v_{\alpha}$ ($\alpha = 1, ..., n$) but not $\bar{v}_{\alpha}$ ($\alpha = 1, ..., n$). This is because in the contour integral formula (3.97), only the factors coming from $P(x)$ give poles but $\bar{P}(x)$ will not. At the fixed point $\vec{\rho}$, the character of the bundle $\mathbf{K}$ will be

$$\mathbf{K}|_{\vec{\rho}} = \sum_{\alpha=1}^{n} \sum_{\mathbb{Q} \in \rho^{(\alpha)}} \chi_{\mathbf{4},v_{\alpha}}(\mathbb{Q}). \qquad (3.100)$$

Since a D8-brane will appear with a $\overline{\text{D8}}$ as a pair, it is natural to use the following notation

$$\bar{v}_{\alpha} = K_{\alpha} v_{\alpha}, \quad \alpha = 1, \dots, n, \quad \mathbf{N} = \sum_{\alpha=1}^{n} (1 - K_{\alpha}) v_{\alpha} \qquad (3.101)$$
where$^{10}$ $K_\alpha$ physically represents the distance between the D8$_\alpha$ and $\overline{D8}_\alpha$ branes. Inserting this to (3.94) and taking the index, we obtain

$$
\begin{align}
\mathcal{Z}_{\mathbf{4};4}^{\mathrm{D8}}[\vec{v}, \vec{\rho}; \vec{K}] &= \prod_{\alpha=1}^{n} \mathcal{Z}_{\mathbf{4};4}^{\mathrm{D8}}[\rho^{(\alpha)}; K_{\alpha}] \prod_{\beta>\alpha} \mathcal{Z}_{K_{\alpha}|K_{\beta}}^{\mathrm{D8-D8}}(v_{\alpha}, \rho^{(\alpha)}|v_{\beta}, \rho^{(\beta)}), \nonumber \\
\mathcal{Z}_{\mathbf{4};4}^{\mathrm{D8}}[\rho; K] &= \prod_{\underline{\mathbb{R}}' \in \rho} \frac{(1 - Kv/\chi_{\mathbf{4},v}(\underline{\mathbb{R}}))}{(1 - v/\chi_{\mathbf{4},v}(\underline{\mathbb{R}}))} \prod_{\underline{\mathbb{R}}, \underline{\mathbb{R}}' \in \rho} g_{\underline{\mathbb{R}}} \left( \frac{\chi_{\mathbf{4},v}(\underline{\mathbb{R}})}{\chi_{\mathbf{4},v}(\underline{\mathbb{R}}')} \right)^{-1}, \nonumber \\
\mathcal{Z}_{K_1|K_2}^{\mathrm{D8-D8}}(x_1, \rho^{(1)}|x_2, \rho^{(2)}) &= \prod_{\underline{\mathbb{R}}' \in \rho^{(2)}} \left( \frac{1 - K_1 x_1/\chi_{\mathbf{4},x_2}(\underline{\mathbb{R}}')}{1 - x_1/\chi_{\mathbf{4},x_2}(\underline{\mathbb{R}}')} \right) \prod_{\underline{\mathbb{R}}' \in \rho^{(1)}} \left( K_2 \frac{1 - K_2^{-1} \chi_{\mathbf{4},x_1}(\underline{\mathbb{R}})}{1 - \chi_{\mathbf{4},x_1}(\underline{\mathbb{R}})/x_2} \right) \nonumber \\
&\qquad \times \prod_{\underline{\mathbb{R}}' \in \rho^{(2)}} A_{\mathbb{C}^4} \left( \frac{\chi_{\mathbf{4},x_1}(\underline{\mathbb{R}})}{\chi_{\mathbf{4},x_2}(\underline{\mathbb{R}}')} \right)^{-1}. \tag{3.102}
\end{align}
$$

The factor $\mathcal{Z}_{\mathbf{4};4}^{\mathrm{D8}}[\rho; K]$ comes from the $U(1|1)$ gauge theory for each $\overline{D8} - \overline{D8}$ stack. The subindex 4 comes from the fact we are using $\mathbf{P}_{\bar{4}} = \mathbf{P}_{123}$ for the square root part. For later use, we introduce

$$
\mathcal{Z}_{\mathbf{4};a}^{\mathrm{D8}}[\rho; K] = \prod_{\underline{\mathbb{R}}' \in \rho} \frac{(1 - Kv/\chi_{\mathbf{4},v}(\underline{\mathbb{R}}))}{(1 - v/\chi_{\mathbf{4},v}(\underline{\mathbb{R}}))} \prod_{\underline{\mathbb{R}}, \underline{\mathbb{R}}' \in \rho} g_{\underline{\mathbb{R}}} \left( \frac{\chi_{\mathbf{4},v}(\underline{\mathbb{R}})}{\chi_{\mathbf{4},v}(\underline{\mathbb{R}}')} \right)^{-1}, \quad a \in \mathbf{4}. \qquad (3.103)
$$

The factor $\mathcal{Z}_{K_1|K_2}^{\mathrm{D8-D8}}(x_1, \rho^{(1)}|x_2, \rho^{(2)})$ comes from the open strings connecting the two different stacks $\overline{D8}_\alpha - \overline{D8}_\alpha$ and $\overline{D8}_\beta - \overline{D8}_\beta$.

**Proposition 3.5** ([15, 16]). The total instanton partition function of K-theoretic $U(n|n)$ magnificent four system is given by summation over $n$-tuple solid partitions,

$$
\mathcal{Z}_{\text{inst.}}^{\text{D8}} = \sum_{\vec{\rho} \in \mathcal{SP}} q^{\vert \vec{\rho} \vert} (-1)^{\sigma_a(\vec{\rho})} \mathcal{Z}_{\mathbf{4};a}^{\text{D8}}[\vec{v}, \vec{\rho}; \vec{K}], \quad a \in \mathbf{4} \qquad (3.104)
$$

where $\sigma_a(\vec{\rho})$ is the sign factor depending on the choice of the orientation associated with the square root bundle.

**One-loop perturbative part.** For the one-loop perturbative part, we need to take the index of the square root part of $\mathring{V}$. To do this, we first omit the singular part, specify an ordering in the parameters $\{v_\alpha\}$ and then take half of them:

$$
\begin{align}
\mathring{V} &= \frac{\mathbf{N}^{\vee}\mathbf{N}}{\mathbf{P}_{\underline{4}}} = \frac{1}{\mathbf{P}_{\underline{4}}} \sum_{\alpha, \beta} (1 - K_{\alpha}^{\vee})(1 - K_{\beta})v_{\beta}/v_{\alpha} \rightarrow \mathring{v} + \mathring{v}^{\vee}, \tag{3.105} \\
\mathring{v} &= \frac{1}{\mathbf{P}_{\underline{4}}} \sum_{v_{\beta} > v_{\alpha}} (1 - K_{\alpha}^{\vee})(1 - K_{\beta})v_{\beta}/v_{\alpha}. \nonumber
\end{align}
$$

<footnote>$^{10}$Later in section 9.1, we will see that this notation of $\{K_\alpha\}_\alpha$ is motivated from the central charge of the MacMahon representation of quantum toroidal $\mathfrak{gl}_1$.</footnote>
The index will then be

$$ \begin{aligned} \mathcal{I}[\hat{\mathbf{v}}] &= \prod_{\alpha < \beta} \mathcal{Z}_{1-\text{loop}}^{\text{D8-D8}}(v_{\alpha}, K_{\alpha} | v_{\beta}, K_{\beta}) := \mathcal{Z}_{1-\text{loop}}^{\text{D8}}, \\ \mathcal{Z}_{1-\text{loop}}^{\text{D8-D8}}(v_1, K_1 | v_2, K_2) &= \exp \left( - \sum_{n>0} \frac{1}{n} \frac{(1-K_2^{-n})(1-K_1^n)}{\mathbf{P}_{\underline{4}}^{[n]}} \left( \frac{v_1}{v_2} \right)^n \right). \end{aligned} \quad (3.106) $$

The one-loop perturbative part can be written using the q-shifted factorial or q-deformed multi-gamma functions in appendix A.1 and (A.11).

## 3.4 Tetrahedron instanton

The instanton partition function of the tetrahedron instanton was first computed in [52]. We also note that the contour integral formula of the $\mathbb{C}^3$-partition function was given in [45, 47]. Let us review the explicit formulas.

The total index of the tetrahedron instanton system resembles the magnificent four system. The different part is that the total observable sheaf $\mathbf{Y}$ will be a sum of the observable sheaves<fnref n="11" /> of $\mathbf{Y}_{\bar{a}}$ corresponding to the theory on $\mathbb{C}_{\bar{a}}^3 \times \mathbb{S}^1$:

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee} \mathbf{Y}}{\mathbf{P}_{\underline{4}}}, \quad \mathbf{Y} = \sum_{a \in \underline{4}} \mathbf{P}_a \mathbf{Y}_{\bar{a}}, \quad \mathbf{Y}_{\bar{a}} = \mathbf{N}_{\bar{a}} - \mathbf{P}_a \mathbf{K}_{\bar{a}} \quad (3.107) $$

which gives

$$ \begin{aligned} \mathbf{V} &= \mathring{\mathbf{V}} + \mathbf{V}_{\text{inst.}}, \quad \mathring{\mathbf{V}} = \sum_{a,b \in \underline{4}} \frac{\mathbf{P}_a^{\vee} \mathbf{P}_b}{\mathbf{P}_{\underline{4}}} \mathbf{N}_{\bar{a}}^{\vee} \mathbf{N}_{\bar{b}}, \\ \mathbf{V}_{\text{inst.}} &= \sum_{a,b \in \underline{4}} \left( -\mathbf{P}_b \mathbf{N}_{\bar{b}} \mathbf{K}_{\bar{a}}^{\vee} - \mathbf{P}_a^{\vee} \mathbf{N}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{b}} + \mathbf{P}_{\underline{4}} \mathbf{K}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{b}} \right), \end{aligned} \quad (3.108) $$

where

$$ \mathbf{N}_{\bar{a}} = \sum_{\alpha=1}^{n_{\bar{a}}} e^{a_{\bar{a},\alpha}} = \sum_{\alpha=1}^{n_{\bar{a}}} v_{\bar{a},\alpha}, \quad \mathbf{K}_{\bar{a}} = \sum_{I=1}^{k_{\bar{a}}} e^{\phi_{\bar{a},I}} = \sum_{I=1}^{k_{\bar{a}}} x_{\bar{a},I}, \quad a \in \underline{4}. \quad (3.109) $$

Note that the observable sheaves can be rewritten as

$$ \mathbf{Y} = \sum_{a \in \underline{4}} \mathbf{P}_a \mathbf{N}_{\bar{a}} - \mathbf{P}_{\underline{4}} \sum_{a \in \underline{4}} \mathbf{K}_{\bar{a}} = \sum_{a \in \underline{4}} \mathbf{P}_a \mathbf{N}_{\bar{a}} - \mathbf{P}_{\underline{4}} \mathbf{K}_{\bar{a}} \quad (3.110) $$

where $\mathbf{K} = \sum_{a \in \underline{4}} \mathbf{K}_{\bar{a}}$ is the vector space introduced in section 3.1. For later use, we decomposed the instanton bundle into components $\mathbf{K}_{\bar{a}}$ so that the instanton partition function after localization has a nice decomposition formula. Similar to the magnificent four setup, we need to take the square root part of $\mathbf{V}$. For the instanton part, we choose the following square root part:

$$ \mathbf{v}_{\text{inst.}} = \sum_{a \in \underline{4}} \left( -\mathbf{P}_a^{\vee} \mathbf{N}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{a}} + \frac{\mathbf{P}_{\underline{4}}}{\mathbf{P}_a^{\vee}} \mathbf{K}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{a}} \right) - \sum_{a \in \underline{4}} \sum_{b \neq a} \mathbf{P}_a^{\vee} \mathbf{N}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{b}} + \sum_{a < b} \mathbf{P}_{\underline{4}} \mathbf{K}_{\bar{a}}^{\vee} \mathbf{K}_{\bar{b}}, \quad (3.111) $$

$$ \mathbf{V}_{\text{inst.}} = \mathbf{v}_{\text{inst.}} + \mathbf{v}_{\text{inst.}}^{\vee} $$

<footnote n="11">¹¹Note here that the subindex $\bar{a}$ of $\mathbf{Y}_{\bar{a}}, \mathbf{N}_{\bar{a}}, \mathbf{K}_{\bar{a}}$ does not mean that these bundles are products of bundles with index $a$ such as $\mathbf{N}_{123} = \mathbf{N}_1 \mathbf{N}_2 \mathbf{N}_3$.</footnote>
where the first two terms give the partition function of the theory on $\mathbb{C}_{\tilde{a}}^3 \times \mathbb{S}^1$ and the last two terms give the bifundamental contributions coming from open strings connecting $\mathbb{C}_{\tilde{a}}^3 \times \mathbb{S}^1$ and $\mathbb{C}_{\tilde{b}}^3 \times \mathbb{S}^1$.

The contour integral formula is obtained by inserting (3.109) into (3.111) and taking the index.

**Proposition 3.6** ([52]). The $k$-instanton partition function of the tetrahedron instanton system is given by the contour integral,

$$ Z_{\underline{k}}^{\mathrm{D6}} = \mathbb{I}[\mathrm{v}_{\mathrm{inst.}}] = \frac{\mathcal{G}^k}{\underline{k}!} \oint \prod_{a \in \underline{4}} \prod_{I=1}^{k_{\tilde{a}}} \frac{dx_{\tilde{a},I}}{2\pi i x_{\tilde{a},I}} \prod_{a,b \in \underline{4}} \prod_{\alpha=1}^{n_{\tilde{a}}} \prod_{I=1}^{k_{\tilde{b}}} \mathcal{V}_a \left( \frac{v_{\tilde{a},\alpha}}{x_{\tilde{b},I}} \right) \\ \times \prod_{a \in \underline{4}} \prod_{I \neq J}^{k_{\tilde{a}}} g_{\tilde{a}} \left( \frac{x_{\tilde{a},J}}{x_{\tilde{a},I}} \right)^{-1} \prod_{a<b} \prod_{I=1}^{k_{\tilde{a}}} \prod_{J=1}^{k_{\tilde{b}}} \mathcal{A}_{\mathbb{C}^4} \left( \frac{x_{\tilde{a},I}}{x_{\tilde{b},J}} \right)^{-1} \quad (3.112) $$

where

$$ \underline{\mathcal{G}}^k = \prod_{a \in \underline{4}} \mathcal{G}_{\tilde{a}}^{k_{\tilde{a}}}, \quad \underline{k}! = \prod_{a \in \underline{4}} k_{\tilde{a}}!. \quad (3.113) $$

The total partition function is then given by

$$ Z_{\mathrm{inst.}}^{\mathrm{D6}} = \sum_{k=0}^{\infty} q^k \sum_{\substack{(k_{\tilde{a}})_{a \in \underline{4}}, \\ \sum_a k_{\tilde{a}} = k}} Z_{\underline{k}}^{\mathrm{D6}}. \quad (3.114) $$

We also note that this contour integral formula is compatible with the symmetries given in (3.26), (3.27), (3.28), (3.29).

The poles of the contour integral are classified by plane partitions:

$$ \begin{gathered} \underline{\vec{v}} = (\underline{\vec{v}}_{\tilde{a}})_{a \in \underline{4}} = (v_{\tilde{a},\alpha})_{a \in \underline{4}}^{\alpha=1, \dots, n_{\tilde{a}}}, \quad \underline{\vec{\pi}} = (\underline{\vec{\pi}}_{\tilde{a}})_{a \in \underline{4}} = (\pi_{\tilde{a}}^{(\alpha)})_{a \in \underline{4}}^{\alpha=1, \dots, n_{\tilde{a}}}, \quad |\underline{\vec{\pi}}| = \sum_{a \in \underline{4}} \sum_{\alpha=1}^{n_{\tilde{a}}} |\pi_{\tilde{a}}^{(\alpha)}|, \\ \{x_{\tilde{a},I}\}_{a \in \underline{4}}^{I=1, \dots, k_{\tilde{a}}} \rightarrow \{\chi_{\tilde{a},v_{\tilde{a},\alpha}}(\mathfrak{D})\}_{a \in \underline{4}, \mathfrak{D} \in \pi_{\tilde{a}}^{(\alpha)}}^{\alpha=1, \dots, n_{\tilde{a}}}, \quad \chi_{abc,v}(\mathfrak{D}) = v q_a^{I-1} q_b^{J-1} q_c^{K-1}. \end{gathered} \quad (3.115) $$

At the fixed points, the character of $K_{\tilde{a}}$ will be

$$ K_{\tilde{a}}|_{\tilde{\pi}_{\tilde{a}}} = \sum_{\alpha=1}^{n_{\tilde{a}}} \sum_{\mathfrak{D} \in \pi_{\tilde{a}}^{(\alpha)}} \chi_{\tilde{a}, v_{\tilde{a}, \alpha}}(\mathfrak{D}), \quad a \in \underline{4}. \quad (3.116) $$

Inserting this in (3.111) and taking the index, we have

$$ Z_{\mathrm{tet. inst.}}^{\mathrm{D6}}[\vec{v}, \underline{\vec{\pi}}] = \prod_{a \in \underline{4}} \prod_{\alpha=1}^{n_{\tilde{a}}} \tilde{Z}_{\tilde{a}}^{\mathrm{D6}}[\pi_{\tilde{a}}^{(\alpha)}] \prod_{a \in \underline{4}} \prod_{1 \le \alpha < \beta \le n_{\tilde{a}}} Z_{\tilde{a}|\tilde{a}}^{\mathrm{D6-D6}}(v_{\tilde{a},\alpha}, \pi_{\tilde{a}}^{(\alpha)} | v_{\tilde{a},\beta}, \pi_{\tilde{a}}^{(\beta)}) \\ \times \prod_{a<b} \prod_{\alpha=1}^{n_{\tilde{a}}} \prod_{\beta=1}^{n_{\tilde{b}}} Z_{\tilde{a}|\tilde{b}}^{\mathrm{D6-D6}}(v_{\tilde{a},\alpha}, \pi_{\tilde{a}}^{(\alpha)} | v_{\tilde{b},\beta}, \pi_{\tilde{b}}^{(\beta)}), \quad (3.117) $$
where

$$
\begin{align}
\tilde{Z}_{\tilde{a}}^{\text{D6}}[\pi] &= \prod_{\tilde{\mathfrak{a}} \in \pi} \frac{1 - q_a v / \chi_{\tilde{a},v}(\tilde{\mathfrak{a}})}{1 - v / \chi_{\tilde{a},v}(\tilde{\mathfrak{a}})} \prod_{\substack{\tilde{\mathfrak{a}} \in \pi \\ \tilde{\mathfrak{a}} \in \pi}} g_{\tilde{a}} \left( \frac{\chi_{\tilde{a},v}(\tilde{\mathfrak{a}})}{\chi_{\tilde{a},v}(\tilde{\mathfrak{a}})} \right)^{-1}, \\
\mathcal{Z}_{\tilde{a}|\tilde{b}}^{\text{D6-D6}}(v_1, \pi^{(1)}|v_2, \pi^{(2)}) &= \prod_{\tilde{\mathfrak{a}} \in \pi^{(1)}} \left( q_b \frac{1 - q_b^{-1} \chi_{\tilde{a},v_1}(\tilde{\mathfrak{a}})/v_2}{1 - \chi_{\tilde{a},v_1}(\tilde{\mathfrak{a}})/v_2} \right) \prod_{\tilde{\mathfrak{a}} \in \pi^{(2)}} \left( \frac{1 - q_a v_1 / \chi_{\tilde{b},v_2}(\tilde{\mathfrak{a}})}{1 - v_1 / \chi_{\tilde{b},v_2}(\tilde{\mathfrak{a}})} \right) \nonumber \\
&\qquad \times \prod_{\substack{\tilde{\mathfrak{a}} \in \pi^{(1)} \\ \tilde{\mathfrak{a}} \in \pi^{(2)}}} \mathcal{A}_{\mathbb{C}^4} \left( \frac{\chi_{\tilde{a},v_1}(\tilde{\mathfrak{a}})}{\chi_{\tilde{b},v_2}(\tilde{\mathfrak{a}})} \right)^{-1}. \tag{3.118}
\end{align}
$$

**Proposition 3.7.** The total instanton partition function of the tetrahedron instanton system is given by summation over plane partitions,

$$
\mathcal{Z}_{\text{inst.}}^{\text{D6}} = \sum_{\vec{\pi}} q^{\|\vec{\pi}\|} \mathcal{Z}_{\text{tetinst.}}^{\text{D6}}[\vec{v}, \vec{\pi}] \quad (3.119)
$$

We remark that, for this case, compared to the magnificent four system, we do not need
any sign factor as proved by [53]. Later we will also see this property in the algebraic formalism.

**Nekrasov factors for 7d theory.** Following the 5d Nekrasov factors in (3.86), we may
define the Nekrasov factors for the tetrahedron instanton system as

$$
\mathbb{N}_{\tilde{a}}(v_1, \pi_1 | v_2, \pi_2) = \frac{\prod_{\tilde{\mathfrak{a}} \in \pi_2} (1 - v_1 / \chi_{\tilde{a},v_2}(\tilde{\mathfrak{a}}))}{\prod_{\tilde{\mathfrak{a}} \in \pi_1} (1 - q_a^{-1} \chi_{\tilde{a},v_1}(\tilde{\mathfrak{a}})/v_2)} \prod_{\substack{\tilde{\mathfrak{a}} \in \pi_1 \\ \tilde{\mathfrak{a}} \in \pi_2}} g_{\tilde{a}} \left( \frac{\chi_{\tilde{a},v_1}(\tilde{\mathfrak{a}})}{\chi_{\tilde{a},v_2}(\tilde{\mathfrak{a}})} \right). \quad (3.120)
$$

It is a part of the following index

$$
\mathbb{I} \left[ \mathbf{N}_{\tilde{a},1}^{\vee} \mathbf{K}_{\tilde{a},2} - q_a \mathbf{K}_{\tilde{a},1}^{\vee} \mathbf{N}_{\tilde{a},2} - \mathbf{P}_{\tilde{a}}^{\vee} \mathbf{K}_{\tilde{a},1}^{\vee} \mathbf{K}_{\tilde{a},2} \right] \quad (3.121)
$$

which represents a different square root from what we used in (3.111). The square root we
use here is symmetric in the sense that we have both $\mathbf{N}_{\tilde{a},i}$ ($i=1,2$) and $\mathbf{K}_{\tilde{a},i}$ ($i=1,2$).

**Lemma 3.8.** We have the following recursion relations for the 7d Nekrasov factor:

$$
\begin{align}
\frac{\mathbb{N}_{\tilde{a}}(v_1, \pi_1 + \mathcal{D} | v_2, \pi_2)}{\mathbb{N}_{\tilde{a}}(v_1, \pi_1 | v_2, \pi_2)} &= \mathcal{W}_{\pi_2, v_2}^{\tilde{a} \vee} (q_a^{-1} \chi_{\tilde{a}, v_1}(\mathcal{D}))^{-1} \tag{3.122} \\
\frac{\mathbb{N}_{\tilde{a}}(v_1, \pi_1 | v_2, \pi_2 + \mathcal{D})}{\mathbb{N}_{\tilde{a}}(v_1, \pi_1 | v_2, \pi_2)} &= \mathcal{W}_{\pi_1, v_1}^{\tilde{a}} (\chi_{\tilde{a}, v_2}(\mathcal{D})), \nonumber
\end{align}
$$

where

$$
\mathcal{W}_{\pi, v}^{\tilde{a}}(x) = (1 - v/x) \prod_{\mathcal{D} \in \pi} g_{\tilde{a}}(\chi_{\tilde{a},v}(\mathcal{D})/x) \propto \prod_{\mathcal{D} \in A(\pi)} (1 - \chi_{\tilde{a},v}(\mathcal{D})/x) \prod_{\mathcal{D} \in R(\pi)} (1 - q_a^{-1} \chi_{\tilde{a},v}(\mathcal{D})/x) \quad (3.123)
$$

and

$$
\mathcal{W}_{\pi, v}^{\tilde{a} \vee}(x) = \left(1 - \frac{x}{v}\right) \prod_{\tilde{\mathfrak{a}} \in \pi} g_{\tilde{a}}\left(q_a \frac{x}{\chi_{\tilde{a},v}(\tilde{\mathfrak{a}})}\right)^{-1}. \quad (3.124)
$$
Moreover, using the results in appendix B.3, we have

$$ \frac{\mathbf{N}_{\bar{a}}(v, \pi + \bar{\tau} | v, \pi + \bar{\tau})}{\mathbf{N}_{\bar{a}}(v, \pi | v, \pi)} = \left( \frac{q_a v}{\chi_{\bar{a},v}(\bar{\tau})} \right) \frac{\mathrm{Res}_{x=\chi_{\bar{a},v}(\bar{\tau})} x^{-1} \mathcal{W}_{\pi+\bar{\tau},v}^{\bar{a}}(q_4^{-1}x)^{-1}}{\mathrm{Res}_{x=\chi_{\bar{a},v}(\bar{\tau})} x^{-1} \mathcal{W}_{\pi,v}^{\bar{a}}(x)^{-1}}. \quad (3.125) $$

We can define the vector U(1) contribution of the D6 theory as

$$ \mathcal{Z}_{\bar{a}}^{\mathrm{D6}}[\pi] = \frac{1}{\mathbf{N}_{\bar{a}}(v, \pi | v, \pi)} \quad (3.126) $$

which resembles the partition function of the pure SYM in the 5d theory. The two factors $\mathcal{Z}_{\bar{a}}^{\mathrm{D6}}[\pi]$ and $\tilde{\mathcal{Z}}_{\bar{a}}^{\mathrm{D6}}[\pi]$ differ by extra Chern-Simons like term and topological term

$$ \mathcal{Z}_{\bar{a}}^{\mathrm{D6}}[\pi] = \prod_{\bar{\tau} \in \pi} \left( -\frac{\chi_{\bar{a},v}(\bar{\tau})}{q_a v} \right) \tilde{\mathcal{Z}}_{\bar{a}}^{\mathrm{D6}}[\pi]. \quad (3.127) $$

The recursion relation of $\tilde{\mathcal{Z}}_{\bar{a}}^{\mathrm{D6}}[\pi]$ is then (see Thm. B.1)

$$ \frac{\tilde{\mathcal{Z}}_{\bar{a}}^{\mathrm{D6}}[\pi + \bar{\tau}]}{\tilde{\mathcal{Z}}_{\bar{a}}^{\mathrm{D6}}[\pi]} = - \frac{\mathrm{Res}_{x=\chi_{\bar{a},v}(\bar{\tau})} x^{-1} \mathcal{W}_{\pi,v}^{\bar{a}}(x)^{-1}}{\mathrm{Res}_{x=\chi_{\bar{a},v}(\bar{\tau})} x^{-1} \mathcal{W}_{\pi+\bar{\tau},v}^{\bar{a}}(q_4^{-1}x)^{-1}}. \quad (3.128) $$

**One-loop perturbative part.** Similar to the magnificent four system, we omit the singular part and choose the following square root part for the perturbative part:

$$ \dot{\mathbf{v}} = \sum_{(b,\beta)>(a,\alpha)} \frac{\mathbf{P}_a^{\vee} \mathbf{P}_b}{\mathbf{P}_4} v_{\bar{b},\beta} / v_{\bar{a},\alpha} \quad (3.129) $$

where we specified an order in the pair of indices $(\bar{a}, \alpha)_{a \in \mathbf{4}}^{\alpha=1, \dots, n_{\bar{a}}}$. Taking the index gives

$$ \begin{gathered} \mathbb{I}[\dot{\mathbf{v}}] = \prod_{(b,\beta)>(a,\alpha)} \mathcal{Z}_{1-\mathrm{loop}}^{\mathrm{D6-D6}}(v_{\bar{a},\alpha}, \bar{a} | v_{\bar{b},\beta}, \bar{b}) := \mathcal{Z}_{1-\mathrm{loop}}^{\mathrm{D6}}, \\ \mathcal{Z}_{1-\mathrm{loop}}^{\mathrm{D6-D6}}(x_1, \bar{a} | x_2, \bar{b}) = \exp \left( - \sum_{n>0} \frac{1}{n} \frac{\mathbf{P}_a^{[n]} \mathbf{P}_b^{-[n]}}{\mathbf{P}_4^{[n]}} \left( \frac{x_1}{x_2} \right)^n \right). \end{gathered} \quad (3.130) $$

For $a=b$, the one-loop factor is written using the multi $q$-shifted factorial or the $q$-deformed triple gamma function. See appendix A.1 and (A.10).

### 3.5 Spiked instanton

The spiked instanton system introduced in [11, 12] can be understood similar to the previous two setups. The total observable sheaf $\mathbf{Y}$ will be a sum of the observable sheaves of $\mathbf{Y}_A$ ($A \in \mathbf{6}$) corresponding to the theory of D4-branes on $\mathbb{C}_A^2 \times \mathbb{S}^1$:

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee} \mathbf{Y}}{\mathbf{P}_{\underline{4}}}, \quad \mathbf{Y} = \sum_{A \in \mathbf{6}} \mathbf{P}_{\bar{A}} \mathbf{Y}_A, \quad \mathbf{Y}_A = \mathbf{N}_A - \mathbf{P}_A \mathbf{K}_A, \quad A \in \mathbf{6} \quad (3.131) $$
which gives $\mathbf{V} = \mathring{\mathbf{V}} + \mathbf{V}_{\text{inst}}$ where<fnref n="12" />

$$ \mathring{\mathbf{V}} = \sum_{A,B \in \mathbf{6}} \mathbf{N}_A^{\mathring{\mathbf{V}}} \frac{\mathbf{P}_A^{\mathring{\mathbf{V}}} \mathbf{P}_{\bar{A}}}{\mathbf{P}_4} \mathbf{N}_B, \qquad \mathbf{V}_{\text{inst.}} = \sum_{A,B \in \mathbf{6}} \left( -\mathbf{N}_A^{\mathring{\mathbf{V}}} \mathbf{P}_{\bar{A}}^{\mathring{\mathbf{V}}} \mathbf{K}_B - \mathbf{K}_A^{\mathring{\mathbf{V}}} \mathbf{P}_{\bar{A}} \mathbf{N}_B + \mathbf{P}_4 \mathbf{K}_A^{\mathring{\mathbf{V}}} \mathbf{K}_B \right), \qquad (3.132) $$

and

$$ \mathbf{N}_A = \sum_{\alpha=1}^{n_A} e^{a_{A,\alpha}} = \sum_{\alpha=1}^{n_A} v_{A,\alpha}, \quad \mathbf{K}_A = \sum_{I=1}^{k_A} e^{\phi_{A,I}} = \sum_{I=1}^{k_A} x_{A,I}, \quad A \in \mathbf{6}. \qquad (3.133) $$

Note again that the total observable sheaf can be rewritten as

$$ \mathbf{Y} = \sum_{A \in \mathbf{6}} \mathbf{P}_{\bar{A}} \mathbf{N}_A - \mathbf{P}_4 \mathbf{K} \qquad (3.134) $$

where $\mathbf{K} = \sum_{A \in \mathbf{6}} \mathbf{K}_A$ is the total instanton bundle introduced in section 3.1. This is a similar situation as (3.110).

Similar to the previous cases, we actually have to extract the square root part. We choose the following decomposition

$$ \mathbf{v}_{\text{inst.}} = - \sum_{A \in \mathbf{6}} \left( \mathbf{P}_{\bar{A}}^{\mathring{\mathbf{V}}} \mathbf{N}_{A}^{\mathring{\mathbf{V}}} \mathbf{K}_{A} - \mathbf{P}_{\text{inf}(\bar{A})}^{\mathring{\mathbf{V}}} \mathbf{P}_{A}^{\mathring{\mathbf{V}}} \mathbf{K}_{A}^{\mathring{\mathbf{V}}} \mathbf{K}_{A} \right) \\ - \sum_{A \in \mathbf{6}} \sum_{B \neq A} \mathbf{N}_{A}^{\mathring{\mathbf{V}}} \mathbf{P}_{\bar{A}}^{\mathring{\mathbf{V}}} \mathbf{K}_{B} + \sum_{A < B} \mathbf{P}_{4} \mathbf{K}_{A}^{\mathring{\mathbf{V}}} \mathbf{K}_{B} \qquad (3.135) $$

where the first line gives the contribution of the affine quiver gauge theory with adjoint mass $q_{\text{inf}(\bar{A})}$ on each $\mathbb{C}_A^2 \times \mathbb{S}^1$ and the second lines gives the folded and crossed instantons contributions.

**Proposition 3.9.** The $\underline{k}$-instanton partition function of the spiked instanton system is given by the following contour integral,

$$ Z_{\underline{k}}^{\text{D4}} = \mathbb{I}[\mathbf{v}_{\text{inst.}}] = \frac{\mathcal{G}^{\underline{k}}}{\underline{k}!} \oint \prod_{A \in \mathbf{6}} \prod_{I=1}^{k_A} \frac{dx_{A,I}}{2\pi i x_{A,I}} \prod_{A \in \mathbf{6}} \prod_{\alpha=1}^{n_A} \prod_{I=1}^{k_A} S_{\bar{A}}\left(\frac{v_{A,\alpha}}{x_{A,I}}\right) \prod_{A \in \mathbf{6}} \prod_{I \neq J} g_{\sup(\bar{A})}\left(\frac{x_{A,I}}{x_{A,J}}\right)^{-1} \\ \times \prod_{A \in \mathbf{6}} \prod_{B \neq A} \prod_{\alpha=1}^{n_A} \prod_{I=1}^{k_B} S_{\bar{A}}\left(\frac{v_{A,\alpha}}{x_{B,I}}\right) \prod_{A < B} \prod_{I=1}^{k_A} \prod_{J=1}^{k_B} A_{\mathbb{C}^4}\left(\frac{x_{A,I}}{x_{B,J}}\right)^{-1} \qquad (3.136) $$

where we define

$$ \underline{\mathcal{G}}^{\underline{k}} = \prod_{A \in \mathbf{6}} \mathcal{G}_{\sup(\bar{A})}^{k_A}, \quad \underline{k}! = \prod_{A \in \mathbf{6}} k_A!. \qquad (3.137) $$

The total instanton partition function is

$$ Z_{\text{inst.}}^{\text{D4}} = \sum_{k=0}^{\infty} q^k \sum_{\substack{(k_A)_{A \in \mathbf{6}} \\ \sum_A k_A = k}} Z_{\underline{k}}^{\text{D4}}. \qquad (3.138) $$

<footnote n="12">¹²One of the defining conditions of the spiked instanton moduli space {$\Upsilon_A = 0$}$_{A \in \mathbf{6}}$ gives rise to the contribution $\sum_{A \in \mathbf{6}} q_A \mathbf{N}_A^{\mathring{\mathbf{V}}} \mathbf{N}_{\bar{A}}$, which is now incorporated in the perturbative part.</footnote>
The contour integral formula is compatible with the symmetries given in (3.15), (3.16), (3.17), (3.19).

Note here that the index in (3.135) is slightly different from the one used in (3.76). The term we used is $\mathbf{P}_{\bar{A}}^{\vee}\mathbf{N}_{A}^{\vee}\mathbf{K}_{A}$ while a straightforward generalization of (3.76) will be

$$ \mathbf{P}_{\inf(\bar{A})}^{\vee} (\mathbf{N}_{A}^{\vee}\mathbf{K}_{A} + q_{A}^{-1}\mathbf{K}_{A}^{\vee}\mathbf{N}_{A}). \qquad (3.139) $$

After taking the index, the contour integral formula differs with (3.77):

$$ \prod_{A \in \mathbf{6}} \prod_{\alpha=1}^{n_A} \prod_{I=1}^{k_A} S_{\bar{A}} \left( \frac{v_{A,\alpha}}{x_{A,I}} \right) = \prod_{A \in \mathbf{6}} q_{\inf(\bar{A})}^{-n_A k_A} \prod_{A \in \mathbf{6}} \prod_{\alpha=1}^{n_A} \prod_{I=1}^{k_A} \mathcal{V}_{\inf(\bar{A})} \left( \frac{v_{A,\alpha}}{x_{A,I}} \right) \mathcal{V}_{\inf(\bar{A})} \left( \frac{q_A x_{A,I}}{v_{A,\alpha}} \right). \qquad (3.140) $$

This overall factor will be eventually related to the topological term for each affine quiver gauge theory. It seems that using the index in (3.135) is natural from the quantum algebraic viewpoint so we will use this. Note that in the limit $S^1 \to pt$, such terms will disappear and have no effect.

The poles of the contour integral (3.136) are classified by 2d partitions:

$$ \begin{gathered} \vec{v} = (v_{A,\alpha})_{A \in \mathbf{6}}^{\alpha=1,\dots,n_A}, \quad \vec{\lambda} = (\vec{\lambda}_A)_{A \in \mathbf{6}} = (\lambda_A^{(\alpha)})_{A \in \mathbf{6}}^{\alpha=1,\dots,n_A}, \quad |\vec{\lambda}| = \sum_{A \in \mathbf{6}} \sum_{\alpha=1}^{n_A} |\lambda_A^{(\alpha)}|, \qquad (3.141) \\ \{x_{A,I}\}_{A \in \mathbf{6}}^{I=1,\dots,k_A} \to \{\chi_{A,v_{A,\alpha}}(\Box)\}_{A \in \mathbf{6}, \Box \in \lambda_{A,\alpha}}^{\alpha=1,\dots,n_A}, \quad \chi_{ab,v(\Box)} = v q_a^{i-1} q_b^{j-1}. \end{gathered} $$

At the fixed points, the character of $\mathbf{K}_A$ will be

$$ \mathbf{K}_A|_{\vec{\lambda}_A} = \sum_{\alpha=1}^{n_A} \sum_{\Box \in \lambda_A^{(\alpha)}} \chi_{A, v_{A,\alpha}}(\Box), \quad A \in \mathbf{6}. \qquad (3.142) $$

Inserting this and taking the index, we have

$$ \begin{aligned} \mathcal{Z}_{\text{spkinst.}}^{\text{D4}}[\vec{v}, \vec{\lambda}] = & \prod_{A \in \mathbf{6}} \prod_{\alpha=1}^{n_A} \tilde{\mathcal{Z}}_{A}^{\text{D4}}[\lambda_A^{(\alpha)}] \prod_{A \in \mathbf{6}} \prod_{\alpha < \beta} \mathcal{Z}_{A|A}^{\text{D4-D4}}(v_{A,\alpha}, \lambda_A^{(\alpha)} | v_{A,\beta}, \lambda_A^{(\beta)}) \\ & \times \prod_{A < B} \prod_{\alpha=1}^{n_A} \prod_{\beta=1}^{n_B} \mathcal{Z}_{A|B}^{\text{D4-D4}}(v_{A,\alpha}, \lambda_A^{(\alpha)} | v_{B,\beta}, \lambda_B^{(\beta)}), \end{aligned} \qquad (3.143) $$

where

$$ \begin{gathered} \tilde{\mathcal{Z}}_{A}^{\text{D4}}[\lambda] = q_{\text{inf}(\bar{A})}^{-|\lambda|} \mathcal{Z}_{A}^{\text{D4}}[\lambda; q_{\text{inf}(\bar{A})}] = q_{\text{inf}(\bar{A})}^{-|\lambda|} \frac{N_A(q_{\text{inf}(\bar{A})}v, \lambda|v, \lambda)}{N_A(v, \lambda|v, \lambda)}, \\ \mathcal{Z}_{A|B}^{\text{D4-D4}}(v_1, \lambda^{(1)}|v_2, \lambda^{(2)}) = \prod_{\Box \in \lambda^{(1)}} S_{\bar{B}}\left(q_B \frac{\chi_{A,v_1}(\Box)}{v_2}\right) \prod_{\substack{\bullet \in \lambda^{(2)} \\ \Box \in \lambda^{(1)}}} S_{\bar{A}}\left(\frac{v_1}{\chi_{B,v_2}(\Box)}\right) \prod_{\substack{\bullet \in \lambda^{(2)} \\ \Box \in \lambda^{(1)}}} \mathcal{A}_{\mathbb{C}^4} \left(\frac{\chi_{A,v_1}(\Box)}{\chi_{B,v_2}(\Box)}\right)^{-1}. \end{gathered} \qquad (3.144) $$

The property and low levels of the U(1) contribution are given in appendix B.4.

**Proposition 3.10.** The total instanton partition function of the spiked instanton system is given by summation over 2d partitions,

$$ \mathcal{Z}_{\text{inst.}}^{\text{D4}} = \sum_{\vec{\lambda}} q^{|\vec{\lambda}|} \mathcal{Z}_{\text{spkinst.}}^{\text{D4}}[\vec{v}, \vec{\lambda}]. \qquad (3.145) $$
**One-loop perturbative part.** Omitting the singular part, we use the following square root part for the one-loop perturbative part:

$$ \dot{\mathbf{v}} = \sum_{(B,\beta)>(A,\alpha)} \frac{\mathbf{P}_A^{\vee} \mathbf{P}_{\bar{B}}}{\mathbf{P}_4} v_{B,\beta}/v_{A,\alpha}, \qquad (3.146) $$

where we specified an order in $(A, \alpha)_{A \in \mathbf{6}}^{\alpha=1, \dots, n_A}$. Taking the index, we have

$$ \begin{gathered} \mathbb{I}[\dot{\mathbf{v}}] = \prod_{(B,\beta)>(A,\alpha)} \mathcal{Z}_{1-\text{loop}}^{\text{D4-D4}}(v_{A,\alpha}, A | v_{B,\beta}, B) =: \mathcal{Z}_{1-\text{loop}}^{\text{D4}}, \\ \mathcal{Z}_{1-\text{loop}}^{\text{D4-D4}}(x_1, A | x_2, B) = \exp \left( - \sum_{n=1}^{\infty} \frac{1}{n} \frac{\mathbf{P}_{\bar{A}}^{[n]} \mathbf{P}_{B}^{-[n]}}{\mathbf{P}_{\bar{4}}^{[n]}} \left( \frac{x_1}{x_2} \right)^n \right). \end{gathered} \qquad (3.147) $$

For $a=b$, the one-loop factor is written using the multi $q$-shifted factorial or the $q$-deformed double gamma function. See appendix A.1 and (A.9).

**Supergroup analogue.** Actually, the contour integral formulas and instanton partition functions for the 5d gauge theories have a supergroup generalization introduced in [93] (see also [94–98] for related works). The explicit contour integral formula for the 5d $\mathcal{N} = 1^*$ U($n_+$|$n_-$) gauge theory is given in appendix D and (D.3). The strategy to derive it is to change all of the characters of $\mathbf{N}$, $\mathbf{K}$ to supercharacters:

$$ \mathrm{sch} \mathbf{N} = \mathrm{ch} \mathbf{N}^{+} - \mathrm{ch} \mathbf{N}^{-}, \quad \mathrm{sch} \mathbf{K} = \mathrm{ch} \mathbf{K}^{+} - \mathrm{ch} \mathbf{K}^{-}. \qquad (3.148) $$

This generalization gives extra vector-like and bifundamental-like contributions which enables us to understand the supergroup gauge theory as a type of quiver gauge theory (see [93, 96, 98]). To obtain the supergroup gauge theory using D-branes, one needs to include ghost/negative D4-branes, denoted as $D4^{-}$, to the system additional to the normal<fnref n="13" /> D4-branes, which we denote $D4^{+}$. For such cases, $D0^{+}$-branes play the role of positive instantons while $D0^{-}$-branes play the role of negative instantons (see [97]).

After changing all the characters to supercharacters, one can obtain the supergroup analog of the gauge origami of the spiked instanton system. For the simplest case, where there are only D-branes on $\mathbb{C}_{12}^2 \times \mathbb{S}^1$, see (D.3) for the contour integral formula. See for example [93] or [99, section 2] for the explicit formulas for general quiver gauge theories. The generalization to the gauge origami system is straightforward so we omit it.

## 3.6 Coupled vortex system

Following the previous sections, it is natural to consider a theory where D2-branes intersect. We propose a D2-analogue partition function of the gauge origami system. The total index is written similarly with $\mathbf{Y}$, but this time it is a sum of $\mathbf{Y}_a$ ($a \in \mathbf{4}$):

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee} \mathbf{Y}}{\mathbf{P}_{\bar{4}}} \quad \mathbf{Y} = \sum_{a \in \mathbf{4}} \mathbf{P}_{\bar{a}} \mathbf{Y}_a, \quad \mathbf{Y}_a = \mathbf{N}_a - \mathbf{P}_a \mathbf{K}_a, \qquad (3.149) $$

<footnote n="13">¹³We also call it positive D-branes.</footnote>
which gives

$$ 
\begin{aligned}
\mathbf{V} &= \mathring{\mathbf{V}} + \mathbf{V}_{\text{inst}}, \quad \mathring{\mathbf{V}} = \sum_{a,b \in \mathbf{4}} \mathbf{N}_a^{\vee} \frac{\mathbf{P}_a^{\vee} \mathbf{P}_b}{\mathbf{P}_{\mathbf{4}}} \mathbf{N}_b \\
\mathbf{V}_{\text{inst.}} &= \sum_{a,b \in \mathbf{4}} (-\mathbf{N}_a^{\vee} \mathbf{P}_a^{\vee} \mathbf{K}_b - \mathbf{K}_a^{\vee} \mathbf{P}_b^{\vee} \mathbf{N}_b + \mathbf{P}_{\mathbf{4}} \mathbf{K}_a^{\vee} \mathbf{K}_b)
\end{aligned}
\quad (3.150) $$

where

$$ \mathbf{N}_a = \sum_{\alpha=1}^{n_a} e^{a_{a,\alpha}} = \sum_{\alpha=1}^{n_a} v_{a,\alpha}, \quad \mathbf{K}_a = \sum_{I=1}^{k_a} e^{\phi_{a,I}} = \sum_{I=1}^{k_a} x_{a,I}. \quad (3.151) $$

Similar to the tetrahedron instanton (3.110) and spiked instanton (3.134) cases, the total observable sheaf can be rewritten using the total instanton bundle

$$ \mathbf{Y} = \sum_{a \in \mathbf{4}} \mathbf{P}_{\bar{a}} \mathbf{N}_a - \mathbf{P}_{\mathbf{4}} \mathbf{K} \quad (3.152) $$

where $\mathbf{K} = \sum_{a \in \mathbf{4}} \mathbf{K}_a$.

Mimicking the other cases, we decompose the instanton part as $\mathbf{V}_{\text{inst}} = \mathbf{v}_{\text{inst}} + \mathbf{v}_{\text{inst}}^{\vee}$ and choose the square root part as

$$ 
\begin{aligned}
\mathbf{v}_{\text{inst}} = & - \sum_{a \in \mathbf{4}} \mathbf{P}_{i(a)j(a)}^{\vee} (\mathbf{N}_a^{\vee} \mathbf{K}_a - q_a^{-1} \mathbf{N}_a \mathbf{K}_a^{\vee} - \mathbf{P}_a^{\vee} \mathbf{K}_a^{\vee} \mathbf{K}_a) \\
& - \sum_{a \in \mathbf{4}} \sum_{b \neq a} \mathbf{N}_a^{\vee} \mathbf{P}_{\bar{a}}^{\vee} \mathbf{K}_b + \sum_{a<b} \mathbf{P}_{\mathbf{4}} \mathbf{K}_a^{\vee} \mathbf{K}_b.
\end{aligned}
\quad (3.153) $$

Here, we denote the three indices in $\mathbf{4} \setminus \{a\}$ as $i(a), j(a), k(a)$ and chose two of them for each $a \in \mathbf{4}$ to define the index. The first line gives the 3d theory on each plane $\mathcal{C}_a \times \mathbb{S}^1$ and the other terms give the bifundamental interactions between the 3d theories.

**Proposition 3.11.** The total instanton partition function of the coupled vortex system is given by

$$ Z_{\text{inst.}}^{\text{D2}} = \sum_{k=0}^{\infty} q^k \sum_{\sum_{a} k_a = k} Z_{\underline{k}}^{\text{D2}}, \quad (3.154) $$

where each contribution is given by the following contour integral,

$$ Z_{\underline{k}}^{\text{D2}} = \mathbb{I}[\mathbf{v}_{\text{inst.}}] = \frac{\mathcal{G}^k}{\underline{k}!} \oint \prod_{a \in \mathbf{4}} \prod_{I=1}^{k_a} \frac{dx_{a,I}}{2\pi i x_{a,I}} \prod_{a,b \in \mathbf{4}} \prod_{\alpha=1}^{n_a} \prod_{I=1}^{k_b} g_{\bar{a}} \left( \frac{v_{a,\alpha}}{x_{b,I}} \right) \prod_{I \neq J} g_{a i(a)j(a)} \left( \frac{x_{a,I}}{x_{a,J}} \right)^{-1} \\ \times \prod_{a<b} \prod_{I=1}^{k_a} \prod_{J=1}^{k_b} \mathcal{A}_{\mathbb{C}^4} \left( \frac{x_{a,I}}{x_{b,J}} \right)^{-1} \quad (3.155) $$

with

$$ \underline{\mathcal{G}}^k = \prod_{a \in \mathbf{4}} \mathcal{G}_{a i(a)j(a)}^{k_a}, \quad \underline{k}! = \prod_{a \in \mathbf{4}} k_a. \quad (3.156) $$
For simplicity, let us consider the case of a U(n) theory on $\mathbb{C}_4 \times \mathbb{S}^1$. We choose $a = 4$, $i(4) = q_2$, $j(4) = q_3$ to be the masses and then we obtain

$$ 
\begin{aligned}
Z_k^{\text{D2}} &= \frac{G_1^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{\alpha=1}^n \prod_{I=1}^k g_4 \left(\frac{v_\alpha}{x_I}\right) \prod_{I \neq J} g_1 \left(\frac{x_I}{x_J}\right)^{-1}, \\
&= \frac{G_1^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{\alpha=1}^n \prod_{I=1}^k \frac{(1 - q_4^{-1} v_\alpha / x_I) \prod_{i=1}^3 (1 - q_i v_\alpha / x_I)}{(1 - v_\alpha / x_I) \prod_{i=1}^3 (1 - q_4^{-1} q_i^{-1} v_\alpha / x_I)} \\
&\quad \times \prod_{I \neq J} \frac{(1 - x_I / x_J) \prod_{i=2}^4 (1 - q_i^{-1} q_1^{-1} x_I / x_J)}{(1 - q_1^{-1} x_I / x_J) \prod_{i=2}^4 (1 - q_i x_I / x_J)}
\end{aligned}
\quad (3.157) $$

The pole structure implies that the fixed points should be classified by the one-dimensional partitions in section 2. Looking at the $g_4(x)$ part, the pole at $x_I = v_\alpha$ starts the growth of the one-dimensional partition, while the zero at $x_I = q_4^{-1}v_\alpha$ terminates the growth in the opposite direction. The $g_1(x)^{-1}$ part is decomposed into

$$ \frac{(1 - x_I / x_J)}{(1 - q_4 x_I / x_J)} \times \frac{(1 - q_4 q_2 x_I / x_J)(1 - q_4 q_3 x_I / x_J)(1 - q_1^{-1} q_4^{-1} x_I / x_J)}{(1 - q_2 x_I / x_J)(1 - q_3 x_I / x_J)(1 - q_1^{-1} x_I / x_J)} \quad (3.158) $$

where the first term gives the contribution of the $\mathcal{N} = 2$ vector multiplet and the second term gives the contributions of the three $\mathcal{N} = 2$ adjoint chiral multiples with adjoint masses $q_1, q_2, q_3$ (in the multiplicative notation).

Generally, we assume that the fixed points are classified by one-dimensional partitions labeled by $\mathbb{Z}_{\ge 0}$ as the vortex partition function [41, 82, 100–102]:

$$ 
\begin{gathered}
\vec{v} = (v_{a,\alpha})_{a \in \mathfrak{q}}^{\alpha=1,\dots,n_a}, \quad \vec{k} = (\vec{k}_a)_{a \in \mathfrak{q}} = (k_a^{(\alpha)})_{a \in \mathfrak{q}}^{\alpha=1,\dots,n_a}, \quad |\vec{k}| = \sum_{a \in \mathfrak{q}} \sum_{\alpha=1}^{n_a} k_a^{(\alpha)}, \quad (3.159) \\
\{x_{a,I}\}_{a \in \mathfrak{q}}^{I=1,\dots,k_a} \rightarrow \{\chi_{a,v_{a,\alpha}}(\Box)\}_{a \in \mathfrak{q}}^{\alpha=1,\dots,n_a}, \quad \chi_{a,v_{a,\alpha}}(\Box) = v_{a,\alpha} q_a^{I-1} \quad (I=1,\dots,k_a^{(\alpha)}).
\end{gathered}
$$

The character of $K_a$ at the fixed points will then be

$$ K_a|_{\vec{k}_a} = \sum_{\alpha=1}^{n_a} \sum_{\Box \in k_a^{(\alpha)}} \chi_{a,v_{a,\alpha}}(\Box) = \sum_{\alpha=1}^{n_a} v_{a,\alpha} \frac{1 - q_a^{k_a^{(\alpha)}}}{1 - q_a}. \quad (3.160) $$

To evaluate the partition function, we need to insert this and take the index. Before doing that let us introduce the D2-version of the Nekrasov factors.

**Nekrasov factors for 3d theory.** We define the D2-version of the Nekrasov factors as

$$ N_a(v_1, k_1 | v_2, k_2) = \frac{\prod_{\Box \in k_2} (1 - v_1 / \chi_{a,v_2}(\Box))}{\prod_{\Box \in k_1} (1 - q_a \chi_{a,v_1}(\Box) / v_2) \prod_{\Box \in k_2} \chi_a \left( \frac{\chi_{a,v_1}(\Box)}{\chi_{a,v_2}(\Box)} \right)}. \quad (3.161) $$

**Lemma 3.12.** The following recursion formulas hold for the 3d Nekrasov factors,

$$ \frac{\mathcal{N}_a(v_1, k_1 + \Box | v_2, k_2)}{\mathcal{N}_a(v_1, k_1 | v_2, k_2)} = \mathcal{U}_{k_2, v_2}^{q\chi} (q_a \chi_{a, v_1}(\Box))^{-1}, \quad \frac{\mathcal{N}_a(v_1, k_1 | v_2, k_2 + \Box)}{\mathcal{N}_a(v_1, k_1 | v_2, k_2)} = \mathcal{U}_{k_1, v_1}^{a} (\chi_{a, v_2}(\Box)) \quad (3.162) $$
with

$$
\begin{align}
U_{k,v}^{a}(x) &= \left(1 - \frac{v}{x}\right) \prod_{\square \in k} \nu_a \left(\frac{\chi_{a,v}(\square)}{x}\right) = \left(1 - \frac{vq_a^k}{x}\right), \tag{3.163} \\
U_{k,v}^{a\vee}(x) &= \left(1 - \frac{x}{v}\right) \prod_{\square \in k} \nu_a \left(q_a^{-1} \frac{x}{\chi_{a,v}(\square)}\right)^{-1} = \left(1 - \frac{x}{vq_a^k}\right) \nonumber
\end{align}
$$

where note that $k + \square = k + 1$.

We note that the U(1) contributions coming from $\mathcal{N}_a(v, k|v, k)$ is trivial (see appendix B.5):

$$
\mathcal{N}_a(v, k | v, k) = 1. \tag{3.164}
$$

**Proposition 3.13.** The coupled vortex partition function is given as follows,

$$
\mathcal{Z}_{\text{vort.}}^{\text{D2}} = \sum_{\vec{k}} q^{|\vec{k}|} \mathcal{Z}_{\text{cpl.vort.}}^{\text{D2}}[\vec{v}, \vec{k}], \quad (3.165)
$$

where

$$
\begin{align*}
\mathcal{Z}_{\text{cpl.vort.}}^{\text{D2}}[\vec{v}, \vec{k}] &= \prod_{a \in \mathbf{4}} \prod_{\alpha=1}^{n_a} \mathcal{Z}_a^{\text{D2}}[k_a^{(\alpha)}; q_{i(a)}, q_{j(a)}] \prod_{a \in \mathbf{4}} \prod_{\alpha<\beta} \mathcal{Z}_a^{\text{D2-D2}}(v_{a,\alpha}, k_a^{(\alpha)} | v_{a,\beta}, k_a^{(\beta)}) \\
&\quad \times \prod_{a<b} \prod_{\alpha=1}^{n_a} \prod_{\beta=1}^{n_b} \mathcal{Z}_{a|b}^{\text{D2-D2}}(v_{a,\alpha}, k_a^{(\alpha)} | v_{b,\beta}, k_b^{(\beta)}),
\end{align*}
$$

$$
\mathcal{Z}_a^{\text{D2}}[k; q_i, q_j] = \frac{\mathcal{N}_a(q_i v, k | v, k) \mathcal{N}_a(q_j v, k | v, k)}{\mathcal{N}_a(v, k | v, k) \mathcal{N}_a(q_i q_j v, k | v, k)}, \quad a \in \mathbf{4}, i, j \in \mathbf{4} \setminus \{a\},
$$

$$
\mathcal{Z}_{a|b}^{\text{D2-D2}}(v_1, k_1 | v_2, k_2) = \prod_{\blacksquare \in k_2} g_{\blacksquare} \left( \frac{v_1}{\chi_{b,v_2}(\blacksquare)} \right) \prod_{\square \in k_1} g_{\square} \left( \frac{q_b \chi_{a,v_1}(\square)}{v_2} \right)^{-1} \prod_{\blacksquare \in k_1} A_{\mathcal{C}4} \left( \frac{\chi_{a,v_1}(\square)}{\chi_{b,v_2}(\blacksquare)} \right)^{-1} \prod_{\blacksquare \in k_2} A_{\mathcal{C}4} \left( \frac{\chi_{a,v_2}(\square)}{\chi_{b,v_1}(\blacksquare)} \right)^{-1} . \tag{3.166}
$$

**One-loop perturbative part.** We choose the following square root part of the perturbative part $\mathring{\mathbf{V}}$ as

$$
\mathring{\mathbf{v}} = \sum_{(b,\beta) > (a,\alpha)} \frac{\mathbf{P}_{\bar{a}}^{\vee} \mathbf{P}_{\bar{b}}}{\mathbf{P}_{\mathbf{4}}} v_{b,\beta} / v_{a,\alpha} \quad (3.167)
$$

which gives

$$
\mathbb{I}[\mathring{\mathbf{v}}] = \prod_{(b,\beta) > (a,\alpha)} \mathcal{Z}_{1-\text{loop}}^{\text{D2-D2}}(v_{a,\alpha}, a | v_{b,\beta}, b) =: \mathcal{Z}_{1-\text{loop}}^{\text{D2}}, \quad (3.168)
$$

$$
\mathcal{Z}_{1-\text{loop}}^{\text{D2-D2}}(x, a | x', b) = \exp \left( - \sum_{n=1}^{\infty} \frac{1}{n} \frac{\mathbf{P}_{\bar{a}}^{[n]} \mathbf{P}_{\bar{b}}^{[-n]}}{\mathbf{P}_{\mathbf{4}}^{[n]}} \left( \frac{x}{x'} \right)^n \right). \quad (3.168)
$$

When $a = b$, the one-loop factor is written using the $q$-shifted factorial or $q$-deformed gamma functions. See appendix A.1 and (A.8) for the explicit formulas.
## 3.7 Decomposition of partition functions

The partition functions of the 3d, 5d, 7d, and 9d theories explained in the previous section can be written in different forms. For each setup, the fixed points are labeled by multi-dimensional partitions introduced in section 2. Each multi-dimensional partition has different descriptions (see section 2), which eventually leads to different decompositions of the partition functions. A similar discussion decomposing the partition functions of 5d theories into 3d theories was done in [103].

### 3.7.1 3d theory

Let us focus on the case when we only have n-stacks of D2-branes on $\mathbb{C}_a \times \mathbb{S}^1$ leading to a $U(n)$ theory. After localization, we obtain

$$ 
\begin{aligned}
\mathbf{Y}_a &= \mathbf{N}_a - \mathbf{P}_a \mathbf{K}_a = \sum_{\alpha=1}^{n_a} v_{a,\alpha} - (1-q_a) \sum_{\alpha=1}^{n_a} v_{a,\alpha} \frac{1-q_a^{k_a^{(\alpha)}}}{1-q_a} = \sum_{\alpha=1}^{n_a} v_{a,\alpha} q_a^{k_a^{(\alpha)}} =: \mathbf{X}_a, \\
\mathbf{X}_a &= \sum_{x \in \mathcal{X}_a} x, \quad \mathcal{X}_a = \{v_{a,\alpha} q_a^{k_a^{(\alpha)}} \mid \alpha = 1, \dots, n_a\}
\end{aligned}
\quad (3.169) $$

The total index of the coupled vortex system will be rewritten as

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee}\mathbf{Y}}{\mathbf{P}_{\mathbf{4}}} = \sum_{a,b \in \mathbf{4}} \frac{\mathbf{P}_{a}^{\vee}\mathbf{P}_{b}}{\mathbf{P}_{\mathbf{4}}} \mathbf{X}_{a}^{\vee}\mathbf{X}_{b}. \quad (3.170) $$

Choosing a square root part and taking the index gives the representation using the $\mathbf{X}_a$-variables. Note that the partition function obtained here will be the same as the partition function in (3.166) up to perturbative factors and topological terms.

### 3.7.2 5d theory

The two-dimensional partition (Young diagram) has two (1, 1)-type descriptions depending on which axis we project the Young diagram. We can rewrite the character of $\mathbf{Y}_{ab}$ ($a < b$) as

$$ 
\begin{aligned}
\mathbf{Y}_{ab} &= \mathbf{N}_{ab} - \mathbf{P}_a \mathbf{P}_b \mathbf{K}_{ab} = \mathbf{P}_a \mathbf{X}_{ab} = \mathbf{P}_b \check{\mathbf{X}}_{ab}, \\
\mathbf{X}_{ab} &= \sum_{x \in \mathcal{X}_{ab}} x, \quad \mathcal{X}_{ab} = \left\{ v_{ab,\alpha} q_a^{i-1} q_b^{\lambda_{ab,i}^{(\alpha)}} \mid \begin{array}{c} \alpha=1, \dots, n_{ab} \\ i=1, \dots, \infty \end{array} \right\}, \\
\check{\mathbf{X}}_{ab} &= \sum_{x \in \check{\mathcal{X}}_{ab}} x, \quad \check{\mathcal{X}}_{ab} = \left\{ v_{ab,\alpha} q_b^{j-1} q_a^{\lambda_{ab,j}^{(\alpha)\mathrm{T}}} \mid \begin{array}{c} \alpha=1, \dots, n_{ab} \\ j=1, \dots, \infty \end{array} \right\}.
\end{aligned}
\quad (3.171) $$

This comes from the following identities

$$ 
\begin{aligned}
v - (1 - q_a)(1 - q_b) \sum_{i=1}^{\infty} \sum_{j=1}^{\lambda_i} v q_a^{i-1} q_b^{j-1} &= (1 - q_a) \sum_{i=1}^{\infty} v q_a^{i-1} q_b^{\lambda_i}, \quad |q_a| < 1, \\
v - (1 - q_a)(1 - q_b) \sum_{j=1}^{\infty} \sum_{i=1}^{\lambda_j^{\mathrm{T}}} v q_a^{i-1} q_b^{j-1} &= (1 - q_b) \sum_{j=1}^{\infty} v q_b^{j-1} q_a^{\lambda_j^{\mathrm{T}}}, \quad |q_b| < 1.
\end{aligned}
\quad (3.172) $$
Note that depending on the analytic region of the q-parameters, the decomposition will differ. Since we have the condition $q_1q_2q_3q_4 = 1$, we can not keep all of the parameters as $|q_a| < 1$, but at least one of the parameters should be $|q_a| > 1$. For example, if we choose an analytic region<fnref n="14" /> $|q_{1,2,3}| < 1$, $|q_4| > 1$, then we can rewrite $\mathbf{Y}_A (A \in \mathbf{6})$ as

$$ \mathbf{Y}_{i4} = \mathbf{P}_i \mathbf{X}_{i4}, \quad \mathbf{Y}_{ij} = \mathbf{P}_i \mathbf{X}_{ij} = \mathbf{P}_j \check{\mathbf{X}}_{ij}, \quad i, j = 1, 2, 3, \qquad (3.173) $$

where for $\mathbf{Y}_{ij}$, we can either use the normal Young diagram or the transpose of it. For $A \in \mathbf{6}$, assume we have the decomposition

$$ \mathbf{Y}_A = \mathbf{P}_{s(A)} \mathbf{X}_A, \qquad (3.174) $$

where $s(A)$ is one of the indices in $A$ and $\mathbf{X}_A$ is the corresponding $\mathbf{X}$-bundle after the decomposition, then the total index of the spiked instanton is

$$ \mathbf{V} = \frac{\mathbf{Y}^{\vee} \mathbf{Y}}{\mathbf{P}_{\mathbf{4}}} = \sum_{A, B \in \mathbf{6}} \frac{(\mathbf{P}_{\bar{A}} \mathbf{P}_{s(A)})^{\vee} (\mathbf{P}_{\bar{B}} \mathbf{P}_{s(B)})}{\mathbf{P}_{\mathbf{4}}} \mathbf{X}_{A}^{\vee} \mathbf{X}_{B} = \sum_{A, B \in \mathbf{6}} \frac{\mathbf{P}_{\mathbf{4}}}{\mathbf{P}_{\bar{s}(A)} \mathbf{P}_{\bar{s}(B)}} \mathbf{X}_{A}^{\vee} \mathbf{X}_{B} \qquad (3.175) $$

where $\bar{s}(A)$ is the other index in $A$, namely $A = s(A)\bar{s}(A)$. Taking the square root part and the index gives the (1,1)-type description of the spiked instanton partition function.

### 3.7.3 7d theory

For the tetrahedron instanton system, since the fixed points are classified by plane partitions, we have two types of descriptions of the partition functions: (2,1) and (1,2)-type.

(2,1)-type. We can rewrite the character of $\mathbf{Y}_{abc}$ ($a < b < c$) as

$$ \begin{align} \mathbf{Y}_{abc} &= \mathbf{P}_a \mathbf{P}_b \mathbf{X}_{abc} = \mathbf{P}_a \mathbf{P}_c \check{\mathbf{X}}_{abc} = \mathbf{P}_b \mathbf{P}_c \check{\check{\mathbf{X}}}_{abc}, \nonumber \\ \mathbf{X}_{abc} &= \sum_{x \in \mathcal{X}_{abc}} x, \quad \check{\mathbf{X}}_{abc} = \sum_{x \in \check{\mathcal{X}}_{abc}} x, \quad \check{\check{\mathbf{X}}}_{abc} = \sum_{x \in \check{\check{\mathcal{X}}}_{abc}} x, \nonumber \\ \mathcal{X}_{abc} &= \{v_{abc,\alpha} q_a^{i-1} q_b^{j-1} q_c^{\pi_{abc,ij}^{(\alpha)}} \mid \alpha=1,\dots,n_{abc}, i,j=1,\dots,\infty\}, \quad \check{\mathcal{X}}_{abc} = \{v_{abc,\alpha} q_a^{i-1} q_c^{k-1} q_b^{\check{\pi}_{abc,ik}^{(\alpha)}} \mid \alpha=1,\dots,n_{abc}, i,k=1,\dots,\infty\}, \nonumber \\ \check{\check{\mathcal{X}}}_{abc} &= \{v_{abc,\alpha} q_b^{j-1} q_c^{k-1} q_a^{\check{\pi}_{abc,jk}^{(\alpha)}} \mid \alpha=1,\dots,n_{abc}, j,k=1,\dots,\infty\}, \tag{3.176} \end{align} $$

where $\pi, \check{\pi}, \check{\check{\pi}}$ are the three possible (2,1)-type descriptions depending on which 2d plane the plane partition is projected. The above decomposition comes from the following identity

$$ v - (1 - q_a)(1 - q_b)(1 - q_c) \sum_{i=1}^{\infty} \sum_{j=1}^{\infty} \sum_{k=1}^{\pi_{i,j}} v q_a^{i-1} q_b^{j-1} q_c^{k-1} = (1 - q_a)(1 - q_b) \sum_{i,j=1}^{\infty} v q_a^{i-1} q_b^{j-1} q_c^{\pi_{i,j}} \qquad (3.177) $$

where we assumed $|q_a|, |q_b| < 1$. Under the condition that only one of the q-parameters obey $|q_a| > 1$ ($\exists! a$), we can always do this (2,1)-description.

For $a \in \mathbf{4}$, let $i(a), j(a), k(a)$ be the three indices of $\bar{a}$ and assume we have the following decomposition

$$ \mathbf{Y}_{\bar{a}} = \mathbf{P}_{i(a)} \mathbf{P}_{j(a)} \mathbf{X}_{\bar{a}} \qquad (3.178) $$

<footnote n="14">¹⁴In this paper, we assume in most cases that only one of the q-parameters will be $|q_a| > 1$.</footnote>
where we omit the check mark on $X_{\bar{a}}$ for simplicity. Then, the total index of the tetrahedron instanton system will be

$$V = \frac{Y^{\vee}Y}{\mathbf{P}_{\mathbf{4}}} = \sum_{a,b \in \mathbf{4}} \frac{(\mathbf{P}_a \mathbf{P}_{i(a)} \mathbf{P}_{j(a)} \mathbf{X}_{\bar{a}})^{\vee} (\mathbf{P}_b \mathbf{P}_{i(b)} \mathbf{P}_{j(b)} \mathbf{X}_{\bar{b}})}{\mathbf{P}_{\mathbf{4}}} = \sum_{a,b \in \mathbf{4}} \frac{\mathbf{P}_{\mathbf{4}}}{\mathbf{P}_{k(a)}^{\vee} \mathbf{P}_{k(b)}} \mathbf{X}_{\bar{a}}^{\vee} \mathbf{X}_{\bar{b}}. \quad (3.179)$$

(1, 2)-type. Under this description, the plane partition is decomposed into multiple Young diagrams extending in one of the three directions. Let us focus on a U(1) theory on $\mathbb{C}_{123}^3 \times \mathbb{S}^1$ for simplicity. For $k \in \mathbb{Z}_{>0}$ we have a Young diagram $\lambda^{(k)}$ and the instanton bundle is written as

$$\mathbf{K}_{123} = \sum_{k=1}^{\infty} \sum_{\square \in \lambda^{(k)}} \chi_{12, vq_3^{k-1}(\square)}, \quad |q_3| < 1 \quad (3.180)$$

Then, we obtain

$$\begin{aligned} \mathbf{Y}_{123} &= v - \mathbf{P}_{123} \sum_{k=1}^{\infty} \sum_{\square \in \lambda^{(k)}} \chi_{12, vq_3^{k-1}(\square)} = \mathbf{P}_3 \left( \frac{v}{1-q_3} - \mathbf{P}_{12} \sum_{k=1}^{\infty} \sum_{\square \in \lambda^{(k)}} \chi_{12, vq_3^{k-1}(\square)} \right) \\ &= \mathbf{P}_3 \left( \sum_{k=1}^{\infty} vq_3^{k-1} - \mathbf{P}_{12} \sum_{\square \in \lambda^{(k)}} \chi_{12, vq_3^{k-1}(\square)} \right) = \mathbf{P}_3 \sum_{k=1}^{\infty} \mathbf{Y}_{12}[vq_3^{k-1}, \lambda^{(k)}], \end{aligned} \quad (3.181)$$

where we used

$$\mathbf{Y}_{ab}[v, \lambda] = v - \mathbf{P}_{ab} \sum_{\square \in \lambda} \chi_{ab,v}(\square). \quad (3.182)$$

This identity shows that the 7d U(1) theory can be understood as a 5d U($\infty$)-theory stacked in one of the transverse direction.<fnref n="15" /> Doing this decomposition for other cases and inserting it in the gauge origami index, we obtain the description in lower dimensional partitions.

### 3.7.4 9d theory

For simplicity, let us consider the U(1|1) theory of the magnificent four system:

$$\mathbf{Y} = \mathbf{N} - \mathbf{P}_{\mathbf{4}}\mathbf{K}, \quad \mathbf{N} = v - \bar{v}, \quad \mathbf{K} = \sum_{\square \in \rho} \chi_{\mathbf{4},v}(\square), \quad (3.183)$$

where $\rho$ is the solid partition. We have three possible descriptions: (3, 1), (2, 2), (1, 3).

(3, 1)-type. We choose the fourth direction to be the direction where the height function is defined and then obtain for $|q_{1,2,3}| < 1$

$$\begin{aligned} \mathbf{K} &= \sum_{i,j,k=1}^{\infty} \sum_{l=1}^{\rho_{i,j,k}} v q_1^{i-1} q_2^{j-1} q_3^{k-1} q_4^{l-1} = \frac{v}{\prod_{a \in \mathbf{4}} (1-q_a)} - \frac{1}{1-q_4} \sum_{i,j,k=1}^{\infty} v q_1^{i-1} q_2^{j-1} q_3^{k-1} q_4^{\rho_{i,j,k}}, \\ \mathbf{Y} &= \mathbf{P}_{123} \sum_{i,j,k=1}^{\infty} v q_1^{i-1} q_2^{j-1} q_3^{k-1} q_4^{\rho_{i,j,k}} - \bar{v} = \mathbf{P}_{123}\mathbf{X} - \bar{v}. \end{aligned} \quad (3.184)$$

<footnote n="15">¹⁵Combining with the discussion in section 9.1, one will see that this property is related to the fact that the MacMahon representation of quantum toroidal $gl_1$ is obtained as infinite tensor products of the Fock representations.</footnote>
For higher rank theories such as $U(n|n)$, we have

$$ \mathbf{Y} = \mathbf{P}_{123}\mathbf{X} - \bar{\mathbf{n}}, \quad \mathbf{X} = \sum_{x \in \mathcal{X}_4} x, \quad \bar{\mathbf{n}} = \sum_{\alpha=1}^{n} \bar{v}_{\alpha} \quad \mathcal{X}_4 = \left\{ v_{\alpha} q_1^{j_1-1} q_2^{j_2-1} q_3^{k_1-1} q_4^{\rho_{i,j,k}^{(\alpha)} |_{\substack{i,j,k=1,\dots,n \\ \alpha=1,\dots,n}}} \right\}. \quad (3.185) $$

**(2, 2)-type.** For example, in this description, we can decompose the solid partition into two Young diagrams $\rho = (\lambda_{12}; \mu_{34})$. The $\lambda_{12}, \mu_{34}$ will be Young diagrams extending in the 12 and 34 directions respectively. We choose this decomposition for simplicity but of course, we can use the quadrality to choose other decompositions. For each box in the Young diagram $(i, j) \in \lambda_{12}$, we have a Young diagram $\mu_{34}^{(i,j)}$ ($\mu_{34}^{\square}$ for $\square \in \lambda_{12}$) and thus

$$ \mathbf{K} = \sum_{(i,j) \in \lambda_{12}} \sum_{\square \in \mu_{34}^{(i,j)}} \chi_{34, v q_1^{i_1-1} q_2^{j_2-1}}(\square). \quad (3.186) $$

We then have

$$ \begin{aligned} \mathbf{Y}_4 + \bar{\mathbf{n}} &= v - \mathbf{P}_{12}\mathbf{P}_{34} \sum_{(i,j) \in \lambda_{12}} \sum_{\square \in \mu_{34}^{(i,j)}} \chi_{34, v q_1^{i_1-1} q_2^{j_2-1}}(\square) \\ &= \mathbf{Y}_{12}[v, \lambda_{12}] + \mathbf{P}_{12} \sum_{\square \in \lambda_{12}} \mathbf{Y}_{34}[\chi_{12, v}(\square), \mu_{34}^{\square}]. \end{aligned} \quad (3.187) $$

**(1, 3)-type.** We decompose the solid partition into sequences of plane partitions as

$$ \rho = (\pi^{(1)}, \dots, \pi^{(l)}, \dots) \quad (3.188) $$

after choosing the specific direction to be 4. Namely, multiple plane partitions are piled up in the fourth direction. We then have

$$ \mathbf{K}_4 = \sum_{l=1}^{\infty} \sum_{\square \in \pi^{(l)}} \chi_{\bar{\mathbf{q}}_4, v q_4^{l-1}}(\square), \quad |q_4| < 1, \quad (3.189) $$

and the character of $\mathbf{Y}_4$ is rewritten as

$$ \begin{aligned} \mathbf{Y} + \bar{\mathbf{n}} &= \mathbf{N} - \mathbf{P}_4 \mathbf{K} = \mathbf{P}_4 \left( \frac{v}{1 - q_4} - \sum_{l=1}^{\infty} \sum_{\square \in \pi^{(l)}} \chi_{\bar{\mathbf{q}}_4, v q_4^{l-1}}(\square) \right) \\ &= \mathbf{P}_4 \left( \sum_{l=1}^{\infty} \left( v q_4^{l-1} - \sum_{\square \in \pi^{(l)}} \chi_{\bar{\mathbf{q}}_4, v q_4^{l-1}}(\square) \right) \right) = \mathbf{P}_4 \sum_{l=1}^{\infty} \mathbf{Y}_{123}[v q_4^{l-1}, \pi^{(l)}]. \end{aligned} \quad (3.190) $$

This description resembles the 7d (1, 2)-type description where 2d partitions were piled up to get 3d partitions.

## 3.8 $qq$-characters from instanton partition functions

### 3.8.1 5d theory

Let us review the $qq$-character of the 5d $\mathcal{N} = 1^*$ theory on $\mathbb{C}_{12}^2 \times \mathbb{S}^1$. Using the recursion formulas of the 5d Nekrasov factors in (3.88), the recursion relation of the $U(1)$ partition
function $\tilde{Z}_{12}^{D4}[\lambda]$ is (see the derivation and the discussion in (B.42))

$$ \frac{\tilde{Z}_{12}^{D4}[\lambda + \Box]}{\tilde{Z}_{12}^{D4}[\lambda]} = - \frac{y_{\lambda, v}^{12}(q_3^{-1}\chi_{12, v}(\Box))y_{\lambda+\Box, v}^{12}(q_4^{-1}\chi_{12, v}(\Box))}{y_{\lambda, v}^{12}(\chi_{12, v}(\Box))y_{\lambda+\Box, v}^{12}(q_3^{-1}\chi_{12, v}(\Box))} \quad (3.191) \\ = -S_{12}(q_4) \frac{y_{\lambda, v}^{12}(q_3^{-1}\chi_{12, v}(\Box))y_{\lambda, v}^{12}(q_4^{-1}\chi_{12, v}(\Box))}{y_{\lambda, v}^{12}(\chi_{12, v}(\Box))y_{\lambda+\Box, v}^{12}(q_3^{-1}\chi_{12, v}(\Box))} $$

where $\Box \in A(\lambda)$ and we used $y_{\lambda+\Box, v}^{12}(x) = S_{12}(\chi_{12, v}(\Box)/x) y_{\lambda, v}^{12}(x)$. Strictly speaking, the $y$-functions in the denominator are singular. The term $y_{\lambda, v}^{12}(x)$ has a zero in $x = \chi_{12, v}(\Box)$ while the term $y_{\lambda+\Box}^{12}(q_{12}x)$ has a pole at $x = \chi_{12, v}(\Box)$. However, since the zero and pole will cancel with each other, we can write in the above way.

We can also write the recursion formula using the dual $y$-functions as

$$ \frac{\tilde{Z}_{12}^{D4}[\lambda + \Box]}{\tilde{Z}_{12}^{D4}[\lambda]} = - \frac{y_{\lambda, v}^{12 \vee}(q_3^{-1} \chi_{12, v}(\Box)) y_{\lambda+\Box, v}^{12 \vee}(q_4^{-1} \chi_{12, v}(\Box))}{y_{\lambda, v}^{12 \vee}(\chi_{12, v}(\Box)) y_{\lambda+\Box, v}^{12 \vee}(q_3^{-1} \chi_{12, v}(\Box))} \quad (3.192) \\ = -S_{12}(q_3) \frac{y_{\lambda, v}^{12 \vee}(q_3^{-1} \chi_{12, v}(\Box)) y_{\lambda, v}^{12 \vee}(q_4^{-1} \chi_{12, v}(\Box))}{y_{\lambda, v}^{12 \vee}(\chi_{12, v}(\Box)) y_{\lambda+\Box, v}^{12 \vee}(q_3^{-1} \chi_{12, v}(\Box))} $$

Note here that we have the property $S_{12}(q_3) = S_{12}(q_4)$.

The recursion relations are then rewritten as

$$ \mathrm{Res}_{x=\chi_{12,v}(\Box)} \left[ y_{\lambda+\Box,v}(q_{12}x) \tilde{Z}_{12}^{D4}[\lambda+\Box] + S_{12}(q_4) \frac{y_{\lambda,v}^{12}(q_3^{-1}x) y_{\lambda,v}^{12}(q_4^{-1}x)}{y_{\lambda,v}^{12}(x)} \tilde{Z}_{12}^{D4}[\lambda] \right] = 0, \quad (3.193) \\ \mathrm{Res}_{x=\chi_{12,v}(\Box)} \left[ y_{\lambda+\Box,v}^{12\vee}(q_{12}x) \tilde{Z}_{12}^{D4}[\lambda+\Box] + S_{12}(q_3) \frac{y_{\lambda,v}^{12\vee}(q_3^{-1}x) y_{\lambda,v}^{12\vee}(q_4^{-1}x)}{y_{\lambda,v}^{12\vee}(x)} \tilde{Z}_{12}^{D4}[\lambda] \right] = 0 $$

which means that the singularity of the first term at $x = \chi_{12, v}(\Box)$ is cancelled by the second term. Including the topological term and taking the instanton summation, we obtain

$$ \langle \hat{y}^{12}(q_{12}x) \rangle_{12} + q S_{12}(q_4) \left\langle \frac{\hat{y}^{12}(q_3^{-1}x) \hat{y}^{12}(q_4^{-1}x)}{\hat{y}^{12 \vee}(x)} \right\rangle_{12}, \quad (3.194) \\ \langle \hat{y}^{12 \vee}(q_{12}x) \rangle_{12} + q S_{12}(q_4) \left\langle \frac{\hat{y}^{12 \vee}(q_3^{-1}x) \hat{y}^{12 \vee}(q_4^{-1}x)}{\hat{y}^{12 \vee}(x)} \right\rangle_{12} $$

where

$$ \langle \hat{O} \rangle_{12} = \frac{1}{\tilde{Z}_{12}} \sum_{\lambda} q^{|\lambda|} \tilde{Z}_{12}^{D4}[\lambda] \mathcal{O}_{\lambda, v}, \quad \tilde{Z}_{12} = \sum_{\lambda} q^{|\lambda|} \tilde{Z}_{12}^{D4}[\lambda]. \quad (3.195) $$

Note here that we are understanding $\hat{O}$ as an operator acting on the fixed points labeled by $(v, \lambda)$, where $v$ is the exponentiated Coulomb branch parameter of the U(1) gauge theory on $\mathbb{C}_{12}^2 \times \mathbb{S}^1$. The subindex 12 denotes on which gauge theory the instanton expectation value is taken. Although we were considering when the bulk theory is the U(1) theory, we can generalize it to U(n) theories. In such cases, the $\hat{y}^{12}(x)$ operator will act as

$$ \hat{y}^{12}(x) \rightarrow \prod_{\alpha=1}^{n} y_{\lambda^{(\alpha)}, v_{\alpha}}^{12}(x), \quad \hat{y}^{12 \vee}(x) \rightarrow \prod_{\alpha=1}^{n} y_{\lambda^{(\alpha)}, v_{\alpha}}^{12 \vee}(x) \quad (3.196) $$

under an instanton background labeled by $(\vec{v}, \vec{\lambda})$.
Due to the property in (3.193), the singularities coming from the first term are canceled by the second term. This procedure to add a term and compensate the pole singularity is called the **iWeyl reflection** [71] and is defined as

$$ \begin{aligned} \hat{y}^{12}(q_{12}x) &\mapsto q\mathfrak{S}_{12}(q_4)\hat{y}^{12}(q_{12}x)\hat{\mathcal{A}}(x)^{-1}, \\ \hat{y}^{12\vee}(q_{12}x) &\mapsto q\mathfrak{S}_{12}(q_3)\hat{y}^{12\vee}(q_{12}x)\hat{\mathcal{A}}^{\vee}(x)^{-1}, \end{aligned} \quad (3.197) $$

where

$$ \hat{\mathcal{A}}(x) = \frac{\hat{y}^{12}(x)\hat{y}^{12}(q_3^{-1}q_4^{-1}x)}{\hat{y}^{12}(q_3^{-1}x)\hat{y}^{12}(q_4^{-1}x)}, \quad \hat{\mathcal{A}}^{\vee}(x) = \frac{\hat{y}^{12\vee}(x)\hat{y}^{12\vee}(q_3^{-1}q_4^{-1}x)}{\hat{y}^{12\vee}(q_3^{-1}x)\hat{y}^{12\vee}(q_4^{-1}x)} \quad (3.198) $$

After this iWeyl reflection, new singularities will appear from the new $\mathfrak{Y}$-functions coming from $\hat{\mathcal{A}}(x)$ and thus, we need to recursively do this procedure. After summing up all the terms appearing after the iWeyl reflection, we will obtain a pole-free function in $x$ which is called the **qq-character**:

$$ \begin{aligned} \hat{\mathcal{F}}^{12}(x) &= \hat{y}^{12}(q_{12}x) + q\mathfrak{S}_{12}(q_4)\hat{y}^{12}(q_{12}x)\hat{\mathcal{A}}(x)^{-1} + \dots \\ \hat{\mathcal{F}}^{12\vee}(x) &= \hat{y}^{12}(q_{12}x) + q\mathfrak{S}_{12}(q_3)\hat{y}^{12\vee}(q_{12}x)\hat{\mathcal{A}}^{\vee}(x)^{-1} + \dots \end{aligned} \quad (3.199) $$

Doing the iWeyl reflection recursively, one can show that the full formula of this **qq-character** is

$$ \begin{aligned} \hat{\mathcal{F}}^{12}(x) &= \sum_{\mu \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{34}^{\mathrm{D}4}[\mu] \hat{y}^{12}(q_{12}x) \prod_{\square \in \mu} \hat{\mathcal{A}}(\chi_{34,x}(\square))^{-1} \\ &= \sum_{\mu \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{34}^{\mathrm{D}4}[\mu] \frac{\prod_{\square \in A(\mu)} \hat{y}^{12}(\chi_{34,x}(\square))}{\prod_{\square \in R(\mu)} \hat{y}^{12}(q_{34}\chi_{34,x}(\square))}, \\ \hat{\mathcal{F}}^{12\vee}(x) &= \sum_{\mu \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{34}^{\mathrm{D}4}[\mu] \hat{y}^{12\vee}(q_{12}x) \prod_{\square \in \mu} \hat{\mathcal{A}}^{\vee}(\chi_{34,x}(\square))^{-1} \\ &= \sum_{\mu \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{34}^{\mathrm{D}4}[\mu] \frac{\prod_{\square \in A(\mu)} \hat{y}^{12\vee}(\chi_{34,x}(\square))}{\prod_{\square \in R(\mu)} \hat{y}^{12\vee}(q_{34}\chi_{34,x}(\square))}. \end{aligned} \quad (3.200) $$

Obviously, $\mathfrak{S}_{12}(q_3) = \mathfrak{S}_{12}(q_4)$ is the one-instanton contribution of the U(1) affine quiver gauge theory on $\mathbb{C}_{34}^2 \times \mathbb{S}^1$ (see (B.41)).

Generally, we can do the same calculation for other 5d theories on $\mathbb{C}_A^2 \times \mathbb{S}^1$ ($A \in \underline{\mathbf{6}}$).

**Proposition 3.14.** The **qq-characters** with respect to the 5d $\mathcal{N} = 1^*$ gauge theory on $\mathbb{C}_A^2 \times \mathbb{S}^1$ where $A \in \underline{\mathbf{6}}$ are given by

$$ \begin{aligned} \hat{\mathcal{F}}^A(x) &= \sum_{\lambda \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{\bar{A}}^{\mathrm{D}4}[\lambda] \hat{y}^A(q_A x) \prod_{\square \in \lambda} \hat{\mathcal{A}}(\chi_{\bar{A},x}(\square))^{-1}, \\ \hat{\mathcal{F}}^{A\vee}(x) &= \sum_{\lambda \in \mathcal{P}} q^{\vert \mu \vert} \tilde{Z}_{\bar{A}}^{\mathrm{D}4}[\lambda] \hat{y}^{A\vee}(q_A x) \prod_{\square \in \lambda} \hat{\mathcal{A}}^{\vee}(\chi_{\bar{A},x}(\square))^{-1}. \end{aligned} \quad (3.201) $$

The iWeyl reflection of $\hat{y}^A(x)$, $\hat{y}^{A\vee}(x)$ is defined as

$$ \begin{aligned} \hat{y}^A(q_A x) &\mapsto q\mathfrak{S}_A(q_c)\hat{y}^A(q_A x)\hat{\mathcal{A}}(x)^{-1}, \\ \hat{y}^{A\vee}(q_A x) &\mapsto q\mathfrak{S}_A(q_d)\hat{y}^{A\vee}(q_A x)\hat{\mathcal{A}}^{\vee}(x)^{-1}, \end{aligned} \quad (3.202) $$

where $\{c, d\} = \bar{A}$ and

$$ \hat{\mathcal{A}}(x) = \frac{\hat{y}^A(x)\hat{y}^A(q_A x)}{\hat{y}^A(q_c^{-1}x)\hat{y}^A(q_d^{-1}x)}, \quad \hat{\mathcal{A}}^{\vee}(x) = \frac{\hat{y}^{A\vee}(x)\hat{y}^{A\vee}(q_A x)}{\hat{y}^{A\vee}(q_c^{-1}x)\hat{y}^{A\vee}(q_d^{-1}x)}. \quad (3.203) $$
<table><caption>**Table 5.** The configuration of the two sets of D4-branes wrapping the 12-plane and 34-plane. From the D4<sub>12</sub> view point, the D4<sub>34</sub>-brane plays the role of the *qq*-character.</caption><thead><tr><th rowspan="2"></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R × S<sup>1</sup></th></tr><tr><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><th>D4<sub>12</sub></th><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td></tr><tr><th>D4<sub>34</sub></th><td>•</td><td>•</td><td>•</td><td>•</td><td>—</td><td>—</td><td>—</td><td>—</td><td>•</td><td>—</td></tr></tbody></table>

Note that the explicit form of the operator $\hat{\mathcal{A}}(x)$ depends on the theory we consider.<fnref n="16" /> We call the *qq*-characters $\hat{\mathcal{T}}^A(x)$, $\hat{\mathcal{T}}^{A\vee}(x)$ the D4$_{\bar{A}}$-brane *qq*-characters (shortly D4$_{\bar{A}}$ *qq*-characters). Namely, the *qq*-characters associated with the 5d theory on the D4$_A$-brane are the D4$_{\bar{A}}$ *qq*-characters.

**D4 *qq*-characters as codimension four defects.** The *qq*-character is understood as a codimension four defect with respect to the corresponding bulk theory. Let us consider $\mathbb{C}_{12}^2 \times S^1$ as the bulk theory. The *qq*-character is understood as a D4-brane spanning the codimension four subspace $\mathbb{C}_{34}^2 \times S^1$ in the transverse direction:

<figure><img src="image_10.png" /></figure>

The brane web construction is given in table 5. This is the reason why we called the *qq*-character of the D4 theory on $\mathbb{C}_A^2 \times S^1$ the D4$_{\bar{A}}$ *qq*-character.

The contour integral formula of the *qq*-character is intuitive. The contour integral comes from the following character:

$$ \begin{aligned} & \mathbf{N}_{34}^{\vee} \frac{\mathbf{P}_{12}^{\vee} \mathbf{P}_{34}}{\mathbf{P}_{4}} \mathbf{N}_{12} - \mathbf{P}_{34}^{\vee} \mathbf{N}_{12}^{\vee} \mathbf{K} - \mathbf{N}_{34}^{\vee} \mathbf{P}_{12}^{\vee} \mathbf{K} + \mathbf{P}_{123}^{\vee} \mathbf{K}_{12}^{\vee} \mathbf{K}_{12} + \mathbf{P}_{134}^{\vee} \mathbf{K}_{34}^{\vee} \mathbf{K}_{34} + \mathbf{P}_{4} \mathbf{K}_{12}^{\vee} \mathbf{K}_{34} \\ & = \mathbf{N}_{34}^{\vee} q_{12}^{-1} (\mathbf{N}_{12} - \mathbf{P}_{12} \mathbf{K}) - \mathbf{P}_{34}^{\vee} \mathbf{N}_{12}^{\vee} \mathbf{K} + \mathbf{P}_{123}^{\vee} \mathbf{K}_{12}^{\vee} \mathbf{K}_{12} + \mathbf{P}_{134}^{\vee} \mathbf{K}_{34}^{\vee} \mathbf{K}_{34} + \mathbf{P}_{4} \mathbf{K}_{12}^{\vee} \mathbf{K}_{34} \end{aligned} \quad (3.205) $$

where $\mathbf{K} = \mathbf{K}_{12} + \mathbf{K}_{34}$ and we included the perturbative part for convention. Note that this character is just a part of (3.135) where the terms including $\mathbf{N}_{34}$, $\mathbf{K}_{34}$ are kept, namely the crossed instanton configuration.

<footnote n="16">¹⁶Later, we will see in the operator formalism that it will be a unique operator (see (4.68)).</footnote>
Consider the case when there is only one transverse D4-brane with $N_{34} = x$. The contour integral is then written as

$$ \frac{\mathcal{G}_4^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{\alpha=1}^n \left(1 - \frac{q_{12}x}{v_\alpha}\right) \prod_{I=1}^k S_{12}\left(\frac{x}{x_I}\right) \prod_{\alpha=1}^n \prod_{I=1}^k S_{34}\left(\frac{v_\alpha}{x_I}\right) \prod_{I<J} \mathcal{A}_{\mathbb{C}^4}\left(\frac{x_I}{x_J}\right)^{-1}. \quad (3.206) $$

The red term comes from the transverse D4$_{34}$-brane while the blue term comes from the D4$_{12}$-brane. One can see that after localization the red term gives the contribution coming from the $\mathcal{Y}$-functions. When $n=1$, after evaluating the poles, one can see we have

$$ \langle \hat{\mathcal{J}}^{12}(x) \rangle_{12} = \mathcal{Z}_{\text{cross. inst.}}^{12|34}, \quad \langle \hat{\mathcal{J}}^{12\vee}(x) \rangle_{12} = \mathcal{Z}_{\text{cross. inst.}}^{12|34}. \quad (3.207) $$

Namely, integrating the transverse D4$_{34}$-brane first, the operators $\hat{\mathcal{J}}^{12}(x)$, $\hat{\mathcal{J}}^{12\vee}(x)$ representing the defect brane arises in the bulk theory and the expectation value of it gives the partition function of the gauge origami system.

### 3.8.2 7d theory

Let us do a similar analysis for the 7d theory. We focus first on the 7d theory on $\mathbb{C}_{123}^3 \times \mathbb{S}^1$. The recursion formula of $\tilde{\mathcal{Z}}_{123}^{D6}[\pi]$ is given from (B.25):

$$ \frac{\tilde{\mathcal{Z}}_{123}^{D6}[\pi + \Box]}{\tilde{\mathcal{Z}}_{123}^{D6}[\pi]} = - \frac{W_{\pi+\Box,v}^{\bar{4}}(q_{123}\chi_{123,v}(\Box))}{W_{\pi,v}^{\bar{4}}(\chi_{123,v}(\Box))} = -q_4 \frac{W_{\pi+\Box,v}^{\bar{4}\vee}(q_{123}\chi_{123,v}(\Box))}{W_{\pi,v}^{\bar{4}\vee}(\chi_{123,v}(\Box))}, \quad (3.208) $$

where $\Box \in \mathcal{A}(\pi)$. It is then rewritten as

$$ \begin{gathered} \mathrm{Res}_{x=\chi_{123,v}(\Box)} \left[ W_{\pi+\Box,v}^{\bar{4}}(q_{123}x)^{-1} \tilde{\mathcal{Z}}_{123}^{D6}[\pi+\Box] + W_{\pi,v}^{\bar{4}}(x)^{-1} \tilde{\mathcal{Z}}_{123}^{D6}[\pi] \right] = 0, \\ \mathrm{Res}_{x=\chi_{123,v}(\Box)} \left[ W_{\pi+\Box,v}^{\bar{4}\vee}(q_{123}x)^{-1} \tilde{\mathcal{Z}}_{123}^{D6}[\pi+\Box] + q_4 W_{\pi,v}^{\bar{4}\vee}(x)^{-1} \tilde{\mathcal{Z}}_{123}^{D6}[\pi] \right] = 0 \end{gathered} \quad (3.209) $$

which shows that the second term cancels the singularity coming from the pole at $x = \chi_{123,v}(\Box)$ of the first term. Including the topological term and taking the instanton summation, we obtain

$$ \begin{gathered} \langle \widehat{\mathcal{W}}^{\bar{4}}(q_{123}x)^{-1} \rangle_{123} + q \langle \widehat{\mathcal{W}}^{\bar{4}}(x)^{-1} \rangle_{123}, \\ \langle \widehat{\mathcal{W}}^{\bar{4}\vee}(q_{123}x)^{-1} \rangle_{123} + q q_4 \langle \widehat{\mathcal{W}}^{\bar{4}\vee}(x)^{-1} \rangle_{123} \end{gathered} \quad (3.210) $$

where

$$ \langle \mathcal{O} \rangle_{123} = \frac{1}{\mathcal{Z}_{123}} \sum_{\pi} q^{|\pi|} \tilde{\mathcal{Z}}_{123}^{D6}[\pi] \mathcal{O}_{\pi,v}, \quad \mathcal{Z}_{123} = \sum_{\pi} q^{|\pi|} \tilde{\mathcal{Z}}_{123}^{D6}[\pi]. \quad (3.211) $$

This combination is a pole-free function at $x = \chi_{123,v}(\Box)$ because the singularity coming from the first term is canceled by the second term. We can define the iWeyl reflection analogous to the D4 case as

$$ \begin{gathered} \widehat{\mathcal{W}}^{\bar{4}}(q_{123}x)^{-1} \mapsto q \widehat{\mathcal{W}}^{\bar{4}}(q_{123}x)^{-1} \widehat{\mathcal{A}}(x)^{-1}, \\ \widehat{\mathcal{W}}^{\bar{4}\vee}(q_{123}x)^{-1} \mapsto q q_4 \widehat{\mathcal{W}}^{\bar{4}\vee}(q_{123}x)^{-1} \widehat{\mathcal{A}}^{\vee}(x)^{-1} \end{gathered} \quad (3.212) $$

where

$$ \widehat{\mathcal{A}}(x) = \frac{\widehat{\mathcal{W}}^{\bar{4}}(x)}{\widehat{\mathcal{W}}^{\bar{4}}(q_4^{-1}x)}, \quad \widehat{\mathcal{A}}^{\vee}(x) = \frac{\widehat{\mathcal{W}}^{\bar{4}\vee}(x)}{\widehat{\mathcal{W}}^{\bar{4}\vee}(q_4^{-1}x)}. \quad (3.213) $$
<table><caption>**Table 6.** The D2₄-brane and D6₁₂₃ are transverse to each other. The D6 (D2)-brane plays the role of the *qq*-character of the D2 (D6) theory.</caption><thead><tr><th></th><th colspan="2">C<sub>1</sub></th><th colspan="2">C<sub>2</sub></th><th colspan="2">C<sub>3</sub></th><th colspan="2">C<sub>4</sub></th><th colspan="2">R &times; S<sup>1</sup></th></tr><tr><th></th><th>1</th><th>2</th><th>3</th><th>4</th><th>5</th><th>6</th><th>7</th><th>8</th><th>9</th><th>0</th></tr></thead><tbody><tr><th>D2<sub>4</sub></th><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&ndash;</td><td>&ndash;</td><td>&bull;</td><td>&ndash;</td></tr><tr><th>D6<sub>123</sub></th><td>&ndash;</td><td>&ndash;</td><td>&ndash;</td><td>&ndash;</td><td>&ndash;</td><td>&ndash;</td><td>&bull;</td><td>&bull;</td><td>&bull;</td><td>&ndash;</td></tr></tbody></table>

We can do this procedure recursively and obtain a pole-free function in $x$ which is a generalization of the 5d case:

$$ 
\begin{aligned}
\hat{\mathcal{F}}^{123}(x) &= \hat{W}^4(q_{123}x)^{-1} + q\hat{W}^4(q_{123}x)^{-1}\hat{\mathcal{A}}(x)^{-1} + \dots \\
&= \hat{W}^4(q_{123}x)^{-1} + q\hat{W}^4(x)^{-1} + \dots, \\
\hat{\mathcal{F}}^{123 \vee}(x) &= \hat{W}^4 \vee(q_{123}x)^{-1} + q\hat{q}_4\hat{W}^4(q_{123}x)^{-1}\hat{\mathcal{A}} \vee(x)^{-1} + \dots \\
&= \hat{W}^4 \vee(q_{123}x)^{-1} + q\hat{q}_4\hat{W}^4 \vee(x)^{-1} + \dots.
\end{aligned} 
\quad (3.214) $$

The full formula is given as

$$ 
\begin{aligned}
\hat{\mathcal{F}}^{123}(x) &= \sum_{k=0}^{\infty} q^k \hat{W}^4(q_{123}x)^{-1} \prod_{i=1}^{k} \hat{\mathcal{A}}(q_4^{i-1}x)^{-1} = \sum_{k=0}^{\infty} q^k \hat{W}^4(q_4^{k-1}x)^{-1}, \\
\hat{\mathcal{F}}^{123 \vee}(x) &= \sum_{k=0}^{\infty} (qq_4)^k \hat{W}^4 \vee(q_{123}x)^{-1} \prod_{i=1}^{k} \hat{\mathcal{A}} \vee(q_4^{i-1}x)^{-1} = \sum_{k=0}^{\infty} (qq_4)^k \hat{W}^4 \vee(q_4^{k-1}x)^{-1}
\end{aligned} 
\quad (3.215) $$

A similar discussion for other 7d theories on $\mathbb{C}_a^3 \times S^1$ can be done and we have the following statement.

**Proposition 3.15.** The *qq*-characters with respect to the 7d $\mathcal{N}=1$ theory on $\mathbb{C}_a^3 \times S^1$ where $a \in \mathbb{Z}_4$ are

$$ 
\begin{aligned}
\hat{\mathcal{F}}^{\bar{a}}(x) &= \sum_{k=0}^{\infty} q^k \hat{W}^{\bar{a}}(q_{\bar{a}}x)^{-1} \prod_{i=1}^{k} \hat{\mathcal{A}}(q_a^{i-1}x)^{-1} = \sum_{k=0}^{\infty} q^k \hat{W}^{\bar{a}}(q_a^{k-1}x)^{-1}, \\
\hat{\mathcal{F}}^{\bar{a} \vee}(x) &= \sum_{k=0}^{\infty} (qq_a)^k \hat{W}^{\bar{a} \vee}(q_{\bar{a}}x)^{-1} \prod_{i=1}^{k} \hat{\mathcal{A}}^{\vee}(q_a^{i-1}x)^{-1} = \sum_{k=0}^{\infty} (qq_a)^k \hat{W}^{\bar{a} \vee}(q_a^{k-1}x)^{-1},
\end{aligned} 
\quad (3.216) $$

where the iWeyl reflection is defined as

$$ 
\begin{aligned}
\hat{W}^{\bar{a}}(q_{\bar{a}}x)^{-1} &\mapsto q\hat{W}^{\bar{a}}(q_{\bar{a}}x)^{-1}\hat{\mathcal{A}}(x)^{-1}, \\
\hat{W}^{\bar{a} \vee}(q_{\bar{a}}x)^{-1} &\mapsto qq_a\hat{W}^{\bar{a} \vee}(q_{\bar{a}}x)^{-1}\hat{\mathcal{A}}^{\vee}(x)^{-1}
\end{aligned} 
\quad (3.217) $$

and

$$ \hat{\mathcal{A}}(x) = \frac{\hat{W}^{\bar{a}}(x)}{\hat{W}^{\bar{a}}(q_a^{-1}x)}, \quad \hat{\mathcal{A}}^{\vee}(x) = \frac{\hat{W}^{\bar{a} \vee}(x)}{\hat{W}^{\bar{a} \vee}(q_a^{-1}x)}. \quad (3.218) $$

We call these *qq*-characters the D2-brane *qq*-characters, shortly D2 *qq*-characters. Namely, the *qq*-character with respect to the D6$_{\bar{a}}$ theory on $\mathbb{C}_{\bar{a}}^3 \times S^1$ is the D2$_a$ *qq*-character.
<figure><img src="image_11.png" /><figcaption>**Figure 7.** D2$_4$-brane and D6$_{123}$-brane. The D2$_4$-brane wraps the $\mathbb{C}_4$ inside $\mathbb{C}^4$ while the D6$_{123}$-brane wraps the $\mathbb{C}_{123}^3$. From the D2-brane viewpoint, the D6 shares the $\mathbb{S}^1$ part and is a 1d line defect which is a codimension two defect. On the other hand, from the D6-brane viewpoint, the D2-brane is a 1d line defect sharing the $\mathbb{S}^1$ part and thus is a codimension six defect.</figcaption></figure>

**D2 $qq$-characters as codimension six defects.** Similar to the D4 theories, we can understand the codimension six defects coming from D2-branes in the transverse direction as D2 $qq$-characters of the D6 theory (see figure 7). The brane web configuration is given in table 6. The D2-brane in the transverse direction shares the $\mathbb{S}^1$ and thus is a line defect, which is a codimension six defect from the D6-brane theory viewpoint.

Mimicking the construction of the D4 case, we propose that the contour integral formula comes from the character

$$ N_4^\vee \frac{\mathbf{P}_{123}^\vee \mathbf{P}_4}{\mathbf{P}_4} N_{123} - \mathbf{P}_4^\vee N_{123}^\vee \mathbf{K} - \mathbf{P}_{123}^\vee N_4^\vee \mathbf{K} + \mathbf{P}_{123}^\vee \mathbf{K}_{123}^\vee \mathbf{K}_{123} + \mathbf{P}_{124}^\vee \mathbf{K}_4^\vee \mathbf{K}_4 + \mathbf{P}_4 \mathbf{K}_{123}^\vee \mathbf{K}_4 \\ = -q_4 N_4^\vee (N_{123} - \mathbf{P}_{123} \mathbf{K}) - \mathbf{P}_4^\vee N_{123}^\vee \mathbf{K} + \mathbf{P}_{123}^\vee \mathbf{K}_{123}^\vee \mathbf{K}_{123} + \mathbf{P}_{124}^\vee \mathbf{K}_4^\vee \mathbf{K}_4 + \mathbf{P}_4 \mathbf{K}_{123}^\vee \mathbf{K}_4, \qquad (3.219) $$

where $\mathbf{K} = \mathbf{K}_{123} + \mathbf{K}_4$. Note that although considering a subpart of the gauge origami system of the spiked instantons gives the D4 $qq$-characters, for the D6 case, we need to consider a gauge origami system where both D2 and D6-branes appear together. Such generalization is straightforward by using

$$ \mathbf{Y} = \sum_{a \in \mathbf{\underline{4}}} \mathbf{P}_{\bar{a}} \mathbf{Y}_a + \sum_{a \in \mathbf{\underline{4}}} \mathbf{P}_a \mathbf{Y}_{\bar{a}}, \qquad (3.220) $$

inserting into $\mathbf{Y}^\vee \mathbf{Y}/\mathbf{P}_4$, and doing the similar analysis in sections 3.3, 3.4, 3.5, 3.6. We omit such discussions and only focus only on (3.219).

Let us consider the case when we have one D2$_4$-brane $\mathbf{N}_4 = x$ and $n$ D6-branes $\mathbf{N}_{123} = \sum_{\alpha=1}^n v_\alpha$. The contour integral is written as

$$ \frac{G_{123}^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{\alpha=1}^n \frac{1}{(1 - q_{123}x/v_\alpha)} \prod_{I=1}^k g_{123} \left(\frac{x}{x_I}\right) \prod_{\alpha=1}^n \prod_{I=1}^k \nu_4 \left(\frac{v_\alpha}{x_I}\right) \prod_{I<j} \mathcal{A}_{\mathbb{C}^4} \left(\frac{x_J}{x_I}\right)^{-1}. \qquad (3.221) $$

The red term comes from the transverse D2-brane, while the blue term comes from the bulk D6-branes. After localization, the red term gives the W-functions. Similar to the D4-case,
integrating out the D2₄-brane leads to a codimension six defect operator $\hat{\mathcal{J}}^{123}(x)$, $\hat{\mathcal{J}}^{123\vee}(x)$ and the expectation value of it with respect to the bulk D6₁₂₃-theory gives the partition function of the gauge origami system where there are D2₄ and D6₁₂₃-branes spanning transversely.

### 3.8.3 3d theory

Let us consider 3d theories on $\mathcal{C}_a \times \mathbb{S}^1$. We focus on the $a = 4$ case. The recursion formula of the 3d partition function is (B.52) (see also (B.55)):

$$ \begin{aligned} \frac{\mathcal{Z}_4^{\mathrm{D2}}[k+\Box; q_1, q_2]}{\mathcal{Z}_4^{\mathrm{D2}}[k; q_1, q_2]} &= -\frac{\mathcal{V}_4(q_3) \mathcal{U}_{k,v}^4(q_1^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^4(q_2^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^4(q_3^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k+\Box,v}^4(q_{123}^{-1}\chi_{4,v}(\Box))}{\mathcal{V}_4(q_{23})\mathcal{V}_4(q_{13}) \mathcal{U}_{k,v}^4(q_{23}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^4(q_{13}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^4(q_{12}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^4(\chi_{4,v}(\Box))}, \\ \frac{\mathcal{Z}_4^{\mathrm{D2}}[k+\Box; q_1, q_2]}{\mathcal{Z}_4^{\mathrm{D2}}[k; q_1, q_2]} &= -\frac{\mathcal{V}_4(q_1)\mathcal{V}_4(q_2) \mathcal{U}_{k,v}^{4\vee}(q_1^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^{4\vee}(q_2^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^{4\vee}(q_3^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k+\Box,v}^{4\vee}(q_{123}^{-1}\chi_{4,v}(\Box))}{\mathcal{V}_4(q_{12}) \mathcal{U}_{k,v}^{4\vee}(q_{23}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^{4\vee}(q_{13}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^{4\vee}(q_{12}^{-1}\chi_{4,v}(\Box)) \mathcal{U}_{k,v}^{4\vee}(\chi_{4,v}(\Box))} \end{aligned} \quad (3.222) $$

where we used $\mathcal{U}_{k+1,v}^4(x) = \mathcal{V}_4(\chi_{4,v}(\Box)/x) \mathcal{U}_{k,v}^4(x)$ and $\mathcal{U}_{k+1,v}^{4\vee}(x) = \mathcal{V}_4(q_4^{-1}x/\chi_{4,v}(\Box))^{-1} \mathcal{U}_{k,v}^{4\vee}(x)$. This is rewritten as

$$ \begin{aligned} \mathrm{Res}_{x=\chi_{4,v}(\Box)} \left[ \mathcal{U}_{k+\Box}^4(q_4 x)^{-1} \mathcal{Z}_4^{\mathrm{D2}}[k+\Box; q_1, q_2] \right. &+ \frac{\mathcal{V}_4(q_3)}{\mathcal{V}_4(q_{23})\mathcal{V}_4(q_{13})} \frac{\mathcal{U}_{k,v}^4(q_1^{-1}x)}{\mathcal{U}_{k,v}^4(q_{23}^{-1}x)} \frac{\mathcal{U}_{k,v}^4(q_2^{-1}x)}{\mathcal{U}_{k,v}^4(q_{13}^{-1}x)} \frac{\mathcal{U}_{k,v}^4(q_3^{-1}x)}{\mathcal{U}_{k,v}^4(q_{12}^{-1}x)} \frac{1}{\mathcal{U}_{k,v}^4(x)} \mathcal{Z}_4^{\mathrm{D2}}[k; q_1, q_2] \right] = 0, \\ \mathrm{Res}_{x=\chi_{4,v}(\Box)} \left[ \mathcal{U}_{k+\Box}^{4\vee}(q_4 x)^{-1} \mathcal{Z}_4^{\mathrm{D2}}[k+\Box; q_1, q_2] \right. &+ \frac{\mathcal{V}_4(q_1)\mathcal{V}_4(q_2)}{\mathcal{V}_4(q_{12})} \frac{\mathcal{U}_{k,v}^{4\vee}(q_1^{-1}x)}{\mathcal{U}_{k,v}^{4\vee}(q_{23}^{-1}x)} \frac{\mathcal{U}_{k,v}^{4\vee}(q_2^{-1}x)}{\mathcal{U}_{k,v}^{4\vee}(q_{13}^{-1}x)} \frac{\mathcal{U}_{k,v}^{4\vee}(q_3^{-1}x)}{\mathcal{U}_{k,v}^{4\vee}(q_{12}^{-1}x)} \frac{1}{\mathcal{U}_{k,v}^{4\vee}(x)} \mathcal{Z}_4^{\mathrm{D2}}[k; q_1, q_2] \right] = 0 \end{aligned} \quad (3.223) $$

Including the topological term and taking the instanton summation, the following function is pole-free in $x = \chi_{4,v}(\Box)$:

$$ \begin{aligned} \left\langle \hat{\mathcal{U}}^4(q_4 x)^{-1} \right\rangle_4 + \mathfrak{q} \frac{\mathcal{V}_4(q_3)}{\mathcal{V}_4(q_{23})\mathcal{V}_4(q_{13})} \left\langle \frac{\hat{\mathcal{U}}^4(q_1^{-1}x)\hat{\mathcal{U}}^4(q_2^{-1}x)\hat{\mathcal{U}}^4(q_3^{-1}x)}{\hat{\mathcal{U}}^4(q_{12}^{-1}x)\hat{\mathcal{U}}^4(q_{13}^{-1}x)\hat{\mathcal{U}}^4(q_{23}^{-1}x)\hat{\mathcal{U}}^4(x)} \right\rangle_4, \\ \left\langle \hat{\mathcal{U}}^{4\vee}(q_4 x)^{-1} \right\rangle_4 + \mathfrak{q} \frac{\mathcal{V}_4(q_2)\mathcal{V}_4(q_1)}{\mathcal{V}_4(q_{12})} \left\langle \frac{\hat{\mathcal{U}}^{4\vee}(q_1^{-1}x)\hat{\mathcal{U}}^{4\vee}(q_2^{-1}x)\hat{\mathcal{U}}^{4\vee}(q_3^{-1}x)}{\hat{\mathcal{U}}^{4\vee}(q_{12}^{-1}x)\hat{\mathcal{U}}^{4\vee}(q_{13}^{-1}x)\hat{\mathcal{U}}^{4\vee}(q_{23}^{-1}x)\hat{\mathcal{U}}^{4\vee}(x)} \right\rangle_4, \end{aligned} \quad (3.224) $$

where

$$ \langle \mathcal{O} \rangle_4 = \frac{1}{\mathcal{Z}_4} \sum_{k \ge 0} \mathfrak{q}^k \tilde{\mathcal{Z}}_4^{\mathrm{D2}}[k; q_1, q_2] \mathcal{O}_{k,v}, \quad \mathcal{Z}_4 = \sum_k \mathfrak{q}^k \tilde{\mathcal{Z}}_4^{\mathrm{D2}}[k; q_1, q_2]. \quad (3.225) $$

Hence, using

$$ \frac{\mathcal{V}_4(q_3)}{\mathcal{V}_4(q_{23})\mathcal{V}_4(q_{13})} = -\prod_{i=1}^{3} \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i}, \quad \frac{\mathcal{V}_4(q_1)\mathcal{V}_4(q_2)}{\mathcal{V}_4(q_{12})} = -q_4 \prod_{i=1}^{3} \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i} \quad (3.226) $$
the iWeyl reflection is analogously defined as

$$ 
\begin{aligned}
\hat{U}^4(q_4 x)^{-1} &\mapsto -q \prod_{i=1}^3 \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i} \hat{U}^4(q_4 x)^{-1} \hat{A}(x)^{-1}, \\
\hat{U}^{4\vee}(q_4 x)^{-1} &\mapsto -q_4 q \prod_{i=1}^3 \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i} \hat{U}^{4\vee}(q_4 x)^{-1} \hat{A}^{\vee}(x)^{-1}
\end{aligned}
\quad (3.227) $$

where

$$ 
\begin{aligned}
\hat{A}(x) &= \frac{\hat{U}^4(q_{12}^{-1}x)\hat{U}^4(q_{13}^{-1}x)\hat{U}^4(q_{23}^{-1}x)\hat{U}^4(x)}{\hat{U}^4(q_1^{-1}x)\hat{U}^4(q_2^{-1}x)\hat{U}^4(q_3^{-1}x)\hat{U}^4(q_4x)}, \\
\hat{A}^{\vee}(x) &= \frac{\hat{U}^{4\vee}(q_{12}^{-1}x)\hat{U}^{4\vee}(q_{13}^{-1}x)\hat{U}^{4\vee}(q_{23}^{-1}x)\hat{U}^{4\vee}(x)}{\hat{U}^{4\vee}(q_1^{-1}x)\hat{U}^{4\vee}(q_2^{-1}x)\hat{U}^{4\vee}(q_3^{-1}x)\hat{U}^{4\vee}(q_4x)}
\end{aligned}
\quad (3.228) $$

Using the iWeyl reflection recursively, we can construct the qq-characters as

$$ 
\begin{aligned}
\hat{\mathcal{F}}^4(x) &= \hat{U}^4(q_4 x)^{-1} + q \left( -\prod_{i=1}^3 \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i} \right) \hat{U}^4(q_4 x)^{-1} \hat{A}(x)^{-1} + \dots, \\
\hat{\mathcal{F}}^{4\vee}(x) &= \hat{U}^{4\vee}(q_4 x)^{-1} + q \left( -q_4 \prod_{i=1}^3 \frac{1 - q_4^{-1} q_i^{-1}}{1 - q_i} \right) \hat{U}^{4\vee}(q_4 x)^{-1} \hat{A}^{\vee}(x)^{-1} + \dots,
\end{aligned}
\quad (3.229) $$

Generally, the complete formula will be

$$ 
\begin{aligned}
\hat{\mathcal{F}}^4(x) &= \sum_{\pi \in \mathcal{P}\mathcal{P}} (qq_4^{-1})^{\lvert\pi\rvert} \tilde{Z}_{123}^{\mathrm{D6}}[\pi] \hat{U}^4(q_4 x)^{-1} \prod_{\mathfrak{a} \in \pi} \hat{A}(\chi_{\bar{4},x}(\mathfrak{a}))^{-1}, \\
\hat{\mathcal{F}}^{4\vee}(x) &= \sum_{\pi \in \mathcal{P}\mathcal{P}} q^{\lvert\pi\rvert} \tilde{Z}_{123}^{\mathrm{D6}}[\pi] \hat{U}^{4\vee}(q_4 x)^{-1} \prod_{\mathfrak{a} \in \pi} \hat{A}^{\vee}(\chi_{\bar{4},x}(\mathfrak{a}))^{-1}.
\end{aligned}
\quad (3.230) $$

A derivation of this will be done using the operator formalism in section 7.

We can do the same analysis for other 3d theories and obtain the following statement.

**Proposition 3.16.** The qq-characters with respect with the 3d theory on $\mathbb{C}_a \times \mathbb{S}^1$ where $a \in \mathfrak{a}$ are

$$ 
\begin{aligned}
\hat{\mathcal{F}}^a(x) &= \sum_{\pi \in \mathcal{P}\mathcal{P}} (qq_a^{-1})^{\lvert\pi\rvert} \tilde{Z}_{\bar{a}}^{\mathrm{D6}}[\pi] \hat{U}^a(q_a x)^{-1} \prod_{\mathfrak{a} \in \pi} \hat{A}(\chi_{\bar{a},x}(\mathfrak{a}))^{-1}, \\
\hat{\mathcal{F}}^{a\vee}(x) &= \sum_{\pi \in \mathcal{P}\mathcal{P}} q^{\lvert\pi\rvert} \tilde{Z}_{\bar{a}}^{\mathrm{D6}}[\pi] \hat{U}^{a\vee}(q_a x)^{-1} \prod_{\mathfrak{a} \in \pi} \hat{A}^{\vee}(\chi_{\bar{a},x}(\mathfrak{a}))^{-1},
\end{aligned}
\quad (3.231) $$

where the iWeyl reflection is defined as

$$ 
\begin{aligned}
\hat{U}^a(q_a x)^{-1} &\mapsto -q \prod_{i=1}^3 \frac{1 - q_a^{-1} q_i^{-1}}{1 - q_i} \hat{U}^a(q_a x)^{-1} \hat{A}(x)^{-1}, \\
\hat{U}^{a\vee}(q_a x)^{-1} &\mapsto -q_a q \prod_{i=1}^3 \frac{1 - q_a^{-1} q_i^{-1}}{1 - q_i} \hat{U}^{a\vee}(q_a x)^{-1} \hat{A}^{\vee}(x)^{-1}
\end{aligned}
\quad (3.232) $$

where

$$ 
\hat{A}(x) = \frac{\hat{U}^a(x)}{\hat{U}^a(q_a x)} \prod_{i=1}^3 \frac{\hat{U}^a(q_a q_i x)}{\hat{U}^a(q_i^{-1} x)}, \quad \hat{A}^{\vee}(x) = \frac{\hat{U}^{a\vee}(x)}{\hat{U}^{a\vee}(q_a x)} \prod_{i=1}^3 \frac{\hat{U}^{a\vee}(q_a q_i x)}{\hat{U}^{a\vee}(q_i^{-1} x)}. \quad (3.233) $$
We call these *qq*-characters the D6-brane *qq*-characters, shortly D6 *qq*-characters. Namely, the *qq*-character with respect to the D2$_a$ theory on $\mathbb{C}_a \times S^1$ is the D6$_{\bar{a}}$ *qq*-character.

**D6 *qq*-characters as codimension two defects.** We can switch the roles of the D-branes for the 7d theory case and consider the D2 theory as the bulk theory and the D6 theory as the defect theory (see figure 7 and table 6). This setup gives rise to a codimension two defect, a line defect in the 3d theory. The D6$_{\bar{a}}$-brane will then give the *qq*-character of the D2$_a$-theory. Let us focus on the case $a = 4$ and $N_{123}^{\vee} = x$. The contour integral formula comes from

$$ \begin{aligned} & N_{123}^{\vee} \frac{\mathbf{P}_4^{\vee} \mathbf{P}_{123}}{\mathbf{P}_{\underline{4}}} \mathbf{N}_4 - \mathbf{P}_4^{\vee} N_{123}^{\vee} \mathbf{K} - \mathbf{P}_{123}^{\vee} N_4^{\vee} \mathbf{K} + \mathbf{P}_{123}^{\vee} \mathbf{K}_{123}^{\vee} \mathbf{K}_{123} + \mathbf{P}_{124}^{\vee} \mathbf{K}_4^{\vee} \mathbf{K}_4 + \mathbf{P}_4 \mathbf{K}_{123}^{\vee} \mathbf{K}_4 \\ & = -q_{123} N_{123}^{\vee} (\mathbf{N}_4 - \mathbf{P}_4 \mathbf{K}) - \mathbf{P}_{123}^{\vee} N_4^{\vee} \mathbf{K} + \mathbf{P}_{123}^{\vee} \mathbf{K}_{123}^{\vee} \mathbf{K}_{123} + \mathbf{P}_{124}^{\vee} \mathbf{K}_4^{\vee} \mathbf{K}_4 + \mathbf{P}_4 \mathbf{K}_{123}^{\vee} \mathbf{K}_4, \end{aligned} \quad (3.234) $$

Similar to the D2 case, this will come from a gauge origami system where both D2 and D6-branes are included. The contour integral is written as

$$ \frac{G_{\bar{1}}^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \prod_{\alpha=1}^n \frac{1}{(1 - q_4 x / v_\alpha)} \prod_{I=1}^k \mathcal{V}_4 \left(\frac{x}{x_I}\right) \prod_{\alpha=1}^n \prod_{I=1}^k g_{\bar{\alpha}} \left(\frac{v_\alpha}{x_I}\right) \prod_{I<J} \mathcal{A}_{\mathbb{C}^4} \left(\frac{x_I}{x_J}\right)^{-1}. \quad (3.235) $$

The red term comes from the transverse D6-brane, while the blue term comes from the bulk D2-brane. After localization, the red term gives the $\mathcal{U}$-functions. Similar to other cases, integrating out the D6$_4$-brane leads to a codimension two defect operator $\hat{T}^4(x)$, $\hat{T}^{4\vee}(x)$ and the expectation value of it with respect to the bulk D2$_4$-theory gives the partition function of the gauge origami system where there are D2$_4$ and D6$_{123}$-branes spanning transversely.

# 4 Free field realizations of contour integral formulas

In the previous section, we introduced the contour integral formulas and non-perturbative partition functions of the magnificent four (3.97), tetrahedron instantons (3.112), spiked instantons (3.112), and the coupled vortex system (3.155). Since we expect BPS/CFT correspondence of these systems, the contour integrals themselves should be related to vertex operators. In this section, we determine the explicit forms of the vertex operators and show that their compositions give free field realizations of the contour integral formulas. The main strategy is to relate D-branes in the physical setup with vertex operators. We discuss the operator formalism of the magnificent four in section 4.1, the tetrahedron instanton in section 4.2, the spiked instanton in section 4.3, and the coupled vortex system in section 4.4. Zero modes of the vertex operators are discussed in section 4.5. In section 4.6, we discuss the relation with graded quivers and that the vertex operators are defined from the quiver $q$-Cartan matrices. We also generalize the discussion to toric Calabi-Yau four-folds and give some conjectures.

The main statement is as follows.
**Theorem 4.1.** For each D-brane (D0, D2, D4, D6, D8), we can define the corresponding vertex operators as

<table><thead><tr><th>D-brane</th><th>space-time</th><th>vertex operator</th><th>reference</th></tr></thead><tbody><tr><td>D0-brane</td><td>S<sup>1</sup></td><td>A(x)</td><td>(4.2)</td></tr><tr><td>D2-brane</td><td>C<sub>a</sub> &times; S<sup>1</sup> (a &isin; 4)</td><td>S<sub>a</sub>(x)</td><td>(4.41)</td></tr><tr><td>D4-brane</td><td>C<sub>A</sub><sup>2</sup> &times; S<sup>1</sup> (A &isin; 6)</td><td>X<sub>A</sub>(x)</td><td>(4.29)</td></tr><tr><td>D6-brane</td><td>C<sub>a</sub><sup>3</sup> &times; S<sup>1</sup> (a &isin; 4)</td><td>W<sub>a</sub>(x)</td><td>(4.15)</td></tr><tr><td>D8-brane</td><td>C<sup>4</sup> &times; S<sup>1</sup></td><td>Z(x)</td><td>(4.6)</td></tr></tbody></table>

In the contour integral formula, the D0-branes giving instanton contributions arise from<sup>17</sup> $A^{-1}(x)$, while other D-branes arise from $S_a(x)$, $X_A(x)$, $W_{\bar{a}}(x)$, $Z(x)$. To include anti D-branes, we need to reverse the power of the operators as $S_a(x)^{-1}$, $X_A(x)^{-1}$, $W_{\bar{a}}(x)^{-1}$, $Z(x)^{-1}$.

## 4.1 Magnificent four

Let us start with the magnificent four system. Actually, the vertex operator of this system was already introduced in [77] in the context of double quiver gauge theory. See also [104].

We first introduce the vertex operator corresponding to the D0-brane which represents the instantons and vortices in the physical setup (see section 3.1). In the algebraic context, it is called the *root current*:

$$A(x) = a_0(x) : \exp \left( \sum_{n \neq 0} a_n x^{-n} \right) : , \quad [a_n, a_m] = -\frac{1}{n} \mathbf{P}_{\frac{1}{4}}^{[n]} \delta_{n+m,0}, \qquad (4.2)$$

where $a_0(x)$ is the zero mode that will be determined later. The right-hand side of the commutation relation of the root current represents the $CY_4$ geometry $C^4$. In the context of quiver W-algebra [60] and double quiver gauge theory [77], this is just the root current of the affine quiver W-algebra which is denoted by the $(\tilde{A}_0, \tilde{A}_0)$ theory in the terminology of [77]. The operator product of the A-operator is

$$A(x)A(x') = a_0(x)\overline{a_0(x')}A_{C^4}\left(\frac{x'}{x}\right)^{-1} : A(x)A(x') : , \qquad (4.3)$$

where we used the convention

$$a_0(x)a_0(x') = \overline{a_0(x)a_0(x')} : a_0(x)a_0(x') : . \qquad (4.4)$$

For the zero modes $a_0(z)$, we impose the condition that the OPE factor will be the same rational function. Namely, using (3.69), we impose

$$\overline{a_0(x)a_0(x')} = \overline{a_0(x')a_0(x)} \qquad (4.5)$$

and then we have the OPE factor symmetric in $x$ and $x'$. We set $\overline{a_0(x)a_0(x')} = 1$ (see section 4.5 for the derivation and explicit form of the zero modes).

<footnote n="17"><sup>17</sup>In this paper, when we write $V(x)^{-1}$ for a vertex operator $V(x)$, we are meaning : $V(x)^{-1}$ :. The normal ordering of them is implicitly imposed.</footnote>
To discuss the operator formalism of the magnificent four system, we need to introduce a vertex operator corresponding to the D8-brane (and $\overline{\text{D}}$8). We denote this vertex operator as $\mathcal{Z}(x)$:

$$ \mathcal{Z}(x) = \mathcal{Z}_0(x) : \exp \left( \sum_{n \neq 0} \mathbf{z}_n x^{-n} \right) : , \quad [\mathbf{z}_n, \mathbf{z}_m] = -\frac{1}{n} \frac{1}{\mathbf{P}_{\mathbf{4}}^{[n]}} \delta_{n+m,0}, \qquad (4.6) $$

where the relation with the root current is

$$ [\mathbf{a}_n, \mathbf{z}_m] = -\frac{1}{n} \delta_{n+m,0}, \quad \mathbf{z}_n = \frac{\mathbf{a}_n}{\mathbf{P}_{\mathbf{4}}^{[n]}}. \qquad (4.7) $$

Explicitly, we have

$$ \begin{aligned} \mathcal{Z}(x) \mathbf{A}(x') &= \mathcal{Z}_0(x) \overline{\mathbf{a}_0(x')} (1 - x'/x) : \mathcal{Z}(x) \mathbf{A}(x') : && (4.8) \\ \mathbf{A}(x') \mathcal{Z}(x) &= \mathbf{a}_0(x') \overline{\mathcal{Z}_0(x)} (1 - x/x') : \mathbf{A}(x') \mathcal{Z}(x) : && \end{aligned} $$

Since, in the magnificent four, the D8-brane appears with the $\overline{\text{D}}$8-brane, we introduce a vertex operator corresponding to the brane-antibrane coupled system as

$$ \tilde{\mathcal{Z}}^K(x) := \frac{\mathcal{Z}(x)}{\mathcal{Z}(Kx)} := \tilde{\mathbf{z}}_0^K(x) : \exp \left( \sum_{n \neq 0} \tilde{\mathbf{z}}_n^K x^{-n} \right) : , \quad \tilde{\mathbf{z}}_n^K = (1 - K^{-n}) \mathbf{z}_n = \frac{1 - K^{-n}}{\mathbf{P}_{\mathbf{4}}^{[n]}} \mathbf{a}_n, \quad (4.9) $$

where $K \in \mathbb{C}^\times$ is a generic parameter. This parameter is the parameter introduced in (3.101) and corresponds to the distance between the D8 and $\overline{\text{D}}$8 branes physically. This gives

$$ \begin{aligned} \tilde{\mathcal{Z}}^K(x) \mathbf{A}(x') &= \tilde{\mathbf{z}}_0^K(x) \overline{\mathbf{a}_0(x')} \frac{1 - x'/x}{1 - K^{-1}x'/x} : \tilde{\mathcal{Z}}^K(x) \mathbf{A}(x') : , \\ \mathbf{A}(x') \tilde{\mathcal{Z}}^K(x) &= \mathbf{a}_0(x') \overline{\tilde{\mathbf{z}}_0^K(x)} \frac{1 - x/x'}{1 - Kx/x'} : \tilde{\mathcal{Z}}^K(x) \mathbf{A}(x') : \end{aligned} \qquad (4.10) $$

For the zero modes, we impose that the contraction with the root current will be the same rational function after analytic continuation:

$$ \tilde{\mathbf{z}}_0^K(x) \overline{\mathbf{a}_0(x')} = K^{-1} \mathbf{a}_0(x') \overline{\tilde{\mathbf{z}}_0^K(x)}. \qquad (4.11) $$

We impose $\mathbf{a}_0(x') \overline{\tilde{\mathbf{z}}_0^K(x)} = 1$ (see section 4.5). Note that we are relaxing the conditions and only imposing conditions on the zero modes of the brane anti-brane coupled vertex operator $\tilde{\mathcal{Z}}^K$ but not the Z-operator.

**Proposition 4.2.** The contour integral formula of the magnificent four system shown in (3.97) is given as

$$ q^k \mathcal{Z}_k^{\text{D8}} = \frac{q^k \mathcal{G}_a^k}{k!} \oint \prod_{I=1}^k \frac{dx_I}{2\pi i x_I} \langle \mathbf{A}_k^{-1} \tilde{\mathbf{Z}}_{\underline{n}}^{K} \rangle \qquad (4.12) $$

where we used

$$ \mathbf{A}_k^{-1} = \prod_{I=1}^k \mathbf{A}(x_I)^{-1}, \quad \tilde{\mathbf{Z}}_{\underline{n}}^{K} := \prod_{\alpha=1}^n \tilde{\mathbf{Z}}^{K_\alpha}(v_\alpha) : , \quad \langle \mathbf{A}_k^{-1} \tilde{\mathbf{Z}}_{\underline{n}}^{K} \rangle = \prod_{\alpha=1}^n \prod_{I=1}^k \frac{1 - K_\alpha v_\alpha / x_I}{1 - v_\alpha / x_I} \prod_{I<j} \mathcal{A}_{\mathbb{C}^4} \left( \frac{x_I}{x_J} \right)^{-1} \qquad (4.13) $$

and $\langle \mathcal{O} \rangle = \langle 0 | \mathcal{O} | 0 \rangle$.
We can generalize the correlation function by including an extra parameter $p = e^{2\pi i\tau}$ as $\text{tr}(p^{L_0}\mathcal{O})$ which will give an elliptic deformation (a torus correlator). $L_0$ is the degree counting operator. Taking the limit $p \to 0$ from this elliptic deformed correlator gives the vacuum expectation value: $\text{tr}(p^{L_0}\mathcal{O}) \xrightarrow{p \to \infty} \langle 0|\mathcal{O}|0\rangle$. Under this elliptic deformation, the arising algebra will be quiver elliptic W-algebra and will be discussed in section 12. For the trigonometric case, we simply use the vacuum expectation value of the operators.

Note that we need to determine an order in the operators when doing explicit computations. In most of the cases, we simply assume $\prod_{I=1}^k \mathcal{O}(x_I) = \mathcal{O}(x_k) \cdots \mathcal{O}(x_2)\mathcal{O}(x_1)$ for an operator $\mathcal{O}(x)$. Actually, because of the zero modes conditions (4.5) and (4.11), the order of the operators is not relevant after analytic continuation. Note also that the OPE of the $\tilde{Z}^K(x)$ operators give the perturbative factor introduced in (3.106):

$$ \tilde{Z}^{K_n}(v_n) \cdots \tilde{Z}^{K_1}(v_1) = \prod_{\beta > \alpha} \mathcal{Z}_{1-\text{loop}}^{\text{D8-D8}}(v_\alpha, K_\alpha | v_\beta, K_\beta) : \prod_{\alpha=1}^n \tilde{Z}^{K_\alpha}(v_\alpha) : . \quad (4.14) $$

## 4.2 Tetrahedron instanton

The tetrahedron system is obtained by adding D6-branes and D0-branes. To reproduce the free field realization of the integral formula in (3.112), we introduce the following vertex operator that represents the D6-brane wrapping $\mathbb{C}_{\tilde{a}}^3 \times S^1$:

$$ W_{\tilde{a}}(x) = w_{\tilde{a},0}(x) : \exp \left( \sum_{n \neq 0} w_{\tilde{a},n} x^{-n} \right) : , \quad [w_{\tilde{a},n}, w_{\tilde{b},m}] = - \frac{1}{n} \frac{\mathbf{P}_{\tilde{a}}^{[n]}}{\mathbf{P}_{\tilde{a}}^{[-n]} \mathbf{P}_{\tilde{b}}^{[n]}} \delta_{n+m,0}, \quad (4.15) $$

where $a \in \mathbb{4}$ and $w_{\tilde{a},0}(x)$ are zero modes. The relation with the root current is

$$ [a_n, w_{\tilde{a},m}] = -\frac{1}{n} \mathbf{P}_a^{[n]} \delta_{n+m,0}, \quad w_{\tilde{a},n} = \frac{a_n}{\mathbf{P}_{\tilde{a}}^{[-n]}}, \quad (4.16) $$

which gives the operator product

$$ \begin{align} A(x)W_{\tilde{a}}(x') &= \overline{a_0(x)w_{\tilde{a},0}(x')}\mathcal{V}_a(x'/x)^{-1} : A(x)W_{\tilde{a}}(x') : \tag{4.17} \\ W_{\tilde{a}}(x')A(x) &= \overline{w_{\tilde{a},0}(x')}a_0(x)\mathcal{V}_a(q_a^{-1}x/x') : W_{\tilde{a}}(x')A(x) : . \nonumber \end{align} $$

Using (3.69), we impose the zero mode conditions as

$$ \overline{a_0(x)w_{\tilde{a},0}(x')} = q_a \overline{w_{\tilde{a},0}(x')}\overline{a_0(x)}. \quad (4.18) $$

Explicitly, we impose $\overline{a_0(x)w_{\tilde{a},0}(x')} = 1$ (see section 4.5). The tetrahedron instanton system is then given as follows.

**Proposition 4.3.** The tetrahedron instanton partition function (3.112) is equivalent to the following vertex operator correlation function after analytic continuation,

$$ q^{\underline{k}} \mathcal{Z}_{\underline{k}}^{\text{D6}} = \frac{q^{\underline{k}} \mathcal{G}^{\underline{k}}}{\underline{k}!} \oint \prod_{a \in \mathbb{4}} \prod_{I=1}^{k_{\tilde{a}}} \frac{dx_{\tilde{a},I}}{2\pi i x_{\tilde{a},I}} \langle \mathbf{A}_{\underline{k}}^{-1} \mathbf{W}_{\underline{n}} \rangle \quad (4.19) $$
where

$$ q^k = \prod_{a \in \mathcal{A}} q^{k_{\bar{a}}}, \quad A_k = \prod_{a \in \mathcal{A}} \prod_{I=1}^{k_{\bar{a}}} A(x_{\bar{a},I}), \quad W_{\underline{n}} := \prod_{a \in \mathcal{A}} \prod_{\alpha=1}^{n_{\bar{a}}} W_{\bar{a}}(v_{\bar{a},\alpha}), \qquad (4.20a) $$

$$ \langle A_{\underline{k}}^{-1} W_{\underline{n}} \rangle = \prod_{a,b \in \mathcal{A}} \prod_{\alpha=1}^{n_{\bar{a}}} \prod_{I=1}^{k_{\bar{b}}} \mathcal{V}_a \left( \frac{v_{\bar{a},\alpha}}{x_{\bar{b},I}} \right) \prod_{(a,I) < (b,J)} \mathcal{A}^{\mathcal{C}^4} \left( \frac{x_{\bar{a},I}}{x_{\bar{b},J}} \right)^{-1} \qquad (4.20b) $$

Note that the order of the operator product in $A_{\bar{k}}$ does not change the result. The OPE of the W-operators give the one loop perturbative part in (3.130):

$$ W_{\bar{b}}(v_{\bar{b},\beta}) W_{\bar{a}}(v_{\bar{a},\alpha}) = \mathcal{Z}_{1-\text{loop}}^{\text{D6-D6}}(v_{\bar{a},\alpha}, \bar{a} | v_{\bar{b},\beta}, \bar{b}) : W_{\bar{b}}(v_{\bar{b},\beta}) W_{\bar{a}}(v_{\bar{a},\alpha}) : . \qquad (4.21) $$

**Relation with magnificent four.** Let us focus on the 7d U(1) theory on $\mathbb{C}_{\bar{a}}^{3} \times \mathbb{S}^{1}$. Starting from the 9d U(1|1) theory of the magnificent four and tuning the parameter $K = q_{a}$, we have

$$ \tilde{Z}^{q_a}(x) := \frac{Z(x)}{Z(q_a x)} \simeq W_{\bar{a}}(x), \qquad (4.22) $$

where the equality $\simeq$ is up to extra zero modes depending on the explicit form. In our notation in section 4.5, this becomes an exact identity. Generally, starting from a 9d U(n|n) magnificent four theory with parameters $(K_{\alpha})_{\alpha=1}^{n}$, $n = \sum_{a \in \mathcal{A}} n_{\bar{a}}$ and tuning

$$ (K_{\alpha})_{\alpha=1}^{n} \rightarrow (K_{\bar{a},\alpha})_{\alpha=1}^{n_{\bar{a}}}, \quad K_{\bar{a},\alpha} = q_{a} \qquad (4.23) $$

we have

$$ \langle A_k^{-1} \tilde{Z}_{\underline{n}}^{K} \rangle = \langle A_k^{-1} W_{\underline{n}} \rangle, \qquad (4.24) $$

and thus, we obtain the tetrahedron instanton system. Note also that setting $K = q_{a}^{-1}$ gives

$$ \tilde{Z}^{q_a^{-1}}(x) := \frac{Z(x)}{Z(q_a^{-1}x)} := W_{\bar{a}}(q_a^{-1}x)^{-1} \qquad (4.25) $$

which allows $W_{\bar{a}}(x)$ to appear in the denominator.

Physically, this property suggests that the D8-branes and anti-D8-branes annihilate in a specific distance under the $\Omega$-background and eventually reproduce the, generally intersecting, D6-branes system [15, 16, 52, 81], which is also interpreted as a tachyon condensation [105].

**Supergroup generalization.** Following the construction of the magnificent four system in (4.9), we can write down the contour integral formula with D6 operators appearing in the denominator:

$$ \therefore \frac{W_{\bar{a}}(v_1) \cdots W_{\bar{a}}(v_n)}{W_{\bar{a}}(u_1) \cdots W_{\bar{a}}(u_m)} : \qquad (4.26) $$

The contour integral formula is proportional to

$$ \oint \prod_{a \in \mathcal{A}} \prod_{I=1}^{k_{\bar{a}}} \frac{dx_{\bar{a},I}}{2\pi i x_{\bar{a},I}} \langle A_{\underline{k}}^{-1} W_{\underline{n}|\underline{m}} \rangle \qquad (4.27) $$