## 3 The Proposed Method
### The LWAS Framework
#### 3.1.2 The Topology-Fused Search Space  
Figure 1 illustrates the search space backbone of LWAS, which also contains an acoustic encoder and an attention-based decoder. By adjusting the encoder output target, the search space can support both the architecture search of the CTC and AED frameworks.  
As mentioned above, there have existed several human-designed architectures that have different advantages in performance or resources, as shown in Figure 2a,c. To integrate the advantages of different existing architectures, we propose fusing the topologies of these architectures into the encoder/decoder block search space, i.e., the topology-fused search space, as shown in Figure 2b,d. Drawing on literature methods, we introduce three kinds of edges into the search space, which correspond to MHA, CNN and FFN modules respectively and allow optional topology connections between them. In this way, the architecture topologies of many existing works can be covered in our search space, e.g., in the encoder search space, Transformer {\(x^{0}\)\(\rightarrow\)\(x^{1}\)\(\rightarrow\)\(x^{3}\)}, Conformer {\(x^{0}\)\(\rightarrow\)\(x^{1}\)\(\rightarrow\)\(x^{2}\)\(\rightarrow\)\(x^{3}\)}, LiteTransformer {\(x^{0}\)\(\rightarrow\)\(x^{1}\)\(\rightarrow\)\(x^{3}\)}, \(x^{0}\)\(\rightarrow\)\(x^{2}\)\(\rightarrow\)\(x^{3}\)}, and LightConv {\(x^{0}\)\(\rightarrow\)\(x^{2}\)\(\rightarrow\)\(x^{3}\)} can be obtained. Moreover, apart from these human-designed architectures, more brand-new architecture topologies also can be obtained in the search space. On top of fused topologies, we provide a set of candidate operations for each edge to allow the potential of different operation combinations. The MHA operations own flexible head numbers, e.g., 4, 8, 16, denoted as MHA4, MHA8, and MHA16. A set of CNN operations with diverse kernel sizes are provided, e.g., 7, 15, 31, denoted as CNN7, CNN15, CNN31. The FFN operations have various hidden dimensions, e.g., 512, 1024, 2048, denoted as FFN512, FFN1024, FFN2048.  
Following the paradigm of NAS, the output of each edge \((i,j)\) is the weighted sum of all candidates \(o^{(ij)}(x)=\sum_{k=1}^{|\mathcal{O}|}c_{k}^{(i,j)}o_{k}^{(i,j)}(x^{i})\) and each node can be calculated based on all its predecessors: \(x^{(j)}=\sum_{i<j}o^{(i,j)}(x^{(i)})\). Different from the fixed block number of human-designed architectures, LWAS further makes the block number elastic and determines it based on a block number search module, which is achieved on top of the continuous relaxation of block outputs:  
\[x^{Out}=\sum_{m=1}^{N}c_{n}x^{L_{n}} \tag{3}\]  
where \(x^{Out}\) is the final output of the encoder/decoder, \(N\) is the maximum number of the encoder/decoder block, \(x^{L_{n}}\) refers to the output of the \(n\)-th block. And then, for each block of the encoder and decoder, the block architecture will be derived from the corresponding search space in a block-wise way.  
Figure 1: The backbone of our proposed search space. The encoder output can be fed into the CTC classifier or the decoder, and in this way, the search space can support the architecture search of the CTC and AED frameworks meanwhile.