## 3 Proposed Method
### Model Architecture  
This paper proposes a disfluency detection network designed for disfluency classification using limited, disfluent speech data. The paper uses the wav2 vec2.0 model to extract contextual speech embeddings from raw audio and achieves good disfluency classification results with limited disfluent data. As shown in Figure 1, the contextual representation of the speech input obtained through the wav2 vec2.0 model is then used to more effectively extract disfluency features through a CNN and Transformer model, and finally, a fully connected layer is used to obtain binary classification results for fluent and disfluent speech. Specifically, the disfluency detection network first employs a single-layer CNN to extract specific disfluent features from speech signals, which are then passed to the Transformer layer for further processing. The subsequent sections describe in detail the structure of the proposed disfluency detection network.  
First, the raw audio is input into the pre-trained wav2 vec2.0 model, obtaining the audio feature representation of the last hidden layer of the wav2 vec2.0 model. As shown  
Figure 1: The architecture of the proposed model.  
in Formula (1), where \(X\in R^{L\times d}\) represents the raw speech signal, L represents the length, and d represents the dimension. \(f_{\text{wav2 vec2.0}}\) represents the pre-trained wav2 vec2.0 model.  
\[Z=f_{\text{wav2 vec2.0}}(X) \tag{1}\]  
Afterwards, local features can be extracted through the CNN by sliding convolution kernels, and these feature representations contain local structures and speech features in the audio, which can be further used for disfluency detection tasks. Specifically, 768-dimensional audio features are inputted into the CNN network structure, which has 50 output channels. The CNN layer applies 50 different filters to the input, each with a width of 2 and with no padding added to the input, producing 50 output feature maps. These feature maps reflect disfluent information in speech signals, such as repetitions, blocks, and prolongations. After the CNN, the obtained features are fed into the Transformer model for more effective feature extraction. The encoder part of the Transformer can be seen as a combination of a multi-head self-attention mechanism and a feed-forward neural network. These network layers can effectively learn the relationships and sequence information between features, generating a more abstract and high-level representation. This part has an input vector dimension of 50 and is composed of two stacked Transformer encoder layers. Each Transformer encoder layer uses the same set of parameters, including 10 multi-head self-attention mechanisms to capture different semantic information.  
\[\hat{Z}=\text{Transformer}(\text{ConvID}(Z)) \tag{2}\]  
Finally, the output of the encoder is classified through a fully connected layer to obtain the results of the disfluency classification.  
Due to the denser and more complex nature of speech data compared to text data, the combined structure of wav2 vec2.0 + CNN + Transformer can fully explore more details and complexities in the speech signal, effectively extracting and classifying speech features. This approach can improve the performance of disfluency detection models and has a certain level of generality and applicability.