# Automated Clinical Impression Generation for Medical Signal Data Searches
## 2 Methods
### Text Generation for EEG Signal Using DeepSleepNet Encoder and a Transformer  
In an information retrieval system, indexing a sentence can be easily implemented using an inverted index [13]. However, because the EEG signal is a continuous value, it is difficult to directly index it into an existing database and use search engines, such as Lucene [2] or Elasticsearch [3]. Therefore, this work proposes a text generator for the signal. Once the signal is described by the generated text, it can be stored using a word-level inverted index [14]. For example, as depicted in Figure 2, the signals named _Doc 1_ already have an impression reported by neurologists, stating \"Abnormal EEG. Excess drowsiness. Low voltage pattern\". However, the signals named _Doc 2_ do not have any report yet. Thus, the goal of this research is to generate an associated report that precisely describes the signal using our text generator and to apply an inverted index using a tokenizer, such as a one-gram or bi-gram tokenizer. To achieve this goal, this work introduces a text generator for word-level inverted indexes that combines convolutional neural networks (CNNs) borrowed from DeepSleepNet [11] and a transformer [12]. DeepSleepNet [11] is a model proposed for detecting sleep stages in time series EEG data. It is composed of both convolutional neural networks (CNNs) and long short-term memory networks (LSTMs), which are used to capture patterns of EEG activity in both local and global areas, respectively. In this work, the CNN portion of the model is only considered since the proposed method utilizes the transformer to learn patterns in the global area. The convolutional layers of DeepSleepNet are designed to detect low- and high-frequency patterns along the time axis using two branches of large and small filters, making it a popular choice for research aiming to capture features from EEG signals [15; 16; 17]. The two-branch CNN structure proposed in [11] is adopted in the suggested method in this work for mapping the signal into a feature space. However, the network is modified to be multi-channel CNNs to accommodate EEG signals recorded from diverse channels. The modified version is named in this work DeepSleepNet Encoder (DE), which is depicted in Figure 3. Given an EEG signal \(S=\{e_{0},\ldots,e_{t-1}\}\), the signal is divided into fixed-length segments, such as every 30 s, yielding segments \(\{e_{0},\ldots,e_{6000-1}\},\ldots,\{e_{t-6000}\ldots,e_{t-1}\}\). Then, each segment is fed into the DeepSleepNet Encoder to extract local features, resulting in a set of features \(F=\{f_{0},\ldots,f_{m}\}\). Transformer [12] is an encoder-decoder model which is based on a self-attention mechanism for sequence transduction, especially machine translation. The self-attention mechanism was proposed to learn a sentence representation by using query \(Q\), key \(K\), and value \(V\). In the encoder, the self-attention mechanism discovers relationships among the inputs using the scaled dot-product attention proposed in [12], which is defined as follows:  
\[\text{Attention}(Q,K,V)=\text{softmax}\big{(}\frac{QK^{T}}{\sqrt{d_{k}}} \big{)}V \tag{1}\]  
Figure 2: Flowcharts for inverted indexing of signals with impressions, both provided by a neurologist (_Doc 1_) and generated by the proposed method (_Doc 2_). where \(Q=FW^{Q}\), \(K=FW^{K}\), and \(V=FW^{V}\) in our problem for the initial layer and \(W\) is the matrix of the trainable parameters. The encoder of the transformer is utilized to discover relationships among the divided signal segments. Additionally, the decoder of the transformer is exploited to capture the correlation among words in the target sentence, as well as to learn the relationship between the words and the input features given by the encoder using the cross-attention mechanism.