## 2 Background and Related Research
### Related Research  
[PERSON] et al. [21] proposed a B-LSTM neural network architecture based on Word2 vec's CBOW architecture. The main goal of the model was to efficiently learn the general context embedding function for variable-length sentence contexts around the target word, making contextual expression very simple. This resulted in cutting-edge results in sentence completion, vocabulary replacement, and word sense clarification, outperforming other techniques of general contextual representation of average word embedding. Another key model was proposed for medical texts by [PERSON] [22]; they developed a model for understanding complex, specific medical structures as medical text contains some of the most specialized information that only field experts can understand. The proposed embedding model is generated in a specialized general corpus with or without features handcrafted by medical experts. [PERSON] et al. [23] proposed a text classification model based on Word2 vec and LSTM to efficiently classify text in the security field. They used a pre-trained Word2 vec model to overcome the high dimensionality suffered by traditional methods. Finally, by training the LSTM classification model, the study extracted text functions and performed patent text classification in the security field. The results show that  
Figure 3: The Bi-long short-term memory (LSTM) process used to capture sequential features. The last hidden layer of the LSTM is extracted as features representing the text. the model can provide classification of 93.48% for these patent texts. This laid a solid foundation for further research and effective use of patents. To increase the classification accuracy, CNN and LSTM have been combined in some studies. The LSTM model and a CNN were used for a variety of natural-language processing (NLP) tasks with surprising and effective results. [PERSON] et al. [24] proposed a hybrid model that uses a very deep CNN and LSTM to overcome previous challenges for sentiment analysis. The proposed model also uses dropout techniques, normalization techniques, and rectified linear units to increase the prediction accuracy. Results show that the proposed hybrid CNN-LSTM model outperforms conventional deep learning and machine learning technologies in terms of precision, recall, F1-score, and accuracy. The study in [25] proposed a text classification model called CNN-COIF-LSTM and experiments that included eight variants, showed that the combination of CNN and LSTM without an activation function or a variant thereof exhibited higher accuracy. [PERSON] et al. [26] proposed a local CNN-LSTM model with a tree structure to measure the steps in emotional analysis. Unlike conventional CNNs, where the entire text is considered as the input, which is divided into multiple regions and useful emotional information is extracted from each region and weighted according to that region; the proposed regional CNN uses a portion of the text as a region. By combining CNN and LSTM, the classification accuracy is further increased because this combination considers both local (regional) information in sentences and long-distance dependencies between sentences. Novel hybrid CNN-LSTM models have been proposed and achieved good results than the previous ones. [PERSON] et al. [27] have proposed a novel hybrid approach that leverages the capabilities of CNN to extract local features and addresses its inherent weakness to express long-term contextual features. The model also tries to tackle the natural weaknesses of LSTM which processes information sequentially and hence making it a worst feature extractor. The hybrid model achieved better results compared to counterparts' models, but the results were not interesting as compared with the models that use attention mechanism. [PERSON] et al. [28] proposed a novel hybrid model that combines different word embedding (Word2 Vec, FastText, and character-level embedding) with various learning approaches (LSTM, Gated recurrent unit (GRU), Bi-LSTM, CNN). The model used CNN and LSTM for feature extraction.