# How Successful Is Transfer Learning for Detecting Anorexia on Social Media?
## 4 Methodology
### Deep Neural Networks
#### 4.1.3 Convolutional Neural Networks  
Convolutional Neural Networks (CNNs) were originally designed for deep learning computer vision tasks, but they have also proven to be useful for NLP tasks [36]. In CNN, every network layer acts as a detection filter for the presence of specific features or patterns present in the original data. The first layers in a CNN detect large features, later layers detect increasingly smaller sets of more abstract features, and the last layer of the CNN can perform a classification by combining all the specific features detected by the previous layers in the input data.  
We tested these networks with different hyperparameter values and the ones that performed best are presented below. We perform 10-fold cross-validation with the training dataset, in this way we get the best hyperparameters over 20 different combinations with Bayesian Optimization. Finally, we used the test set to predict and evaluate the predictions using the best hyperparameters which are presented below. We used 100 units in the case of CNN and 150 units in the case of LSTM. BiLSTM uses 100 units in each LSTM. A batch size of 128 for BiLSTM and CNN and 64 for LSTM was employed. A max-pooling layer was appended to the CNN model. For all neural networks, a dense layer of size 50 with Rectified Linear Unit (ReLU) activation and a dropout function was added to help prevent overfitting. Specifically, we use a dropout rate of 0.25. This was followed by a dense layer with a size of 1 to represent the number of classification classes with a sigmoid function determining the output. Figures 1-3 show the architectures followed for the LSTM, BiLSTM and CNN models, respectively.  
For all previous neural networks, we used an embedding layer as input. Word embedding is a form of representing words and documents using a dense vector representation. The position of a word within the vector space is learned from the text and is based on the words that surround the word. This allows words that are used in a similar way to have a similar representation, capturing their meaning.  
The vocabulary of the corpus is composed of 12,313 unique words and Table 1 shows the number of words included in the Spanish pre-trained embeddings ([[https://github.com/dccuchile/spanish-word-embeddings](https://github.com/dccuchile/spanish-word-embeddings)]([https://github.com/dccuchile/spanish-word-embeddings](https://github.com/dccuchile/spanish-word-embeddings))).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Corpus** & **Algorithm** & **Dimension** & **Included** & **Non-Included** \\ \hline Spanish Unannotated & FastText & 300 & 11,032 & 1281 \\ Corpora & FastText & 300 & 10,469 & 1844 \\ Wikipedia Spanish & FastText & 300 & 10,394 & 1919 \\ Spanish Billion Word & GloVe & 300 & 10,394 & 1919 \\ Corpus & & & & \\ \hline \hline \end{tabular}
\end{table}
Table 1: Analysis of the number of words included in word embeddings.  
Figure 1: LSTM architecture.  
Figure 2: BiLSTM architecture.  
Figure 3: CNN architecture.  
After analyzing the existing embeddings in Spanish in our experiments we used the embeddings extracted from the Spanish Unannotated Corpora [37] because this is the model that contains the most words from our vocabulary.