# Sentiment Analysis Based Requirement Evolution Prediction
## 5 Discussion  
Our hierarchical joint model achieved results competitive with the software requirements changing analytical model of ASUM and the hierarchical aspect sentiment model of HASM. We highlighted the identification of hierarchical features and sentiments where this facilitates product feature requirements evolution predictions from users' reviews. In our work, hierarchical features requirement evolution prediction can be divided into two core components: supervised LSTM deep learning for sentence-level sentiment analysis and unsupervised topic modeling for hierarchical features identification. Thus, we analyzed the impact of different word embeddings, activation functions, and sizes of hidden layers for LSTM sentiment analysis and discussed a prior of the Dirichlet distribution for HLDA topic modeling. Word embedding is a distributed representation of words, which is combined with a word and its context to represent a word as real valued, dense, and having low-dimensional vectors and greatly alleviated the data sparsity problem [24]. Syntactic or semantic properties of the word are potentially described by each dimension of word embedding. The semantic representation of sentences and texts was captured by the use of word embedding. Pre-trained embeddings improved the model's performance. Further, combining different pre-trained word embedding with deep neural networks brought new inspiration to various sentiment analysis tasks. We plugged the readily available embedding of sentences into the LSTM network framework and used it as the only feature to avoid \begin{table}
\begin{tabular}{l l l} \hline \hline
**Hierarchy** & \multicolumn{3}{c}{**Features and Sentiments**} \\ \hline \multirow{3}{*}{Level-0} & features: & interface, update, power, usability, performance \\  & pos: & handy, could, pretty, friendly, usable \\  & neg: & frustrating, confusing, crash, fault, inconvenient \\ \hline \multirow{3}{*}{Level-1} & features: & permission, log, option, connection, availability \\  & pos: & smart, install, graphical, adaptive, instant \\  & neg: & error, slow, boring, stupid, block \\ \hline \multirow{3}{*}{Level-2} & features: & popup, buttons, text, account, login \\  & pos: & attractive, compatible, support, control, search \\  & neg: & small, missing, confusion, poor, useless \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Hierarchical requirement features with sentiment orientation. manual feature engineering efforts for sentimental orientation classification. Polarity was predicted on either a binary (positive or negative) or multivariate scale using sentiment polarity classification techniques. The compositional distributed vector representations for review sentences were obtained by concatenating their word embedding representations. The resulting high-level distributed representations were used as the individual features to classify each sentence-sentiment pair. As a result, the LSTM network's performance can be improved by choosing the optimal hyperparameters of the dimension of pre-trained embeddings for the word and paragraph vector training. The proper selection of the activation function is an effective way to initialize the weight matrix \(W\) to reduce the gradient vanishing and gradient explosion effect. The activation functions used in LSTM for sentiment analysis are sigmoid and tanh. The update of the state value is described as Equation (5) in the form of addition with the sigmoid and tanh function to solve the problem of long-term dependencies in LSTM better. The size of hidden layers in LSTM depends on the application domain and context. A model with many hidden layers will resolve non-linearly separable data. For the sentiment classification task with the deep neural network model, we assumed that the data required a non-linear technique. The introduction of the hidden layer(s) of deep LSTM network made it possible for the deep network to exhibit non-linear behavior. The different size of hidden layers in LSTM affected the results of the sentiment classification task.