# Stepwise Multi-Task Learning Model for Holder Extraction in Aspect-Based Sentiment Analysis
## 2 Related Works
### Holder Extraction  
A holder extraction task involves extracting a direct or indirect holder or source of the sentiment represented in the text [21]. Figure 5 shows examples of when sentiment holders are direct and indirect.  
In Sentence (1) in Figure 5, a sentiment word \"_enjoyed_\" has a direct holder of \"_[PERSON]_\", while in Sentence 2, the holder \"_[PERSON]_\" for a sentiment word \"_not like_\" is mentioned by \"_[PERSON]_\" which is indirectly described.  
A holder extraction task is also referred to as opinion source identification [21], source extraction [9], opinion holder extraction [10], or opinion entity identification [11]. [PERSON] and [PERSON] [22] employed the holder extraction task and proposed an approach to a sequence labeling problem. A holder is defined as three items below. This task is distinguished from the named entity recognition (NER) task due to the third item.  
1. A holder is generally a noun phrase.
2. A holder must contain semantic sentiment or be an entity capable of expression.
3. A holder must be related to sentiment.  
[PERSON] et al. [9] and [PERSON] and [PERSON] [11] experimented with sentiment word, target, and holder extraction tasks together using a CRF tagger and binary classifier. [PERSON] and [PERSON] [10] improved accuracy by modeling the relationship between sentiment word and holder within a text based on the insight that one holder may have multiple sentiment words. However, the above approach has a disadvantage that a vast range of feature engineering procedures is required.  
Accordingly, [PERSON] and [PERSON] [23] simplified the feature engineering procedure by applying LSTM which is a neural network model. However, it demonstrated poorer performance compared to machine learning approaches due to insufficient training data. [PERSON] et al. [12] used a multi-task learning model for target extraction and holder extraction task. For resolving the issue of insufficient data, a shared layer was trained with a semantic role labeling model. The output of the semantic role labeling layer was used as input for task-specific layers, which is the last layer of the multi-task learning model. Data insufficiency was resolved by reducing the number of input data to simplify the process.  
Using a multi-task learning model to simultaneously employ two tasks of target and holder extraction from one text increased training efficiency and solved the issue of lack of data, but such a method entails the following three limitations. First, the entire model  
Figure 4: An example for sentence-level ABSA containing aspects: _FOOD (colored in blue)_ and _PRICE (colored in red)_.  
Figure 5: Examples of direct and indirect sentiment holders.  
becomes dependent on errors generated in a semantic role labeling layer. Second, sentiment word information, which can be a clue for target and holder extraction tasks, was not used. Third, the contextual information of text data was changed to semantic role information, thus causing information loss.  
Therefore, in this paper, sentiment word and target extraction tasks are simultaneously executed for the holder extraction task. Sentiment word and target information become a clue for holder extraction. Furthermore, the extraction task was not replaced with another task to avoid information loss.