Human Behavior Assessment using Ensemble Models (2020)

Abstract: Behavioral analysis is a pertinent step in today’s automated age. It is important to judge a statement on a variety of parameters before reaching a valid conclusion. In today’s world of technology and automation, Natural language processing tools have benefited from growing access to data in order to analyze the context and scenario. A better understanding of human behaviors would empower a range of automated tools to provide users a customized experience. For precise analysis, behavior understanding is important. We have experimented with various machine learning techniques, and have obtained a maximum private score of 0.1033 with a public score of 0.1733. The methods are described as part of the ALTA 2020 shared task. In this work, we have enlisted our results and the challenges faced to solve the problem of the human behavior assessment.

## introduction human behavior assessment is an important computation task that automates the task of detecting human behavior from textual data. the behavior in the text depends on many parameters. some of these include words of different types including attitude and appraisal @xcite . the use of evaluative language allows for a greater deal of solidarity in the text @xcite . various rule-based algorithms can be used to evaluate the essence of the sentence. the sentence judgment can be divided into two sections viz. social esteem and social sanction. the former comprises normality, capacity, and tenacity. whereas the latter includes veracity and propriety. sentence classes, their meaning, and sample explanation are included in various approaches include natural language processing tools to extract the sentiment or detect human behavior from the text. the work by @xcite , describes various aspects of the sentiment analysis and opinion mining problem. since the above task belongs to the natural processing domain, it brings along various difficulties, including coreference resolution, negation handling among many @xcite . to classify the sentences into the above 5 classes, we have formulated the same into a machine learning multi-classification task. this paper investigates different approaches for the human behavior assessment, as part of the australasian language technology association (alta) 2020 shared task @xcite . the rest of the paper is divided as follows. the related works are enlisted in section 2. the dataset description is given in section 3. the experimental setup is given in section 4. the experimentation details are described in section 5. results and analysis are tabulated in section 6. finally, we conclude with discussion and conclusion in section 7. ## related works on experimental investigation of the problem, we have found that the given problem closely resembles the multi-class human sentiment analysis such as the multi-class sentiment analysis using clustering and scoring @xcite . the work by @xcite , uses the semantic analysis and clustering on a bag of nouns to identify the class of the sentiments based on the textual description. other works show the use of multi-class class svm foot_0 @xcite which employs topic adaptive learning method to produce more generic and abstract based systems. there also exists machine learning systems that perform discourse analysis @xcite 2017) on the description to map out the sentiment. some works also show that systems are performing better if there is a fusion of more than one architecture like that of the gme-lstm(a) foot_1 @xcite and @xcite , which uses multi-phased architecture and thereby takes the advantage of those methods as well as the concept of word-level and fine level fusion techniques to surpass other state-of-the-art techniques. as a part of this experimentation, we have used ensemble models to tackle different aspects of the problem. starting from the xlnet pretraining as given in section 4.1 to decision tree classifier is discussed in section 4.4 and up to xgboost (in section 4.5). these were used in different phases of feature generation, multi-class classification, analysis, and validation. ## dataset the labeled dataset foot_2 for the alta 2020 shared task was provided by the organizers. the dataset included single, multiple, or no labels for a single sentence as the output label. the train data contains a total of 200 instances of labeled data, whereas the test set contains 100 instances. the dataset provided was based on the semeval 2018 ait disc dataset 4 (mohammad et al., 2018). for the purpose of experimentation, we have worked with both sets of data, with and without preprocessing. preprocessing steps include removal of punctuation and stop words. since the data provided to us by the organizers is quite small as discussed in section 3, we employed the use of machine learning techniques instead of data craving deep learning methods. for the word embeddings, we have experimented with the xl-net @xcite pre-trained embeddings and the freely available spacy foot_4 word embeddings. ## experimentation as discussed in section 4, we have used various machine learning techniques for the given multiclassification problem and have used feature vectors generated from different deep learning approaches as discussed in sections 4.1 and 4.2. the generated sentence vectors of each sentence are fixed to a length of 300. for reasons attributed to computational cost and efficiency, we have used polynomial features of degree 2 in our experiments. the results obtained using different approaches are tabulated in ## results and analysis the result from the experimentation, as discussed in section 5 are tabulated in section 3. as we can see from ## discussion and conclusion in our work, we have worked with various deep learning algorithms and fusion techniques to study and investigate human behavior. we have also set up the analogy between the human sentiment analysis and behavior in section 2. we have also trained our system based on various architectures and the best results can be referred to in section 2. as the dataset size was not so significant, the system is not trained on complex deep learning-based architectures. from future works can involve a rule-based approach for the same problem statement. such an approach would be able to provide much better results even on a smaller dataset. various techniques could be used to improve on the dataset size, and a deep learning architecture can be developed to cater to the same.