# Quest_SA: Preprocessing Method for Closed-Ended Questionnaires Using Sentiment Analysis through Polarity

Authors: Amala Jayanthi M; Elizabeth Shanthi I
Journal: Mobile Information Systems
 (2022)

---

## Abstract

Sentiment analysis is a prominent research topic in natural language processing, with applications in politics, news, education, product review, and other sectors. Especially in the education sector, sentiment analysis can assist educators in finding students’ feelings about a course on time, altering the teaching plan appropriately and timely to improve the quality of education and teaching. For students, the sentiment analysis can identify emotions, academic performance, behaviour, and so on; the primary purpose of this research paper is to analyze students’ emotions, self-esteem, and efficacy based on closed-ended questionnaires. This paper proposes Quest_SA, which uses the sentiment analysis technique to identify students’ emotions based on the answer provided by a closed-ended questionnaire. The polarity value is assigned for each questionnaire scale. The students’ responses are then gathered using a closed-ended questionnaire, and the student’s emotions are classified using a polarity-based method of sentiment analysis. Finally, sentiment scores and emotion variance were used to evaluate the outcomes. According to the sentiment ratings, students have favourable sentiments and emotions such as unhappy, somewhat happy, and happy. The real-world closed-ended questionnaires such as emotional intelligence, Eysenck, personality, self-determination scale, self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness questionnaires were used to examine the academic performance with the proposed sentiment analysis. This study inferred that the proposed sentiment analysis preprocessing method with polarity scores is as accurate as the standard value calculation.


---

## Body

## 1. Introduction

Sentiment analysis is a technique for detecting polarity and recognizing emotion toward a certain object, such as a person, a concept, or an activity. The purpose of sentiment analysis is to determine people’s opinions, identify the emotions they express, and categorize them as positive, negative, or neutral. Natural language processing (NLP) and machine learning (ML) techniques are used by sentiment analysis systems to identify, retrieve, and synthesize information and opinions from large amounts of text [1].In general, sentiment analysis was done at three levels: document, sentence, and aspect. Document Level Sentiment Analysis discovers the user sentiments by evaluating the entire document. The goal of sentence-level research is to establish the polarity of individual sentences rather than the entire document; as a result, it is more precise. Finally, aspect-level sentiment analysis identifies elements or attributes mentioned in reviews and categorizes users’ reactions to them. The architecture of a broad sentiment analysis system is shown in Figure1.Figure 1
General sentiment analysis architecture[2].The whole system may employ a set of lexicons and linguistic resources. The document analysis module is a critical component of the system design since it employs linguistic resources to annotate preprocessed documents with sentiment annotations. The system's output—positive, negative, or neutral—is represented by annotations in several visualization tools. Depending on the sentiment analysis form, annotations may be utilized in various ways. For example, in document-based sentiment analysis, annotations may be applied to the entire document; in sentence-based sentiment analysis, annotations can be applied to specific sentences; and in aspect-based sentiment analysis, annotations can be applied to certain subjects or entities.Sentiment analysis has been used in various settings to achieve a variety of goals, most notably in professional and economic networks. A few examples of well-known sentiment analysis business applications include product and service reviews [3], financial marketing approaches [4], and customer relationship management [5]. The most common use of sentiment analysis in social media apps is to analyze a company’s reputation on Twitter or Facebook [6] and investigate people’s reactions to a crisis, for example, COVID-19 [7]. Another important application area is politics [8], where sentiment research might aid candidates in their election campaigns.Sentiment analysis and opinion mining have got a lot of attention in the educational community [9]. Unlike the previously stated sectors of social and commercial networks, which focus on a single user, education sentiment analysis research covers a variety of views, including teachers/instructors, students/learners, decision-makers, and institutions. Sentiment analysis is largely used to improve teaching, management, and assessment by examining learners’ attitudes and behaviour toward courses, platforms, institutions, and teachers.Sentiment analysis is utilized to investigate the relationship between learners’ sentiments and drop-out rates in massive open online courses and the relationship between performance and retention and learners’ emotions [10]. Finally, sentiment analysis has examined several teacher-related aspects expressed in student reviews or comments on discussion forums in terms of teacher viewpoints [11].Students are frequently obliged to engage in postcourse questionnaires at the end of each academic term to obtain information about their experiences. This procedure allows teachers and administrators to review student assessments and improve learning processes. There are both closed- and open-ended questions on the survey. Closed-ended questions, frequently used in Likert-scale inquiries, try to capture students’ evaluations in numerical ratings. Students can provide written comments or ideas in response to open-ended questions, which reflect their personal views and perceptions. This paper considers the closed-ended questions for identifying students’ emotions using sentiment analysis. The students’ responses are collected using a closed-ended questionnaire, and the students’ emotions are specified using a polarity-based sentiment analysis algorithm. The outcomes were assessed using sentiment scores and emotion variance. According to the sentiment ratings, students have positive sentiments and emotions such as unhappy, somewhat happy, and happy.The remainder of this study paper is structured as follows: the research backdrop is described in Section2, which includes sentiment analysis and a questionnaire. After that, in Section 3, the recommended methodology is explained. Finally, in Section 4, the proposed work’s performance is evaluated using standard questionnaires such as the Oxford happiness inventory, self-determination scale, Rosenberg’s self-esteem, self-efficacy, emotional intelligence, Eysenck personality questionnaire, and positive and negative affect schedule, and the conclusion and future work of this research work are presented.


## 2. Related Work

### 2.1. Sentiment Analysis

Sentiment analysis [12] evaluates emotional representation through language that comprises acquiring dynamic data, processing and analyzing data, and classifying a piece of text. The three main sentiment analysis tasks are facial expression identification, polarity detection, and affective computing [13]. Text sentiment analysis is a realistic approach for emotion mining in natural language processing widely used in public opinion monitoring, artificial intelligence, and corporate analytics. The three primary methods for text sentiment analysis are a machine learning-based technique, a dictionary-based approach, and a hybrid approach [14].In machine learning-based techniques, sentiment classifiers are trained using a prelabeled data set. A classifier can be created to determine the polarity of textual inputs using methods such as naive Bayes, support vector machine, maximum entropy, and Word2vec, which are commonly used in sentiment analysis [15]. The dictionary-based approach uses a predeveloped lexicon, which includes the contradiction of words or phrases, to compute sentiment ratings and detect the polarity of a given text. The sentiment score is based on open source or bespoke sentiment dictionaries and can be computed using numerous semantic criteria [16]. A hybrid approach to sentiment classification combines machine learning and dictionary approaches. In general, the machine learning-based method is more effective although it takes a long time to classify the data [17]. The dictionary-based technique, on the other hand, has the advantage of not requiring any training data to determine sentiment and is substantially faster than machine learning in terms of computing time.Customer product review [18], sale predictions [19], social media data [20], sarcasm detection [21], and the economic domain [22] are just a few examples of where sentiment analysis has been used. In the subject of education, sentiment analysis has recently gained interest. Reference [23] employed a lexicon-based approach to judging document-level polarity on students’ feedback to evaluate teachers.Reference [24] introduced sentiment analysis provided by students at the end of a teacher evaluation course. The text processing capability of KNIME was utilized to build a pipeline for analyzing student feelings. This method recommends categorizing feedback as good, negative, or neutral using a sentiment score. Reference [25] proposed a hybrid technique for analyzing student input emotions that blends machine learning and lexicon-based methodologies. Textual feedback, usually given at the end of a course, provides useful insights into the general level of teaching and suggests practical ways to enhance teaching methods. Reference [26] planned to evaluate students’ text feedback and estimate instructional success levels using a lexicon-based technique. A lexicon of English sentiment phrases is built to get the polarity of terms as a linguistic source. In a sentiment-based eSystem,(i) for film reviewing, client happiness is measured using sentiment analysis with hybrid fuzzy and deep neural network [27], (ii) for modern business, knowledge discovery and sentiment analysis is used [28]. Selection for the best SVM hyperparameter values is done by applying natural optimizing techniques [29]. (iii) for nontraditional learning, expansion of hybrid reality-based education is done [30].


### 2.2. Open-Ended Questionnaire

Open-ended questions are survey questions that allow respondents to respond in an open text format, conveying their complete understanding, feelings, and knowledge. It implies that the answer to this question is not limited to a few options. Open-ended questions are commonly used in qualitative market research. A question with an open-ended response allows the viewer to answer depending on their knowledge and experience. The viewer’s detailed and elaborate knowledge leaves the potential for additional discussion and improvement. An open-ended question provides opportunities for both the researcher and the respondent to learn.Figure2 shows some examples of open-ended questionnaires.Figure 2
Example of the open-ended questionnaire.The open-ended questionnaire has many merits, but it is difficult to analyze and organize the data into reports. Too many questions can directly harm the response rate. Moreover, the open-ended questionnaire may provide irrelevant information.


### 2.3. Closed-Ended Questionnaire

A questionnaire is a research tool that consists of questions or other prompts designed to gather data from a respondent. There are two types of questionnaires: structured and unstructured questionnaires. Quantitative data were collected via structured questionnaires. Quantitative questionnaires are used to evaluate or verify the accuracy that has already been developed. The questionnaire is meticulously constructed and designed to collect precise data. It also starts a formal investigation, contributes data, double-checks previously gathered data, and aids in invalidating any previous idea. Unstructured surveys are used to gather qualitative information. For example, qualitative questionnaires are used when collecting exploratory data to prove or reject a theory. They employ a minimal structure and a few branching questions, but nothing restricts a respondent’s options. To acquire specific responses from people, the questions are more open-ended. This research work considers structured quantitative questionnaires (closed-ended questionnaires). An investigation of the association between self-esteem and students’ academic performance was done in [31]. The authors of [32] worked on research contemplated on educational data mining.Closed-ended questions, such as “yes” or “no” or multiple-choice questions, require respondents to choose from a limited set of predefined responses. Closed-ended inquiries are frequently used to gather statistical data from responders. It can take various shapes, but they are all driven by the requirement for respondents to have particular choices. Figure3 depicts many sorts of closed-ended questions.Figure 3
Types of closed-ended questions.Table1 shows the sample questions and the Likert scale used in each questionnaire.Table 1
Sample questions.
QuestionnaireSample questionsScaleEmotional intelligence1. I realize immediately when I lose my temperNot at all2. I can reframe bad situation quicklyRarely3. I am always able to motivate myself to do difficult tasksSometimes4. I am always able to see a thing from the other person’s viewpointOften5. I am an excellent listenerVery oftenEysenck personality1. Does your mood often go up and down?2. Do you take much notice of what people think?Yes3. Are you a talkative person?No4. If you say you will do something, do you always keep your promise, no matter how inconvenient it might be?5. Do you ever feel just miserable for no reason?Self-determination scale1A. I always feel like I choose the things I do11B. I sometimes feel that it is not really me choosing the things I do22A. My emotions sometimes seem alien to me32B. My emotions always seem to belong to me43A. I choose to do what I have to do53B. I do what I need to do, but I do not feel like it is really my choiceGeneral self-efficacy1. I can always manage to solve difficult problem if I try hard enough.Very slightly or not at all2. If someone opposes me, I can find the means and ways to get what I wantA little3. It is easy for me to stick to my aim and accomplish my goalsModerately4. I am confident that I could deal efficiently with unexpected eventsQuite a bit5. Thanks to my resourcefulness, I know how to handle unforeseen situationsExtremelyRosenberg’s self-esteem1. On the whole, I am satisfied with myselfStrongly agree2. At times, I think I am no good at allAgree3. I feel that I have a number of good qualitiesDisagree4. I am able to do things as well as most other people.Strongly disagree5. I feel I do not have much to be proud ofPositive and negative affect schedule1. InterestedVery slightly or not at all2. DistressedA little3. ExcitedModerately4. UpsetQuite a bit5. StrongExtremelyOxford happiness1. I do not feel particularly pleased with the way I am (R)Strongly disagree2. I am intensely interested in other peopleModerately disagree3. I feel that life is very rewardingSlightly disagree4. I have very warm feelings towards almost everyoneSlightly agree5. I rarely wake up feeling rested (R)Moderately agreeStrongly agreeThe Oxford happiness questionnaire was developed by psychologists [33] at Oxford University. In the Oxford questionnaire, the (R) indicates reverse scoring. For example, if the student gives “1,” cross it out and change it to “6.” The emotional intelligence questionnaire is a self-evaluation tool. Self-awareness, self-regulation, motivation, empathy, and social skills are the five characteristics that characterize emotional intelligence, according to [34]. The self-esteem of an individual is assessed using Rosenberg’s self-esteem scale. The score of negative question items is inverted for analysis such that the positive and negative things have the same meaning. The final test result might be between 10 and 40. A person with a score of less than 14 has a problem with low self-esteem and needs assistance. The Eysenck personality test is a self-reporting tool [35]. It has 48 items: 12 for each of the personality traits of neuroticism, extraversion, and psychoticism and 12 for the lying scale. “Yes” or “no” is the binary response to each inquiry. Each dichotomous item was given a value of 1 or 0, with a maximum score of 12 and a minimum of 0. The self-determination scale (SDS) was developed to examine how self-determined people perform individually. It is thus regarded as a reasonably stable feature of people’s personalities that reflects (1) increased awareness of their feelings and sense of self and (2) a sense of control over their behaviour. The general self-efficacy scale is a 10-item psychometric scale that assesses optimistic self-beliefs in one’s ability to cope with various life challenges. Positive and negative affect schedule (PANAS) is a scale of several words that express feelings and emotions. The overall score is computed by adding 10 positive items together and then 10 negative items. For both sets of objects, the scores range from 10 to 50. A greater total positive score suggests a stronger beneficial influence. A lower total negative score suggests a lesser level of negative impact.


## 2.1. Sentiment Analysis

Sentiment analysis [12] evaluates emotional representation through language that comprises acquiring dynamic data, processing and analyzing data, and classifying a piece of text. The three main sentiment analysis tasks are facial expression identification, polarity detection, and affective computing [13]. Text sentiment analysis is a realistic approach for emotion mining in natural language processing widely used in public opinion monitoring, artificial intelligence, and corporate analytics. The three primary methods for text sentiment analysis are a machine learning-based technique, a dictionary-based approach, and a hybrid approach [14].In machine learning-based techniques, sentiment classifiers are trained using a prelabeled data set. A classifier can be created to determine the polarity of textual inputs using methods such as naive Bayes, support vector machine, maximum entropy, and Word2vec, which are commonly used in sentiment analysis [15]. The dictionary-based approach uses a predeveloped lexicon, which includes the contradiction of words or phrases, to compute sentiment ratings and detect the polarity of a given text. The sentiment score is based on open source or bespoke sentiment dictionaries and can be computed using numerous semantic criteria [16]. A hybrid approach to sentiment classification combines machine learning and dictionary approaches. In general, the machine learning-based method is more effective although it takes a long time to classify the data [17]. The dictionary-based technique, on the other hand, has the advantage of not requiring any training data to determine sentiment and is substantially faster than machine learning in terms of computing time.Customer product review [18], sale predictions [19], social media data [20], sarcasm detection [21], and the economic domain [22] are just a few examples of where sentiment analysis has been used. In the subject of education, sentiment analysis has recently gained interest. Reference [23] employed a lexicon-based approach to judging document-level polarity on students’ feedback to evaluate teachers.Reference [24] introduced sentiment analysis provided by students at the end of a teacher evaluation course. The text processing capability of KNIME was utilized to build a pipeline for analyzing student feelings. This method recommends categorizing feedback as good, negative, or neutral using a sentiment score. Reference [25] proposed a hybrid technique for analyzing student input emotions that blends machine learning and lexicon-based methodologies. Textual feedback, usually given at the end of a course, provides useful insights into the general level of teaching and suggests practical ways to enhance teaching methods. Reference [26] planned to evaluate students’ text feedback and estimate instructional success levels using a lexicon-based technique. A lexicon of English sentiment phrases is built to get the polarity of terms as a linguistic source. In a sentiment-based eSystem,(i) for film reviewing, client happiness is measured using sentiment analysis with hybrid fuzzy and deep neural network [27], (ii) for modern business, knowledge discovery and sentiment analysis is used [28]. Selection for the best SVM hyperparameter values is done by applying natural optimizing techniques [29]. (iii) for nontraditional learning, expansion of hybrid reality-based education is done [30].


## 2.2. Open-Ended Questionnaire

Open-ended questions are survey questions that allow respondents to respond in an open text format, conveying their complete understanding, feelings, and knowledge. It implies that the answer to this question is not limited to a few options. Open-ended questions are commonly used in qualitative market research. A question with an open-ended response allows the viewer to answer depending on their knowledge and experience. The viewer’s detailed and elaborate knowledge leaves the potential for additional discussion and improvement. An open-ended question provides opportunities for both the researcher and the respondent to learn.Figure2 shows some examples of open-ended questionnaires.Figure 2
Example of the open-ended questionnaire.The open-ended questionnaire has many merits, but it is difficult to analyze and organize the data into reports. Too many questions can directly harm the response rate. Moreover, the open-ended questionnaire may provide irrelevant information.


## 2.3. Closed-Ended Questionnaire

A questionnaire is a research tool that consists of questions or other prompts designed to gather data from a respondent. There are two types of questionnaires: structured and unstructured questionnaires. Quantitative data were collected via structured questionnaires. Quantitative questionnaires are used to evaluate or verify the accuracy that has already been developed. The questionnaire is meticulously constructed and designed to collect precise data. It also starts a formal investigation, contributes data, double-checks previously gathered data, and aids in invalidating any previous idea. Unstructured surveys are used to gather qualitative information. For example, qualitative questionnaires are used when collecting exploratory data to prove or reject a theory. They employ a minimal structure and a few branching questions, but nothing restricts a respondent’s options. To acquire specific responses from people, the questions are more open-ended. This research work considers structured quantitative questionnaires (closed-ended questionnaires). An investigation of the association between self-esteem and students’ academic performance was done in [31]. The authors of [32] worked on research contemplated on educational data mining.Closed-ended questions, such as “yes” or “no” or multiple-choice questions, require respondents to choose from a limited set of predefined responses. Closed-ended inquiries are frequently used to gather statistical data from responders. It can take various shapes, but they are all driven by the requirement for respondents to have particular choices. Figure3 depicts many sorts of closed-ended questions.Figure 3
Types of closed-ended questions.Table1 shows the sample questions and the Likert scale used in each questionnaire.Table 1
Sample questions.
QuestionnaireSample questionsScaleEmotional intelligence1. I realize immediately when I lose my temperNot at all2. I can reframe bad situation quicklyRarely3. I am always able to motivate myself to do difficult tasksSometimes4. I am always able to see a thing from the other person’s viewpointOften5. I am an excellent listenerVery oftenEysenck personality1. Does your mood often go up and down?2. Do you take much notice of what people think?Yes3. Are you a talkative person?No4. If you say you will do something, do you always keep your promise, no matter how inconvenient it might be?5. Do you ever feel just miserable for no reason?Self-determination scale1A. I always feel like I choose the things I do11B. I sometimes feel that it is not really me choosing the things I do22A. My emotions sometimes seem alien to me32B. My emotions always seem to belong to me43A. I choose to do what I have to do53B. I do what I need to do, but I do not feel like it is really my choiceGeneral self-efficacy1. I can always manage to solve difficult problem if I try hard enough.Very slightly or not at all2. If someone opposes me, I can find the means and ways to get what I wantA little3. It is easy for me to stick to my aim and accomplish my goalsModerately4. I am confident that I could deal efficiently with unexpected eventsQuite a bit5. Thanks to my resourcefulness, I know how to handle unforeseen situationsExtremelyRosenberg’s self-esteem1. On the whole, I am satisfied with myselfStrongly agree2. At times, I think I am no good at allAgree3. I feel that I have a number of good qualitiesDisagree4. I am able to do things as well as most other people.Strongly disagree5. I feel I do not have much to be proud ofPositive and negative affect schedule1. InterestedVery slightly or not at all2. DistressedA little3. ExcitedModerately4. UpsetQuite a bit5. StrongExtremelyOxford happiness1. I do not feel particularly pleased with the way I am (R)Strongly disagree2. I am intensely interested in other peopleModerately disagree3. I feel that life is very rewardingSlightly disagree4. I have very warm feelings towards almost everyoneSlightly agree5. I rarely wake up feeling rested (R)Moderately agreeStrongly agreeThe Oxford happiness questionnaire was developed by psychologists [33] at Oxford University. In the Oxford questionnaire, the (R) indicates reverse scoring. For example, if the student gives “1,” cross it out and change it to “6.” The emotional intelligence questionnaire is a self-evaluation tool. Self-awareness, self-regulation, motivation, empathy, and social skills are the five characteristics that characterize emotional intelligence, according to [34]. The self-esteem of an individual is assessed using Rosenberg’s self-esteem scale. The score of negative question items is inverted for analysis such that the positive and negative things have the same meaning. The final test result might be between 10 and 40. A person with a score of less than 14 has a problem with low self-esteem and needs assistance. The Eysenck personality test is a self-reporting tool [35]. It has 48 items: 12 for each of the personality traits of neuroticism, extraversion, and psychoticism and 12 for the lying scale. “Yes” or “no” is the binary response to each inquiry. Each dichotomous item was given a value of 1 or 0, with a maximum score of 12 and a minimum of 0. The self-determination scale (SDS) was developed to examine how self-determined people perform individually. It is thus regarded as a reasonably stable feature of people’s personalities that reflects (1) increased awareness of their feelings and sense of self and (2) a sense of control over their behaviour. The general self-efficacy scale is a 10-item psychometric scale that assesses optimistic self-beliefs in one’s ability to cope with various life challenges. Positive and negative affect schedule (PANAS) is a scale of several words that express feelings and emotions. The overall score is computed by adding 10 positive items together and then 10 negative items. For both sets of objects, the scores range from 10 to 50. A greater total positive score suggests a stronger beneficial influence. A lower total negative score suggests a lesser level of negative impact.


## 3. Methodology

Sentiment analysis is a computational study that evaluates individuals’ thoughts, assessments, and opinions regarding persons, situations, entities, concepts, activities, and items and their characteristics. Its goal is to find underlying opinions on a specific entity automatically. Sentiment analysis is mainly used for commercial applications such as product reviews, recommendations, marketing analysis, and public relations [36]. In the field of education, sentiment analysis is the process of determining a student’s feelings. In education, sentiment analysis can help with learning process improvement, performance improvement, study discontinuance reduction, teaching process improvement, and course satisfaction.Emotion is commonly defined as a person’s mental state, including attitudes, feelings, and actions. Nowadays, public sentiment on a particular context can be easily known by extracting the opinions from a wealth of publicly available information on platforms such as Facebook, Youtube, Twitter, and Instagram.Emotion detection, Reddit, Twitter, and others are becoming increasingly popular as a new study horizon in NLP. It could also be used in health services (as a tool for psychoanalysis), education (identifying learner dissatisfaction), and other fields [37]. This paper proposes Quest_SA to find students’ affective traits using polarity-enabled sentimental analysis in a closed-ended questionnaire. Figure 4 shows the architecture of the proposed Quest_SA.Figure 4
Quest_SA architecture.The proposed Quest_SA contains two phases: online and offline processes. The students are requested to take an online closed-ended questionnaire in the online phase.The following seven kinds of questionnaires are used: emotional intelligence, Eysenck personality, self-determination scale, self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness. The selected scale value is converted into a polarity value for each question. Table2 shows the polarity assignment for the questionnaire scale.Table 2
Polarity score.
QuestionnaireSealeStandard valueProposed polarity valueEmotional intelligenceNot at all1−2Rarely2−1Sometimes30Often41Very often52Eysenck personalityYes11No0−1Self-determination scale11−222−1330441552General self-efficacyVery slightly or not at all1−2A little2−1Moderately30Quite a bit41Extremely52Rosenberg’s self-esteemStrongly agree42Agree31Disagree2−1Strongly disagree1−2Positive and negative affect scheduleVery slightly or not at all1−2A little2−1Moderately30Quite a bit41Extremely52Oxford happinessStrongly disagree1−3Moderately disagree2−2Slightly disagree3−1Slightly agree41Moderately agree52Strongly agree63In the offline phase, the sentiment analyzer uses the polarity value to predict the students’ emotions. The polarity values are given based on the positive and negative sense. The positive Likert scale is given a positive score, and negative Likert scale is assigned a negative score.


## 4. Proposed QUEST_SA: Questionnaire Evaluation Using Sentiment Analysis

In this section, the performance of the proposed work is analyzed. Seven real-world questionnaires are used in this experiment. Table3 shows the details of the questionnaire and the calculation in determining the result is shown.Table 3
Questionnaire details.
QuestionnaireNo. of questionsScaleScore calculationsResultResult with polarityEmotional intelligence connects a person’s knowledge process to their emotional processes151 = Not at allSum all scale values for each item<34 = Low35–55 = Average>56 = High–ve Score = low2 = Rarely0 = Average3 = Sometimes+ve Score = high4 = Often5 = Very oftenEysenck personality measures the personality domain481 = YesSum all scale values of psychoticism (PM), extroversion (En), and neuroticism (Nm)The biggest value of psychoticism, extroversion, and neuroticism (Pm, En, and Nm)The positive value of psychoticism, extroversion, and neuroticism (Pm, En, and Nm)0 = NoSelf-determination scale assesses individual differences in the extent to which people tend to function in a self-determined way101Sum all result scale values and divide by 10<3 = Low>3 = High+ve = high2–ve = Low345General self-efficacy is a self-report measure of self-efficiency101 = Very slightly or not at allSum all result scale values<20 = Low>20 = High–ve Score = low+ve Score = high2 = A little3 = Moderately4 = Quite a bit5 = ExtremelyRosenberg’s self-esteem measures global self-worth104 = Strongly agreeSum all result scale values<14 = Low15–25 = Normal>26 = High–ve Score = low3 = Agree2 = Disagree0 = Normal1 = Strongly disagree+ve Score = highPositive and negative affect schedule is a self-reported measure of affect201 = Very slightly or not at allSum all positive (PS) and negative (NS) affect scale valuesPS > NS = positivePS = +ve score positive2 = A littleNS > PS = negative3 = ModeratelyNS = +ve score negativePS = NS = false4 = Quite a bit5 = ExtremelyOxford happiness is used to predict the happiness score of the person291 = Strongly disagreeSum all scale values and divide by the total number of questions1–2: Not happy–ve Score = happy2 = Moderately disagree2–3: Somewhat unhappy3 = Slightly disagree3–4: Not particularly happy or unhappy0 = Moderately happy4 = Slightly agree4: Somewhat happy+ve Score = unhappy5 = Moderately agree4–5: Happy6 = Strongly agree5–6: Very happy6: Too happyThe same standard calculation given in Table3 is calculated for each questionnaire with the respective polarity value shown in Table 2. The result is given based on the polarity: if the result has negative polarity, then the scale is low; else, if the result has positive polarity, then the scale is high; and if the result is zero, then the scale is moderate. For instance, in Rosenberg’s self-esteem, if the score is negative, the result is low self-esteem, and if the score is positive, the result is high self-esteem.


## 5. Results and Discussions

This section evaluates the performance of the proposed through experiments. The research work uses seven kinds of questionnaires such as emotional intelligence (EI), Eysenck personality (EP), self-determination scale (SDS), general self-efficacy (GSE), Rosenberg’s self-esteem (RSE), positive and negative affect schedule (PNAS), and Oxford happiness (OH) and collects response from 1,000 students. The collected response was analyzed based on the standard and polarity-based evaluation. Finally, the obtained results are calculated and evaluated using MAE (mean absolute error) and accuracy.Table4 shows the emotional intelligence standard and the proposed polarity-based results. In addition, it shows the comparison of the result for all possible results. The result shows that the percentage of result deviation is very low between the standard evaluation and the proposed evaluation. Figure 5 shows the EI questionnaire scale value: low, average, and high. Table 5 gives the sample code of SentimentAnalyzer.Table 4
Emotional intelligence details.
Emotional intelligenceStandard evaluationProposed evaluationResultCountResultCountLow10Low10Average0High0Average290Low10Average265High15High700Low0Average0High700Figure 5
Emotional intelligence (standard vs. proposed).Table 5
Sample code of SentimentAnalyzer.
/∗∗ To change this license header, choose License Headers in Project Properties.∗ To change this template file, choose Tools | Templates∗ and open the template in the editor.∗/package servlet1;import java.util.Properties;import org.ejml.simple.SimpleMatrix;import edu.stanford.nlp.ling.CoreAnnotations;import edu.stanford.nlp.neural.rnn.RNNCoreAnnotations;import edu.stanford.nlp.pipeline.Annotation;import edu.stanford.nlp.pipeline.StanfordCoreNLP;import edu.stanford.nlp.sentiment.SentimentCoreAnnotations;import edu.stanford.nlp.trees.Tree;import edu.stanford.nlp.util.CoreMap;/∗∗∗∗ @author jayanthi∗/public class SentimentAnalyzer{static Properties props;static StanfordCoreNLP pipeline;public void initialize(String path){// creates a StanfordCoreNLP object, with POS tagging, lemmatization, NER, parsing, and sentimentprops = new Properties();props.setProperty(“parse.model”, path+”edu\\stanford\\nlp\\models\\lexparser\\englishPCFG.ser.gz”);props.setProperty(“sentiment.model”, path+”edu\\stanford\\nlp\\models\\sentiment\\sentiment.ser.gz”);props.setProperty(“annotators”, “tokenize, ssplit, parse, sentiment”);pipeline = new StanfordCoreNLP(props);//LexicalizedParser lp = LexicalizedParser.loadModel(“edu/stanford/nlp/models/lexparser/englishPCFG.ser.gz”);}public SentimentResult getSentimentResult(String text) {SentimentResult sentimentResult = new SentimentResult();SentimentClassification sentimentClass = new SentimentClassification();if (text ! = null && text.length() > 0) {// run all Annotators on the textAnnotation annotation = pipeline.process(text);for (CoreMap sentence: annotation.get(CoreAnnotations.SentencesAnnotation.class)) {// this is the parse tree of the current sentenceTree tree = sentence.get(SentimentCoreAnnotations.SentimentAnnotatedTree.class);SimpleMatrix sm = RNNCoreAnnotations.getPredictions(tree);String sentimentType = sentence.get(SentimentCoreAnnotations.SentimentClass.class);sentimentClass.setVeryPositive((double)Math.round(sm.get(4)∗100d));sentimentClass.setPositive((double)Math.round(sm.get(3)∗100d));sentimentClass.setNeutral((double)Math.round(sm.get(2)∗100d));sentimentClass.setNegative((double)Math.round(sm.get(1)∗100d));sentimentClass.setVeryNegative((double)Math.round(sm.get(0)∗100d));sentimentResult.setSentimentScore(RNNCoreAnnotations.getPredictedClass(tree));sentimentResult.setSentimentType(sentimentType);sentimentResult.setSentimentClass(sentimentClass);}}Return sentimentResult;}}Table6 shows the MAE and accuracy comparison of EI for different responses. Again, the lower number of the responses (200 and 400) produces a lower error.Table 6
MAE and accuracy for emotional intelligence.
No. of responsesMAEAccuracy2006.67954008.6796.75600129780014.6797.25100016.6797.5Table7 shows the Eysenck personality standard and the proposed polarity-based results. Figure 6 shows the questionnaire scale value for psychoticism, extroversion, and neuroticism. The standard and polarity evaluation produce the same result for all scale values.Table 7
Eysenck personality questionnaire results.
Eysenck personalityStandard evaluationProposed evaluationResultCountResultCountPsychoticism35Psychoticism35Extroversion0Neuroticism0Extroversion665Psychoticism0Extroversion665Neuroticism0Neuroticism∗300Psychoticism0Extroversion0Neuroticism300Figure 6
Scale value for EP (standard vs. polarity).Table8 shows the MAE and EP accuracy comparison for a different number of responses. Again, the result produces zero error and 100% accuracy for all different numbers of responses.Table 8
MAE and accuracy for Eysenck personality.
No of responsesMAEAccuracy20001004000100600010080001001,0000100Table9 shows the self-determination scale standard and the proposed polarity-based results. Figure 7 shows the SDS questionnaire scale value: low and high.Table 9
Self-determination scale result.
Self-determination scaleStandard evaluationProposed evaluationResultCountResultCountLow240Low215High25High760Low20High740Figure 7
Scale value for SDS (standard vs. polarity).Table10 shows the MAE and SDS’s accuracy comparison for a different number of responses.Table 10
MAE and accuracy self-determination scale.
No. of responsesMAEAccuracy200595.54001096.256001095800596.251,000294Table11 shows the MAE and accuracy comparison of GSE for a different number of responses.Table 11
MAE and accuracy for general self-efficacy.
No. of responsesMAEAccuracy200597.54001596.256002096.678002596.911,0002597.5Table12 shows the general self-efficacy standard and proposed polarity-based result. Again, the standard and polarity evaluation results for low-scale values. Figure 8 shows the GSE questionnaire scale value low and high.Table 12
Self-efficacy result.
General self-efficacyStandard evaluationProposed evaluationResultCountResultCountLow30Low30High0High970Low25High945Figure 8
Scale value for GSE (standard vs. polarity).Table13 shows Rosenberg’s self-esteem standard and proposed polarity-based result. Figure 9 shows the RSE questionnaire scale value: low, normal, and high.Table 13
Rosenberg’s self-esteem result.
Rosenberg’s self-esteemStandard evaluationProposed evaluationResultCountResultCountLow0Low0Normal0High0Normal145Low15Normal120High10High855Low15Normal20High820Figure 9
Scale value for RSE.Table14 shows the MAE and accuracy comparison of RSE for a different number of responses.Table 14
MAE and accuracy for Rosenberg’s self-esteem.
No. of responsesMAEAccuracy2005.3914001092.560013.392.580016.6793.121,0002094Table15 shows the positive and negative affect schedule standard and the proposed polarity-based result. The standard and polarity evaluation produce the same result for all the scale values.Table 15
Positive and negative affect schedule result.
Positive and negative affect scheduleStandard evaluationProposed evaluationResultCountResultCountPositive805Positive805Negative0Neutral0Negative125Positive0Negative125Neutral0Neutral70Positive0Negative0Neutral70Figure10 shows the PANAS questionnaire scale value: positive, negative, and neutral.Figure 10
Scale value for PANAS (standard vs. polarity).Table16 shows the MAE and accuracy comparison of PANAS for the different numbers of responses. The result produces zero error and 100% accuracy for all different numbers of responses.Table 16
MAE and accuracy for positive and negative affect schedule.
No. of responsesMAEAccuracy20001004000100600010080001001,0000100Table17 shows the Oxford happiness questionnaire standard and proposed polarity-based result. Figure 11 shows the OH questionnaire scale value happy, moderately, happy, and unhappy. Table 18 gives MAE and the accuracy of the Oxford happiness questionnaire.Table 17
Oxford happiness result.
Oxford happinessStandard evaluationProposed evaluationResultCountResultCountHappy250Happy200Moderately happy35Unhappy15Moderately happy745Happy50Moderately happy690Unhappy5Unhappy5Happy0Moderately happy0Unhappy5Figure 11
Scale value Oxford happiness (standard vs. polarity).Table 18
MAE and accuracy for Oxford happiness.
No. of responsesMAEAccuracy2003.35884006.6787.56006.6787.3380013.3487.51,00013.3489.5


## 6. Conclusion

The task of sentiment analysis for questionnaire data was the focus of this study. The main goal was to develop a mechanism for analyzing questions and students’ emotions based on closed-ended responses. Quest SA is a tool for assessing questionnaire sentiments and students’ emotions proposed in this paper. The students’ replies are gathered using a closed-ended questionnaire, and the students’ emotions are identified using polarity-based sentiment analysis in this study. The performance of the study task is evaluated using seven real-time surveys (emotional intelligence, Eysenck personality, self-determination scale, general self-efficacy, Rosenberg’s self-esteem, positive and negative affect schedule, and Oxford happiness). The suggested Quest_SA accurately predicts students’ emotions compared to established questionnaire evaluation methods. The proposed system’s accuracy is comparable to that of the traditional method. When opposed to traditional evaluation, categorizing the result is simple. Because the traditional evaluation with range of values takes long time than the proposed evaluation with polarity score, multimodal SA techniques are probably going to be in high demand in the near future.Table15 shows the MAE and OH accuracy comparison for a different number of responses. Again, the results proved that the proposed system works similarly to the traditional system with good accuracy.



---

*Source: 4733550-2022-09-20.xml*