Combining goal model with reviews for supporting the evolution of apps

IET SoftwareVolume 14, Issue 1 p. 39-49 Research ArticleOpen Access Combining goal model with reviews for supporting the evolution of apps Yuzhou Liu, Yuzhou Liu College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorLei Liu, Lei Liu College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorHuaxiao Liu, Corresponding Author Huaxiao Liu liuhuaxiao@jlu.edu.cn College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorShanquan Gao, Shanquan Gao College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this author Yuzhou Liu, Yuzhou Liu College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorLei Liu, Lei Liu College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorHuaxiao Liu, Corresponding Author Huaxiao Liu liuhuaxiao@jlu.edu.cn College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this authorShanquan Gao, Shanquan Gao College of Computer Science and Technology, Jilin University, Changchun, 130012 People's Republic of China Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Changchun, 130012 People's Republic of ChinaSearch for more papers by this author First published: 01 February 2020 https://doi.org/10.1049/iet-sen.2018.5192Citations: 3AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract To support the iterative development process of Apps, the goal model is not only established to describe the requirements at the early stage but also used for identifying the updating strategy in every iteration. In this process, reviews from users provide valuable information for developers to analyse the model with users sentiments. In this study, the authors combine the goal model with reviews for supporting the evolution of Apps. First, the authors introduce the reviews into the goal model as a new factor by comparing keywords. Second, the users sentiments in reviews are mined, and two kinds of information are gained by analysing the model to help developers make decisions on which goals to be improved in next version: one kind of information is about users sentiments on the goals to evaluate whether users like them; another kind is the impact of updating one goal to others. To validate the proposed approach, they conducted experiments and a survey based on the Apps in Google Play. The results show that the proposed approach can establish relationships between goals and reviews reasonably and further provide useful information for optimising the evolution strategy of the App. 1 Introduction Goal-oriented requirements engineering has been paid much attention since it emerged, and goal model (GM) is proved to be an effective way to describe requirements and capture the interactions between them [1, 2]. In recent years, GM was broadly adopted for supporting different kinds of software and achieved good results, such as information system [3], self-adaptive system [4], as well as Apps [5]. Mobile application (App) is a software system carried on mobile terminals and grows in popularity rapidly [6]. Compared with traditional software, Apps place more emphasis on user experience and often release new versions continually with shorter iteration cycle, so that they could respond to the demands of markets quickly for retaining existing users and further attracting new ones [7, 8]. Although the GMs can be well used for Apps to formulate and organise requirements at the early stage of the development, there are some challenges for using them to identify the updating strategy in each iteration of the development. In general, there are two main questions must be considered for GM to support the evolution of Apps [9-11]: How to evaluate the level of goal achievement in order to retain users? How to decide which part (goal) of the App needs to be improved for attracting users? User reviews in App stores provide data resource for answering the above questions [12]. As the typical feedback of users, reviews contain a significant amount of requirement-related information [13, 14]. By analysing reviews, developers can gain users experience on the App, identify the goals concerned by users and further evaluate these goals with users sentiments [15]. However, an App may receive hundreds of reviews per day and there are many redundant data, which means it is not doable for developers to read each review manually; additionally, reviews are always provided in form of natural language, so it is hard to extract users opinions from them. To use reviews efficiently, many (semi-)automated approaches and tools have been proposed to classify and summarise reviews, such as SUR-Miner [9], CLAP [16], AR-miner [17]. While these approaches and tools are efficient, it would be desirable to further introduce the information in reviews to the GM, so that it can be used for analysing the model directly for the evolution of Apps. In fact, goals in GM have the multidimensional hierarchy characteristics [18], which make them more adaptive for analysing with reviews. As the model is construed by developers, a GM gives the requirements of Apps clearly and systematically by decomposing them into goals at different levels of abstraction, ranging from high-level, strategic ones (such as ‘provide a way for making friends’) to low-level, implementation ones (such as ‘let user search photos by inputting keywords’). Meanwhile, the goals can cover different kinds of features of Apps, including the functional and non-functional ones (such as ‘easy to use’ or ‘safety’). Similarly, reviews also give comments on the product from many aspects at kinds of abstraction levels, so they can be associated with goals at different levels. Such as the reviews of Instagram shown in Table 1, the R1 and R2 are relatively abstract and can be related to high-level goals, while R3–R6 are more concrete and give comments on low-level goals, and R7 pays attention to the usability of the App, which is a non-functional goal. Table 1. Example of reviews for the App ‘Instagram’ Contents of the review R1 i love this app it helps me contact friends R2 it lets me follow famous persons that inspire me, thank you R3 no other app can be better than Instagram! Enjoy using it, you can improve your picture to your taste R4 it is a nice to edit photos, and you can whilst make videos yourself R5 enjoy seeing posts from all over the world easily. But every time I try to upload multiple pictures the app crashes and I lose all the picture I've selected, please fix it R6 too bad I cannot add photos from the web when I want to share them R7 i love it. Just fun and easy to use In this paper, we propose an approach to (i) automatically establish relationships between reviews and goals to construct the GM with reviews (R-GM); (ii) mine users sentiments from reviews to identify users' concerns and analyse whether users like the product in term of a certain goal (question 1), and then analyse the model based on users sentiments to help developers adjust the priorities of goals in the evolution process (question 2). For example, ‘edit photos’ and ‘share photos’ are two goals of the Instagram, and our approach can connect the reviews {R3, R4} and {R5, R6} to the two goals separately, and further point out that the product achieves the goal ‘edit photos’ of the App well based on the information in R3 and R4 while there exist problems for the goal ‘share photos’ according to R5 and R6. Thus, the developers should pay more attention to the goal ‘share photo’ and update it in the next version of the App. Unlike other approaches, we combine GM with reviews and take the information in reviews as a new factor of the model, this provides additional insights to use reviews from the perspective of requirement, and also make the GM more adaptive for supporting the development of Apps, especially for the evolution process. We thoroughly evaluate our approach with 410,512 reviews of six Apps from different categories on Google Play. Specially, we evaluate the method for identifying reviews related to goals by taking it as a classification problem; and the users sentiments extracted from reviews automatically are compared with the ones gained manually; then, a survey was conducted on 24 developers to analyse the usefulness of information provided by our approach and get suggestions from them. The paper is organised as follows. Section 2 presents the work related to our approach. Section 3 gives the methods for combining the GM with reviews and Section 4 presents the quantified analysis method of the model. Finally, the experiments and the conclusion are shown in Sections 5 and 6, respectively. 2 Related work Our work combines the analysis of GM with reviews mining, so we give the discussion on the related work from these two directions. 2.1 Studies on the GM The GM has been paid much attention in requirements engineering and its studies are always a hot research field [19-21]. Here, we summarise the research literatures on goal modelling briefly and systematic literature is given in [1, 2]. On one hand, much research focuses on the GM itself and involves several research subjects in general. First, some research targets specific categories of requirements, such as security [5], trust [22] and risk [23]. Second, the GM is studied for different development processes, for example, Lin et al. propose a light-weight Goal Net-based method to model goal requirements in agile software development process [18]. Third, the questions on characters of the GM are still considered, such as dependability [24], traceability [25] and conflict detection [26]. In these research, the ones related us most are introducing new factors into the GM: Hayashi et al. [27] introduce concerns into the GM to support the understanding of goal refinement and Mendonça et al. [24] considers the factor of time in analysing the model. Inspired by these researches, our work introduces reviews as a new factor into the GM for better supporting the evolution of Apps. On the other hand, many research combines the GM with other research fields. Mohammadi et al. [28] propose a framework for combining GMs and problem frames, so that the goals of the system can be documented together with the corresponding knowledge of the system's context; Asadi et al. [29] study both GM and feature model for the development lifecycle of software product lines, they give a description logic (DL)-based approach to represent both models and their relations in a common DL knowledge base. Similar to these research, our work combines the analysis of GM with the methods of reviews mining and provides a new way to extend the studies of the GM. 2.2 Reviews mining Reviews mining aims at gaining valuable information from reviews and helping developers understand users opinions efficiently. The research on these subjects is often given from two aspects: classify reviews and summarise information. As there are always non-informative reviews, methods are given to distinguish informative reviews from the noisy and irrelevant ones [17], and further classify reviews into different categories. For example, Panichella et al. [11] present a taxonomy of reviews for software maintenance and evolution, and they merged natural language processing (NLP), text analysis, and sentiment analysis to propose an automatic classification approach; Maalej and Nabil [30] introduce several probabilistic techniques to classify reviews into four categories including rating, user experience, requirements and community; Guzman et al. [31] study the performance of individual machine learning algorithms and it ensembles for automatically classifying reviews. Similar to these research works, our work also needs to filter useless data, but we achieve this step by analysing the relationships between reviews and goals in the model. In this way, the reviews are classified into goals and if such relationships does not exist, the reviews are filtered. Users sentiments in reviews are important for improving Apps [32, 33], and lots of research proposes methods to extract and summarise such information. Fu et al. [34] propose WisCom to identify users' major concerns and their preferences, and then find reasons why users like or dislike an App; Vu et al. [35, 36] propose keyword-based/ phrase-based approaches to acquire user opinions; Guzman and Maalej [15] give an approach to extract fine-grained App features and users sentiments in reviews for analysing values of the single features. Meanwhile, some short texts analysis method can be used to summarise the topics of reviews [37, 38] and a number of tools are given to help developers understand the main information of a large number of reviews, such as CLAP [16], WisCom [10] and SUR-Miner [9]. These research focus on analysing reviews and the objects to be evaluated in them are also gained from reviews. Different from them, our work aims at using the information in reviews to evaluate goals in the GM. As the GM is used to give requirements, it describes the product accurately. Thus, introducing reviews into the model provides a way to mine reviews from the perspective of requirements engineering, and the information gained in this process can be used in the development of Apps directly. Besides, our approach fully uses the relations between goals in the model, this enables us to analyse and summarise reviews as a whole based on their potential relationships. 3 Establishment of R-GM When developing an App, the developers could construct the GM with kinds of methods, such as i*/Tropos [39] and NFR [40], KAOS [41]. Although the representations of these models may be different and have their own characters, they also share many commonalities. At the same time, some types of relationships between goals are important for analysing the GM with reviews while others are not. Thus, we summarise the basic commonalities of these methods in describing goals and use them with typical relationships between goals to give the definition of the GM, so that our approach based on it can be combined with most of existing methods to support the iterative development process of Apps. 3.1 Definitions of GM Usually, a GM includes a set of goals and it is organised in tree-like structures, in which goals are decomposed with AND/OR connectors to represent the alternative solutions for fulfilling the super-goals and the goals in different branches could also have contribution relations. Thus, we define the GM as below: Definition.(GM): A GM is specified as a 3-tuple, , where Gset is a set of goals; the set C is contribution relations between goals and the set D are decomposition relations of goals. We give the specific illustration of each part in GM, respectively. A goal (G) in Gset is represented as a 4-tuple, where Id refers to the name of G; Des is a text that describes G in natural language; Type is the category of G; Priority defines the importance of degree of G in the GM. Usually, goals are classified into two types: hard goals present functional requirements; and soft goals describe non-functional requirements. The priorities of goals can be divided in different ways to meet the demands of practice projects. The priority of a goal is assigned by developers initially, and one important purpose of our approach is to help developers adjust it according to the response of users during the evolution process. Note that the kinds of priorities could be different so we do not limit them. The set denotes positive or negative contributions between goals. One relation can be , where must be a soft goal. The relation gives the impact degree between two goals. From Make to Break, the contributions are from totally positive to totally negative. For example, Fig. 1 gives part of the GM of the App ‘Instagram’, a social communication app, and there is a contribution relation ‘Some’ between and . It means that the improvement of may have a negative impact on achieving . When the App can be used easily, it would have fewer restrictions on the behaviours of users, and the safety of the App may be declined. In addition, as Unknown means the impact degree is unknown, we ignore such relations in this work. Fig. 1Open in figure viewerPowerPoint Part of GM of the App ‘Instagram’ The set represents decomposition relation, : the satisfaction of a target goal in an AND-decomposition requires that all source goals need to be satisfied; IOR is used to specify that satisfaction of at least one source goal satisfies the target goal; while XOR specifies that exactly one of the source goals is necessary to satisfy the target. For example, the in Fig. 1 is decomposed into with ‘And’ relation, it means that developers should satisfy both and for satisfying ; while is further decomposed into with ‘IOR’ relation, so can be satisfied by satisfying either or . Furthermore, if we only consider the decomposition relations D in the GM, a decomposition tree (denoted by DTree) can be gained to represent the hierarchies of goals directly, and we use it as the basis to search the GM when analysing relationships between goals and reviews (introduced in Section 3.2). 3.2 Analyse relationships between goals and reviews The core idea of our approach is introducing reviews into the GM, so that users sentiments in reviews can be used in the analysis of GM. Thus, we suppose that the GM of one App has already been constructed well by its developers at the early stage of the development. Then, in the evolution process of the Apps, we analyse the relationships between goals and reviews by comparing their contents and establish R-GM for helping developers identify the updating strategy. However, developers tend to use unified words to specify the same function of the App to avoid ambiguity, while the users may use different words when mentioning it. For example, for the function ‘Send message’, users may use ‘post message’, ‘convey information’, or ‘throw texts’ to express it. This means that the words used to describe a goal G in the GM could be not as diverse as the ones in reviews, so it would not achieve good performance to establish relationships between goals and reviews by simply matching the words in them. To solve this problem, we establish an extensible Vocabulary for each goal as the basis for discovering its related reviews and use an iterative way to establish R-GM, as is shown in Fig. 2 We first analyse the descriptions of goals in the GM to extract informative words and expand them to gain Vocabulary preliminarily; also, the reviews are handled in parallel, including pre-processing and extracting keywords. Fig. 2Open in figure viewerPowerPoint Process for establishing R-GM The relationships between goals and reviews are identified. We adopt the bottom-top traversal strategy to search the DTree of GM, which gives the hierarchy of the model and calculate the relevancy between each goal and reviews to identify whether they are related. When one relationship between a review and the goal is established, we enrich the Vocabulary of the goal with the keywords of the review, then step (ii) is implemented again for the goal until no new review is identified related to it. Based on the above process, we gain a set of relevant reviews for each goal G in GM, denoted by <G, >. Next, we give a detailed introduction of each step in this process. 3.2.1 First, goals and reviews are handled separately For one goal, the words with particular part of speech (POS ), including {nouns, verbs, adjectives, adverbs}, give the main information of G. For example, in the descriptions of two goals and in Fig. 1, the words are more informative for the hard goal and are more important for the soft goal . These words can be used to describe the goals simply for discovering their relevant reviews, and we gain them from a goal by either analysing its description manually or tagging the POS of words with automatic tools directly, such as NLTK [42]. Then, the Vocabulary of the goal is initialised by these words. Considering that the Vocabulary still contains a small number of words, we further expand the words in it with their synonyms. For a word , if , , is added into Vocabulary. is calculated by the path similarity between words based on Word.Net in this paper, the higher value indicates the similarity of the two words are higher; α is a pre-defined threshold to control the expanding process. For example, the word ‘photo’ in the Vocabulary of G12 is expanded with words such as , and the word ‘easy’ in the Vocabulary of G3 is expanded with words such as . For a review, we pre-process it and extract its keywords. A review R often consists of more than one sentence, and we split it into sentence-level for the analysis. There are mainly three reasons: Some sentences are informative while others are not, for example, the first sentence ‘No other app can be better than Instagram!’ in R3 (Table 1) is useless for evaluating the goals of the App, while its second sentence is useful. Different sentences may give comments on different goals in GM, such as the two sentences in R4 (Table 1) are related to ‘Post photo (G13 in Fig. 1)’ and ‘Edit video(G14)’, respectively. Different sentences may contain totally different sentiments. For example, the first sentence in R5 (Table 1) is quite positive while the second sentence is relatively negative. We generate a review instance from each sentence as the basic unit to be analysed to improve the accuracy. Note that all the items ‘reviews’ mentioned in the subsequent introduction on our approach are sentence-level review instances, both for identifying goal-related reviews and analysing users sentiments on goals (introduced in Section 4). Then, basic pre-processing is conducted on each sentence: the non-English parts are removed; the mis-spelled words, acronyms, and abbreviations in the sentence are modified by utilising a custom dictionary for the most frequent out-of-dictionary words given in [35]; and word stemming is conducted; also, we adopt Stanford parser to identify the negation of terms in the reviews and remove them [16, 33]. Furthermore, similar as the descriptions of goals, only a few words are important to express the main information of a review, so we tag the POS of words in one sentence-level review by NLTK, and extract the words with {nouns, verbs, adjectives, adverbs} to establish its KeywordSet. 3.2.2 Second, identify goal-related reviews. In this step, the relevancy between a goal G and a review r is calculated for identifying their relationship. The relevancy is determined by the similarities of words in the Vocabulary of the goal and the KeywordSet of the review. In the two sets, the words have been tagged with their POSs, so we define three parameters , , by calculating the similarity between words with same POS. For example: Then, the formula to calculate is given as mentioned below where are the weights of the parameters satisfying and developers can set them according to the practice situation. In general, the POSs of words emphasised by goal and soft goal are usually different according to our observation, so for them should be also different: gives the functional requirements that are often expressed by nouns and verbs, so and are assigned higher weights; while introduce the non-functional requirements that are usually emphasised by adjectives and adverbs, so maybe more important for . For a goal G and a review r, if is bigger than the threshold , we establish a relationship between G and r as . In step (ii), we search the DTree of GM with a bottom-top traversal strategy to discover all the related reviews for each goal. In this process, a review can be related to more than one goal because one sentence can still give comments on different goals, for example, the first sentence of R5 in Table 1: ‘seeing posts’ makes it related to G1 in Fig. 1 while ‘easily’ means it also related to G3, so it would appear in both and . However, to avoid using the information of reviews repeatedly, we limit that a review cannot be related to two goals with decomposition relation, in which the children goal is given for achieving the parent goal. That is if there exists , r is not further compared with the ancestor nodes of G in DTree. 3.2.3 Third, enrich the Vocabulary of the goal. In fact, this step could be taken as a part of step (ii), but we separate it to emphasise its importance in the iterative process. In step (ii), once we identify relationships between a review r and a goal G, the Vocabulary of G is expanded with the KeywordSet of r: for , if , w is added into Vocabulary. In this way, Vocabulary can be enriched continually to adapt to the expression habits of users. For example, the second sentence of R3 in Table 1 is related to in Fig. 1 because ‘improve pictures’ is similar to ‘edit photo’, so the Vocabulary of would be enriched with the words ‘improve’ and ‘picture’. Then, the step (ii) is implemented for the goal according to the results of step (iii) to find more reviews related to the goal. After analysing the goals in GM with all the reviews by our approach, the reviews that are not related to any goals are taken as non-informative ones and filtered out, while the informative reviews are introduced into GM as a new factor of goals, and R-GM is constructed for analysing goals based on users sentiments. 4 Analysing R-GM for evaluating apps R-GM integrate the GM with reviews, and it can provide information for answering the questions given in Section 1. For evaluating whether a goal is achieved well (question 1), we gain users sentiments on goals from its related reviews, and further summarise the sentiments according to the decomposition relations among goals. The result of each goal helps developers to know how many users pay attention to the goal and whether they like it. For determining which goal to be updated (question 2), we further analyse the impact of each goal on other goals based on contribution relations in the R-GM, and evaluate whether it is positive or negative to update the goal. Combining above information together, developers can optimise the evolution strategy of the App to better retain and attract users. 4.1 Quantify the users sentiments in reviews In R-GM, each goal G is related with a set of reviews , so we first extract users sentiments from each review in . In App Stores, a review is given with the rating, which reflects the users’ sentiments directly. However, the rating aims at commenting on the overall App rather than concrete features of the App. Thus, we extract two variables from the text of a review and use them with rating together to quantify the users’ sentiments on the goal: Emotion, which reflects the emotion of the user when giving reviews. Usually, strong emotions may lead to bias in the review. So we use Emotion to evaluate whether the review is tendentious and help to alleviate the affection, , the smaller value indicates the review can reflect the truth better. Senti, which is sentiment expressed in the text of reviews directly, and from the most negative to the most positive. These two variables can be mined from reviews based on Pattern.en in NLP. Specifically, using the function sentiment() imported from Pattern.en to analyse a review can return the two values: the first is the Senti and the second is the Emotion. Then, we define the formula to quantify users sentiments in a review: where is the weight of Emotion and it is set according to characteristics of Apps in different domains. For example, the Emotion of reviews from a ‘Game’ App may not be important because developers need to focus on users emotion when they play the game; while the Emotion of reviews for an ‘Education’ App is important b