# An Interactive Tool for Goal Model Construction using a Knowledge Graph

Shahin Abdoul-Soukour, William Aboucaya, Nikolaos Georgantas

► To cite this version:

Shahin Abdoul-Soukour, William Aboucaya, Nikolaos Georgantas. An Interactive Tool for Goal Model Construction using a Knowledge Graph. REFSQ 2025 - 31st International Working Conference on Requirement Engineering: Foundation for Software Quality, Apr 2025, Barcelona, Spain. pp.15. hal-04907365v2

HAL Id: hal-04907365

https://inria.hal.science/hal-04907365v2

Submitted on 2 Apr 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# An Interactive Tool for Goal Model Construction using a Knowledge Graph

Shahin Abdoul Soukour<fnref n="1" />[0009-0005-5026-712X], William Aboucaya<fnref n="1" />,<fnref n="2" />[0000-0003-2413-6968], and Nikolaos Georgantas<fnref n="1" />[0000-0001-5704-4889]

**Abstract.** [*Context and motivation*] The goal model is an essential model in Goal-Oriented Requirements Engineering. It is used to describe the system's goals using a hierarchical structure in which high-level goals are refined into more specific ones.

*[Question/problem]* Constructing a goal model for a new application can present challenges, demanding considerable time and effort. Although there have been attempts to automate or semi-automate the construction of goal models, these tasks remain complex and manual.

*[Principal ideas/results]* This paper presents an interactive graphical tool that leverages a domain Knowledge Graph (KG) to assist the application designer in creating goals derived from this knowledge, thereby facilitating the creation of goal models. We use semantic similarity measurement and Natural Language Inference (NLI) to effectively extract and align triples from the KG with the high-level initial goals formulated by the application designer. The extracted triples undergo sentiment analysis and Graph-to-Text (G2T) generation to build meaningful sub-goals. Nevertheless, processing KGs with Natural Language Processing (NLP) techniques can be a lengthy process. We introduce a restriction-based approach to bound the exploration of the KG to the most promising nodes. By tuning KG exploration bounds while using our tool in a case study, we analyze the trade-off between the quality of the resulting goal model and time performance, which is a key factor for an interactive approach.

*[Contribution]* Our paper highlights the relevance of our restriction-based approach for information retrieval from KGs to facilitate goal model generation in an interactive way. It proposes a tool that implements this approach.

**Keywords:** Goal modeling · Goal-Oriented Requirement Engineering · Requirement Engineering · Knowledge Graph · Natural Language Processing · Natural Language Inference · Graph-to-Text

# 1 Introduction

**Goal-Oriented Requirements Engineering (GORE) [15,16,17]** is a specific approach to Requirements Engineering (RE). RE is the first step in the design

<footnote n="1">¹ Inria, Paris, France {abdoul-shahin.abdoul-soukour,william.aboucaya,nikolaos.georgantas}@inria.fr</footnote>

<footnote n="2">² DRM – Université Paris Sciences et Lettres, Paris, France</footnote>
process of a software system. GORE's main focus is on identifying, capturing, and modeling stakeholders' goals for such a system. Various goal-oriented modeling methods have been proposed, such as [4,11,16]. The **goal model** is the central model of the GORE methods. It is a structural approach that enables goals to be defined and analyzed, and helps to understand the stakeholders needs. It provides a representation of goals in an explicit way, where high-level goals (strategic, global) are refined (or decomposed) into lower-level ones (operational, local, design-specific) until a clear understanding of the system is reached. Ultimately, the leaf goals of the goal model are elementary goals and are considered as specifications of software components. Building a goal model for a new application is a complex and challenging task, requiring both time and effort from the application designer [16]. Complexity comes from both identifying the goals of the new system and organizing them into a goal model.

A few approaches have proposed to generate a goal model in an automatic or semi-automatic manner by extracting goals from a small text corpus. Most approaches apply Natural Language Processing (NLP) techniques on various text sources. Das et al. [8] propose an end-to-end framework that uses several NLP techniques in order to transform automatically a set of unstructured natural language requirement specifications into goal models, supporting goal refinement, associations among goals, resources and softgoals. In this case, requirements are already provided by stakeholders; the focus is on organizing them in a structured way. Casagrande et al. [6] collect abstracts of research publications related to a specific system category. They then use these texts to extract goals by looking for text parts containing goal-related keywords; from these goals a goal taxonomy is created. However, other text parts that may contain relevant domain-specific information for creating a goal are not considered. Both of the above approaches attempt to build a complete goal model from the reference text sources. The application designer only gets involved at the end of the process to check the goal model and make any necessary changes. Recent research has considered Large Language Models (LLM) in the field of RE, offering the potential to drive RE processes [2]. Nakagawa et al. [18] presented an approach for semi-automatically generating a goal model based on a LLM that iteratively refines the goal model. However, the information extracted from the LLM may lack precision for a particular domain. The intervention of domain experts is necessary to effectively validate and refine the goal model.

In contrast to these solutions, our approach proposes using domain Knowledge Graphs (KG) to interactively assist the application designer in creating and refining their goals into a goal hierarchy, inspired from the KG. We consider that domain KGs are extremely valuable resources for developing goal models, as they can help identify relevant facts on a given topic. Goals describe a condition of the target system that should be achieved or avoided [16]. Relevant facts extracted from KGs can inspire target conditions related to goals [1].

To efficiently explore a domain KG, our method uses NLP techniques based on pre-trained language models. Our approach identifies areas of interest inside a KG by measuring semantic textual similarity between a designer's high-level
goal and KG triples, and uses Natural Language Inference (NLI) to determine textual entailment relations between triples and the high-level goal. Such entailment relations can emulate relations between a goal and potential subgoals. Additionally, sentiment analysis is performed to detect positive and negative connotations in triples, implying target conditions that should be achieved or avoided, respectively. Finally, Graph-to-Text (G2T) generation is employed to convert triples into coherent text for facilitating the designer in their creation of new goals. Through step-by-step interaction with our solution, the designer can refine their initial goals into a goal hierarchy from the KG, which may be complemented by data from other sources to create a complete goal model.

An initial version of our approach was published in [1]. In the present paper, we report on the development of an interactive graphical tool that implements our approach. This tool enables real users to use our approach and incorporates several small improvements to our solution, pointed out in Sections 3 and 4, which present the tool workflow and the evaluation, respectively.

Nevertheless, most importantly in this paper, we enhance our solution with the capability to bound the exploration of a KG to the most promising triples. This aims at accelerating the search inside a KG, thus improving the time performance of our approach, which is a key feature for an interactive tool. More specifically, we constrain the maximum depth of exploration of the graph for each information retrieval process and the number of neighbors from a given triple explored at each step of the exploration process.

Based on this approach, we assess the trade-off between the quality of the resulting goal model and time performance. In particular, we use our tool to design a flood management system similar to the one proposed in [1], for the purpose of which we have built a KG related to flood management. More specifically, we identify the share of additional (ir)relevant information retrieved by the more exhaustive versions of our graph exploration algorithm based on beam search, as well as the time-performance cost induced.

The code and KG used to produce the results of this paper are available at
https://github.com/ShahinAbdoulSoukour/Knowledgeable_Goal_Modeling

2 Background and related work

2.1 Background

Natural Language Inference (NLI) involves examining a pair of sentences,
usually called a *premise* and a *hypothesis*, and determining the relationship be-
tween them. The relationship can be classified in one of three categories: en-
tailment (the hypothesis can be inferred from the premise), contradiction (the
hypothesis directly contradicts or opposes the premise) or neutral (if the rela-
tionship between the premise and the hypothesis is neither an entailment nor a
contradiction). Several methods have been proposed to deal with NLI tasks [26].
In this context, we will focus on transformer-based language models specifi-
cally trained for this purpose. These models are well-suited for NLI tasks due
to their contextual understanding of a sentence or a pair of sentences, and of-
ten outperform other methods. In this paper, NLI tasks are performed using a
RoBERTa<sub>LARGE</sub> model trained using the SNLI [3], MNLI [28], ANLI [21] and
FEVER-NLI [20] corpuses<fnref n="3" />. Performances of this model on the different datasets
used for training are available in [21].

**Graph-to-Text** (G2T) aims at generating fluent texts from knowledge stored as a graph. The quality of G2T models is typically evaluated by comparing the generated text to a human-written reference using metrics like BLEU [22] or BERTScore [30]. For evaluating the *factual faithfulness* of the generated texts, specialized metrics like Data-QuestEval [24] or FactSpotter [29] are used. Most advanced G2T models use Transformer architectures and are trained on datasets like DART [19] or WebNLG [7], which contain pairs of graph-based facts and their text summaries. This paper employs a T5<sub>BASE</sub> [23] model trained on the WebNLG'20 dataset<fnref n="4" />. Performances of this model on multiple metrics, including the *factual faithfulness* of the generated texts in relation to the original data, are available in [29].

**Goal model evaluation** is an important part of GORE. This process involves evaluating the quality of goal models. Evaluation methods are either qualitative (i.e., experts manually inspect the goal model to identify issues and inconsistencies) or quantitative (i.e., using metrics to assess the goal model's quality, such as complexity and goal achievement) [12]. For automated or semi-automated goal model generation, both qualitative and quantitative evaluations can be used to compare the goals generated by the tool with those validated by the experts [6,27], or to use expert feedback to improve the goal model [18]. On the other hand, *formal verification* can be used to formalize the goal model, allowing for its analysis and validation of correctness [5,16,17].

## 2.2 Related work

The automatic or semi-automatic generation of a goal model is a challenging
task, and several approaches have been proposed to tackle this problem. Most
approaches apply NLP techniques on different text sources.

Güneş *et al.* [14] proposed an approach to automatically generate and vi-
sualize goal models from user stories using heuristics. The evaluation process
involved expert feedback, heuristic refinement, and experiments measuring the
similarity between manual and automated models. Case studies and academic
experiments assessed the tool’s usefulness for agile practitioners. The resulting
goal models are typically small, focusing on organizing pre-formulated user sto-
ries. A tool called ArTu [13] was implemented for this purpose.

Das *et al.* [8] proposed an end-to-end framework that uses various NLP tech-
niques to automatically transform a set of unstructured natural language re-
quirement (uNLR) specifications into goal models. The framework follows pre-
cise steps to identify goal model artifacts and their relationships across multiple

<footnote n="3">³ Model available at https://huggingface.co/ynie/roberta-large-snli_mnli_fev er_anli_R1_R2_R3-nli</footnote>

<footnote n="4">⁴ Model available at https://huggingface.co/Inria-CEDAR/WebNLG20T5B</footnote>
uNLR statements. However, it may miss some goal model concepts and struggle with identifying relationships between goals and goal decompositions that span multiple statements.

In both approaches above, requirements are already provided by stakeholders; the focus is on organizing them in a structured way.

Casagrande et al. [6] use NLP to extract goals from the abstracts of research publications related to a specific system category. They extract sentences that contain goal-related keywords and generate syntax trees. A rule-based syntactic pattern is employed to extract a goal as a triplet, which is then transformed into a structured format. A goal taxonomy is created using iterative centrality re-ranking, positioning goals based on their occurrence. This taxonomy serves as a preliminary goal model. However, other sentences that may contain domain-specific information pertinent to the goal creation are not considered. Some goals may be missed in the resulting goal model.

Recent research has explored the use of LLMs in the field of RE, offering the potential to drive RE processes [2] and generate goal models.

Nakagawa et al. [18] introduced a semi-automatic approach using an LLM within a MAPE-K loop (an autonomic loop for implementing self-adaptive systems) to generate and refine goal models iteratively. The LLM creates the initial model, validates it, provides feedback, and integrates new goals based on expert input. However, the generated models may be generic, requiring expert intervention for domain-specific refinement and validation.

In contrast to existing methods, we introduce an interactive tool designed to assist application designers at each step of creating a goal model rather than validating and adjusting an entire goal model at the end. Our tool effectively leverages knowledge from a domain KG to suggest relevant information that inspires goal creation. Moreover, designers can introduce additional goals from other sources to each part of the goal model, enabling a comprehensive and flexible approach to goal modeling. Our tool offers real-time interaction enabling designers to make adjustments throughout the goal modeling process, which results in a more precise and contextually accurate goal model.

## 3 Tool workflow

We elaborate in this section the workflow of our interactive graphical tool for assisting an application designer in building a goal model for a new application by leveraging a pre-existing domain KG. The user interface of the tool is presented in the Figures 1, 2 and 3).

<figure><img src="image_2.png" /><figcaption>Fig. 1: Screen capture from the proposed tool showing the parent goal creation.</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>Fig. 2: Screen capture from the proposed tool showing the creation of a subgoal based on information related to its parent goal.</figcaption></figure>

The following paragraphs present the tool’s process, which aims to facilitate the refinement of a given goal. We explain the steps of the process, which are also depicted in Figure 4 with their respective numbers. We also illustrate the different steps with an example from the tool’s execution for the flood management scenario. The KG that we have built for this scenario is partially depicted in Figure 5. The illustration part is *italicized* in each paragraph to accentuate the distinction with the part explaining the process:

1. **Triple inference.** Inference rules<fnref n="5" /> are applied to complement the KG, focusing particularly on inheritance relationships, addressing cases where the NLI fails to identify the fact expressed by the triple, despite its potential contribution to the goal.

2. **Formulate a goal and provide parameters.** The application designer formulates a non-composite goal. If the goal is complex, it should be split into multiple elementary goals. Then, the designer inputs *maximum depth* and *beam width* to limit the depth and width of the exploration of the KG (see the parent goal creation form in Figure 1). In the following illustration, we will use the high-level goal (or parent goal) "Anticipate the impact of floods on people", a maximum depth of 3 and a beam width of 5.

3. **Search for anchor triples in the KG.** The tool identifies relevant triples in the KG based on the formulated goal. Querying the KG can be challenging, requiring SPARQL knowledge and understanding of the KG’s structure, or

<footnote n="5">⁵ An example is the rule "IF *subject predicate parent* AND *child rdf:type parent* THEN *subject predicate child*"</footnote>
<figure><img src="image_4.png" /><figcaption>Fig. 3: Screen capture from the proposed tool showing the visualization of an example goal refinement.</figcaption></figure>

exact keyword matches. Visual representations of the KG can aid information
retrieval but may become overwhelming and challenging to navigate as the
KG grows in size and complexity. To overcome this, our approach employs
SBERT [25] to compute the semantic textual similarity between each triple
of the KG and the expressed goal. We only consider the best results, i.e., the
triples whose similarity with the goal is above $c \times \text{best score}$, with $c \in [0, 1]$
a confidence parameter$^6$. If the number of triples found is lower than an
arbitrary value$^7$, we reduce the confidence parameter to a second threshold
$c_2$ to capture more triples. These triples are considered as **anchor triples**
in the KG. In our example, the anchor triples are: "Flood Causes Health and
safety risks", "Flood Causes Loss of life", "Flood warning system Analyzes
Forecasting" and "River flood warning system Analyzes Data".

4. Identify connotation in anchor triples by applying sentiment analysis. The tool performs sentiment analysis using a RoBERTa model<fnref n="8" />. This step aims at determining the sentiment of anchor triples in order to distinguish between facts to achieve and facts to avoid. To enhance entailment detection that comes in the next step, negatively connoted triples are prefixed with "Prevent that". In our example, anchor triples "Flood Causes Health and safety risks" and "Flood Causes Loss of life" are flagged as negative and are therefore transformed respectively into "Prevent that Flood Causes Health and safety risks" and "Prevent that Flood Causes Loss of life".

<footnote n="6">⁶ Higher values of *c* enhance the quality of the retrieved triples while reducing their number.</footnote>

<footnote n="7">⁷ Here, we use 4 as the arbitrary parameter. However, this value can be increased or decreased depending on whether it is more important to provide a high number of data or only highly relevant resources.</footnote>

<footnote n="8">⁸ Available at https://huggingface.co/cardiffnlp/twitter-roberta-base-sentiment-latest</footnote>
<figure><img src="image_5.png" /><figcaption>Fig. 4: Steps of our approach to facilitate goal models generation using a KG</figcaption></figure>

5. **Test entailment between anchor triples and the goal.** The tool uses NLI to evaluate entailment between each anchor triple as premise and the submitted goal as hypothesis. If an anchor triple entails the goal, the fact expressed by the triple can be considered as contributing to the goal. The resulting entailing anchor triples are then presented to the designer, who can use them as inspiration to create new subgoals in order to refine the goal model. In *our example*, the anchor triples “Prevent that Flood Causes Loss of Life” and “Prevent that Flood Causes Health and Safety Risks” are entailed with the goal. The non-entailing anchor triples are “Flood warning system Analyzes Forecasting” and “River flood warning system Analyzes Data”.

6. **Add context to non-entailing anchor triples if there is no entailment.** In cases where there is no entailment between an anchor triple and the goal, a possibility is that an implicit entailment might still exist depending on a context which has not been expressed. To tackle this, the tool concatenates each non-entailing anchor triple with a neighbor triple (selected as described in the next step) to form a **contextualized triple**. In *our example*, we can produce the contextualized triples “Flood warning system Recommends Evacuation of residents. Flood warning system Analyzes Forecasting”, “Flood warning system Predicts Flood. Flood warning system Analyzes Forecasting”, etc.

7. **Beam search for context triples.** To identify potentially relevant context triples, the tool computes semantic textual similarity between the goal and the different versions of the contextualized triple. It then selects the *β* context triples that produce the highest similarity of the contextualized triple to the goal, with *β* equal to **beam width**, and discards the other neighbor triples. We choose this criterion, as computing semantic textual similarity is less
time-consuming than entailment detection that is applied in the next step. Furthermore, this technique allows to perform a beam search, which is more time-effective than an exhaustive exploration of the graph.

8. **Test entailment between the contextualized triples and the goal.**
The tool tests entailment of the goal and retains the contextualized triple with the highest entailment score. If entailment has been achieved, it proceeds to the next step. If entailment does not hold, it repeats the process (Steps 6 to 8) by adding another context triple to the already contextualized triple. This iterative contextual enrichment continues as long as the entailment score increases. The process stops if entailment is reached, if the entailment score decreases, or if the number of added context triples becomes equal to the **maximum depth** parameter. For instance, these contextualized triples can be retained as they lead to an entailment: "Flood warning system Recommends Evacuation of residents. Flood warning system Analyzes Forecasting". However, these contextualized triples cannot be retained even with added context triples, as it does not lead to an entailment (the entailment score decreased): "Consumer cameras Generate Data. River flood warning system Generates Alerts. River flood warning system Analyzes Data".

9. **Apply G2T to transform the entailing contextualized triples into texts.**
The tool applies G2T to transform the entailing (both single and contextualized) triples into meaningful texts that can help the application designer in creating subgoal(s). The prefix `[ACHIEVE]` is added to texts resulting from anchor triples with positive or neutral connotations, indicating that the information in the text contributes to the achievement of the goal. Conversely, the prefix `[AVOID]` is added to texts resulting from anchor triples with negative connotations, indicating an obstacle or an undesirable result to be avoided in order to reach the goal. The entailing triples and the generated texts are then displayed to the application designer as shown in Figure 2. For these contextualized triples "Flood warning system Recommends Evacuation of residents. Flood warning system Analyzes Forecasting", the generated text is: `[ACHIEVE] the flood warning system recommends evacuation of residents and analyzes forecasting.

10. **Create subgoal(s).** Drawing inspiration from (some or all) the texts output by the tool in the previous step, the application designer creates one or more subgoals and adds them to the goal model. Indeed, the designer can freely review, adopt or modify these texts. Furthermore, the designer should indicate which triples among the ones suggested by the tool were used for subgoal creation (see checkboxes in the Figure 2). In this way, the history of utilization of triples will be taken into account by the tool in its future suggestions of triples. In our example, by selecting the entailing triples provided by the tool "Flood Causes Loss of life" and "Flood Causes Health and safety risks", the designer can formulate a new subgoal of type [AVOID]: "Avoid losses of lives and other risks for health and safety in case of flood". The new subgoal will be displayed as shown in Figure 3.
<figure><img src="image_6.png" /><figcaption>Fig. 5: Partial view of the domain-specific KG related to flood management. A visualization of the complete KG is available at https://github.com/ShahinAbdoulSoukour/Knowledgeable_Goal_Modeling/raw/main/flooding_graph_V2.png</figcaption></figure>

11. Repeat the process for constructing the goal model. The interaction between the application designer and the tool can be repeated several times, where each time the designer can input a new goal to the tool, e.g., one of the subgoals created in the previous iteration. This allows the designer to refine and extend the goal hierarchy by leveraging the KG knowledge. The designer may also include in this process goals inspired from other sources.

# 4 Tool development and evaluation

We have developed our tool by relying on FastAPI, a web framework for building APIs with Python, and on Google Charts for displaying a goal model as an organization chart. For performing the various NLP inferences, we rely on the HuggingFace environment.

We present in this section the evaluation of our tool, and more specifically the evaluation of our approach for bounding the exploration of a KG for improving the time performance our tool. For this evaluation, we use our tool for creating goal models for a flood management system based on our hand-crafted KG (see Figure 5). The KG is composed of 90 triples (61 nodes linked by 28 different types of edges). It provides an overview of the basic principles of flood management and risks mitigation. It covers the possible causes of floods, techniques for detecting them, the anticipated effects of floods on residents and infrastructure, and measures to prevent or to minimize these impacts. For our experiments, we used a configuration of our tool with confidence parameters $c = 0.85$ and $c_2 = 0.65$ as the arbitrary thresholds for semantic similarity. Our experiments were performed on an Nvidia 3080 RTX GPU with 32 GB RAM.

To evaluate the quality and effectiveness of our tool, we used it to construct eight goal models (GM). In this process, we adjusted two core parameters (**maximum depth** and **beam width**) to control the depth of exploration
<table><caption>Table 1: Goal model construction performance and relevance metrics</caption><thead><tr><th>GM Nº</th><th>max. depth</th><th>beam width</th><th>NET</th><th>NSET</th><th>M10</th><th>NLG</th><th>MD</th><th>T (sec)</th><th>R (%)</th></tr></thead><tbody><tr><td>1</td><td>1</td><td>3</td><td>9</td><td>2</td><td>1</td><td>1</td><td>1</td><td>10.715</td><td>22.222</td></tr><tr><td>2</td><td>3</td><td>1</td><td>44</td><td>20</td><td>11</td><td>5</td><td>5</td><td>11.858</td><td><strong>45.454</strong></td></tr><tr><td>3</td><td>3</td><td>2</td><td>50</td><td>18</td><td>10</td><td>4</td><td>5</td><td>13.283</td><td>36.000</td></tr><tr><td>4</td><td>3</td><td>3</td><td>49</td><td>19</td><td>10</td><td>4</td><td>5</td><td>13.735</td><td>38.775</td></tr><tr><td>5</td><td>3</td><td>5</td><td>46</td><td>19</td><td>10</td><td>4</td><td>5</td><td>13.290</td><td>41.304</td></tr><tr><td>6</td><td>5</td><td>3</td><td>54</td><td>20</td><td>11</td><td>5</td><td>5</td><td>13.670</td><td>37.037</td></tr><tr><td>7</td><td>1</td><td>1</td><td>9</td><td>2</td><td>1</td><td>1</td><td>1</td><td>10.460</td><td>22.222</td></tr><tr><td>8</td><td>99</td><td>99</td><td>75</td><td>29</td><td>14</td><td>7</td><td>5</td><td>16.941</td><td>38.666</td></tr></tbody></table>

within the KG and to filter relevant triples (triple neighbors) using beam search. We performed the experiment for multiple values of each of these two parameters, including a baseline with the two parameters set to 99 to replicate the behavior of an unconstrained version of the tool. Each generated goal model progressively refined the initial high-level goal, “Anticipate the impact of flooding on people” allowing us to capture variations in structure and relevance across different configurations (see Table 1). After constructing these goal models, we extracted the coverage of relevant triples for each goal model, particularly focusing on the number of entailing triples (i.e., triples that can contribute to the high-level goal) provided by our tool (NET), the number of entailing triples selected by the designer for goal refinement (NSET), the number of direct or indirect refinements of the high-level goal (called M10 in [9]), and the number of leaf goals (NLG). Moreover, we measured the depth of the goal hierarchy (called MD in [10]) and also the execution time for each refinement in a goal model. We use NET and NSET metrics to quantify both the number of facts highlighted by the tool and, out of these, the number of facts which are relevant to help refine high-level goals into more specific ones. Then, we calculated the mean execution time (T)<fnref n="9" />. Next, we calculated the relevance (R), which indicates how effectively the selected entailing triples contributed to the refinement of the goal model (see Equation 1).

$$R = \left( \frac{NSET}{NET} \right) * 100 \qquad (1)$$

Regarding the performance of our approach, we noticed, as expected, an increase in execution time when the parameter values are higher. The first plot (Figure 6a) highlights how different values affect the relevance. If the value is

<footnote n="9">⁹ N.B.: After publication of our previous work [1], a bug was identified which led the tool to behave as if the **maximum depth** parameter was set to 1, even though this parameter had not yet been implemented. The intended behavior was for the tool to function as if **maximum depth** was set to +∞. This bug has since been fixed, and the tool now correctly explores the graph. However, due to this past issue and the differences in the graph used for knowledge extraction, results – particularly time-based metrics – should not be compared between the two papers.</footnote>
<figure><img src="image_7.png" /><figcaption>Fig. 6: Parameter impact on goal model quality and performance (closer to the top-left corner is better)</figcaption></figure>

high (closer to 100%), it suggests that most entailing triples are useful for goal formulation. This means that KG exploration and entailment filtering are effectively identifying triples that the designer can leverage directly. If the value is low, it signifies that only a few entailing triples are useful in formulating the goals. This might indicate that some triples do not contribute significantly to the high-level goal or are not relevant. The parameters could be adjusted to focus on more relevant triples. Then, the second plot (Figure 6b) highlights how parameter choices affect the final number of leaf goals produced.

To illustrate the results obtained, Figure 7 provides an example of a goal model produced during the evaluation for the configuration with max depth = 3 and beam width = 1.

The analysis suggests that the optimal configuration for constructing an initial goal model with a balanced execution time and high relevance is the configuration #2, with a **maximum depth of 3** and a **beam width of 1**. Using these parameters, we obtain a **Relevance score of 45.454 %**, which is the highest value in our experiment, while **keeping the execution time acceptable** ($T \approx 11.858$ seconds). However, this configuration is not able to help us produce as many goals as the configuration with a **maximum depth** and **beam width** of 99, therefore inducing an existing loss of relevant information.

The designer can complete each part of the generated goal model by either using our tool with different parameter adjustments or adding goals from other sources. This approach ensures that the goal model remains dynamic and adaptable to changing needs.
<figure><img src="image_8.png" /><figcaption>Fig. 7: Screen capture from the proposed tool showing the goal model created using configuration #2.</figcaption></figure>
# 5 Threats to validity

Concerning the scalability of our approach, the step where we identify anchor points using semantic textual similarity has a complexity which is linear with the number of triples in the graph. While this problem is irrelevant in the context of our evaluation where the KG used contains 119 triples after simple inferences, our method clearly shows its limits in the context of an open KG such as Wikidata<fnref n="10" /> or DBpedia<fnref n="11" />. Therefore, our method can only be applied to relatively small, domain-specific KGs. Depending on the context, such graphs could be either found as open data or derived from an open KG by selecting only the triples fulfilling certain criteria (e.g., within a certain distance from a given entity, not linked to irrelevant parts of the topic, etc.). While we built ourselves the KG for the flood management scenario used in our evaluation, we plan to investigate in the future the possibility of extracting domain-specific KGs from large open KGs.

Additionally, while we conduct a quantitative analysis of the information highlighted by our tool and its relevance, we do not assess the overall quality of the goal models produced. We consider that this issue should be tackled by conducting an experimental evaluation of goal models produced by volunteers using our tool or a baseline alternative. The evaluation could be made based on qualitative criteria (e.g., hierarchical consistency or completeness of the goal model), which would help better characterize the impact of the use of our methodology on goal model creation.

# 6 Conclusion

In this paper, we present an interactive graphical tool that can leverage domain KGs to assist application designers in creating goal models. Besides implementing for real users the KG exploration algorithm proposed in our previous work, our tool incorporates a new feature supporting depth limitation and beam search during the KG exploration process. This improvement is designed to avoid unnecessary and costly explorations, thereby accelerating the overall exploration process, which is essential for interactivity with the designer.

To validate this feature, we constructed multiple goal models by using our tool with varying depth limitation and beam search parameters. Each model was evaluated on three main criteria: structure of the goal models, processing efficiency, and quantitative metrics of effective usage of KG knowledge. This allowed us to analyze the trade-off between tool execution time and effectiveness in leveraging the KG.

As future work, we plan to perform an empirical evaluation of our tool to assess the degree of facilitation for the designer and the quality of the resulting goal models.

<footnote n="10">¹⁰ ~ 16 billion triples at the time of writing of this article.</footnote>

<footnote n="11">¹¹ ~ 1.2 billion triples in the English version at the time of writing of this article.</footnote>
**Disclosure of Interests** The authors have no competing interests to declare that are relevant to the content of this article.

References

1. Abdoul Soukour, S., Aboucaya, W., Georgantas, N.: Leveraging Knowledge Graphs for Goal Model Generation. In: 7th Workshop on Natural Language Processing for Requirements Engineering (NLP4RE). Winterthur, Switzerland (2024)

2. Arora, C., Grundy, J., Abdelrazek, M.: Advancing requirements engineering through generative ai: Assessing the role of llms. In: Generative AI for Effective Software Development, pp. 129–148. Springer (2024)

3. Bowman, S.R., Angeli, G., Potts, C., Manning, C.D.: A large annotated corpus for learning natural language inference. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. ACL, Lisbon, Portugal (2015)

4. Bresciani, P., Perini, A., Giorgini, P., Giunchiglia, F., Mylopoulos, J.: Tropos: An agent-oriented software development methodology. Autonomous Agents and Multi-Agent Systems 8 (2004)

5. Cailliau, A.: Software requirements engineering: A risk-driven approach. Ph.D. thesis (2018)

6. Casagrande, E., Woldeamlak, S., Woon, W.L., Zeineldin, H.H., Svetinovic, D.: Nlp-kaos for systems goal elicitation: Smart metering system case study. IEEE Transactions on Software Engineering 40(10), 941–956 (2014)

7. Castro Ferreira, T., Gardent, C., Ilinykh, N., van der Lee, C., Mille, S., Mous-sallem, D., Shimorina, A.: The 2020 bilingual, bi-directional WebNLG+ shared task: Overview and evaluation results (WebNLG+ 2020). In: Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web. Association for Computational Linguistics, Dublin, Ireland (Virtual) (2020)

8. Das, S., Deb, N., Cortesi, A., Chaki, N.: Extracting goal models from natural language requirement specifications. Journal of Systems and Software 211 (2024)

9. Espada, P., Goulão, M., Araújo, J.: Measuring complexity and completeness of kaos goal models. In: Workshop on Empirical Requirements Engineering (EmpiRE 2011). IEEE (2011)

10. Espada, P., Goulão, M., Araújo, J.: A framework to evaluate complexity and completeness of kaos goal models. In: Advanced Information Systems Engineering: 25th International Conference, CAiSE 2013. Springer, Valencia, Spain (2013)

11. Franch, X., López, L., Cares, C., Colomer, D.: The i* Framework for Goal-Oriented Modeling, pp. 485–506. Springer International Publishing (Jul 2016)

12. Gralha, C., Araújo, J., Goulão, M.: Metrics for measuring complexity and completeness for social goal models. Information Systems 53, 346–362 (2015)

13. Günes, T., Öz, C.A., Aydemir, F.B.: ArTu: a tool for generating goal models from user stories. In: 29th International Requirements Engineering Conference (RE). IEEE (2021)

14. Güne, T., Aydemir, F.B.: Automated goal model extraction from user stories using nlp. In: 28th International Requirements Engineering Conference. IEEE (2020)

15. van Lamsweerde, A.: Goal-oriented requirements engineering: a guided tour. In: Fifth IEEE International Symposium on Requirements Engineering (2001)

16. van Lamsweerde, A.: Requirements Engineering: From System Goals to UML Models to Software Specifications. Wiley Publishing, 1st edn. (2009)

17. Letier, E.: Reasoning about agents in goal-oriented requirements engineering (2002)
18. Nakagawa, H., Honiden, S.: Mape-k loop-based goal model generation using generative ai. In: 31st International Requirements Engineering Conference Workshops (REW). IEEE (2023)

19. Nan, L., Radev, D., Zhang, R., Rau, A., Sivaprasad, A., Hsieh, C., Tang, X., Vyas, A., Verma, N., Krishna, P., Liu, Y., Irwanto, N., Pan, J., Rahman, F., Zaidi, A., Mutuma, M., Tarabar, Y., Gupta, A., Yu, T., Tan, Y.C., Lin, X.V., Xiong, C., Socher, R., Rajani, N.F.: DART: Open-domain structured data record to text generation. In: Proceedings of the 2021 Conference of the North American Chapter of the ACL: Human Language Technologies. ACL (2021)

20. Nie, Y., Chen, H., Bansal, M.: Combining fact extraction and verification with neural semantic matching networks. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 33, pp. 6859–6866 (Jul 2019)

21. Nie, Y., Williams, A., Dinan, E., Bansal, M., Weston, J., Kiela, D.: Adversarial NLI: A new benchmark for natural language understanding. In: Proceedings of the 58th Annual Meeting of the ACL. ACL (2020)

22. Papineni, K., Roukos, S., Ward, T., Zhu, W.J.: Bleu: A method for automatic evaluation of machine translation. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. ACL, USA (2002)

23. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research 21 (140), 1–67 (2020)

24. Rebuffel, C., Scialom, T., Soulier, L., Piwowarski, B., Lamprier, S., Staiano, J., Scoutheeten, G., Gallinari, P.: Data-QuestEval: A Reference-less Metric for Data-to-Text Semantic Evaluation. In: 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). ACL, Punta Cana, Dominican Republic (2021)

25. Reimers, N., Gurevych, I.: Sentence-bert: Sentence embeddings using siamese bert-networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics (Nov 2019)

26. Schlegel, V., Nenadic, G., Batista-Navarro, R.: Beyond leaderboards: A survey of methods for revealing weaknesses in natural language inference data and models. arXiv preprint arXiv:2005.14709 (2020)

27. Shimada, H., Nakagawa, H., Tsuchiya, T.: Goal model construction based on user review classification. In: REFSQ Workshops (2019)

28. Williams, A., Nangia, N., Bowman, S.: A broad-coverage challenge corpus for sentence understanding through inference. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). ACL (2018)

29. Zhang, K., Balalau, O., Manolescu, I.: FactSpotter: Evaluating the Factual Faithfulness of Graph-to-Text Generation. In: Findings of EMNLP 2023 - Conference on Empirical Methods in Natural Language Processing. Singapore (2023)

30. Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y.: BERTScore: Evaluating text generation with BERT (2020)