SW‐Context: a model to improve developers’ situational awareness

IET SoftwareVolume 14, Issue 5 p. 535-543 Research ArticleOpen Access SW-Context: a model to improve developers' situational awareness Leandro Ferreira D'Avila, Leandro Ferreira D'Avila orcid.org/0000-0003-2602-6076 Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this authorJorge Luis Victória Barbosa, Corresponding Author Jorge Luis Victória Barbosa jbarbosa@unisinos.br Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this authorKleinner Silva Farias de Oliveira, Kleinner Silva Farias de Oliveira orcid.org/0000-0003-1891-3580 Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this author Leandro Ferreira D'Avila, Leandro Ferreira D'Avila orcid.org/0000-0003-2602-6076 Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this authorJorge Luis Victória Barbosa, Corresponding Author Jorge Luis Victória Barbosa jbarbosa@unisinos.br Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this authorKleinner Silva Farias de Oliveira, Kleinner Silva Farias de Oliveira orcid.org/0000-0003-1891-3580 Applied Computing Graduate Program (PPGCA), University of Vale do Rio dos Sinos (UNISINOS), São Leopoldo – 93.022-000, BrazilSearch for more papers by this author First published: 01 October 2020 https://doi.org/10.1049/iet-sen.2018.5156Citations: 2AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinked InRedditWechat Abstract This study proposes SW-Context, a model focused on defining context information for software, and storing this information in contextual history. The proposed model provides qualitative information concerning source code and design issues to support the daily activities of software developers, thereby improving their situation awareness. SW-Context was evaluated through a case study in industry, in which 12 developers used it to support maintenance activities throughout one month. After this period, all developers filled out a qualitative questionnaire, which was formulated based on the well-established technology acceptance model. The obtained results suggest that 67% of developers completely agree with the usefulness and ease of use of the SW-Context. These results are encouraging and show the potential for implementing SW-Context in broader software maintenance scenarios. 1 Introduction Situational awareness plays a central role in many software engineering activities, for example, providing qualitative information concerning source code and design issues to support daily maintenance tasks. The lack of qualitative information about the source code under maintenance may not favour the execution of such tasks. Usually, software developers need to manipulate source code to implement changes requested by customers, or even to solve bugs. In practice, software developers often need to fix bugs, add new features and evolve fast-changing enterprise systems, while keeping their software architecture in shape. In addition, separated software-development teams should share qualitative information (e.g. key design decisions) to ensure not only quality issues of the software under maintenance, but also better support time-consuming and error-prone tasks, including bug fixing without any context information [1 ]. Additionally, developers also need to deal with other issues such as organisational factors that may impact on the situational awareness about design decisions [2 ], for example, poor documentation, turnover of specialists and deadlines agreed with the customers. Software context might be briefly defined as information that software developers need to perform properly software-development tasks. This information can be related to different dimensions, including design metrics, technologies, organisational issues, among others [3-5 ]. All need to be adequately made available at the time of maintenance to enhance the assertiveness of the changes; otherwise, code anomalies and hence architectural degradation can arise. For example, to implement change requests developers should know which modules implement a functionality, how to change such modules properly, preventing the emergence of inconsistencies between the proposed architecture and source code. In recent years, some situational awareness-raising mechanisms such as software analytics have been proposed to provide actionable, real-time qualitative information [6-8 ]. Menzies [9 ] highlights that software analytics can produce high-value data by filtering large volume of low-value data. However, little has been done to evaluate how such software analytics might improve the daily work of developers while performing maintenance activities in real-world settings. In fact, implications and insights on how specific qualitative information may impact on developers' daily work are severely lacking. This qualitative information is typically represented as actionable analytics. Briand et al. [10 ] highlight the pivotal role of context-driven software engineering research to close the gap between research and industrial needs. To address this lack of qualitative information, this study proposes SW-Context, a model focused on defining context information to improve developers' situational awareness of software under maintenance. It means, the model aims to improve the state of mind in which developers are aware of the elements of software, have an understanding as to the environment's greater meaning, and can anticipate (or plan to change) these elements in the near future. These elements are part of the context information defined by SW-Context and are described below: Design patterns : This group of information aims to advise the developer about the current architecture of application. The developer should correlate the change requests with a possible software artefact in terms of architecture (e.g. factory) and extend this pattern avoiding degradation. History of last changes : By checking the history of last changes, the developers would identify similar previous changes and more easily find the source code artefact to be changed or discover some code examples. Customer usage of features and performance execution : The aim of this information is to get the developers aware about the possible impacts of a wrong implementation, for example, for the customers who are using the application. At this point, the developer can correlate the variation of time responses with the history of changes and more easily identify the involved artefacts. Software metrics : This information usually represents the current status of the application under maintenance regarding unit test coverage, lines of code and other relevant metrics. Depending on development method and technology, these metrics can impact the software architecture and must be considered during the maintenance activities. By supporting the developers with actionable information (created based on the elements described before), they can properly make design decisions during the maintenance activities. SW-Context stores qualitative information sequentially, forming a contextual history. The model provides a dashboard with contextual information that enables developers and teams to properly deal with the design decisions. SW-Context was evaluated through a case study in industry, in which 12 developers used a prototype for supporting maintenance activities throughout one month. Later, all developers filled out a qualitative questionnaire, which was formulated based on the well-established technology acceptance model (TAM) [11-13 ]. The results suggest that 67% of developers fully agree with the usefulness, and ease of use of the SW-Context's prototype. This result is encouraging and shows the potential for implementing SW-Context in broader software maintenance scenarios. The contributions of this paper are: (i) a model for improving developers' situational awareness about the current state of key design decisions made; and (ii) insights about how software analytics may be used to support maintenance activities. The remainder of this paper is organised as follows. Section 2 discusses the related works. Section 3 describes the SW-Context model. Section 4 describes the implementation aspects of SW-Context. Section 5 describes the case study performed, as well as introduces the results and insights. Finally, Section 6 presents concluding remarks and future work. 2 Related work This section introduces some related studies that explore the use of qualitative information through the concept of software context for supporting developers' situation awareness (Section 2.1). Moreover, this section discusses evaluation criteria, which were used to provide a comparative analysis of the related work (Section 2.2). Based on these evaluation criteria, we identify some research opportunities related to the objective of this study (Section 2.3). The search for related studies was based on the usage of the context definition [1 ] applied to software development. Additionally, studies that would define a model for context to software development were considered. The main sources were International Workshop on Context for Software Developers, Scholar – Google academic, Elsevier – Science Direct, IEEE – Xplore and ACM – Digital Library Springer – International Publishing AG. The terms used in our search were 'Software Context', 'Software Development' and 'Software Analytics'. 2.1 Related works using context on software development The related works described in this section cover the usage of context information related with the source code and widely context related to organisation factors as follows. 2.1.1 Context information related to development tasks Kersten and Murphy [14 ] defined task context as the information that a software developer needs to know to complete that task. The task context is used by Mylyn to improve the productivity of software developers when they need to switch from a current task to another. The plug-in slices the project vision showing only the relevant artefacts to complete the on-going task. Parnin and Gorg [15 ] followed the same line of task management and recovery, proposing an approach for capturing relevant contexts for a task from programmer's interactions with an integrated development environment (IDE). This information is then used to aid the programmer recovering the mental state associated with a previous task and to support the development activities using recommendation techniques. Leano et al. [16 ] defined a task context using a hybrid approach. The edited files are considered as part of task context and developers spend a lot of time to identify these files. The idea of the hybrid technique is to use a combination of information retrieval (IR), data mining and textual analysis techniques to determine the files which should be edited to accomplish a task. For example, when a developer starts editing an artefact, similar stored traces related to the artefact will be analysed to provide a possible set of files to be changed. To determine the full set of files that are likely to be changed for a task, the following approaches can be used: Commit Graphs, Crowd Sourcing and Expert inputs. Latoza et al. [17 ] highlighted software-development tasks require many types of context. For example, developers must know where specific functionalities are implemented, how to change them properly, keeping consistent with an architecture previously defined. When calling a functionality, developers must understand what assumptions it makes the system state and the effects that this execution may cause. Sawadsky and Murphy [18 ] addressed the issue related to discovering relevant code examples from the web through the Fishtail Eclipse plug-in, using the history of the programmers' interactions with source code and related artefacts. The key parts of Fishtails architecture include: (i) a manager component of task context for tracking the degree-of-interest of program elements; (ii) a query generator component to construct keywords for program elements in the task context; (iii) a query executor component for sending queries to a search engine and prioritising results; and (iv) a result display component which surfaces search results within the IDE. 2.1.2 Context information related to code search Holmes and Murphy [19 ] proposed an Eclipse plugin, which was named as Strathcona. It considers as structural context the same artefact elements of task context; however, the usage of this kind of context is to suggest code samples to a developer by querying a repository to find similar usage of source code. The main idea behind this study is to facilitate the usage of frameworks and APIs. Strathcona works by extracting the structural context of source code entities. This structural context includes the methods' signature, the declared type and parent type, the methods called, the name of fields accessed, and the types referred by each method. Martie and Hoek [20 ] used contexts to help developers on code searching activity. Searching for source code on-line is a common activity in programming. Algorithms, language examples and API usage examples are all searched during programming. The authors have developed a new code search engine, called CodeExchange, to directly support query reformulation. The interface of CodeExchange presents meta-data about the code results, highlights semantic properties of the code results and allows the programmer to use these meta-data and properties directly to reform the ongoing query. Tillmann et al. [21 ] identified the use of context information in the Code Hunt tool. Code Hunt is a web-based serious gaming platform in which players write code to advance through levels. The types of context identified in Code Hunt were categorised as follows: earlier solved coding duels, the secret code segment of the coding duel, the playing history of the coding duel, the coding duel being modified by the player, the input–output pairs reported to the player and the hint reported to the player. 2.1.3 Context information related to software operation and test coverage Zhang et al. [8 ] describe a Microsoft project that identifies high-impact program execution patterns from a large number of trace streams based on sequences of function calls that happen during program execution. This project helps developers to identify a performance-bottleneck through analytics techniques reducing the human investigation effort by 90%. The build process of StackMine was done using an iterative flow between researchers and practitioners, where the constant feedback was valuable information to product improvements and usability. Haron and Syed-Mohamad [22 ] proposed the test and defect coverage analytics model, which focuses on integrating test coverage and defect coverage data to assist managers to make decisions. These data are extracted from source code artefacts under test, unit test cases from test suit tools and reported defects from a defect tracking database. After analysing these data, the model shows the result through a dashboard named Test Analytics Dashboard. This dashboard demonstrates the information using a Bubble chart applying a qualitative approach. The model was implemented as an Eclipse plugin and its evaluation was performed by a case study on the Apache POI. The proposed model was compared to some test coverage tools such as Clover and SonarQube. 2.1.4 Context information related to organisation factors and processes Antunes and Gomes [3 ] proposed an approach for capturing contextual information from developers where it should take into account the whole environment that supports their work. This means that contextual information should be retrieved from all the applications the developer typically deals with, such as an IDE, a set of office tools, a Personal Information Manager, and even the operating system itself. Most of the tools referred provide some kind of plug-in integration, which largely facilitate IR. The information gathered across the various applications should then be centralised and coherently integrated within the context model, so that a snapshot of the context model of the user is available at any point in time. Antunes et al. [5 ] considered that software context developer takes into account all dimensions that characterise the work environment of the developer. These dimensions can be represented as a layered model with four main layers: personal, project, organisation and domain. The authors consider that the main sources of contextual information are project management tools. These tools store a big amount of explicit and implicit information about resources produced during a software development project, for example, how the involved people relate with these resources and how the resources relate to each other. Antunes et al. developed a prototype, in the form of an Eclipse plug-in, to show how the context information can be integrated into an IDE and used to help developers. 2.2 Comparative analysis of the related works This section presents a comparison of the analysed works considering four characteristic groups. Table 1 organises these characteristics as evaluation criteria and presents a comparison of the studies. The comparison criteria are: Context definition: It evaluates how the study has defined formally the context information and what kind of information was used on this. Table 1. Comparison of the related work Type of study: This criterion considers the type of research method applied. The types considered are survey, case study, controlled experiment, mapping study and others. Type of model: This criterion evaluates the type of model proposed by the analysed studies, including architecture, conceptual model, ontology and algorithm. Visualisation: This criterion evaluates the approach of providing context information. The considered approaches are analytics dashboard, structured data and views. Table 1 shows that all studies consider context information related to software artefacts and developers. Only Kersten and Murphy [14 ] defined formally context, but their definition is limited to the developer's interaction with development IDE, disregarding other aspects such as project information or software operations. Antunes and Gomes [3 ] and Antunes et al. [5 ] defined context for software development more widely, considering the main aspects listed in the current literature. The studies considered that the context information should contain more than the data extracted from the developer's interaction with the development IDE. This means the context information should consider personal aspects, organisational factors and projects where the requirements, assumptions and constraints make up the framework of information capable of improving the productivity of developers. The dimension of software operation is not considered by the studies cited, and there is no formal definition or context model in the form of ontology or class diagram. Holmes and Murphy [19 ] and Parnin and Gorg [15 ] organised their studies as case studies. Kersten and Murphy [14 ] conducted the study through the execution of a controlled experiment. The remaining studies were organised with a concept different from that covered by the criterion type of study like discussions of context types, identification of context in existing tools and creation of tools to handle context data. Considering the type of model criteria, it was found that none of the studies defined ontology or class diagram to represent software context. Most studies proposed algorithms as the main contribution. In terms of visualisation, most studies deal with structured data and views in their models, except for Zhang et al. [8 ]. This comparison allowed the identification of opportunities and possible contributions to the definition of context for software development. The next section discusses these opportunities. 2.3 Identification of research opportunities The comparative study performed among the related works allowed to determine the following research opportunities: the formal definition of context for the software-development domain, the management of histories of software contexts and the analysis of these histories to infer contextualised information to help the developers. A broader approach related to context definition allows the SW-Context model to improve the awareness of development teams about the contextual information surrounding the software artefacts. Based on the formal definition of context for software, the evaluation of the contextual modifications of the artefacts can be applied using context awareness techniques. These contextual modifications might indicate, for instance, the degradation of an application architecture. The analysis of these modifications may trigger some actions and warnings to developers about relevant contextual changes of the application. An analysis of context history using a Software Analytics approach is also characterised as a relevant research question [9 ]. An analysis and information model can be created to provide information through dashboards. These dashboards might help developers to make decisions during their software maintenance activities. The usage of the Software Analytics approach allows to obtain insights and information that help the developers and architects to identify cause–effect relationships from the contextual information. It would be possible, for example, to evaluate which factors might have affected or caused severe degradation of enterprise application architecture. 3 SW-Context This section presents SW-Context, a model to support developers with actionable, qualitative information from a history of software contexts. The model addresses some fundamental issues, including what information should form the concept of software context, how to structure such quantitative and qualitative information so that actionable information may be produced, and when developers should receive information from analytical queries to the context history. Consequently, they can properly make design decisions during maintenance activities. Suppose a scenario where a development team should monitor the measures of a design metric, such as cyclomatic complexity [23 ] of source code. If the value of this metric reaches a particular threshold set by the team, a warning must be triggered. In this way, the architects can promptly anticipate actions to avoid severe design problems, such as architecture degradation and generation of code anomalies. This early warning seeks to pinpoint specific risks to keep, for example, the architecture in shape. 3.1 SW-Context model Fig. 1 shows the conceptual model of the SW-Context, in which its key entities and relationships are expressed. The model is organised around five dimensions, which represent relevant information for software architects, including artefact information (centre of model), people, projects, operation and maintenance. Fig. 1Open in figure viewerPowerPoint Model of SW-Context These dimensions were perceived as important when examining the related works (Section 2) and the studies proposed by Antunes and Gomes [3 ] and Petersen and Wohlin [24 ]. These dimensions encompass information that characterises the work environment of software developers. When seamlessly integrated, as shown in Fig. 1, they produce context information. Each entity that composes the SW-Context model is described below: Artefact: It identifies the main entity of the model. The classes, methods, includes, and other source code artefacts are represented by this entity; InteractionContext: It represents the context information of the artefact in a timestamp; ChangeLog: It contains the identification of artefact's changes (versioning) and the description of changes; Project: The change requests of source code can have different start points like projects and are represented by this entity; Issue: This entity represents the customer issues that are also origins from software's change requests; Developer: It identifies the person who performed the changes in the source code; DesignPatterns: Entity that represents the design patterns implemented in the source code; MaintainabilityParams: Entity that represents the maintainability metrics of source code, such as lines of code, unit test coverage, cyclomatic complexity and others depending on the technology of an artefact; CustomerUsage: The customers who are using the features implemented by the artefact are covered by this entity; ExecutionTimes: It is the execution time of the artefact in environments of software production. 3.2 SW-Context architecture Fig. 2 presents a layer-based architecture of the SW-Context. It illustrates the SW-Context's design and how its modules relate to each other to store, evaluate and retrieve the context data. Each module is described as follows: Input data services: This module (item 1) consists of the layer of data input services, which allows the reception of different types of context information. SW-Context can get as input five different sources of information (External Data), including artefact data, project data, operation data, issues from incident management and developer data. Fig. 2Open in figure viewerPowerPoint SW-Context architecture Context module: This module (item 2) stores and manages the external data supplied by the input data services. Moreover, it supports the contextual analysis of the collected data by evaluating the data changes and performing the actions parametrised in the Configuration module. Configuration module : This module (item 3) allows the parameterisation of software metrics, and the association of actions related to these parameters. This module allows the architect to specify values of threshold that specific metrics can achieve. If a specified threshold is reached, the developer will receive alerts to draw attention to some strategic information. For example, exceeding the threshold value of a metric could indicate the existence of some disregard for architectural rules. Storage module: This module (item 4) stores context history (received by the Context module ) and SW-Context settings (received by the Configuration module ). This information is stored in the databases named Context history and Config (Item 4). This module supports three other modules (Context module, Qualitative analysis module and Configuration module ) as a persistence layer. It means that the settings entered into the system by the architect are stored and made available to the other modules through this module. The development teams can represent their design decisions and metrics' limit values to guarantee that the development practices are being implemented. For example, the team may define that the unit test coverage for classes must be at least 75% or the maximum lines of code must be 50 rows by method. From these limits, the development team can monitor the evolution of the application architecture, and take actions as early as possible, when the metrics trend is close to the set values. The development teams feed the model with the threshold values for some primary metrics (e.g. cyclomatic complexity, lines of code, number of dependencies, among others, but not limited to them) of an artefact. They can create heuristics based on the composition of such primary metrics having specifics project decisions represented in the SW-Context model. The types of metric compositions provided by the model are as follows: o Simple composition : The primary metrics can be aggregated into a new composition where the values of these metrics are compared to the limits; o Proportional composition : This composition allows the aggregation of two primary metrics in a division operation allowing a more detailed view on global measures, e.g. lines of code divided by the number of methods; o Weighted composition : The model allows the multiplication of the primary metrics by a calculation factor and subsequent multiplication of all recalculated metrics. Both primary and composed metrics can be parametrised against a variation index and periodicity. This feature allows the monitoring of a possible architectural degradation of an application over time, enabling the proactive activities of developers. Based on these types of composition, the architects and developers would define heuristics that represent their daily challenges by aggregating multiple metrics. For example, the development team can define the bigger the number of dependencies of the artefact, the bigger the unit test coverage. The primary metrics and compositions can be associated to a variation index. This feature allows the proactive monitoring of a possible architectural degradation of an application. Table 2 shows some configuration examples of the variation index where t