# Questionnaire Model for Paraconsistent Quality Assessment of Software Developed in SalesForce

Luiz Roberto Forçan, Jair Minoro Abe, Luiz Lima, Samira Sestari Nascimento

► To cite this version:

Luiz Roberto Forçan, Jair Minoro Abe, Luiz Lima, Samira Sestari Nascimento. Questionnaire Model for Paraconsistent Quality Assessment of Software Developed in SalesForce. IFIP International Conference on Advances in Production Management Systems (APMS), Aug 2020, Novi Sad, Serbia. pp.333-340, 10.1007/978-3-030-57993-7_38. hal-03630859

HAL Id: hal-03630859

https://inria.hal.science/hal-03630859

Submitted on 5 Apr 2022

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Questionnaire Model for Paraconsistent Quality
Assessment of Software Developed in Salesforce

Luiz Roberto Forçan✉[0000-0001-7376-830X], Jair Minoro Abe[0000-0003-2088-9065],
Luiz Antônio de Lima[0000-0003-4228-2387], Samira Sestari Nascimento[0000-0001-7429-6033]

Paulista University, São Paulo, Brazil
luforcan@gmail.com.br, jairabe@uol.com.br,
luiz@wcisp.com.br, samirasestari@gmail.com

**Abstract.** The article suggests the use of the Paraconsistent Decision Method (MPD) to improve the analysis of data captured in a standardized SUS - System Usability Scale questionnaire. The paraconsistent evaluation allows you to measure the usability quality of the provider registration software developed on the Salesforce platform. The data obtained through the questionnaire must be processed to be submitted to the Para-analyzer algorithm of The Paraconsistent Annotated Logic Eτ - Logic Et. Logic Eτ is used as a non-classical logic that allows analyzing the opinions of users considering their uncertainties, inaccuracies, ambiguities, and subjectivities that are inherent to human values. Through the Para-analyzer algorithm, we intend to obtain a consensus of the opinions of the experts on the usability of the software. The model with Logic Eτ allows being used in addition to the statistical treatment provided by the SUS method, improving the analysis of the data. With the result of this analysis, it is intended to diagnose usability problems contributing to the improvement of software development

**Keywords:** Software Usability Quality, Paraconsistent Logic Annotated Eτ, Cloud Computing, Salesforce.

# 1 Introduction

To be successful in this competitive market one vital point is the usability assessment. It is essential to measure the degree of user satisfaction that uses the software product developed in the Salesforce tool in your daily life.

There are several approaches to assess the usability of the software: the System Usability Scale (SUS) questionnaire is widely used for usability testing, reliability, learning capacity, appearance, and other aspects. A newer scale for measuring usability is the usability metric for user experience with few items that determine reliability and usefulness.

For the present study, we will use the SUS questionnaire format, and the main advantages of using these instruments for distance evaluation are objectivity in the collection of information, replicability of the instrument in other studies and quantification of results from the participants' responses. [11].
Logic $Eτ$ because it is a non-classical logic, allows the analysis of the subjective opinions of experts considering their uncertainties, inaccuracies, ambiguities, and subjectivities that are inherent to human values. [2] Other approaches allow you to do this analysis, such as the AHP Analytic Hierarchy Process. It is observed that the AHP method, although with peculiarities that aim to solve these possible problems, is based on the classical methods of decision-making, which consider the human opinions accurate and well defined (false and true). [7] [14].

Considering that it is proposed of the ISO/IEC 25010 standard [12] complements the study as qualitative attributes distributed in eight main characteristics: Functional adequacy, Performance efficiency, compatibility, usability, reliability, safety, maintenance, and portability. The standard recommends the qualitative feature of usability, which is divided into six sub characteristics: Accessibility, Apprehension, Aesthetics of the user interface, Operability, Protection against user errors, and Appropriate Recognition.

This article presents the results of a survey, applying the SUS questionnaire, to the specialists who use the software registration of providers, measuring the favorable and contrary evidence for each question of the questionnaire.

The research presents a model to assist in the decision-making process of evaluating the data collected on usability using Logic $Eτ$, through the Para-analyzer algorithm, using logical criteria that enable technical validation, and the input parameters are established by the opinions of the experts, consolidating a collective logic of these opinions, converted into mathematical terms.

# 2 Literature Revision

## 2.1 SalesForce

Salesforce (SF), which stands for "SalesForce" was founded in 1999 in California, USA by Marc Benioff and Parker Harris, and its main predominant point is the customer relationship management or Customer Relationship Management (CRM) system. It proposes to offer a platform in the clouds where the entire computational resource is located. [9] The SF platform has become attractive with the relationship management solution due to outsourcing CRM on the Internet, providing the CRM Web Service where it offers customizable, easy-to-implement software that easily engages the user, ensuring data integrity and security. [15]

## 2.2 Software Usability

According to the iso/IEC 25010 usability standard is the ability of the software to be easily operated, understood, easy to learn, and with an attractive user interface. ISO/IEC25010 [12] defines the following usability characteristics of software product quality: Accessibility, Apprehension, User Interface Aesthetics, Operability, User Error Protection, and Intelligibility or appropriate recognition.
To evaluate the usability of software, questionnaires such as SUMI, SUS, WAMMI, SUPR-Q, CEG, and NPS are used. With these questionnaires, usability, reliability, and
learning capacity and appearance tests are performed in software. For this article, we opted for the System Usability Scale -SUS questionnaire because it is the most popular choice for evaluating software usability.[11]

## 2.3 System Usability Scale (SUS)

The Usability Scale System - SUS [8] questionnaire is used to assess the level of usability of the system. The SUS scale consists of 10 specific questions to allow an analysis of the usability and ease of learning characteristics of a system. Each question has five answer options that follow the 5-point Likert scale: from 1 (I totally disagree) to 5 (I totally agree) where 3 means neutrality. This instrument allows objectivity in the collection of information and evaluation of the results of the experts' answers, through statistical calculations.[6] [11]

## 2.4 Analytical Hierarchical Process (AHP)

The Analytical Hierarchical Process (AHP) method is used to assist in decision making, evaluating multiple objectives and criteria in problems characterized by complexity and subjectivity. Alternatives are evaluated in the face of a complex decision problem. [16]

The AHP assumes that the decision-makers can provide accurate answers by comparing criteria and alternatives. However, due to the uncertainty, incompleteness, and subjectivity of the information, it is difficult to provide accurate judgments.[7] [14]

## 2.5 Paraconsistent Annotated Evidential Logic E<sub>τ</sub>

Paraconsistent logic is a category on non-classical logics that the Contradiction (or Non-contradiction) Principle is not valid in general. So, in this type of logic, theories based on it, there are propositions p and ¬p (the negation of p) both true [1]. One category of paraconsistent logic, namely the paraconsistent annotated evidential logic E<sub>τ</sub>, besides being paraconsistent, it is also paracomplete. Such logic is capable of handling inconsistent, imprecise, and paracomplete data.

In everyday life, the data from various sources can be contradictory, making room for uncertainties, resulting in contradictions that prevent decision-making.

In the Paraconsistent annotated evidential Logic E<sub>τ</sub>, each Proposition P associates a Favorable Degree of Evidence (μ) and a Contrary Degree of Evidence (λ). The Degrees of Evidence are real values between 0 and 1 that denote, in the case of the Favorable Degree of Evidence, the evidence that proposition P is true, and in the case of the Contrary Evidence Degree denotes contrary when proposition P is not true.

It is called Degree of Uncertainty Gin (μ, λ) of an annotation (μ, λ) to any of the degrees of inconsistency or paracompleteness.

It is called Gce Degree of Certainty (μ, λ) of an annotation (μ, λ) to any of the degrees of truth or falsehood. Depending on the Values of the Degrees of Evidence, four extreme states can be: true, false, inconsistent, and paracomplete as shown in Figure 2.
The Para-analyzer algorithm is composed of a set of information collected through a research questionnaire for decision-making analysis [5] The definition of the Para-consistent Decision Method (MPD), proposed in the studies, reflects the method, through Paraconsistent Logic, to use contradictory information, obtaining results that help in decision making. [1] [4] Figure represents the Unitary Square of the Cartesian Plan (QUPC), the degrees of certainty and contradiction, grouped into twelve states, which graphically presents the Para-analyzer algorithm

<table><caption>**Figure 1** - Unitary Square of the Cartesian Plane (Source: [1])</caption><thead><tr><th>Extreme states</th><th>Symbol</th><th>Non-extreme states</th><th>Symbol</th></tr></thead><tbody><tr><td>True</td><td>V</td><td>Quasi-true tending to Inconsistent</td><td>QV→T</td></tr><tr><td>False</td><td>F</td><td>Quasi-true tending to Paracomplete</td><td>QV→⊥</td></tr><tr><td>Inconsistent</td><td>T</td><td>Quasi-false tending to Inconsistent</td><td>QF→T</td></tr><tr><td>Paracomplete</td><td>⊥</td><td>Quasi-false tending to Paracomplete</td><td>QF→⊥</td></tr><tr><td></td><td></td><td>Quasi-inconsistent tending to True</td><td>QT→V</td></tr><tr><td></td><td></td><td>Quasi-inconsistent tending to False</td><td>QT→F</td></tr><tr><td></td><td></td><td>Quasi-paracomplete tending to True</td><td>Q⊥→V</td></tr><tr><td></td><td></td><td>Quasi-paracomplete tending to False</td><td>Q⊥→F</td></tr></tbody></table>

It is observed, then, that with the Para-analyzer algorithm, in addition to the four extreme states, it is possible to determine eight more non-extreme states. The Figure 1 shows the states that represent the non-extreme states in the para-parser algorithm. The Para-analyzer algorithm works with four external control values defined in the application environment, represented in Figure 3: $V_{cve}$ - Veracity control value, $0 \le V_{cve} \le 1$. $V_{cfa}$ - Falsehood control value, $-1 \le V_{cfa} \le 0$. $V_{cic}$ - Inconsistency Control Value, $0 \le V_{cic} \le 1$ e $V_{cpa}$ - Paracompleteza control value, $-1 \le V_{cpa} \le 0$

<figure><img src="image_2.png" /><figcaption>**Figure 2** - Lattice $\tau$ (Source: [2])</figcaption></figure>

<caption>**Figure 3** - Control Values (Source: [3])</caption>
## 3 Methodology

This research intends to evaluate the usability through the data collected through the AHP questionnaire. The users' responses will be submitted to the algorithm para-analyzer, the results of which will be published in an upcoming article after application and analysis of the results. The objective of this research is exploratory because it is intended to build knowledge about the application of Logic Et in the evaluation of usability, through the process of presenting them to a group of specialists who work with the software registration of providers, is based on the research as a survey.

The target audience of interest in this research is the professionals in the area of registration of providers who work with the software in their daily lives, hereinafter called specialists. Participants are not identified with personal data in this research. The questionnaire will be forwarded to the selected specialists via e-mail and telephone contact for the acceptance of participation.

Nine specialists working in a health care company in the city of São Paulo were selected, divided into three groups, adopting the position as a grouping criterion, according to Table 1.

Like Logic Et, it works with values between 0 and 1. The experts' answers will be mapped to logic Et's values. The value 0 is assigned to the answer "I totally disagree" and the value 1 to the answer "I totally agree", where 0.5 means "Indifferent" and so on, according to Table 2.

<table><caption>**Table 1** - Classification of Expert Groups</caption><thead><tr><th>Group</th><th>Occupation</th><th>Number of Interviewees</th></tr></thead><tbody><tr><td>A</td><td>Coordinators</td><td>3</td></tr><tr><td>B</td><td>Senior Registry Analysts</td><td>3</td></tr><tr><td>C</td><td>Junior Registry Analysts</td><td>3</td></tr></tbody></table>

Source: Authors

<table><caption>**Table 2** - Conversion of values</caption><thead><tr><th>Answer</th><th>Score</th><th>Value Logic Et</th></tr></thead><tbody><tr><td>I totally agree</td><td>5</td><td>1</td></tr><tr><td>Agree</td><td>4</td><td>0,75</td></tr><tr><td>Indifferent</td><td>3</td><td>0,5</td></tr><tr><td>Disagree</td><td>2</td><td>0,25</td></tr><tr><td>I totally disagree</td><td>1</td><td>0</td></tr></tbody></table>

Source: Authors

The research instrument was composed of ten closed questions based on the SUS questionnaire, with a Likert scale [10][13] according to Table 3. The questions for usability evaluation were mapped to the factors of Logic Et, according to Table 3.

With the result of the questionnaire, we obtain the agreement or degree of favorable evidence (μ) and the disagreement or evidence contradicts (λ) for each opinion of a specialist regarding the usability of the software. With the data collected from the groups of specialists, through the questionnaire, it is intended to feed a database to be
submitted to the Algorithm Para-analyzer of Logic Eτ, according to the model in Table 4.

As an example of an application of Logic Eτ's MPD method, the following simulation is produced, considering that the 9 experts, who are called Expert 1, Expert 2, ..., Expert 9 in the database, answered two questions: Factor 1 (F1) - I think I would like to use this application frequently and F2 - I found this application unnecessarily complicated. Data entries are obtained for the Algorithm Para-analyzer, according to the database shown in Table 4.

By applying the Para-analyzer algorithm, developed in an Excel spreadsheet, the result is obtained according to Figures 4 and 5. The result of the Global Analysis of the Para-Analyzer algorithm (0.87; 0.25) indicates that it is in the "Totally True" region resulting in a viable result. The Para-analyzer algorithm processed a consensus in the opinion of the nine experts, presenting the following results for the factors: F1 (0.75; 0.25) and F2 (1.00; 0.25). It is understood that the usability of the software is within a viable quality standard.

<table><caption>**Table 3 - SUS Questionnaire**</caption><thead><tr><th>Factor - Question</th><th>Factor - Question</th></tr></thead><tbody><tr><td>F1 - I think that I would like to use this system frequently.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td><td>F6 - I thought there was too much inconsistency in this system.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td></tr><tr><td>F2 - I found the system unnecessarily complex.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td><td>F7 - I felt very confident using the system.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td></tr><tr><td>F3 - I thought the system was easy to use.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td><td>F8 - I found the system very complicated to use.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td></tr><tr><td>F4 - I think that I would need the support of a technical person to be able to use this system.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td><td>F9 - I would imagine that most people would learn to use this system very quickly.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td></tr><tr><td>F5 - I found the various functions in this system were well integrated.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td><td>F10 - I needed to learn a lot of things before I could get going with this system.<br/>( ) I totally disagree ( ) Disagree ( ) Indifferent<br/>( ) Agree ( ) I totally agree</td></tr></tbody></table>

Source: Adapted from [8]

<table><caption>**Table 4 - Database Model formed μ and λ assigned by the experts for each factor of SUS questionnaire**</caption><thead><tr><th rowspan="3">Factor</th><th colspan="6">Group A</th><th colspan="6">Group B</th><th colspan="6">Group C</th></tr><tr><th colspan="2">Expert 1</th><th colspan="2">Expert 2</th><th colspan="2">Expert 3</th><th colspan="2">Expert 4</th><th colspan="2">Expert 5</th><th colspan="2">Expert 6</th><th colspan="2">Expert 7</th><th colspan="2">Expert 8</th><th colspan="2">Expert 9</th></tr><tr><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th><th>μ</th><th>λ</th></tr></thead><tbody><tr><td>F1</td><td>0,75</td><td>0,25</td><td>1,00</td><td>0,25</td><td>0,75</td><td>0,25</td><td>0,50</td><td>0,25</td><td>0,75</td><td>0,25</td><td>1,00</td><td>0,25</td><td>0,75</td><td>0,25</td><td>0,50</td><td>0,25</td><td>0,75</td><td>0,75</td></tr><tr><td>F2</td><td>1,00</td><td>0,25</td><td>0,75</td><td>0,25</td><td>1,00</td><td>0,25</td><td>0,75</td><td>0,25</td><td>1,00</td><td>0,00</td><td>1,00</td><td>0,25</td><td>0,75</td><td>0,00</td><td>0,75</td><td>0,25</td><td>1,00</td><td>1,00</td></tr></tbody></table>

Source: Authors
<table><caption>**Figure 4** - Final result of the questionnaire analysis (Source: Authors)</caption><thead><tr><th rowspan="2">Factor</th><th colspan="2">Resulting degrees</th><th rowspan="2">Decision</th></tr><tr><th>μ</th><th>λ</th></tr></thead><tbody><tr><td>F1</td><td>0,75</td><td>0,25</td><td>Not conclusive</td></tr><tr><td>F2</td><td>1,00</td><td>0,25</td><td>Viable</td></tr><tr><td>Global analysis</td><td>0,87</td><td>0,25</td><td>Viable</td></tr></tbody></table>

<figure><img src="image_3.png" /><figcaption>**Figure 5** - Overall analysis of questionnaires (Source: Authors)</figcaption></figure>

# 4 Conclusions

Current usability assessment processes do not take into account inconsistency or contradiction. In a real situation, both contradiction and inconsistency appear due to the conditions of the environment in which the questionnaire was applied.

Therefore, the existence of conflicts is part of the usability evaluation process. The greater the involvement of specialists with the software to solve their work processes daily, the higher the levels of conflicts, contradictions, and inconsistencies to which they will be subject.

The research presents a paraconsistent model to assist in the decision-making process of usability evaluation using Logic Et, through the Para-analyzer algorithm, using logical criteria that allow a technical validation, and the input parameters are established by the opinions of the experts consolidating the input values into a collective logic of all, converted into mathematical terms.

The research presents a paraconsistent model based on the perspectives that provide satisfaction to the specialists, assisting in the decision-making process of usability evaluation using logic Et. That can be used to validate and present a new technique of dealing with expert opinions in situations of blurring or even inconsistency in the use of the software. Possible future situations are relevant for decision making that ensure the quality of usability assessment and the success of software. Both the analysis and the result of this evaluation will be presented in a future article.
**5 References**

1. Abe, J. M.; Da Silva Filho, J. I.; Celestino, U.; Corrêa, de A. H. Lógica Paraconsistente Anotada Evidential Et. Santos, Editora Comunicar, 2011.

2. Abe, J. M. (2010). Paraconsistent logics and applications. 4th International Workshop on Soft Computing Applications, (pp. 11-18).

3. Abe, J. M., Akama, S., & Nakamatsu, K. (2015). Introduction to Annotated Logics: Foundations for Paracomplete and Paraconsistent Reasoning. Cham: Springer International Publishing.

4. Abe, J. M., Paraconsistent Intelligent-Based Systems - New Trends in the Applications of Paraconsistency. In: Springer (Ed.), Book Series: Intelligent Systems Reference Library (Vol. 94, p. 306). Switzerland: Springer.

5. Akama, S. (Ed.). (2016). Towards Paraconsistent Engineering (Vol. 110). Springer International Publishing.

6. ALAMER, Reem A et al. L3MS: A Lightweight Language Learning Management System using Mobile Web Technologies. p. 1–2, 2015.

7. Bolturk, E., Kahraman, C. A novel interval-valued neutrosophic AHP with cosine similarity measure. Soft Comput 22, 4941–4958 (2018). https://doi.org/10.1007/s00500-018-3140-y

8. Brooke, J. (1996). SUS: A ‘quick and dirty’ usability scale. In P. W. Jordan, B. Thomas, B. A. Weerdmeester, & I. L. McClelland (Eds.), Usability evaluation in industry (pp. 189–194). London, UK: Taylor and Francis.

9. BUYYA, Rajkumar; BROBERG, James; GOSCINSKI, Andrzej (org.). Cloud Computing: principles and paradigms. Hoboken, New Jersey: John Wiley & Sons, 2011.

10. Derham, P. A. J. (2011). Using preferred, understood, or effective scales? How to scale presentations effect online survey data collection. Australasian Journal of Marketing & Social Research, 19(2), 13-26

11. Jeff Sauro, James R. Lewis. Quantifying the User Experience (Second Edition), 2016. ISBN 978-0-12-802308-2. 2016 Elsevier

12. ISO/IEC 25010:2011. Systems and software engineering — Systems and software Quality Requirements and Evaluation (SQuaRE) — System and software quality models. https://www.iso.org/standard/35733.html. Acesso em 25/05/2020.

13. Likert, R. (1932). A technique for the measurement of attitudes. Archives of Psychology, 22(140), 1-55.

14. Mahdi Saeedpoor, Amin Vafadarnikjoo Corrigendum to "Multicriteria renewable energy planning using an integrated fuzzy VIKOR & AHP methodology: The case of Istanbul" [Energy 35(6) (2010) 2517-2527] Energy, Volume 79, 1 January 2015, Pages 536-537

15. Manchar, Anuradha & Chouhan, Ankit. (2017). Salesforce CRM: A new way of managing customer relationships in a cloud environment. 1-4. 10.1109/ICECCT.2017.8117887.

16. Vaidya, Omkarprasad & Kumar, Sushil. (2006). Analytic Hierarchy Process: An Overview of Applications. European Journal of Operational Research. 169. 1-29. 10.1016/j.ejor.2004.04.028.