# SIERA: Improving Data Visualization for Learning Assessment in Educational Organisations

Manuel J. Ibarra, Cristhian Serrano, Juan C. Muñoz

► To cite this version:

Manuel J. Ibarra, Cristhian Serrano, Juan C. Muñoz. SIERA: Improving Data Visualization for Learning Assessment in Educational Organisations. 17th International Conference on Informatics and Semiotics in Organisations (ICISO), Aug 2016, Campinas, Brazil. pp.191-196, 10.1007/978-3-319-42102-5_21. hal-01646568

HAL Id: hal-01646568

https://inria.hal.science/hal-01646568

Submitted on 23 Nov 2017

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# SIERA: Improving Data Visualization for Learning Assessment in Educational Organisations

Manuel J. Ibarra, Cristhian Serrano, and Juan C. Muñoz

School of Informatics and Systems Engineering, Micaela Bastidas National University of Apurimac, 121 Arenas Av., Apurimac, Peru
{manuelibarra, cristhiansj, juancarlosmunozmiranda}gmail.com

**Abstract.** Encouraging a culture of evaluating students’ knowledge, makes it possible to promptly discover weaknesses in the teaching and learning process. Once weaknesses have been identified then teachers can apply methodological strategies to reverse this situation and improve the student learning process. This paper describes a proposed strategy that supports the evaluation process. We propose to use an Excel file in which the evaluator fills in the answers provided by the student, then the system processes the data to rate the assessment for each student and generates statistical reports. The data is exported to a CSV file and then uploaded into a database server using the HTTP protocol. To validate this proposal we used focus group and daily meetings with teachers and students in evaluation process in Apurimac-Peru. The results highlight that using the tool makes it possible to have information everywhere and every time.

**Keywords:** learning assessment, educational organisations, data visualization, Excel VBA.

## 1 Introduction

To meet the *first commitment*, one of the *Eight Commitments School Management* given by the government [1], the school (IE: Institución Educativa in Spanish) organize and propose periodically assessments for students; it means, they implement an evaluation culture for students and propose corrective measures for poor academic performance in the evaluated areas. According to Gonzales et al. [2], an evaluation culture could be defined as the set of values, agreements, traditions, beliefs and thoughts that an educational community attaches to the action of evaluation. Bolseguí and Fuguet [3], point out that the evaluation culture is an evolving concept that refers to the need to evaluate on an ongoing basis; for them, the assessment is a complex and multidimensional process that includes different components: vision, values, behaviours, routines, organizational and social context, past and present experiences, epistemological, theoretical and methodological.

Organisational complexity is very high because the evaluation processes and functions require multi-systems collaborations, and has a high reliance on effective information communication among DRE (from Spanish Dirección Regional de Educación), UGEL (from Spanish Unidad de Gestión Educativa), IE, Director, Teachers
and therefore faces many challenges regarding pragmatic interoperability such as
information collision, policy obstacles, and procedure mismanagement.

Once evaluated, it is necessary to analyze and interpret the results, for which it is
important that the data is properly displayed; this topic has been widely discussed by
the authors in the area of data visualization [4, 5]. Most frequently, a key feature of
such an approach is showing relationships between different data groups of a provided
statistical selection: in order to compare relative proportions between various indicators
[6, 7].

This article presents a strategy to qualify student test primary and secondary level
in *Math* and *Communication* areas in Educational Institutions of Apurimac-Peru,
showing the obtained results by statistical charts sharing achievement levels for each
student, this information is useful to teachers who can use improvement strategies for
underperforming students. In addition, the tool also allows you to display statistical
graphs of the results by district, by level, by grade, by area; so that the Directors of
DRE, UGEL and IE can make decisions to improve student learning.

Section II of this article presents and discusses the work related to the subject of semiotics of educational organizations and data visualization; section III explains the design and implementation of the proposed strategy; section IV explains the evaluation methodology used to validate the proposed strategy; finally, section V describes the conclusions and future work of this research.

## 2 Related Works

*Semiotics* is the study of meaning of sign processes and meaningful communication. This includes the study of signs and sign processes (semiosis), analogy, metaphor, symbolism, signification, and communication. Within the organisational semiotics literature, three forms of activities are discussed, namely *substantive, communication* and *control* [8]. While much attention has been paid to substantive norms in business process modelling [9], relatively less research has focused on control norms, especially in virtual environments. Process controls are embedded constraints that serve as conditions for performing related activities [10].

Effah [9] publishes an article titled "Virtual Process Control Modelling in Organisational Semiotics: A Case of Higher Education Admission", this study explores Web-based virtual process control modelling based on organisational semiotics, Web modelling language (WebML), and higher education admission process. The study contributes to organisational semiotics research, which has so far focused more on substantive norm and less on control norm modelling.

On the other hand, *Data Visualization* is a general term that describes any effort to help people understand the significance of data by placing it in a visual context. Patterns, trends and correlations that might go undetected in text-based data can be exposed and recognized easier with data visualization software. There are techniques for facilitating data selection in the data transformation process [11, 12], techniques for selecting chart type and visual components (e.g., line style, point face, axis range) automatically in the visual mapping process [13, 14]; and techniques for changing visual effects to clarify the user's viewpoint and assertion easily [15] in the view transformation process.
Matsushita et al. [16] make a research titled “Interactive Visualization Method for Exploratory Data Analysis”. They propose an interactive visualization method suitable for exploratory data analysis. The method extracts parameters for drawing from a series of user requirements written in a natural language and redraws the drawn chart interactively according to the change in the user’s viewpoint. In this type of data analysis, statistical charts are employed to help the user understand the target data, and displayed chart is redrawn interactively according to the user’s input inspired by looking at the chart.

# 3 Design and Implementation

For this work we used the sign-based Semiotic Communication and the System Oriented Approach of Semiotic Organisation which includes Sign System Oriented Approach. Sign System Oriented Approach studies media (spoken language, texts, instruments, computer interfaces) as sign systems, and see the use of these media by people as based on systems of narration and interpretation. User interaction with media (texts, computer interfaces, instruments) is observed, as well as communication between people at work.

We also used a Visual Communication approach to represent the obtained results, which has various types of codes that help determine the message. The argument can be made that visual communication operates similar to a language sign system.

## 3.1 Educational Business Process Management for Assessment

The process starts when DRE proposes a schedule for evaluations, then Specialists develop indicators and questions for the test. Then each UGEL's Director, print the tests and distributes to each IE. Then The Director of each IE, assign evaluators (students) for each grade and section. The evaluation day, students take the exam, then teachers fill answers in Excel sheet to qualify each test using Excel Macro (just clicking a button). Then, Director and teachers of IE can see the graphical results generated by the system. Then, Excel file is very heavy in size (about 10 Mb), so we convert the data to a lightweight format file “.CSV” (about 5k), so that file is easier to transfer by Internet. Finally, the System processes updated data and generate new reports for users. The process is shown in Fig. 1.
<figure><img src="image_2.png" /><figcaption>Fig. 1. Business Process Management for assessment application</figcaption></figure>

## 3.2 Input and Output of Data Visualization

**Designing Excel File.** An Excel File was designed to fill the answers marked by students, as shown in Fig 2. Every question can be marked with "A", "B", "C", "X" (when students mark two or more answers) or "-" (blank, when a student did not mark any answer). Mathematics and Communication have 23 questions, every section has one to forty five students approximately, and in the primary level there are six grades named "First", "Second", "Third", "Forth", "Fifth" and "Sixth"; and in the secondary level there are five grades named "First", "Second", "Third", "Forth", "Fifth".

**Designing Reports.** After the teacher fill out the answers, the system use functions in Excel Macro to rate the assessment using values assigned in Table 1. An example of the Report is shown in Fig. 3.

<figure><img src="image_3.png" /><figcaption>Fig. 2. Sheet to fill answers</figcaption></figure>

<figure><img src="image_4.png" /><figcaption>Fig. 3. Report using Semaphore Indicators</figcaption></figure>

**Assigning Reached Level.** Each test has 23 questions for each area. To determine the level a student has reached, specialists produce distribution of scores, as shown in Table1. For example, if a student gets a score of 5, then, the level acquired is "Initiation"; if a student gets a score of 12, then, the level acquired is "In Process"; and if a student gets a score of 20, then, the level acquired is "Achieved".
<table><caption>**Table 1.** Distribution of reached level according to punctuation and colour assigned</caption><thead><tr><td>Item</td><td>Reached level</td><td>Punctuation</td><td>Description</td></tr></thead><tbody><tr><td>0</td><td>Initiation</td><td>0-8</td><td>Red</td></tr><tr><td>1</td><td>In Process</td><td>9-15</td><td>Orange</td></tr><tr><td>2</td><td>Achieved</td><td>16-23</td><td>Green</td></tr></tbody></table>

## 4 Evaluation of the Proposed Strategy

The validation of the tool was conducted with Education Specialist (workers) of the DRE-Apurimac through a focus group. The two events took place on: May 15 and September 14, 2015; in a meeting room of the Pedagogical Management Area of DRE. The participants were five specialists in evaluation. All participants had significant years of experience in monitoring and supervising educational schools.

Before starting the activity, the developed system was briefly shown to each specialist in evaluation. A simulation of the software functionality was then done. An operator, using the SIERA system, entered the “modular code” of the IE, then, the System generates the list of students of each grade and sections. The specialist fill out the answers for each student and then the system show the results graphically, then, he exports the result to CSV file and finally sends the result to the datacentre.

After this simulated process, the Education Specialists provided feedback, suggestions and opinions, and agreed that the system can definitely help improve decision making and perception (awareness) of the assessment results. When asked: “will the use of the designed software help you to make it easier the data visualization of evaluation assessment results?”, they all answered that in their opinion that assumption was valid. Then they were asked: “would the decision-making speed and quality of decisions have been better if the Director would have had a support tool that is visual and provides the appropriate suggestions?”, they all replied that a positive answer would be valid.

There were two evaluations until August 31 2015 using SIERA system; 614 schools and 38338 students were evaluated in the first assessment; 961 schools and 68075 students in the second assessment.

## 5 Conclusion and Future Work

Education Specialists agreed that having this type of information in the SIERA system would help to know the levels reached (Beginning, In Process, and Achieved) by students in every area, level, grade and district; also, Director could have accurate information when making decisions based on the learning achievement indicators. The time required to have results of the assessment is reduced significantly and the information is available at every time and everywhere whenever Internet connection is available. Likewise the participant Specialists agreed that, a priori, they can expect a significant improvement in the process of student learning and the quality of decisions, because Directors would have accurate information when making decisions.
For the future work, we need to define some key performance indicator to measure
the improvement speed and then compare current way of working with the improved
way of working. We hope to continue the experimentation process of using the other
Departments to improve the application.

**Acknowledgments.** Thanks to the Education Specialists of Pedagogical Management Area of the Regional Education Apurimac in Peru, especially Edith Montalvo, Alexander Serrano, Dámaso A. Sanchez and José O. Arohuillca.

References

1. http://www.minedu.gob.pe/campanias/pdf/manual-de-gestion-escolar-2015_10marzo_alta.pd (last visited 5 February 2016)

2. González, J.R., Soledad, M., Montoya, R., Rivera, J.A.: Cultura de evaluación en instituciones educativas. Perfiles Educativos 33(131), 42–63 (2011)

3. Bolsegui, M., A. Fuguet, A.: Cultura de evaluación: una aproximación conceptual. Investigación y Postgrado 21(1), 77–98 (2006)

4. Guchev, V., Massimo, M., Giuseppe S.: Design guidelines for correlated quantitative data visualizations. Proceedings of the International Working Conference on Advanced Visual Interfaces, ACM (2012)

5. Keim, D., Kohlhammer, J., Ellis, G., Mansmann, F.: Mastering the Information Age: Solving Problems with Visual Analytics. Eurographics Association (2010)

6. Spence, R.: Information Visualization: Design for Interaction. Pearson Educational Limited (2007)

7. Wilkinson, L.: The Grammar of Graphics. Springer Verlag New York, Second Edition (2005)

8. Liu, K.: Semiotics in information systems engineering. Cambridge Univ. Press (2000)

9. Effah, J., Liu, K.: Virtual Process Modelling Informed by Organisational Semiotics: A Case of Higher Education Admission. In: Proceedings of ICISO, IFIPAICT 426, 42–51 Springer, Heidelberg (2014)

10. Brambilla, M., Ceri, S., Fraternali, P., Manolescu, I.: Process modeling in Web applications. ACM Transactions on Software Engineering and Methodology 15(4), 360–409 (2006)

11. Ahlberg, C., Shneiderman, B.: Visual information seeking: Tight coupling of dynamic query filters with starfield displays. Proceedings of the SIGCHI conference on Human factors in computing systems, ACM (1994)

12. Derthick, M., Harrison, J., Moore, A., Roth, S.F.: Efficient multi-object dynamic query histograms. In: Proceedings of Information Visualization, IEEE Symposium, pp. 84–91 (1999)

13. Mackinlay, J.D.: Automating the design of graphical presentations of relational information. Readings in Intelligent User Interfaces (1998)

14. Fasciano, M., Lapalme G.: Postgrape: a system for the generation of statistical graphics and text. Proceedings of the Eighth International Workshop on Natural Language Generation (1996)

15. Mittal, V.O.: Visual prompts and graphical design: A framework for exploring the design space of 2-D charts and graphs. AAAI/IAAI (1997)

16. Matsushita, M., Kato, T.: Interactive visualization method for exploratory data analysis. Information Visualisation. In: Proceedings of The 5th International Conference, IEEE (2001)