# Collecting Qualitative Data During COVID-19

Vanessa Cesário, Valentina Nisi

► To cite this version:

Vanessa Cesário, Valentina Nisi. Collecting Qualitative Data During COVID-19. 18th IFIP Conference on Human-Computer Interaction (INTERACT), Aug 2021, Bari, Italy. pp.377-381, 10.1007/978-3-030-85607-6_41. hal-04291203

HAL Id: hal-04291203

https://inria.hal.science/hal-04291203v1

Submitted on 17 Nov 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons CC BY 4.0 - Attribution - International License
<figure><img src="image_2.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Collecting Qualitative Data During COVID-19

Vanessa Cesário¹[0000-0002-0938-9476] and Valentina Nisi¹[0000-0002-8051-3230]

¹ ITI/LARSyS, IST University of Lisbon, Lisbon, Portugal

vanessa.cesario@iti.larsys.pt

**Abstract.** The current pandemic situation leads researchers to reflect on conducting qualitative research, completely changing how they conduct participatory research. As it became clear that the pandemic would last many months, researchers started to redesign their planned research in digital spaces through social media channels and participatory online tools. From communicating with participants over Zoom (or other similar applications) to sharing information on exclusive online groups, digital platforms have become, for many, the only way to work, learn, or be entertained. This situation offered a significant opportunity to think creatively about research engagement and reflect on which aspects truly require researchers to be “on the ground” to conduct face-to-face participatory sessions to gather qualitative data. Qualitative researchers must use this opportunity to reflect while using digital tools for distance research. This paper is inspired by the work the authors are conducting in MEMEX – a European-funded project promoting social inclusion by developing collaborative storytelling tools related to cultural heritage and at the same time facilitating encounters and interactions between communities at risk of social exclusion. Thus, the work here presented reflects on the digital tools and techniques to collect qualitative data when the researchers cannot meet the participants face-to-face due to pandemics safety measures or other restrictions.

**Keywords:** COVID-19, Qualitative Data, Focus Groups, Collaboration, Digital Tools, Co-design

## 1 Introduction

While collecting quantitative data has been used online for several decades, collecting qualitative data has been challenging. Yet the global pandemic has oddly helped to push technologies to make qualitative data collection more accessible to a wide range of users: COVID-19 imposed lockdowns and social distancing, changing how researchers collect qualitative data during these uncertain and stressful times. Much qualitative research usually relies on face-to-face interaction: ethnographic types of interviews (structured, semi or unstructured), focus groups, and fieldwork are typical methods to gather qualitative data that depend on face-to-face interaction. Researchers usually plan to conduct fieldwork using traditional in-person methods, but now they are reconsidering it because participants cannot meet them face-to-face. These traditional face-to-face methods are being transformed into a “distance” method not to put projects on hold. Back in 2017, Braun and colleagues [1] created helpful guidance on what digital techniques have to offer, what types of research questions are best suited to be answered
through digital tools, as well as specific ethical questions that require consideration (practical, technical, and privacy challenges). While the ideal co-design face-to-face sessions may no longer be an option, the authors of this poster – researchers in the co-design field working in the MEMEX project – researched and reported techniques to collect qualitative data when the subjects cannot be face-to-face due to pandemics or other restrictions. This poster addresses some important questions and challenges regarding online methods for collecting qualitative data, raising more questions for future research.

## 2 Data Generation Techniques

Below we report on data generation techniques – focus group, co-creation tools, storing and analysing data, photo-taking, and visuals – inspired by the impossibility to run qualitative studies with participants from the MEMEX project. This section reads like a set of techniques that can be used in any qualitative research when participants have access to computers, internet access, and smartphones.

**Focus groups.** This is a standard technique in qualitative research because they are helpful to track initial reactions to the form of a concept or product [2, 3]. Focused groups can be moved online, with the aid of videoconferencing tools such as *Zoom*, *Teams*, *GoToMeeting*, *Google Hangouts* or virtual environments such as *Virbela* and *Mozilla*. Even before pandemics, *WhatsApp* and *Facebook* started to be used to conduct interviews and sometimes focused groups. Online or virtual focused groups have increased in popularity to capture ideas and opinions from a wider demographic group [4], allow greater accessibility for specific populations, and minimise costs and programming issues. Videoconferencing is a close substitute to face-to-face interviews. It can enable the data to be collected over large geographical areas even when social distancing measures are not in place [5]. As long as participants can access a computer with reliable internet, these tools allow individuals to talk to each other, see the moderator, and view a shared picture or document on the screen. Ideally, participants can access the session by clicking on a link without installing any software. These tools allow groups into private spaces (breakout rooms) where the moderator can enter and exit; however, they require careful management. As a helpful note, it is always handy to have a phone number to call participants if they struggle to enter or re-enter the main session. Researchers can use online polling tools to keep participants engaged at critical points (e.g. *Pollev*, *Mentimeter*). The polls can be used as a mechanism to keep users engaged, alongside collaboration tools, and gather accurate data for discussion.

**Co-creation tools.** In a certain way, digital tools such as *Miro*, *Mural*, *Padlet* replace the whiteboard or flipcharts used in face-to-face sessions. These tools can support videoconferencing and perform exercises with participants throughout the session. Also, the participant can group and collaborate on co-design sessions synchronously or asynchronously with a group of participants. Researchers can use these tools for collaborative brainstorm activities. Participants write down ideas as virtual post-it notes, keeping track of inspirations or solutions during the session by plotting post-it notes in a matrix or map to prioritise items. The researcher should send the participants a link to access a visual workspace where they can collaborate simultaneously.
**Storing and Analysing Data.** While it is an option in presential focused groups to record the session through video and audio, which allows a later checking of the non-verbal communication of the participants, it must be the norm in online ones. Researchers record online session to analyse the behaviours and the conversations of the participants. Recordings of videoconferences sessions provide more focused data than face-to-face sessions; the camera usually focuses on the upper part of the body, emphasising facial expression recording and gestures. These situations can provide a better understanding of how participants express and feel. Storing and analysing video data can require quite a lot of storage space and a reasonably powerful computer. Recording on smartphones quickly becomes problematic – even with the right app, storing and transferring recordings and file security quickly becomes a problem. However, if one is just concerned with the audio of the interview through a computer, one can do this with the native Voice Recorder app of the computer. Note that there are tools that automatically transcribe audio for us, but their accuracy is not always the best and is still not a substitute for professional transcription. As noted, there are many technical and practical challenges concerning videoconferencing and recordings.

**Photo-taking and visuals.** Pandemic constraints allow for a limited presence outside, wearing masks and strict rules about social distancing. These constraints affect photo-taking activities in various ways: (i) limited presence outdoors: depending on the country and pandemic gravity, people are not allowed to spend time outside their apartments, or limited during some hours of the day; (ii) the use of masks can influence expressivity of subjects or possibly their interest or ability in performing their routines; this can bias the observations collected through the photographic data; (iii) social distancing also affect behaviours of people and norms and their presence in public open and closed spaces. To make up for these shortcomings, participants should be allowed to use photos they have taken previously, by other people, or even allowed to find data on the internet. Participants should be encouraged to look through phone camera rolls, old photo albums to select photographs to discuss with others within the activity. However, it is not all about the digital. Methods such as drawings, paper diaries, collages, letters, cultural probes are not digital methods and can be used online. Asking participants to create such visuals or videos may help represent their feelings. The essential point is that the photographs or visuals represent participants’ thoughts, experiences, and feelings, being in line with the activity’s prompt.

## 3 Reflections

This poster questions the above set of tools for gathering qualitative data. We, as qualitative researchers, draw questions about trust and report, chat functionalities, technology training, and the digital divide. We believe these reflections will lead to a lively and productive discussion in the INTERACT community. It highlights challenges on the new possibilities for participation to inform the development of interactive technologies, raising more questions for future research on how remote research can have an enormous impact as time and technology progresses.
*Trust and rapport.* Gaining trust and rapport with participants is a crucial part of getting valuable data. It helps both the researcher and the participant think about each other's feelings, making it easier to feel a human connection. It is more challenging to create this trust and rapport online as we are missing many social cues. Also, it might be more difficult to break the ice over videoconferencing when participants and the moderator never met face-to-face. Researchers have less option for ice-breaking activities, and sometimes for technical reasons, participants do not have access to the camera, and the interview just relies on audio.

*Chat functionalities.* Interesting questions arise about the chat functionalities provided by such online tools. Does it enable participants to raise questions by allowing them time to reflect? Is it a space to share opinions that participants would not have shared in front of more experienced people? More studies are needed to answer these questions.

*Technology training.* Technology provides new possibilities for participation, disrupting, and shifting power. Participants must be aware of using the technology to be used in the online session. Researchers should support this by providing training on the technology, running a “test run” to ensure everyone is comfortable with the actual data collection time frame. Nonetheless, researchers should make the process as welcoming as possible, have ice breakers, and plan to share a cup of tea/coffee virtually!

*Digital divide.* The drawbacks of online tools are the influence of the digital divide and access to the internet and powerful devices. Do participants have access to these tools, and are they familiar with their use? Are all participants at the same tech-savvy level? Researchers should also be mindful of the participants and ensure that all the participants have equal access and opportunity to work with the requirements. If the aim is to prioritise the involvement of marginalised participants, qualitative researchers need to question what is really “rethinking participatory approaches” for the communities they work with. What lessons are we learning about digital participatory methods that can modify our research? Researchers should be highly cautious in celebrating the opportunities offered by the internet and digital devices because of the existing digital divide in some countries: marginalised groups that cannot have a voice due to their lack of availability and sophistication of broadband use. Infrastructural and socioeconomic disparities remain determined by geographic location, education, age, and income. The issue that participatory researchers face in embracing internet-powered participation is that it reinforces the gap between access to the internet and digital devices and deepens digital inequalities. Thus, it is crucial to make methodological negotiations in participatory approaches not to affect the core underpinnings of participatory methods and development ethics.

**Acknowledgements.** This project has received funding from the European Union's Horizon 2020 research and innovation programme under grant agreement No 870743 and the ARDITI's postdoctoral scholarship M1420-09-5369-FSE-000002.
References

1. Braun, V., Clarke, V., Gray, D. eds: Collecting Qualitative Data: A Practical Guide to Textual, Media and Virtual Techniques. Cambridge University Press, Cambridge, UK ; New York, NY (2017).

2. Cesário, V.: Guidelines for Combining Storytelling and Gamification: Which Features Would Teenagers Desire to Have a More Enjoyable Museum Experience? In: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems. p. SRC03. ACM, New York, NY, USA (2019). https://doi.org/10.1145/3290607.3308462.

3. Cesário, V., Matos, S., Radeta, M., Nisi, V.: Designing Interactive Technologies for Interpretive Exhibitions: Enabling Teen Participation Through User-Driven Innovation. In: Human-Computer Interaction - INTERACT 2017. pp. 232–241. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67744-6_16.

4. Del Carpio Ramos, H.A., Del Carpio Ramos, P.A., Tarrillo Carrasco, J.E., García-Peñalvo, F.J., Morante Dávila, M.A.: Online instrument: Perception of virtual learning of the doctorate in the context of COVID-19. In: Eighth International Conference on Technological Ecosystems for Enhancing Multiculturality. pp. 211–218. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3434780.3436603.

5. Wattenberg, T.L.: Online focus groups used as an accessible participatory research method. In: Proceedings of the 7th international ACM SIGACCESS conference on Computers and accessibility. pp. 180–181. Association for Computing Machinery, New York, NY, USA (2005). https://doi.org/10.1145/1090785.1090819.