# The key role of panel managers in enhancing panelist engagement

Sarah Blain, Justine Levy

► To cite this version:

Sarah Blain, Justine Levy. The key role of panel managers in enhancing panelist engagement. ESRA Conference 2025, European Survey Research Association, Jul 2025, Utrecht, Netherlands. hal-05269212

HAL Id: hal-05269212

https://sciencespo.hal.science/hal-05269212v1

Submitted on 19 Sep 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
# The key role of panel managers in enhancing panelist engagement

ESRA Conference - 16 July 2025

Sarah BLAIN & Justine LÉVY
# The CDSP, a reference data center for the humanities and social sciences

* The socio-political data center, a Support and Research Unit created in 2005 by Sciences Po and the CNRS

* Data production, preservation, documentation, and dissemination tasks

* Center of expertise for quantitative and qualitative methods related to major national and European research infrastructures

* Three main data production projects currently underway: ESS, CRONOS and ELIPSS
## Some background informations about the 2 panels

<table><thead><tr><th></th><th>EQIPSS<br>(French survey)</th><th>CRONOS<br>(European survey)</th></tr></thead><tbody><tr><td>Year launch</td><td>2012</td><td>2021</td></tr><tr><td>Collect mode</td><td>web only</td><td>web and paper</td></tr><tr><td>Number of respondents<br>at the time being</td><td>2 009<br>Refresh in progress<br>for 1 000 new panelists</td><td>1 287</td></tr><tr><td>Frequency of surveys</td><td>10 per year</td><td>5 per year</td></tr><tr><td>Participation rate (%)</td><td>more than 80%</td><td>around 60%</td></tr><tr><td>Employment status of panel<br>managers in CDSP</td><td colspan="2">Internalized but on short-term renewable contracts, due to<br>uncertainty regarding the continuation of projects</td></tr></tbody></table>
Two panels, two logics of engagement

<table><thead><tr><th></th><th>EIPSS<br>(French survey)</th><th>CRONOS<br>(European survey)</th></tr></thead><tbody><tr><td>Incentive monetary</td><td>Unconditional & occasional<br>(10€ twice a year)</td><td>Unconditional & regular<br>(5€ prepaid before each wave)</td></tr><tr><td>Reminder strategy</td><td>Batch reminders (emails) &<br>tailored reminders (postal mails)</td><td>Batch reminders (SMS, email)</td></tr><tr><td>Reminder phone calls</td><td colspan="2">Follow-up calls to specific categories of panelists<br>(with the help of student employees)</td></tr><tr><td>Feedback channel</td><td>Open-ended questions at end of<br>survey, mails and phone calls</td><td>Indirect via email replies<br>and phone calls</td></tr><tr><td>Engagement logic</td><td>Community-driven</td><td>Transactional: incentive as<br>participation lever</td></tr></tbody></table>
**Panel managers:**
beyond administrative tasks, a mediating role

* **Operational backbone:** coordinating survey campaigns, translating questionnaires (CRONOS), testing questionnaire, managing reminders to panelists and incentive distribution, tracking panel attrition, responding to technical inquiries

* **Relational gatekeepers:** personalized outreach to panelists, emotional labor (negotiating politeness, disappointment, confusion), handling complaints and making each panelist feel heard

* **Mediators:** from panelists to researchers (feedback, frustrations), from researchers to panelists (protocols, incentives, updates, newsletters)
Invisibles...
but essential in the data production chain

* In longitudinal and cross-sectional survey systems, panel managers remain largely **unnoticed in scholarly literature**. More broadly, it is the data collection phases and the work of its agents that have been little studied (Maurice, 2018).

* Their role activates a subtle but consequential relation of service, shaped by expectations, (un)civilities, and sometimes power struggles (Jeantet, 2003 and Krieger, 2018)

* Sociological studies on public statistics surveyors (Maurice, 2018) and polling institutes (Caveng, 2009), as well as on research technicians and engineers (Wacquet, 2022), shed light on the “little hands” and intermediaries in research, which share the common trait of being undervalued or invisible, yet indispensable
## Our perspective

* For the ELIPSS panel, we present the incentive protocol and the quarterly newsletter and explore explicit respondent feedback via emails exchanges and open-ended questions, through categorization of the topics covered

* For the CRONOS panel, we examine implicit feedback through email exchanges - moments where panelists make their expectations, frustrations, and relational norms visible

> We are not discussing phone reminders today, but they are an important part of the relationship with panelists
ELIPSS: A variety of tools to
enhance panelist engagement and
understanding respondents'
feedback
# Incentive protocol for ELIPSS panelists

* **Two unconditional incentive campaigns per year since 2021**
The response rate to the survey with which the incentive is sent is on average **7.7 points higher** than that of the previous survey

* **Dependence on incentives remains relative**, as panelists receive no gratification in 8 out of 10 surveys and are never informed in advance of incentive campaigns

* **Requests for gift vouchers remain very marginal** (1-5 requests per year), as do expressions of gratitude after receiving them, by email or in the free text field at the end of the questionnaire (10-15 messages in the two months following each incentive campaign)

* The protocol adopted for ELIPSS has struck a **balance between financial constraints, maintaining a high response rate, and moderating attrition**

<figure><img src="image_2.png" /></figure>
## Bonjour,

Toute négligeance nous reprendre en ce que chaleureusement pour votre engagement aquis d'ELIPSS, et tout particulièrement pour votre participation à l'encadrement d'élèves de panel et février Vos Conditions de Vie en 2025. Il est nous parmi d'activer les données sociodémographiques (âge, genre, revenu, de tous les participantes) et nous assurer que le panel est l'unique organisme de la société française de cette nouvelle édition de la newsletter, vous trouverez :

* le compte rendu d'une publication récente telle d'une enquête passée sur ELIPSS

* Retrait d'espérances avec une dictionnaire vous ayant soumis un questionnaire

* les premiers résultats d'une enquête qui vous a été proposé récemment

N'hésitez pas à nous faire parvenir vos retours, commentaires et suggestions, nous restons à votre disposition.

## Publication récente d'une enquête passée sur ELIPSS

En juin 2023, nous vous avons soumis un questionnaire pour demander aux personnes de la section de l'École des Lettres et de la Médecine de l'Université de Lille, laboratoire (ProTemps CNRS) à l'atelier de conférence à l'Université de Lille, laboratoire (CERAPS CNRS) et a récemment en décembre 2024 :

Les futurs montants que la population française est enceinte divise sur la question de Négligez la fertilité et des personnes LGBT+ : Mettez une acceptation croissance de Négligez de gain et de l'homosexuisme, les discrétionnés et les violences restent d'ores et non à l'abri.

## ELIPSS soutient les jeunes chercheurs !

<figure><img src="image_3.png" /></figure>

Depuis 2024, un appel à projets et local chaque année augmente des dictionnaires qu'il subsequently nous permet une questionaire afin d'apprécier les retours et also une aperçu d'avenir sur leur contexte, ce que soit leur université ou école de retraitement en France. Sur les deux derniers projets paru sur ELIPSS, ce qui est d'ailleurs, a été d'ajouter au ouragan de la réflexion et de l'effort deि les modèles de questions en juillet et septembre 2024. Nous vous proposons d'inscrire un projet sur Constance Fréthy, face des fondations de Térapot, 2024, outre de son projet de recherche et de ses premiers résultats d'equity.

## Premiers résultats des enquêtes pour lesquelles vous avez été sollicité(e)

Pour apprécier le focus sur Constance Fréthy, nous vous présentons les premiers résultats de nos enquêtes : "École à l'Université sur les collèges de France", diffusée en septembre 2024. Les questions suivantes vous demandent également :

Notes : l'importance des collèges suivant si vous devrez choisir un collège (pour vous enregistrer ou pour cesser des envois de vos prises et amés) ?

<figure><img src="image_4.png" /></figure>

L'envie : 37,25 % des répondantes et répondants sont des étudiants à l'université (plus de 300 % en envie de se présenter à un collège d'Etudes ou d'Etudes supérieures, mais pas d'actuel ou d'actives)

## Informations pratiques

Préférez-vous de tout changement de programme, c'est, par ailleur et par la suite, à l'uni versité celtique ?

PEEL ELIPSS - CERP - 37 rue du Blandin, 7077 PARS

Un site ELIPSS est à votre disposition, nous vous pourrions vous communiquer des informations : "les genres ELIPSS", les formes de la recherche, etc...

# Communication strategies for panelists on ELIPSS: the quarterly newsletter

## Newsletters in panel surveys

* A practice that is becoming widespread

* A form of "counter-gift"

* Transparency regarding the use of data who serves to **enhance the image of research**

* Reassure respondents who sometimes confuse scientific surveys, polls, and marketing studies

* A tool for building respondent loyalty

## ELIPSS newsletter structure

* A news from a past study (press or radio article, publication of a report)

* A focus on an ELIPSS profession or a portrait of a researcher

* The initial results from a recent survey
# Why do panelists contact us?

Focus on emails received from panelists (from may 2024 to june 2025)

Emails received; n=269

<figure><img src="image_5.png" /></figure>

Mobilization of interpersonal skills and adaptability to convince participants to stay in the panel or to assist them in responding
ELIPSS open-ending questions to close:
between criticism and useful comments

* Open question at the end of each questionnaire:
  Do you have any comments or suggestions regarding this
  questionnaire (technical difficulties, interest in the themes, etc.)?

* An analysis of the panelists' comments was initiated in 2022 by the last panel manager and presented at the previous ESRA conference (Elie, 2023)

* Complex and time-consuming work, but very interesting results: allows panelists to give their opinion (usually negative), overcome any frustration they may feel when answering closed questions, and suggest relevant changes for longitudinal surveys

<caption>Distribution of comments</caption>

(average 2023-2024)

<figure><img src="image_6.png" /></figure>
# ELIPSS open-ending questions for feedback

## Focus on negative comments from the annual survey on living conditions (2025)

### Comments on the substance; n=142

<figure><img src="image_7.png" /></figure>

- Lack of feedback on surveys 7,0%

- Questioning the validity of a question 5,6%

- Too much politics 4,2%

- Lack of interest or appetite for the topic 3,5%

- Absence of an important aspect of a subject 2,8%

- Wishes to leave the panel 0,7%
# ELIPSS open-ending questions for feedback

## Focus on negative comments from the annual survey on living conditions (2025)

Comments on the form; n=59

<figure><img src="image_8.png" /></figure>
Limitations and biases in the current treatment
of ELIPSS questionnaires' open-ended questions

* Manual categorization of responses creates interpretation biases, which can be perpetuated and amplified with the arrival of a new panel manager

* Until now, “neutral” responses such as “N/A” or “thank you” have been removed from the corpus of comments, **but they should be counted**

* Sociodemographic data are currently not integrated into open-ending comments data files, limiting cross-analysis. In accordance with our commitments regarding anonymity, panelists IDs are modified so that they cannot be traced back to their authors. The integration of sociodemographic data can therefore be considered.
# Prospects for improving the processing of panelists' comments on ELIPSS

* To standardize the processing of responses and limit analysis bias, **actual work with a student on automated categorization of comments using RStudio**

* At this stage, the script distinguishes between three categories: positive, neutral, and negative comments

* To go further, we plan to use a methodology combining Python scripts and AI, without ignoring the fact that textual analysis models are not perfect

* A cross-analysis of comments with socio-demographic data (geographical area, age group, gender, level of education, socio-professional category) **opens up more scientifically relevant perspectives**
# CRONOS-3: Analysis of the panelists relationship to the incentive through e-mail exchanges
Communication with panelists

Mails received between the 3rd and 5th wave, n = 96

<figure><img src="image_9.png" /></figure>

As you can see, this a work in progress. The sample being small, it is hard to draw real conclusions yet. The following is hypothetical.
# Incentive Management: A driver of panel engagement and a source of vulnerability

* **35 panelist emails** explicitly concern the incentive issue (non-reception, delay, suspected theft).

* Most respondents had **high participation rates** (4 or 5 waves out of 5).

* The tone remains mostly **polite and cooperative**, even when frustration is expressed.

The incentive is **not perceived as a bonus**, but as an **expected exchange** for participation.
Missing or delayed incentives trigger:

* Repeated complaints, sometimes over several waves

* Conjectures about postal reliability or technical bugs

* Emotional reactions ranging from regret to disengagement threats
Message typology by tone category

<figure><img src="image_10.png" /></figure>

* This tone diversity highlights the complexity of the incentive issue: it's not just transactional, it's relational.

* Incentive-related frustrations, if left unaddressed, jeopardize long-term collaboration and data continuity.
Age and gender trends in incentive requests

<table><thead><tr><td>Age Group</td><td>Implicit Requests</td><td>Explicit Requests</td><td>Key Insight</td></tr></thead><tbody><tr><td>Under 50 (n = 12)</td><td>5</td><td>7</td><td>More direct, task-focused communication</td></tr><tr><td>50 and older (n = 23)</td><td>13</td><td>10</td><td>Preference for softened tone and open-ended remarks</td></tr></tbody></table>

Female, 25: "I've answered each time. Only one cheque has arrived. Could you let me know if this is normal?" → Tone: Fact-based-pragmatic, direct inquiry

Male, 68: "I'm wondering whether the gift voucher policy has changed... just checking." → Tone: Suggestive-proactive, implied request
Age and Gender trends in incentive requests

<table><thead><tr><td>Tone Category</td><td>Female (n=20)</td><td>Male (n=15)</td><td>Trend</td></tr></thead><tbody><tr><td>Courteous-Cooperative</td><td>9</td><td>5</td><td>More frequent among women</td></tr><tr><td>Ambivalent-Frustrated</td><td>7</td><td>5</td><td>Expressed evenly across genders</td></tr><tr><td>Suggestive-Proactive</td><td>3</td><td>4</td><td>Slightly more common among men</td></tr><tr><td>Conditional-Assertive</td><td>1</td><td>1</td><td>Very rare, balanced</td></tr></tbody></table>
Question wording: format, meaning and trust

Among the 38 emails analyzed (waves 3 to 5), only 3 messages explicitly criticize the questionnaire's formulation or relevance. Yet these messages reveal deep-seated concerns about meaning, legitimacy, and transparency:

<table><thead><tr><td>Theme</td><td>Excerpt (translated)</td><td>Tone</td></tr></thead><tbody><tr><td>Contextual misalignment</td><td>“Because I'm retired, some questions didn't apply. I would have liked to complete my answers.”</td><td>Reflective-Constructive</td></tr><tr><td>Civic uncertainty</td><td>“Are our responses used to inform policy? Are these contributions actually reaching political actors?”</td><td>Civic-Skeptical</td></tr><tr><td>Ambiguity → perceived manipulation</td><td>“Your immigration question is vague; is it about my city, my region, my country? This feels misleading.”</td><td>Critical-Assertive</td></tr></tbody></table>
# A human relationship at the heart of a longitudinal survey

## Personalized and respectful communication

* 100% of emails analyzed include clear, respectful politeness formulas.

* Messages express trust and familiarity: panelists know who they're speaking to, and feel comfortable voicing concerns without hostility.

## A relational functioning

* The panelists don't treat surveys as institutional tasks, they view them as **reciprocal engagement** with a trusted referent.

* In several emails, respondents **thank** the manager personally or express loyalty to the survey effort, sometimes even downplaying the incentive itself.

* Tone categories show a strong presence of **courteous-cooperative and suggestive-proactive voices**, favoring dialogue over complaint.
# Overall conclusion

## From incentives to integrity

* In CRONOS, monetary incentives sustain participation - but their management exposes vulnerabilities if mishandled

* In ELIPSS, symbolic gestures (newsletters, open comments) encourages their engagement by offering them the opportunity to give their opinion on questionnaires and access survey results. Dependence on incentives remains relative

* Across both panels, what matters is not just what is given, but how it is given and how it is acknowledged

## Rethinking engagement

* Designing longitudinal surveys requires attention to relational norms, transparency, and participant voice

* Panel managers are not just technicians, **but active agents shaping the quality and sustainability of research**
Thank you for your attention

<figure><img src="image_11.png" /></figure>

CRONOS

sarah.blain@sciencespo.fr

<figure><img src="image_12.png" /></figure>

IPSS

justine.levy@sciencespo.fr
Bibliography

* Caveng, R. (2009) Inversement des positions et réenchantement de l'interaction. La relation d'enquête dans les sondages et les études de marchés. *Actes de la recherche en sciences sociales* 178: 88—98.

* Elie, M. Open-ending questions to close. *ESRA Conference 2023, European Survey Research Association, Jul 2023, Milan, Italy.* (hal-04392400)

* Jeantet, A. (2003). « À votre service ! » La relation de service comme rapport social. *Sociologie du Travail*, 45(2), 191-209. https://doi.org/10.4000/sdt.31416

* Krieger, U. (2018). *A penny for your thoughts - the use of cash incentives in face-to-face surveys*. https://madoc.bib.uni-mannheim.de/45606/

* Maurice, J. (2018). Enquêtrices enquêtées. L'ordre négocié du travail de collecte des enquêtrices de l'Insee. *Bulletin of Sociological Methodology/Bulletin de Méthodologie Sociologique*, 137-138(1), 157-175. https://doi.org/10.1177/0759106318761612 (Original work published 2018)

* Waquet, F. (2022). Dans les coulisses de la science : Techniciens, petites mains et autres travailleurs invisibles. CNRS éditions.