Acquisition of data for dissemination

Julie Baron, Arianna Caporali, Thomas Merly-Alpa, Julie Lenoir

► To cite this version:

Julie Baron, Arianna Caporali, Thomas Merly-Alpa, Julie Lenoir. Acquisition of data for dissemination. 2023, pp.12. hal-05097289

HAL Id: hal-05097289

https://hal.science/hal-05097289v1

Submitted on 4 Jun 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>
<figure><img src="image_2.png" /></figure>

PROCEDURES MAD

ACQUISITION
OF DATA FOR
DISSEMINATION

DOCUMENT ID: PRCMAD2

VERSION: 2 - 11/2023

CREATOR: BARON JULIE

APPROVED BY: MERLY-ALPA THOMAS, LENOIR JULIE

DATALAB - SERVICE DES ENQUÊTES ET DES SONDAGES -
QUETELET PROGEDO DIFFUSION

in ed

DataLab

QUETELET

PRO
GE
DO

DIFFUSION
# 1. Introduction

The French Institute for Demographic Studies (INED)<fnref n="1" /> is a research institute mandated by the Ministry of Higher Education and Research and the Ministry of Solidarity and Health to study the world's populations. INED researchers collect and analyse data to study all aspects of populations. Through an open approach to demography, INED researchers draw on a wide range of disciplines including economics, history, geography, sociology, anthropology, biology, and epidemiology, and conduct research on themes as varied as the trajectories of couples, migratory paths, fertility, contraception, family relations, employment, and employee-employer relations<fnref n="2" />.

In accordance with the principles of Open Science and the "Loi pour une République Numérique"<fnref n="3" />, one of INED's core missions is to make the data collected by INED researchers available to others. It is vital to allow secondary analysis of data, their verification, and the replicability of studies by sharing them with the scientific and academic community. This sharing of data makes knowledge accessible, encourages collaboration between researchers and new discoveries. It discourages scientific fraud and is useful for teaching analytical methods. In addition, it benefits data producers by increasing awareness of their work through citation, justifying high survey costs through data reuse, and allowing for the testing of collection methods<fnref n="4" />.

# 2. WHO and WHAT

INED researchers mostly conduct quantitative surveys. Their data collection is supported by INED's Surveys Department<fnref n="5" />. The Surveys Department works with the research team to define the different stages of the survey and their timing, to design the survey, and to develop the protocol, questionnaire and sampling plan; it provides tailor-made computer applications for data collection, keeps track of data collection and corrects the collected data<fnref n="6" />. Within this department is the INED DataLab<fnref n="7" />, a body specializing in data documentation and providing access to INED surveys. More generally, the DataLab is dedicated to the growth and visibility of the data available at INED, as well as to the dissemination of reliable and accurate data for research in demography and, more broadly, in social sciences.

The DataLab only makes available the quantitative surveys produced by INED researchers after a 3-year embargo<fnref n="8" /> to allow exclusive data exploitation by the research teams at Ined, before a broader release for other uses. More specifically, these surveys are those for which INED is the project manager or, in the case of a partnership, the co-project manager, and/or those for which INED is responsible to the CNIL<fnref n="9" />. These are the surveys for which the Institute has made a significant investment in their design and conduct, which is therefore a guarantee

<footnote n="1">¹ https://www.ined.fr/en/.</footnote>

<footnote n="2">² See Appendix 1: Classification of INED Surveys by topic.</footnote>

<footnote n="3">³ https://www.legifrance.gouv.fr/jorf/id/JORFTEXT000033202746</footnote>

<footnote n="4">⁴ Caporali Arianna, Morisset Amandine, Legleye Stéphane, "Providing Access to Quantitative Surveys for Social Research: The Example of INED", *Population*, 2015/3 (Vol. 70), p. 537-566. DOI: 10.3917/popu.1503.0567. URL: https://www.cairn-int.info/journal-population-2015-3-page-537.htm</footnote>

<footnote n="5">⁵ https://ses.site.ined.fr/en/</footnote>

<footnote n="6">⁶ See Appendix 3 of the document "Ined DataLab" (PRCMAD1).</footnote>

<footnote n="7">⁷ https://databab.site.ined.fr/en/</footnote>

<footnote n="8">⁸ https://www.ined.fr/fichier/rte/General/Institut/science-ouverte/Note-H-Leridon-2021-recommandations-mise-disposition-donnees-enquetes-science-sociale-sur-les-personnes.pdf</footnote>

<footnote>For an English translation of this document, See Appendix 2: Procedure for releasing survey data.</footnote>

<footnote n="9">⁹ https://www.cnil.fr/</footnote>
of data and metadata quality. In the case of partnered surveys, the precise terms and conditions for dissemination and handling by the DataLab are defined in the partnership agreement<fnref n="10" />.

It receives the data and the documentation associated with the surveys. It puts the data in the form of "Scientific Use Files"<fnref n="11" />, produces the survey metadata in DDI 2.5 format<fnref n="12" /> and disseminates the following files:

- Open access metadata via its survey catalogue Datalned<fnref n="13" /> and the Quetelet-Progedo-Diffusion catalogue Dataprogedo<fnref n="14" />.

- Restricted access data files and metadata, via the Quetelet-Progedo-Diffusion<fnref n="15" /> data warehouse. The DataLab also manages access requests.

INED does not have a data acquisition policy as such. As a producer and disseminator, it must make available all the surveys it produces and respond to requests made by researchers.

### 3. HOW - Data and Documentation acquisition procedure

At the beginning of the main data collection for a survey at Ined, the DataLab contacts the teams responsible for the survey to anticipate the data release. The objective is to document each ongoing survey as it progresses to avoid information loss. To achieve this, an initial discussion with the research teams establishes the modalities and the schedule for dissemination of the survey:

- Which documents will appear directly accessible on the catalog without access restrictions (no documents / only the questionnaire / multiple documents such as the code dictionary, survey presentation, etc.)?

- Whether the metadata documentation in DDI at the variable level will be directly available on the catalog or only the metadata without variable level.

Following this discussion, three guides and a form are then distributed to the research teams.

The form is a .docx document in which information about the survey (metadata), will be recorded. This form is formatted according to the DDI standard and is named after the identifier of the survey to be documented.

- The MAD01¹⁶ guide helps teams understand how to complete the form.

- The MAD02¹⁷ guide outlines the expected list and content of mandatory or recommended documents accompanying the future data file release.

<footnote n="10">¹⁰ See Appendix 3 "Excerpts from a standard partnership agreement, dissemination and survey documentation sections."</footnote>

<footnote n="11">¹¹ See the document "Survey Data Anonymisation and Pseudonymisation" (PRCMAD4).</footnote>

<footnote n="12">¹² See the document "L'application des principes FAIR pour la mise à disposition des enquêtes à l'Ined" (PRCMAD6).</footnote>

<footnote n="13">¹³ https://data.ined.fr/index.php/home</footnote>

<footnote n="14">¹⁴ https://data.progedo.fr/</footnote>

<footnote n="15">¹⁵ http://quetelet.progedo.fr/.</footnote>

<footnote n="16">¹⁶ Julie Baron, Julie Lenoir, Aurélie Santos (Collab.) et al., 2023, "Guides Mise à Disposition d'Enquête". n°MAD01, Aubervilliers : http://hdl.handle.net/20.500.12204/AYi60PA8Lg0aT10Rui9w</footnote>

<footnote n="17">¹⁷ Julie Baron, Julie Lenoir, Aurélie Santos et al., 2023, "Guides Mise à Disposition d'Enquête". n°MAD02, Aubervilliers : http://hdl.handle.net/20.500.12204/AYi6yvFjLg0aT10Rui9u</footnote>
* The MAD03¹⁸ guide describes best practices for formatting data files and tracing recodes in preparation for their future transmission to the DataLab.

Once these guides are distributed, the DataLab regularly reconnects with the teams to monitor the progress of the documentation steps throughout the production chain. Specifically, at the end of the collection, the engineers from the survey and sampling service responsible for sample selection are asked to complete information regarding this in the form. During the file cleansing phase, they are again asked to complete information regarding cleansing, response rates, and any produced weights, if applicable. Research teams and support engineers add information about the collection protocol and then about the survey's summary and funders during the exploitation of the results for communications and publications. The editions service teams responsible for collecting publications from the surveys create a dynamic bibliographic link linked to the survey via its identifier. In total, nearly 50 fields are completed through collaborative and continuous filling supervised by the DataLab.

At the end of the three-year embargo, the Data Availability Officer<fnref n="19" /> acquires, via a password-protected NextCloud link, the data, the form, and the previously prepared documentation. As files are shared, the Documentation and Data Availability Officer is notified by email that they can access a new folder, allowing them to retrieve the researcher's shared content.

To be delivered to DataLab, the folder must include at least:

1. The metadata form completed

2. Data and documents tracing the operations that have taken place on the database (modifications made, deletions of variables, recoding of variables etc.)

3. Documents containing information on the use of the data (weights, etc.)

4. Survey instruments (e.g., questionnaire, response cards)

5. Additional documentation (e.g. reports, articles, user guides, codebooks).

These data and documents are deposited in several file formats. The list of accepted formats and all documents to be delivered is public and made known to the research team<fnref n="20" /> via the 3 guides mentioned upstream.

Once the Data Availability Officer has retrieved the data and documentation files, they store them on a file server that is part of a shared network with a high-security access control system. This file server is hosted on a different network than from the one mentioned above. Access to this file server is also governed by filtering rules that complement the access control rules of the network share and is only accessible by the Data Availability and Documentation Manager. Documentation is completed to describe the status of the file<fnref n="21" />. Once data acquisition is complete, the work of preparing the data for the survey can continue.

<footnote n="18">¹⁸ Julie Baron, Julie Lenoir, Géraldine Charrance et al., 2023, "Guides Mise à Disposition d'Enquête ". n°MAD03, Aubervilliers : http://hdl.handle.net/20.500.12204/AYi6wVFwLg0aT10Rui9n</footnote>

<footnote n="19">¹⁹ See the document "Ined Datalab" (PRCMAD1).</footnote>

<footnote n="20">²⁰ See Appendix 4: Data and Documentation Acquisition - List of accepted formats.</footnote>

<footnote n="21">²¹ See Appendix 5: Documentation Acquisition - List of transmitted documents and Appendix 6: Data Acquisition - Information on transmitted databases.</footnote>
# Summary and description of Appendices

## Appendix 1 - Classification of INED Surveys by topic. P6

This document presents the classification of INED surveys by theme. These themes were created during the renovation of the INED website in 2014. A working group was formed to make proposals that were then discussed at a meeting of the heads of INED's Research Units, with an Editorial Committee. These themes aim to popularize INED's research themes and organize the website. When a survey is made available, it is categorized under one of INED's 8 topics (or under several topics) using DDI tag 2.2.1.2. A second classification using the CESSDA controlled vocabulary themes is also performed when archiving the documentation via the docx. form, using the same DDI tag 2.2.1.2. As such the survey is classified both internally at INED and at the European level using the CESSDA themes.

## Appendix 2 - Procedure for releasing survey data. P7

This diagram and the explanations it contains are taken from an internal INED charter called "Recommendations on making availability quantitative social science survey data on individuals (INED DEQP Charter), revised version of 9 April 2021 written by H. Leridon, Emeritus Researcher at INED and Member and Correspondent of the Academy of Sciences (Institut de France, Paris). This charter was validated by INED's Scientific Committee of Council on 22 April 2021 and is not yet public. It is intended to implement the texts and recommendations on the imperatives of "open science" relating to data, specifically data from quantitative social science surveys of individuals.

## Appendix 3 – EXCERPTS FROM A STANDARD PARTNERSHIP AGREEMENT - Dissemination and survey documentation sections. P8

This appendix transcribes excerpts from partnership agreements established by the legal department of Ined in collaboration with the Datalab, to specify the modalities of data file dissemination and metadata documentation produced during a survey;

## Appendix 4 – DATA AND DOCUMENTATION ACQUISITION - List of accepted formats. P9

This document presents all the formats accepted by the DataLab for the transmission of files. The research team that transmits the documentation and databases must ensure that they are in formats that can be manipulated by the DataLab with the technical means at its disposal (OfficeSuite, SAS software, R software, StatTransfer, Foxit Phantom software, Nesstar Publisher). This list is published on the DataLab website and communicated to the research team when it contacts the DataLab to prepare the provision of data.

## Appendix 4 – DOCUMENTATION ACQUISITION - List of transmitted documents. P10

This document is used to structure the exchange of files with the research team and to record the transmission of documents. Once the research team has deposited the files on the Cloud and the data and documentation availability manager has downloaded and stored them on their secure server, this document is completed to record the documentation acquired. It provides a log of this transfer of responsibility between the research team and the DataLab.
Appendix 5 – DATA ACQUISITION - Information on transmitted datasets. P11

Like the previous document, this document is used to structure the exchange of files with the research team and to record the transmission of databases. Once the research team has deposited the files on the Cloud and the data and documentation availability manager has then downloaded and stored them on their secure server, this document is completed to record the data files acquired. It provides a log of this transfer of responsibility between the research team and the DataLab.
# CLASSIFICATION OF INED SURVEYS BY TOPICS

## FR

1. Couples, famille, sexualité

- Formation des couples, Pacs, Mariage

- Séparation, Divorce

- Parentalité

- Sexualités

2. Naissance, fécondité, contraception

- Fécondité, nombre d'enfants

- PMA, Stérilité

- Contraception, IVG

- Politiques familiales

3. Espérance de vie, mortalité, santé

- Causes de décès

- Tables de mortalité

- Inégalités de santé

- Politiques de santé

4. Migrations internationales, discrimination, intégration

- Flux migratoires

- Populations immigrées

- Identité

5. Habitat, environnement, territoire

- Mobilité interne

- Sans-domiciles, mal-logés

- Ségrégations spatiales

6. Genre, inégalités hommes-femmes

- Emploi et genre

- Violences

- + thèmes transversaux : santé, famille, etc.

7. Ages de la vie, vieillissement

- Enfance, jeunesse

- Cycle de vie

- Retraite

- Fin de vie

8. Populations du monde

- Population mondiale

- Population par pays

- Transition démographique

## EN

1. Couples, family, sexuality

- Formation of couples, Pacs, Marriage

- Separation, Divorce

- Parenthood

- Sexuality

2. Birth, fertility, contraception

- Fertility, number of children

- IVF, Sterility

- Contraception, abortion

- Family policies

3. Life expectancy, mortality, health

- Causes of death

- Life tables

- Health inequalities

- Health policies

4. World migrations, discrimination, integration

- Migration flows

- Immigrant populations

- Identity

5. Housing, environment, territory

- Internal mobility

- Homelessness, poor housing

- Spatial segregation

6. Gender, gender inequalities

- Employment and gender

- Violence

- + cross-cutting themes: health, family, etc

7. The stages of the life cycle, ageing

- Childhood, youth

- Life cycle

- Retirement

- End of life

8. The world's populations

- World population

- Population by country

- Demographic transition
# PROCEDURE FOR OPENING UP SURVEY DATA

<figure><img src="image_3.png" /></figure>

In order to take into account the amount of work involved in the design of a survey, data collection, of data, and the control and preparation of the data as well as the responsibility of the "collectors" (researchers and ITAs) for the quality and comprehensibility of open access data, it is suggested that each research institution or university establishes rules within the following limits. The rules presented in this document are those chosen by INED.

1- The end of collection is defined as the collection of the last individual information from respondents in the normal collection process, regardless of any ancillary operations (such as checks) that may take place afterwards;

2- The collected survey data must be stored in a data warehouse along with all documentation required to use them. Unless there are exceptional difficulties, the database must be technically usable no later than two years after the end of collection. The storage of the database can be done under embargo, with use of the data reserved for the designers of the survey until the end of the embargo (see Point 4);

3- The period between the end of collection and the existence of a technically usable database must allow for the creation of files, the storage of data, the control of data, the construction of derived variables, the development of weightings, and the documentation of all the stages of data collection and processing;

4- Once the database is technically usable, it must be opened up no later than three years after the end of collection, except in exceptional circumstances; the duration of the embargo is set by the head of the institution within the limit indicated, after consultation with the survey managers. Opening up, which means making the data available to anyone requesting access, must be done in compliance with the rules of the General Data Protection Regulation (GDPR);

5- The embargo period must allow those responsible for the survey to prepare a first publication, which may be a "data paper", presenting the survey and to be cited by all users of the data, or an article including the first results;

6- The exceptional circumstances mentioned in Points 2 and 4 may result either from unforeseeable circumstances (for example, failure of the organization in charge of storage to meet deadlines), or from difficulties intrinsic to the type of collection or data involved (such as cohort or panel data requiring repeat data collection);

7- The list of researchers, research teams, and other persons protected by the embargo must be drawn up before the end of the data collection period; it may be extended with the agreement of the survey managers;

8- The same rules apply to subsequent waves of data collection and matching, or to the development of derived variables if they are entered into the initial database, which is strongly encouraged; in the latter case, the embargo period cannot exceed one year from the time the data were documented and entered into the database;

9- Any publication using the survey data, including during the embargo period, must specify where the data are stored and their conditions of access, and respect the rules of citation of the database used.

<footnote>*Except in exceptional circumstances</footnote>
# EXTRACT FROM A STANDARD PARTNERSHIP AGREEMENT

## Dissemination and survey documentation sections

### ARTICLE 5 - THE VARIOUS SURVEY FILES

Various files will be derived from the project, each adhering to different rules for dissemination and usage: (...)" the "production and research file",(...).

Any transfer of these files to third parties must comply with the current provisions of the aforementioned Law No. 51-711 of June 7, 1951.

### 5.3. Production and Research File (FPR)

The production and research file is an individual data file derived from the Survey, devoid of direct identification elements, and undergoes statistical processing to limit the risk of re-identification through variable recombination.

The creation of the production and research file will be carried out at Ined by DataLab of the Surveys and Polls Department, using the study file stripped of any information allowing direct identification of individuals.

The FPR will be finalized within three (3) years after the end of data collection.

The content of the FPR should be as comprehensive as possible without compromising anonymity.

### ARTICLE 9 - PUBLICATIONS – COMMUNICATIONS

#### 9.2. Modes of Distribution of Files and Data to Third Parties

##### 9.2.1 Provision of the Production and Research File

The production and research file and its contents will be disseminated to [...] after a period of [...].

The production and research file will be referenced and accessible through the Quetelet-Progedo
Diffusion platform.

##### 9.2.5 Provision of Survey Metadata

The survey metadata will be disseminated by [...] to [...] after a period of [...].

The survey metadata will be referenced and accessible through the Data Ined and Dataprogedo platforms.
# DATA AND DOCUMENTATION ACQUISITION

## List of accepted formats

<figure><img src="image_4.png" /></figure>

### DATA FILES:

* SAS (\*.sas, \*.sas7bdat)
  (Preferred)

* Comma Separated Values (CSV)
  file (\*.csv)

* STATA (\*.dta)

* SPSS (\*.sav)

### METADATA FILES:

* Adobe Portable Document
  Format PDF/A, PDF (\*.pdf)

* MS Word (\*.doc, \*.docx)

* OpenDocument Text (\*.odt)

* Rich Text Format (RTF) (\*.rtf)

* Hypertext Mark-up Language
  (XML) (\*.xml)

* PEG (jpeg, .jpg)

* JPEG 2000 (jp2, .jpux)

* PNG (\*.png)

* MS Excel (\*.xls, .xFORMATION)

* OpenDocument Spreadsheet
  (\*.ods)
# DOCUMENTATION ACQUISITION

## List of transmitted documents

<figure><img src="image_5.png" /></figure>

Research team:

Survey:

Survey ID:

Date:

Person in charge of data and metadata transmission:

<figure><img src="image_6.png" /></figure>

<footnote>*Documents in red are mandatory</footnote>
# DATA ACQUISITION

## Information on transmitted databases

<figure><img src="image_7.png" /></figure>

Research team:

Survey:

Survey ID:

Date:

Person in charge of data and metadata transmission:

DATABASE NAME:

LAST MODIFIED:

FILE FORMAT:

SIZE:

OBSERVATIONS:

VARIABLES:

VERSION DATE:

DATABASE NAME:

LAST MODIFIED:

FILE FORMAT:

SIZE:

OBSERVATIONS:

VARIABLES:

VERSION DATE:

DATABASE NAME:

LAST MODIFIED:

FILE FORMAT:

SIZE:

OBSERVATIONS:

VARIABLES:

VERSION DATE: