Package: pandem-source
PANDEM - SourceIntegrate heterogeneous publuc health surveillance data sources into a coherent data base for PANDEM-2 project.DescriptionIdentify, map and integrate multiple pandemic related data into a coherent pandemic-management database. Developed within the H2020 project PANDEM-2, Pandem-Source allows users to systematically capture, standardize and analyze data coming from international and national surveillance databases, participatory surveillance projects, social networks and mass media. This tool is focused on flexibility so adding new sources or variables can be easily done as is required during a pandemic episode.Target usersPublic health data experts and data managers needing integrate several surveillance data.ApproachData integration follow a semantic approach. Data sources need to be described using a Data Labelling Schema (DLS) file that specifies the acquisition chennel (URL, git, local file or script) the format and how to map the input data to Pandem-Source variables. If further customistions are needed, the user can define them using simple python scripts.The DLS ensures each source and variable is properly documented knwing its origin, meaning and data quality.Supported sourcesPandem-Source is designed to be flexible and extensible so new sources can be easily added by end users. In order to demontstrate this principle. The following sources are supported Out of the BoxSources for indicatorsCOVID19 Data Hubseesource definitionECDC Atlasseesource definitionECDC COVID19 Datasetssee source definitions forvariants,age group,daily cases,goverments measures,vaccinationECDC COVID19 Simulated dataseesource definitionSerotrackerseesource definitionOpen Sky Nerworkseesource definitionInfluenza Netseesource definitionMediSysseesource definitionTwitterseesource definitionUser provided data. You can use this data template to easily integrate your own real or ficticious datasets.Sources for referentialsEurostats NUTSseesource definitionOur airportsseesource definitionGeonamesseesource definitionICD-10-CMseesource definitionUser Interface - timeseries explorerUser Interface - Integration dashboardRequirementsPython 3.7 or higherR 3.6.3 or higherDocker (optional for supporting machine learning article classification)Installing from pipIf you want to customize the installation folder (defaulted to ~/.pandemsource) you need to set the environment variable PANDEM_HOME to a different folderpip install pandem-source

python -m pandemsource setup --installInstalling for contributorsIf you want to customize the installation folder (defaulted to ~/.pandemsource) you need to set the environment variable PANDEM_HOME to a different foldergit clone https://github.com/pandem2/pandem-source

cd pandem-source

make init

make install

source env/bin/activate

python -m pandemsource setup --installDefining sources o monitorPANDEM-Source comes with a list of predefined sources it can monitor, but they are not all enabled by default.
In order to use them them you have to manually activate them with the 'setup' command.See all avaiable sourcespython -m pandemsource setup -hActivating a particular source# Activating ECDC covid19 dataset monitoring
python -m pandemsource setup --ecdc-covid19

# Activating preloaded 2023 functional excercise data
python -m pandemsource setup --pandem-2-2023-fxRunning Pandem-SourceSet the PANDEM_HOME variable to a local foder (only if you have customized the default installation folder)export PANDEM\_HOME=your data folder hereRunning monitoring and dashboardpython -m pandemsource start -dAccessing the dashboard fromhttp://localhost:8001to see the progress and integrated time seriesTroubleshootingIf the commandpython -m pandemsource start -d --no-nlpdoes not work. Please make sure you've installedR 3.6.3 or higher. If R is properly installed, please make sure you've also installed required dependencies with:(In your terminal)sudoaptinstalllibxml2-devlibsodium-devlibssl-devlibcurl4-openssl-devlibgdal-devlibfontconfig1-devlibharfbuzz-devlibfribidi-dev(In the R interpreter)install.packages(c("epitweetr","dplyr","shiny","plotly","DT","jsonlite","httr","XML","ggplot2","epitweetr","reticulate","seqinr","readr"))PANDEM 2PANDEM-2 is a H2020 EU-funded project that aims to develop new solutions for efficient, EU-wide pandemic management. The goal of PANDEM-2 is to prepare Europe for future pandemics through innovations in training and to build capacity between EU member states responding to pandemics on a cross-border basis.