Package: nyctibius
Nyctibius - Streamlining sociodemographic data harmonizing.The Python package Nyctibius is designed to streamline the complex task of gathering and consolidating sociodemographic data from various sources into a cohesive relational database. Nyctibius empowers users to effortlessly unify custom data sets from diverse socio-demographic sources, ensuring that they can work with up-to-date and comprehensive information in a seamless manner. This package facilitates the process of creating a harmonized repository of socio-demographic data, simplifying data management and analysis for users across various domains.FeaturesSeamlessly retrieve data from online data sources through web scraping.Effortlessly extract data from diverse sources, consolidating it into a cohesive relational database.Conduct precise queries and apply transformations to meet specific criteria.Effectively manage data inconsistencies and discrepancies for enhanced accuracy.Support for various data formats, including .csv, .xlsx, .xls, .txt, and zip files, ensuring versatility in sourcing information.InstallationFor full documentation, please refer to theNyctibius documentation.You can install the Nyctibius package using pip. Make sure you have Python 3.x installed on your system; the package requires Python version 3.7 or higher.pipinstallnyctibiusUsageTo use the Nyctibius package, follow these steps:Import the package in your Python script:fromnyctibiusimportHarmonizerCreate an instance of theHarmonizerclass:harmonizer=Harmonizer()Extract data from online sources and create a list of data information:url='https://www.example.com'depth=0ext='csv'list_datainfo=harmonizer.extract(url=url,depth=depth,ext=ext)harmonizer=Harmonizer(list_datainfo)Load the data from the list of data information and merge it into a relational database:results=harmonizer.load()Import the modifier module and create an instance of theModifierclass:fromnyctibius.db.modifierimportModifiermodifier=Modifier(db_path='../../data/output/nyctibius.db')Perfom modifications:tables=modifier.get_tables()print(tables)Import the querier module and create an instance of theQuerierclass:fromnyctibius.db.querierimportQuerierquerier=Querier(db_path='data/output/nyctibius.db')Perform queries:df=querier.select(table="Estructura CHC_2017").execute()print(df)Supported Data SourcesThe package supports the following sources:Colombian microdata links from National Administrative Department of Statistics (DANE)Local filesOpen data sourcesPlease note that accessing data from these organizations may require authentication or specific credentials. Make sure you have the necessary permissions before using the library.LicenseThe Nyctibius package is open-source and released under theMIT License. Feel free to use, modify, and distribute this library in accordance with the terms of the license.AcknowledgementsWe would like to thank the following entities for providing the data used and the economic financial support for the development of this package:National Administrative Department of Statistics (DANE)Barcelona Supercomputing Center (BSC)Universidad de los AndesContactFor any questions, suggestions, or feedback regarding the package please contact:Erick lozano,
Email:es.lozano@uniandes.edu.coDiego Irreño,
Email:dirreno@unal.edu.coDisclaimerThis library is not officially affiliated with or endorsed by any of the mentioned official organizations. The data provided by this library is sourced from publicly available information and may not always reflect the most current or accurate data. Please verify the information with the respective official sources for critical use cases.