Package: pydataweaver
The Pydata Weaver ProjectThe Pydataweaver is a Python tool that offers a simple to use, clean and a robust data integration platform.The PydataWeaver supports data integration of spatial datasets (Raster and Vector data), as well as tabular datasets.Problem solving in science involves and requires studying entities using a broad range of associations among the entities under study. These associations are obtained through collecting and integrating various sources and forms of data.Since these heterogenous datasets are collected by various scientists, the datasets are domain based or centered around a unique subset of problems.The pydataweaver bridges the gap scientist's face of not having readily unified datasets that can be used for multi dimension feature analysis. The data weaver handles the finding and integration of heterogeneous datasets forming a new dataset.DependenciesThis package requires Python 3.5+, recommends Python 3.6+ and depends on the following packages:retriever
 PyMySQL>=0.4
 psycopg2-binary
 gdal
 future
 numpydoc
 pandasThey can be installed usingpip.sudopipinstall-rrequirements.txtThe package supports the following database management systems (DBMS):DBMSSpatial DatasetsTabular DatasetsPostgreSQLYesYesSQLiteNoYesInstallingpipinstallpydataweaverInstalling From SourceEither use pip to install directly from GitHub:pipinstallgit+https://git@github.com/weecology/pydataweaver.gitor:Clone the repositoryFrom the directory containing setup.py, run the following command:pip install .. You may need to includesudoat the beginning of the
command depending on your system (i.e.,sudo pip install .).More extensive documentation for those that are interested in developing can be foundhereUsing the Command LineAfter installing the package, runpydataweaverupdate to download the latest available dataset scripts.
To see the full list of command line options and datasets runpydataweaver --help.$ pydataweaver --helpusage:pydataweaver[-h][-v][-q]{help,ls,citation,license,join,update}...positionalarguments:{help,ls,citation,license,join,update}sub-commandhelphelplsdisplayalistallavailabledatasetscitationviewcitationlicenseviewdatasetlicensesjoinintegratedatausingadatapackagescriptupdatedownloadupdatedversionsofdatapackagescriptsoptionalarguments:-h,--helpshowthishelpmessageandexit-v,--versionshowprogram'sversionnumberandexit-q,--quietsuppresscommand-lineoutputTo get a list of available dataset usepydataweaver ls$ pydataweaver lsAvailabledatasets:11breed-bird-routes-bioclimmammal-community-bioclimmammal-community-massesmammal-community-sites-all-bioclimmammal-community-sites-bioclimmammal-community-sites-harvard-linear-featuresmammal-community-sites-harvard-linear-features-soilsmammal-community-sites-harvard-soilmammal-diet-mammal-life-historymammal-sites-bioclim-1-2portal-plot-species....To view the citaion of the datasets use `pydataweaver citation [dataset-name]
Running pydataweaver with no citation will provide the citation for the tool.$ pydataweaver citation mammal-diet-mammal-life-historyDataset:mammal-diet-mammal-life-historyDescription:Integrateddatasetofmammal-life-histandmammal-dietCitations:mammal-life-hist:S.K.MorganErnest.2003.....mammal-diet:KisslingWD,DalbyL,FlojgaardC,LenoirJ,...Integrating DataTo integrate data, run pydataweaver join [data package name] and provide the connection configurations.pydataweaverjoinpostgres-husage:pydataweaverjoinpostgres[-h][--user[USER]][--password[PASSWORD]][--host[HOST]][--port[PORT]][--database[DATABASE]][--database_name[DATABASE_NAME]][--table_name[TABLE_NAME]]datasetpositionalarguments:datasetfilenameoptionalarguments:-h,--helpshowthishelpmessageandexit--user[USER],-u[USER]EnteryourPostgreSQLusername--password[PASSWORD],-p[PASSWORD]Enteryourpassword--host[HOST],-o[HOST]EnteryourPostgreSQLhost--port[PORT],-r[PORT]EnteryourPostgreSQLport--database[DATABASE],-d[DATABASE]EnteryourPostgreSQLdatabasename--database_name[DATABASE_NAME],-a[DATABASE_NAME]Formatofschemaname--table_name[TABLE_NAME],-t[TABLE_NAME]FormatoftablenameTo use the pydataweaver with postges .pgpass file setpydataweaverjoinpostgresor with command line configurations suppliedpydataweaverjoinpostgres-uname-of-user-hhost-name-ddatabase-to-useContributionIf you find any operation that is not supported by this package, feel free to create a Github issue. Additionally, you are more than welcome to submit a pull request for a bug fix or additional feature.If you find any operation that is not supported by this package, feel
free to create a Github issue. Additionaly you are more than welcome to submit
a pull request for a bug fix or additional feature.Please take a look at theCode of Conductgoverning contributions to this project.AcknowledgmentsDevelopment of this software was funded bythe Gordon and Betty Moore
Foundation's Data-Driven Discovery
InitiativethroughGrant GBMF4563to Ethan White.