Saravanan666/pypi-org huggingface dataset:
Package: csv-batcher
csv-batcherA python-based, multiprocess CSV batcher suitable for
use with dataframes or other tools that deal with large CSV files (or those that require timely processing).Installationpip install csv-batcherGitHubDocumentationhttps://tangledpath.github.io/csv-batcher/csv_batcher.htmlUsageArguments sent to callback function can be controlled by
creating pooler withcallback_withand the CallbackWith enum
values:As dataframe rowfromcsv_batcher.csv_poolerimportCSVPooler,CallbackWith# Callback function passed to pooler; accepts a dataframe row#   as a pandas Series (via apply)defprocess_dataframe_row(row):returnrow.iloc[0]pooler=CSVPooler("5mSalesRecords.csv",process_dataframe_row,callback_with=CallbackWith.DATAFRAME_ROW,pool_size=16)pooler.process()### As dataframe```pythonfromcsv_batcher.csv_poolerimportCSVPooler,CallbackWith# Used in DataFrame.apply:defprocess_dataframe_row(row):returnrow.iloc[0]# Callback function passed to pooler; accepts a dataframe:defprocess_dataframe(df):foo=df.apply(process_dataframe_row,axis=1)# Or do something more complicated....returnlen(df)pooler=CSVPooler("5mSalesRecords.csv",process_dataframe,callback_with=CallbackWith.DATAFRAME,pool_size=16)pooler.process()### As CSV filename```pythonfromcsv_batcher.csv_poolerimportCSVPooler,CallbackWithdefprocess_csv_filename(csv_chunk_filename):# print("processing ", csv_chunk_filename)df=pd.read_csv(csv_chunk_filename,skipinitialspace=True,index_col=None)foo=df.apply(process_dataframe_row,axis=1)returnlen(df)defprocess_as_dataframe(df):foo=df.apply(process_dataframe_row,axis=1)returnlen(df)defprocess_dataframe_row(row):returnrow.iloc[0]pooler=CSVPooler("5mSalesRecords.csv",process_dataframe,callback_with=CallbackWith.CSV_FILENAMEchunk_lines=10000,pool_size=16)pooler.process()DevelopmentLintingruffcheck.# Find linting errorsruffcheck.--fix# Auto-fix linting errors (where possible)Documentation# Shows in browser
  poetry run pdoc csv_batcher
  # Generates to ./docs
  poetry run pdoc csv_batcher -o ./docsTestingclear;pytestPublishingpoetry publish --build -u __token__ -p $PYPI_TOKEN
Package: csv_bleach
clean your CSVs!This command line tool cleans CSV files by:converting encoding to utf-8detecting the delimiter and safely converting it to a commacasting all variables to json form, i.e. integers, floats, booleans, string or null.installpip install csv-bleachand run likepython -m csv_bleach my-data.csvThe only option is the output file name, by default it will be your original file name with.scsvextension.You will now be able to parse your CSV safely with a simple script like:importjsondefparse_row(text):returnjson.loads(f"[{text}]")defparse_file(file):rows=map(parse_row,file)header=next(rows)forrowinrows:yielddict(zip(header,row))withopen("my-data.scsv")asf:foriteminparse_file(f):print(item)
Package: csvblend
csvblend: Python CSV Merge LibraryJoin or merge multiple CSVs.csvblend is amemory constantPython library to merge multiple CSVs based on a list of columns.NOTE: csvblend requires SQLite version 3.24.0 (2018-06-04) or better:python -c 'import sqlite3; print(sqlite3.sqlite_version)'Basic merge usage:>>>fromcsvblendimportMergeFiles>>>columns=["a","b","c"]>>>indexes=["a"]>>>withMergeFiles(columns,indexes)asmf:...mf.merge(open(csvfile1))...mf.merge(open(csvfile2))...mf.merge(open(csvfile3))...forrowinmf.rows():...print(row)Features|Installation|Usage|Contributing|LicenseFeaturesSQLite(RDBMS) under the hood.Affected row count (created or updated) - useful to show changes between CSVs.No external dependencies.csvblend officially supports Python 3.6+.InstallationTo install csvblend, simply run:$pipinstall-Ucsvblend
✨🖇✨UsageFor documentation, see./docs/README.md.ContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.LicenseThis project is licensed under theMIT License.
Package: csvcat
csvcat reads one or more comma separated value text file (a CSV file)
and outputs some or all of the data in the same format.  It includes
options for limiting and reordering the columns in the output, as well
as skipping repeated headers.InstallationThe simplest way to install is using pip:$ pip install csvcatIf you have already downloaded the source and unpacked it, you can
also install by running:$ sudo python setup.py installUsageRuncsvcat--helpfor a complete description of all options,
including examples of how to use the program in various modes.ExamplesThe inputs tocsvcatare any number of CSV files, and the output
is CSV data printed to standard output.  The examples listed below
assume two simple CSV files.$ cat testdata1.csv
"Title 1","Title 2","Title 3"
1,"a",08/18/07
2,"b",08/19/07
3,"c",08/20/07

$ cat testdata2.csv
Title 1,Title 2,Title 3
40,D,08/21/07
50,E,08/22/07
60,F,08/23/07When given no options,csvcatsimply prints the contents of an input
file to standard output.$ csvcat testdata1.csv
Title 1,Title 2,Title 3
1,a,08/18/07
2,b,08/19/07
 3,c,08/20/07To select which columns should be included in the output, use the--columnsoption.  Columns are identified by their number,
beginning with0.  Column numbers can be listed in any order, so
it is possible to reorder the columns of the input data, if needed.$ csvcat --columns 2,0 testdata1.csv
Title 3,Title 1
08/18/07,1
08/19/07,2
08/20/07,3Different output formats can be selected by using the--dialectoption.  There are only two dialects available by default, but thecsvmodule API supports registering additional dialects.$ csvcat --dialect excel-tab testdata1.csv
Title 1 Title 2 Title 3
1       a       08/18/07
2       b       08/19/07
3       c       08/20/07To merge multiple files, only including a single set of headers, use
the--skip-headersoption:$ csvcat --skip-headers --columns 2,0 testdata1.csv testdata2.csv
Title 3,Title 1
08/18/07,1
08/19/07,2
08/20/07,3
08/21/07,40
08/22/07,50
08/23/07,60
Package: csvcleaner
CSV Cleaner is an Apache 2.0 licensed Python library that removes rows
containing blacklisted words from a CSV file.Instructions`python >>> import csvcleaner >>> f = csvcleaner.CSVCleaner() >>>f.run('/path/to/file.csv')`Whenrunis called, CSV Cleaner will loop through each row within the CSV file and search for blacklisted words.When a row is rejected because it contains a blacklisted word, it’s moved to a[name]-rejected.csvfile. Accepted rows are moved to a[name]-accepted.csvfile. Both files are saved in the same directory as the original CSV file.InstallationTo install CSV Cleaner, simply run:`bash $ pip install csvcleaner `ParmateresCSVCleaneraccepts several parameters:`python >>> import csvcleaner >>> f =csvcleaner(blacklist=[],replace_chars=[],configure=True, lowercase=True, strict=False) `#### blacklistAlistof characters or words that are used to determine if a row is rejected.Default:[](unlessconfigureisTrue)#### replace_charsAlistof words or characters that are replaced by a space in order to make word detection more accurate and effective.Default:[](unlessconfigureisTrue)#### configureWhenTrue, CSV Cleaner will use recommended lists forblacklistandreplace_chars. These recommended lists will only be used ifblacklistandreplace_charsare ommitted during class instantiation or contain an empty list. Set toFalseif you intend to supply custom lists forblacklistandreplace_chars.Default:True.#### lowercaseWhenTrue, all characters and strings will be converted to lowercase for more accurate word detection. When a row is inserted into[name]-accepted.csvor[name]-rejected.csv, its original case remains. Set toFalseif case matching is important.Default:True.#### strictWhenTrue, rows thatmaycontain (e.g., fuzzy matches) blacklisted words or characters are rejected.Default:False.BlacklistCSV Cleaner includes a blacklist that’s used whenconfigureisTrueandblacklistis left empty. This blacklist is maintained by [Shutterstock](https://github.com/shutterstock/)
on [Github](https://github.com/shutterstock/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words).
Package: csvcompare
CSVCompare: Compare CSV Files=============================.. image:: https://api.travis-ci.org/grantjenks/csvcompare.svg:target: http://www.grantjenks.com/docs/csvcompare/`CSVCompare`_ is an Apache2 licensed module for CSV file comparison.Review------- Missing diff re: column- setup.py- register command- sdist command- upload command- MANIFEST.in file- tox.ini file- .travis.yml file- travis-ci.org- pyinstaller --onefile --name csvcompare app.py- REST interfaceTODO----- Package django app with web interface- docsRelease Workflow................- Make changes.- Bump version.- tox- git commit -m "Bump version to ___"- git tag -a v___ abchash -m v___- git push- git push --tags- python setup.py sdist upload- cd docs && make html- Upload docsFeatures--------- Pure-Python- Fully Documented- 100% Test Coverage- Developed on Python 2.7- Tested on CPython 2.6, 2.7, 3.2, 3.3, 3.4 and PyPy 2.5+, PyPy3 2.4+Quickstart----------Installing CSVCompare is simple with`pip <http://www.pip-installer.org/>`_::$ pip install csvcompareYou can access documentation in the interpreter with Python's built-in helpfunction::>>> import csvcompare>>> help(csvcompare)Tutorial--------TODOReference and Indices---------------------* `CSVCopmare Documentation`_* `CSVCompare at PyPI`_* `CSVCompare at Github`_* `CSVCompare Issue Tracker`_.. _`CSVCompare Documentation`: http://www.grantjenks.com/docs/csvcompare/.. _`CSVCompare at PyPI`: https://pypi.python.org/pypi/csvcompare.. _`CSVCompare at Github`: https://github.com/grantjenks/csvcompare.. _`CSVCompare Issue Tracker`: https://github.com/grantjenks/csvcompare/issuesCSVCompare License------------------Copyright 2015 Grant JenksLicensed under the Apache License, Version 2.0 (the "License");you may not use this file except in compliance with the License.You may obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, softwaredistributed under the License is distributed on an "AS IS" BASIS,WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.See the License for the specific language governing permissions andlimitations under the License.
Package: csv-compare-tool
:mag_right: CSV-CompareA python package for Jamf MacAdmins in mindThe csvcomparetool allows you to find differences between two CSV files based on a specified column identifier. It provides a simple way to compare the contents of two CSV files and identify which records are present in one file but not in the other.Best PracticeWhen comparing CSV files for differences, be sure to provide the CSV with more entries second. For example, if CSV-1 has a list of 34 names, and CSV-2 has a list of 40, CSV-2 should be set as the second passed CSV path in order for differences to show as expected.InstallationUsageImport classCreate objectValidate pathsValidate columnFind differencesHandle differences as neededExamplesPython projectCLIInstallationYou can install CSV-Compare usingpip:pipinstallcsvcomparetoolUsageTo use CSV-Compare in your Python project, follow these steps.Import theCSVComparerclass from the packagefromcsvcomparetoolimportCSVComparerCreate aCSVComparerobjectNoteTheCSVComparerobject requires two paths and a column identifier. The column identifier is the title of the column you are pulling from.For example, 'Full Name' would be the column identifier for full names.csv1_path="path/to/first.csv"csv2_path="path/to/second.csv"column="identifier_column"comparer=CSVComparer(csv1_path,csv2_path,column)Validate the paths to ensure the specified CSV files existifnotcomparer.validate_paths():print("CSV file paths are invalid. Please check the file paths and try again.")returnValidate the column exists in the provided CSV filesifnotcomparer.validate_columns():print("Provided column not found in CSV. Check the columns and try again.")returnFind the differences between the two CSV filesdifferences=comparer.find_differences()Print or process the differences as neededfordifferenceindifferences:print(f"Record '{difference}' is present in CSV2 but not in CSV1.")ExamplesPython ProjectfromcsvcomparetoolimportCSVComparercsv1_path="path/to/first.csv"csv2_path="path/to/second.csv"column="identifier_column"comparer=CSVComparer(csv1_path,csv2_path,column)ifnotcomparer.validate_paths()ornotcomparer.validate_columns():print("CSV file paths are invalid, or the column identifier does not exist. Check the file paths and columns and try again.")else:differences=comparer.find_differences()fordifferenceindifferences:print(f"Record '{difference}' is present in CSV2 but not in CSV1.")Command Line toolFirst, clone this repository to a local directory on your machinegitclonehttps://github.com/liquidz00/csv-compare.gitNavigate to the cloned repo location (local directory where you chose to save the repository)cd/path/to/repo/src/csvcomparetool/Lastly, run the following commandpythoncli.py/csv/path/one/csv/path/twocolumnidentifier
Package: csv-compare-tool-liquidz00
:mag_right: CSV-CompareA python package for Jamf MacAdmins in mindCSV-Compare allows you to find differences between two CSV files based on a specified column identifier. It provides a simple way to compare the contents of two CSV files and identify which records are present in one file but not in the other.:construction: This project is currently under constructionIt has not yet been uploaded to pypi.org at present. Attempting to install via pip will not work presently.
Package: csv-converter-valtensir
A  CSV <> JSON converterA CSV to JSON and JSON to CSV converter.IntroductionA project to learning porpuses.What this project can do1 - Read a csv file or a folder with csv's and convert them to JSON.2 - Read a json file or a folder with json's and convert them to CSV.This project is a program running on terminal, preferably install with pipx:pipxinstallcsv_converter_valtensirHelpUsage:csv_converter[OPTIONS]Options:-c,--convertTEXTFormatyouwanttoconvert.cc->ConvertCSVtoJSON;cj->ConvertJSONtoCSV.-i,--inputTEXTPathwheretofindthefilestoconvert.-o,--outputTEXTPathwheretheconvertedfileswillbesaved.-d,--delimiterTEXTSeparatorusedtosplitthefiles.-p,--prefixTEXTPrefixusedtoprependtothenameoftheconvertedfilesavedondisk.Thesuffixwillbeanumberstartingfrom0.ge:file_0--helpShowthismessageandexit.
Package: csvcool
Cool way to deal with ‘headed’ CSVRules of the parser1 - First row is a list of column names.
2 - All rows have the same numbers of columns.
3 - Empty column names are ignored.
4 - Columns without name are ignored.Example:name0name1name3v0v2v3v4Equivalent:name0name1name3v0v2Use:>>> import csvcool
>>> cool = csvcool.read(open("path/to/csv"))
>>> cool[0]["name0"]
v0
>>> cool[0]["name0"] = 1
>>> cool[0]["name0"]
1
>>> csvcool.write(cool, open("path/to/csv", "w"))
Package: csvcubed
csvcubedcsvcubed project provides a command line tool which make it straightforward to turn a CSV into 5-star linked data (CSV-W)By publishing 5-star linked data and leveraging open standards for data, we believe that we can help ensure that statistical data is discoverable, comparable and analysable by automated tools. We hope that this standards-based approach will unlock network effects which accelerate data analysis by making it easier to collate, compare and contrast data from different sources.All our work depends on open standards; however it isn't just for open data. Share your data with the world or keep it private, the choice is yours.Getting started immediatelyGet going with csvcubed immediately by installing csvcubed using pip.pipinstallcsvcubedFrom there you'll have access to thecsvcubedcommand line tool which features sub commandsbuildandinspectto create CSV-Ws from CSV and inspect CSV-Ws.Become well acquainted to csvcubed with ourquick start, which includes written instructions as well as transcribed videos.User Documentationcsvcubed has extensive user documentation which tracks the release of csvcubed while it is in its beta phase. Ourdocumentationcan always be improved, so treat bad docs as a bug report.Related PackagesNameDescriptioncsvcubedThe key library helping to transform tidy-data into qb-flavoured CSV-W cubes.csvcubed-modelsModels and RDF serialisation functionality required by the csvcubed and csvcubed-pmd packages.csvcubed-pmdTransforms a CSV-qb into RDF which is compatible with the Publish My Data platform.csvcubed-devtoolsShared test functionality & dev dependencies which are commonly required.Developer DocumentationMore detailed developer documentation for this project can be foundhere.How to report bugsWe welcome and appreciate bug reports. As we are trying to make this tool useful for all levels of experience, any level of bug or improvement helps others. To contribute to making csvcubed better, check out ourbug reporting instructions.
Package: csvcubed-devtools
csvcubed - devtoolsShared test functionality & dev dependencies which are commonly required.Part of thecsvcubedproject.Thedevtoolspackage contains common packages necessary for development ofcsvcubedsuch as:behave - for behaviour testing / BDDdocker - to execute docker containers, used to transform and test csvcubed outputs.black - auto-formatter to ensure consistent code formatting/style.It also contains shared functionality to support testing, e.g. commonbehavetest steps like checking that files exist, using temporary directories when testing as well as tools to copy test files to & from docker containers.InstallationThis package should be installed as adev dependencyto ensure that end-users ofcsvcubedare not required to install development tools such as docker.Adding a packageDependencies are installed in theDocker containeron a container-wide basis. If you're adding a new package, first run:poetryadd<some-package>And once that has completed, if you are working inside the docker dev container, you must rebuild the container before the packages will be available for your use.
Package: csvcubed-models
csvcubed - modelsModels shared by multiple packages in the csvcubed family.Part of thecsvcubedproject.This package contains functionality to:Support serialisation of python models to RDFProvide models for relevant RDF ontologies such asSKOS,qbandDCAT.Enable deserialisation of JSON to instances of python dataclasses.Adding a packageDependencies are installed in theDocker containeron a container-wide basis. If you're adding a new package, first run:poetryadd<some-package>And once that has completed, if you are working inside the docker dev container, you must rebuild the container before the packages will be available for your use.
Package: csvcubed-pmd
csvcubed - PMDTools to transform a CSV-qb into RDF which is compatible with thePublish My Dataplatform.Part of thecsvcubedproject.It is anticipated that this library will only be for internal IDP-D usage and does not need to be installed or distributed to external data producers.Adding a packageDependencies are installed in theDocker containeron a container-wide basis. If you're adding a new package, first run:poetryadd<some-package>And once that has completed, if you are working inside the docker dev container, you must rebuild the container before the packages will be available for your use.
Package: csv-data-generator
csv_data_generatorAfter looking around for a quick way to generate CSVs for testing and not finding any I decided to create this CLI tool.Usage is very simple, install with:$ pip install csv_data_generatorRun it with:$ csv-data-genBy default it's going to generate a 5x5 CSV:col1,col2,col3,col4,col5
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5
1,2,3,4,5It just sends the data to stdout, so you can pipe it to a file or clipboard.You can give it a number of rows and columns:$ csv-data-gen  --rows 10 --columns 150You can give also make it use random words as filler instead of numbers:$ csv-data-gen  --rows 10 --columns 150 --fill-words
Package: csvdb
Csv reader like PEP 249 API.RequirementsPython 3.xInstallation$ pip install csvdbExampleimport csvdb
conn = minitds.connect('/foo/bar/baz.csv')
cur = conn.cursor()
cur.execute('field1,field2')
for r in cur.fetchall():
   print(r[0], r[1])
conn.close()
Package: csvdbclass
CSV DB CLASSA python package for easy python database usage through CSVs.Usage:>>> from csvdbclass import csvDB
>>> data = csvDB("data")
>>> data.show()
data
Empty DataFrame
Columns: []
Index: []
>>> data.db = pandas.DataFrame({
...     "Name" : ['joe', 'joe2', 'joe3']
... })
>>> data.show()
data
   Name
0   joe
1  joe2
2  joe3
Package: csv-db-package
csv_db_packagePackage having the local server module to upload a csv file and perform CRUD operations on the data shown using different features through database.Installationpip install csv_db_packageSTEPS TO INSTALL & MODIFY THE PROJECT-Activate your python venv
-Install the requirements given:
python
pip install -r requirements.txtMysql is used as a database engine here.
-Set env variables$env:HOST= your hostname for mysql$env:USER= your username$env:PASSWORD= your password-Run server inside directory by using command
python fileserver.py-example tableFirst columnSecond columnContent CellContent CellupdateContent CellContent Cellupdate© 2022 Ankita LiyaThis repository is licensed under the MIT license. See LICENSE for details.
Package: csvddata-importer
Documentationhttps://data-importer.readthedocs.ioInstallation$ pip install csvdata-importerCSV Data ImporterA configurable file to database table importer.
It can be used to import data from a csv file to a postgres
or sqlite database.You build a mapping json file and then use that to create or update tables
on the database.Mapping file Example{"DATABASE_URL":"postgresql://user:pwd@host:5432/db","TARGET_TABLE":"quote","IF_EXISTS":"append","COLUMN_MAPPING":[{"COLUMN_NAME":"source","FILE_COLUMN_NAME":"source","MAPPING_TYPE":"CONSTANT","CONSTANT_VALUE":"test-data"},{"COLUMN_NAME":"quote_value","FILE_COLUMN_NAME":"close",}]}Importing data using the mapping fileThe overall mapping configurationColumns and their function inf the mapping fileColumn NameExampleDescriptionDATABASE_URLpostgresql://user:pass@host:5432/dbThe database we are connecting toTARGET_TABLEquoteThe table in that database we are updating or creatingFILE_PATH/test.csvThe file we are importingIF_EXISTSappendThe action taken on the table if it is non empty. Can be one of fail, append.COLUMN_MAPPINGarray of columns see next tableThe columns and how they are mappedThe column mappingsColumns mappingsColumn NameExampleDescriptionCOLUMN_NAMEquote_valueThe table column we are updatingFILE_COLUMN_NAMEquoteThe column in the csv we are reading fromMAPPING_TYPEDIRECTThe type of mapping we are doing. See belowCONSTANT_VALUEFile ImportFor use when we are evaluating or mapping a constant to the table.Different types of mappingsDIRECT
In this case we read the value form the file and write it directly to the table row column.CONSTANT
In this case it is a constant value and we are just updating the table with the value we find in the CONSTANT_VALUE mapping.EVALUATION
Here we are evaluating some code to create the data.
Package: csv-deta
ALPHA version! work in progressLightweight utility to upload and download CSV files data into\from Deta Baseit's based oncsvtodetatool by@Fredy Somyimprovements:more optionsupload about twice as fast (put() method is used)bulk load: up 25 \ down 1000 rows per iterationinstallpipinstall-Ucsv-detausecsv-detauploadPATH_TO_CSV_FILE[--dbnameDETA_DB_NAME][--projectDETA_PROJECT_KEY]by defaultDETA_DB_NAMEparam will be created fromPATH_TO_CSV_FILEfilename without ext,
andDETA_PROJECT_KEY-- from the same name's environment variable;
for other options see alsocsv-deta -htododownload dataupload speed optimizationasync transfervalidate source csv (?)enjoy, star it, and donate