Saravanan666/pypi-org huggingface dataset:
Package: csv-intersection
csv_intersectionIntroductioncsv_intersection is basically a PyPI Package to get common rows from two tsv/csv files. Many a time we need common data from two tsv/csv files so using csv_intersection package will prolly solve this problem.This package currently have one functioncsv_intersectionwhich returns common rows between two csv/tsv in list of OrderedDict.Default delimiter is","and default encoding is"UTF-8"Functioncsv_intersection(path_of_tsv1, path_of_tsv2, delimiter, encoding)Table of contentsSetupUsageDependenciesContactsSetuppip install csv-intersectionUsagefrom csv_intersection import csv_intersection

common_rows = csv_intersection(tsv1, tsv2)

# if delimiter is other than ","
common_rows = csv_intersection(tsv1,tsv2,delimiter = delimiter_of_file)

# if encoding is different other than UTF-8
common_rows = csv_intersection(tsv1, tsv2, encoding = encoding_of_file)DependenciesIt requires csv module only which is a built-in module in Python.ContactsAuthorGithubLinkedinTwitter
Package: csvio
csvio: Python Library for processing CSV filescsvio is a Python library that provides a wrapper around Python’s built incsv.DictReaderandcsv.DictWriter, for ease of reading and
writing CSV files.Rows in a CSV are represented and processed as a list of dictionaries. Each
item in this list is a dictionary that represents a row. The key, value pairs
in each dictionary is a mapping between the column and its associated row value
from the CSV.InstallationpipinstallcsvioDocumentationReadthedocsReading CSVsCSV file contents:Supplier,Fruit,QuantityBigApple,Apple,1BigMelons,Melons,2LongMangoes,Mango,3SmallStrawberries,Strawberry,4>>>fromcsvioimportCSVReader>>>reader=CSVReader("fruit_stock.csv")>>>reader.fieldnames['Supplier','Fruit','Quantity']>>>len(reader.rows)4>>>importjson>>>print(json.dumps(reader.rows,indent=4))[{"Supplier":"Big Apple","Fruit":"Apple","Quantity":"1"},{"Supplier":"Big Melons","Fruit":"Melons","Quantity":"2"},{"Supplier":"Long Mangoes","Fruit":"Mango","Quantity":"3"},{"Supplier":"Small Strawberries","Fruit":"Strawberry","Quantity":"4"}]Writing CSVs>>>fromcsvioimportCSVWriter>>>writer=CSVWriter("fruit_stock.csv",fieldnames=["Supplier","Fruit","Quantity"])>>>row1={"Supplier":"Big Apple","Fruit":"Apple","Quantity":1}>>>writer.add_rows(row1)>>>rows2_3_4=[...{"Supplier":"Big Melons","Fruit":"Melons","Quantity":2},...{"Supplier":"Long Mangoes","Fruit":"Mango","Quantity":3},...{"Supplier":"Small Strawberries","Fruit":"Strawberry","Quantity":4}...]>>>writer.add_rows(rows2_3_4)>>>len(writer.pending_rows)4>>>len(writer.rows)0>>>writer.flush()>>>len(writer.pending_rows)0>>>len(writer.rows)4Once flush is called a CSV file with the namefruit_stock.csvwill be
written with the following contents.Supplier,Fruit,QuantityBigApple,Apple,1BigMelons,Melons,2LongMangoes,Mango,3SmallStrawberries,Strawberry,4Readthedocs for more example code
Package: csv-json-converter-mtba
CSV-JSON ConverterConversor de CSV para JSON / JSON para CSV.IntroductionTrabalho desenvolvido para a disciplina Python para Ciência de Dados do curso de pós-graduação em Ciência de Dados e Big Data da PUC Minas - 2021.What this project can doLê arquivoscsv/jsone os converte paracsv/json.
Este projeto é um programa de execução no terminal.
Instalar preferencialmente com pipx:pipxinstallcsv_json_converter_mtbaPara usar, digite:csv_json_converter--helpEsse comando listará todas as opções disponíveis.
Package: csv-jsonl
csv-jsonlA convenient module for writing a list of dictionaries or list of lists to a.jsonl-formattedtext file, suitable for ingestion byBigQueryand other services.csv-jsonlis built on top of Python's built-incsvmodule. It allows you to specify afieldnameslist to add a bit of assurance. Otherwise, no schema-handling is offered.Why not Just Use csv Files?If you are here asking that question, I'm guessing you have not spent exciting times attempting to clean up poorly-formattedcsvfiles (I'm looking at you, Excel).Other Data FormatsBasically supports anything with a__getitem__, as well as dataclasses. Seetestfor everything.Installationpip install csv-jsonlUsageList of Dictonaries>>>fromcsv_jsonlimportJSONLinesDictWriter>>>l=[{"foo":"bar","bat":1},{"foo":"bar","bat":2}]>>>withopen("foo.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesDictWriter(_fh)...writer.writerows(l)...>>>d={"foo":"bar","bat":1}>>>withopen("bar.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesDictWriter(_fh)...writer.writerow(d)...>>>fromcollectionsimportOrderedDict>>>od=OrderedDict([('foo','bar'),('bat',1)])>>>withopen("qux.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesDictWriter(_fh)...writer.writerow(od)...>>>fieldnames=["foo","bar"]# keys = ["foo", "bat"] expect fail>>>withopen("baz.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesDictWriter(_fh,fieldnames=fieldnames)...writer.writerows(l)...ExpectValueErrorList of Lists>>>fromcsv_jsonlimportJSONLineslistWriter>>>l=zip(["foo","bar","bat"],range(3),range(3))>>>withopen("foo.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesListWriter(_fh)...writer.writerows(l)...>>>l=zip(["foo","bar","bat"],range(3),range(3))>>>withopen("bar.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesDictWriter(_fh)...writer.writerow(next(l))...>>>fieldnames=["baz","qux","quux"]>>>l=zip(["foo","bar","bat"],range(3),range(3))>>>withopen("foo.jsonl","w",encoding="utf-8")as_fh:...writer=JSONLinesListWriter(_fh,fieldnames=fieldnames)...writer.writeheader()...writer.writerows(l)...
Package: csvkey
csvkeyFor a very limited situation such that you don't need performance and
want to use CSV as database.ExamplefromcsvkeyimportConnectionimportpandasaspdimportnumpyasnp# prepare DataFramedata=pd.DataFrame()data['A']=pd.Series([1,2,3],dtype='int')data['B']=pd.Series([4,5,6],dtype='float32')data['C']=pd.Series([7,8,9],dtype='float64')# register databaseconn=Connection()conn.initialize(data,r'C:\TEST\database.csv',primary=['A','B'],unique=['C'],notnull=['C'])# database.csv and the configuration file (default: csv.conf) are generated in C:\TEST\# set 'A' and 'B' columns as a primary key# values in 'C' column must be unique and not NaN# connect to database.csvconn.connect(r'C:\TEST\database.csv')conn.df.dtypes# dtypes are preserved# change values in conn.dfconn.df.loc[0,'C']=8conn.commit()# raise ValueError because 8 is not uniqueconn.df.loc[0,'C']=np.nanconn.commit()# raise ValueError because NaN is not allowed in 'C'conn.df.loc[0,'C']=-1conn.commit()# OKconn.df.loc[2,['A','B']]=[1,4]conn.commit()# raise ValueError because primary keys are duplicatedInstallationpip install csvkeyRequirementspandaspyyaml
Package: csvkit
csvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.It is inspired by pdftk, GDAL and the original csvcut tool by Joe Germuska and Aaron Bycoffe.Important links:Documentation:https://csvkit.rtfd.org/Repository:https://github.com/wireservice/csvkitIssues:https://github.com/wireservice/csvkit/issuesSchemas:https://github.com/wireservice/ffs
Package: csvkitcat
csvkitcat is a library that supplements (and depends on) the greatwireservice/csvkit. It exists because csvkit is no longer adding new tools to its core functionality.New tools and utilities:csvcount: just count recordscsvflatten: transpose records in a format easier to view in text filescsvnorm: normalize whitespace/newlines and character casingcsvsed: likesed, but done at the columnar levelcsvslice: likexsv slicecsvxplit: split a given column by a pattern into n new columns…and more random tools!See more tools and explanation at csvkitcat’sRead The DocsImportant linksDocumentation:https://csvkitcat.readthedocs.io/Repository:https://github.com/dannguyen/csvkitcatIssues:https://github.com/dannguyen/csvkitcat/issuesCHANGELOG:docs/CHANGELOG.rstCSVKit documentationcsvkit is a suite of command-line tools for converting to and working with CSV, the king of tabular file formats.It is inspired by pdftk, GDAL and the original csvcut tool by Joe Germuska and Aaron Bycoffe.Important links:Documentation:http://csvkit.rtfd.org/Repository:https://github.com/wireservice/csvkitIssues:https://github.com/wireservice/csvkit/issuesSchemas:https://github.com/wireservice/ffsDev notesHow to publish:https://realpython.com/pypi-publish-python-package/pythonsetup.pysdistbdist_wheel&&twineuploaddist/*
Package: csvlineparser
csvlineparserA CSV Parser that cleans a messy CSV file, with optional Excel export...
The CSV Parser is based on a BNF grammar, and the parser is built using a recursive descent parser.
Parsing line by line, character by character...See in the example folder for example_parser.py and the files in the in and out folders.From the Exampledefmain():""" Specify the input and output file paths """in_file_path='in/sample-raw-input.csv'out_file_path='out/sample-clean.csv'""" Optionally add an excel output file """out_excel_path='out/sample-clean.xlsx'"""Build the sha256 of the header column to make sure the input file has the expected header. Especially useful ifthe input file is provided by a third party or different occasions, when the input might have changed over time."""expected_header_sha="c551587bfca31449332989a377d59eb5f7bb6f1fb39dce862930094340299723""""Per column, specify the original name in the csvfile, the datatype, if the column is required, if the column shouldbe ignored, a possible rename and a column expander. The column expander is used to split a column into multiplenew cells.In the example, we rename the misspelled column 'stattus' to 'status', and we split the column 'created_on' intothree new columns 'created_on_year', 'created_on_month' and 'created_on_day'. We also split the column 'subtotal'into two new columns 'subtotal' and 'subtotal_ccy'. And the column 'tax' is ignored, because it is not needed."""column_definition=[['order_id','string',True,False,None,None],['stattus','string',True,False,'status',None],['created_on','datetime',True,False,None,ColumnExpander(['created_on','created_on_year','created_on_month','created_on_day'],YearMonthDaySplitter())],['customer_name','string',True,False,None,None],['items_count','int',True,False,None,None],['subtotal','double',True,False,None,ColumnExpander(['subtotal','subtotal_ccy'],PriceCCYSplitter())],['tax','double',True,True,None,None],['discounts_total','double',True,False,None,ColumnExpander(['discounts_total','discounts_total_ccy'],PriceCCYSplitter())],['customer_comment','string',False,False,None,None],]""" At least the column_definitions are required to build the parser config """default_parser_config=DefaultParserConfig(column_definition,expected_header_sha)csv_line_parser=CsvLineParser(default_parser_config)csv_line_parser.parse_csv(in_file_path,out_file_path,out_excel_path)if__name__=='__main__':main()BackgroundBased onhttps://secondboyet.com/articles/csvparser.htmlby Julian M Bucknall, rewritten in Python by raoulsson.BNF Grammar:csvFile ::= (csvRecord)* 'EOF'
csvRecord ::= csvStringList ('\n' | 'EOF')
csvStringList ::= rawString [',' csvStringList]
rawString := optionalSpaces [rawField optionalSpaces)]
optionalSpaces ::= whitespace*
whitespace ::= ' ' | '\t'
rawField ::= simpleField | quotedField 
simpleField ::= (any char except \n, EOF, \t, space, comma or double quote)+
quotedField ::= '"' escapedField '"'
escapedField ::= subField ['"' '"' escapedField]
subField ::= (any char except double quote or EOF)+InstallationClone and run 'make init'. Or, without cloning, to install it as a module, 
'pip install csvlineparser' should work as well...
Package: csvm
CSVMCSVM is a file format that allows you to save multiple tables, add comments, and include metadata for both tables and columnsFile format explanationTransform a.csvinto a.csvmWe start with a.csvfirst_name,last_name,instrument
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drumsTo transform it into a.csvmyou addtable attributes. Tha available table attributes are:[_columns_]to define the column names[_data_]to define the tabular data[_table_meta_.any_key.with_nested_keys]to add table metadata[_columns_meta_.any_key.with_nested_keys]to add column metadata.For now we add the attributes_columns_and_data[_columns_]
first_name,last_name,instrument
[_data_]
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drumsAdd comments and blank linesEvery line that starts with a#is ignored when parsing the file# Pink Floyd band members

[_columns_]
first_name,last_name,instrument

[_data_]
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drumsAdd table metadata# Pink Floyd band members

[_table_meta_.reference]
"https://en.wikipedia.org/wiki/Pink_Floyd#Band_members"

[_columns_]
first_name,last_name,instrument

[_data_]
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drumsAdd column metadata# Pink Floyd band members

[_table_meta_.reference]
"https://en.wikipedia.org/wiki/Pink_Floyd#Band_members"

[_column_meta_.unique]
true,true,false

[_columns_]
first_name,last_name,instrument

[_data_]
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drumsSave more than one tableIf you want to save more than one table you must use anamespacefor each table.# Table 1 - Pink Floyd band members

[members._table_meta_.reference]
"https://en.wikipedia.org/wiki/Pink_Floyd#Band_members"

[members._column_meta_.unique]
true,true,false

[members._columns_]
first_name,last_name,instrument

[members._data_]
Syd,Barret,"guitar, vocals"
David,Gilmour,"guitar, vocals"
Roger,Waters,"bass, vocals"
Richard,Wright,"keyboards, vocals"
Nick,Mason,drums

# Table 2 - Pink Floyd albums

[albums._columns_]
name,year
[albums._data_]
Animals,1997
"The Dark Side of the Moon",1973Read a.csvmfileInstallationpip install csvmimportcsvm# Load TableCollection (if multiple tables) or Table (if one table)tc=csvm.read_csvm("pink_floyd.csvm")print(tc.tables)members=tc['members']# get Table# Access Table attributes if exist (i.e. Table.columns, Table.column_meta, Table.table_meta, Table.data, Table.name)print(members.data)
Package: csvmagic
csvmagicCommand line utilities to work with csv files.Contentscsvread- Display csv rows in a more human-readable format.csvcut- Extract or rearrange columns of a csv file.csvgrep- Find rows matching a pattern in a csv file.csvalign- Print a csv file with its columns aligned.csvsql- Query one or more csv files.csvcsv- Convert a csv file from one variant to another.InstallationWith pip:pip install csvmagicWithdpm:dpm install https://github.com/markuskimius/csvmagic.gitWith git clone, after installing the dependencies with pip:pip install getopts
git clone https://github.com/markuskimius/csvmagic.gitLicenseApache 2.0
Package: csv-mAP-calculator
Function to calculate mean average precision (mAP) in CSV format for post detection prediction calculation.Requirementspython 3.*, numpy, pandas, cython(optinal), pyximport(optional)Installationpip install CSV-mAP-calculator
pip install CSV-mAP-calculator -i https://pypi.python.org/simpleUsage example:Path to CSV-files:fromCSV_mAP_calculatorimportget_csv_mAPgt_file='csvs/gt.csv'predict_file='csvs/predictions.csv'mean_ap,average_precisions=get_csv_mAP(gt_file,predict_file,iou_threshold=0.5)Or numpy arrays of shapes(N, 6)and(M, 7).fromCSV_mAP_calculatorimportget_csv_mAPimportpandasaspdgt=pd.read_csv('csvs/gt.csv',header=None,names=['img_path','x1','y1','x2','y2','conf','label']).valuespred=pd.read_csv('csvs/predictions.csv',header=None,names=['img_path','x1','y1','x2','y2','label']).valuesmean_ap,average_precisions=get_csv_mAP(gt,pred)Input files formatAnnotation CSV-file:'img_path','x1','y1','x2','y2','label'
path/imgname1.jpg,0,0,511,511,cat1
path/imgname2.jpg,122,247,666,799,cat2
...Detection CSV-file:'img_path','x1','y1','x2','y2','conf','label'
path/imgname1.jpg,0,0,511,511,0.8958333,cat1
path/imgname1.jpg,121,32,511,242.5,0.9998,cat2
path/imgname2.jpg,0,0,511,511,0.8958333,cat3
...Return should be like:Numberoffilesinannotations:7283Numberoffilesinpredictions:7282Uniqueclasses:3Detectionslength:7282Annotationslength:7283cat1|0.917434|2445cat2|0.861768|2400cat3|0.930730|2438mAP:0.9033110.903310916116078
Package: csvmedkit
The unofficial extended family of csvkit, i.e. even more tools for command-line data parsing and wrangling.Status as of 2020-11-11: Alpha, but working toward release version with these utilities:csvflatten: reformat data for easier browsing of “wide” datacsvheader: add/alter a data file’s column namescsvnorm: normalize unprintable characters and whitespacecsvpivot: do pivot tablescsvsed: do sed substitution on a per-column basiscsvslice: return rows by index, including head/tail to get first/last n rowsRead the in-progress docs at:csvmedkit.readthedocs.ioRead TODOS:TODOS.mdfor an informal roadmap.The old, now archived csvkitcat repo:dannguyen/csvkitcatNote: many of the data examples incorporate the original csvkit tools, including a couple of post 1.0.6 fixes. Until csvkit makes a new official release, here’s what I used:$ pip install git+https://github.com/wireservice/csvkit.git@73d5bdc4a2f5c07b91737ea007bb3510f970aad7
Package: csvmimesis
Generate dummy tables using the mimesis data providerDescriptionThis is a small library to generate dummy tables based on the providers of the mimesis librarySimilar projects:
* csvfaker:https://github.com/pereorga/csvfakerNoteThis project has been set up using PyScaffold 3.0.3. For details and usage
information on PyScaffold seehttp://pyscaffold.org/.
Package: csvmonitor
Python Client for CSVMonitor.comThis is a thin wrapper around the Pythonrequestsmodule to trigger HTTP beacons on thecsvmonitor.comSoftware as a Service (SaaS) site. To use it in your project, install the csvmonitor module using pip:python -m pip install csvmonitorThen, after registering at csvmonitor.com, create a beacon and copy the organization and beacon keys for the beacons you wish to trigger. Use thefire_beaconfunction to trigger the beacons, and you're done!from csvmonitor import fire_beacon

. . .

ok = somefunc()

fire_beacon(CSVM_ORG_ID, BEACON_KEY, 'success' if ok else 'error')
Package: csv-mqtt
CSV-MQTTA CSV to MQTT connector. Reads data from csv file and pushes it to a MQTT Broker.Many a times, we want to read a particular .csv file and then form a MQTT payload to publish it to cloud. ( Probably when working in IoT).So the project as always came in due to a necessity. The idea is simple. The process will read the .csv file wherein the very first row contains the columns headers. ( Please add column headers before using the code or be ready to see something strange.)
Then the same column headers are treated as Keys and the corresponding row values as Values in a JSON object while forming a MQTT Payload.For example, If the .csv has column headers as,
Name, Age and SchoolThen the resulting MQTT Payload will be of the form,{
"Name" : "dummy_name",
"Age" : "dummy_value",
"School" : "dummy_school"
}One can install the package using pippip3 install csv-mqttOnce installed, they can use the package by using the importfrom csv_mqtt.csv_mqtt import CsvMqttTheCsvMqttcan be used to create objects for creating a pipeline for csv-mqtt.For examples, one can check the example directory in the project.
Package: csv-normalizer
InfoThis is just a simple script that ensures all your .csv files have always same columns and in the same order. Probably
one of the most common issues with .csv files:Some system doesn't respects the columns ordersSome system doesn't adds a column when there is no data for such columnThe script/program resolves both cases in a simple way, process:Process:Normalize: ensure order of columns is always same, add missing columns with empty data.Example, you have a meteorologic station that should always generate a .csv with the following columnsTemperature, Humidity, Radiation, Wind, Wind gustBut sometimes one of the sensors doesn't have data and instead of sending all the columns to the .csv it generates partial .csvTemperature, Humidity, Wind, Wind gustIn this case the software that process the .csv could fail, so you can use the csv_normalizer to ensure the .csv file is alwaysTemperature, Humidity, Radiation, Wind, Wind gustIn this case the csv_normalizer will add the missing column with empty data.
Also the csv_normalizer will ensure the order of the columns is always the same.Returns always a dict/json like, with the 'ok' or 'fail' list of processed files.
examples:{'failed':[],'ok':[{'export_path':'C:\\temp\\csv_export\\business-financial-data-jun-2021-quarter.csv','import_path':'C:\\temp\\csv_import\\business-financial-data-jun-2021-quarter.csv'}]}# Example when nothing was processed:{'failed':[],'ok':[]}Example config:[common]csv_import_folder=C:/temp/csv_importcsv_export_folder=C:/temp/csv_exportcsv_export_headers='Series_reference', 'Period', 'ELEE'csv_delimiter=;csv_encoding=utf-8# You can use column types, like int64, np.float64 if you want to specify# Or you can use type object if you don't want conversion or avoid NaN errors# https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.read_csv.html# example: {'column name': 'object'}'dtype'={}Usageusage:csv_normalizer[-h][-c[CONFIG_INI]][--version[VERSION]][--no_rename[NO_RENAME_OLD]][--write_config]optionalarguments:
-h,--helpshowthishelpmessageandexit-c[CONFIG_INI],--config_ini[CONFIG_INI]csv_normalizeriniconfigurationfile
--version[VERSION]Printversionandexit--no_rename[NO_RENAME_OLD]Donotrenameto.oldtheoriginalfile
--write_configWriteconfigurationwithdefaultvalues,usefultogetaconfigfiletomodifyExample usage on Linuxcsv_normalizer -c .\csv_normalizer.iniOn windows:csv_normalizer.exe -c .\csv_normalizer.iniAdding option to not rename the original files:csv_normalizer -c .\csv_normalizer.ini --no_renameBy default csv_normalizer will rename the original files to .old so if you run the program again, it will not process same files again.Installpipinstall--usercsv_normalizer# or for root accountpipinstallcsv_normalizerAuthorAuthor: Pablo EstigarribiaProject site:https://github.com/CoffeeITWorks/csv_normalizer
Package: csvObject
csvObjectSimple Objected based approach to using data from a csvAbout The ProjectcsvObject is just a simple object base approach to loading in a csv file and using the data. In more complex cases when
doing actual data analysis or changing the structure of the file, usingpandasmakes much more sense. But if all you want is to have a light weight way of extracting the data from the csv file parsed
into an object with a few basic options without Numpy then this may be of interest.All the source code can be found at thecsvObject git repositorywith
additional information found on thedocs site
Package: csv_object_reader
HomepageVisit the home ofcsv_object_readeron the web:https://github.com/fireyone29/csv_object_readerSummaryA simple module which mimics thereaderclasses from python’scsvmodule but returns an object whose attributes are the members of the
header instead of a list or dict.TheObjectReaderalso provides a way to make fields required to be
present and required to be not empty.HistoryVersion 0.1.1 (2015-02-08)Minor documenation fix.Version 0.1 (2015-02-08)First release.Version 0.0 (2015-02-08)Initial revision.  Basic interface and functionality.