Package: csvquery
csvqueryA versatile Python package that allows you to execute MongoDB-style queries on CSV files and interact with them like SQL tables.Downloadingpip install csvqueryPackage contentsopen_csv(str path[, str delimiter = ","])Produces aDatasetfrom a CSV file:fromcsvqueryimportopen_csvdataset=open_csv("path/to/file.csv")get_csv(str url[, str delimiter = ","])Produces aDatasetfrom a URL:fromcsvqueryimportget_csvdataset=get_csv("http://example.com/api/data.csv")parse_csv(str string[, str delimiter = ","])Produces aDatasetfrom a string:fromcsvqueryimportparse_csvstring="name,age\nJohn,34\nJane,33"dataset=parse_csv(string)OperatorsStores all the valid query operator keywords as attributes. Using this class is optional as you can just use the keyword strings instead.equal= "eq"not_equal= "neq"less_than= "lt"greater_than= "gt"less_than_or_equal= "lte"greater_than_or_equal= "gte"inside= "in"_not= "not"_and= "and"_or= "or"ComparisonsStores some common comparison operators.integersA lambda function to compare integer values.data.index("age",Comparisons.integers)floatsA lambda function to compare floating-point values.data.index("rate",Comparisons.floats)stringsA lambda function to compare strings alphabetically.data.index("name",Comparisons.strings)defaultAn alias for thefloatscomparison.get_date_comparison(str format_string)Returns a function that compares dates based on the format string. Seehttps://strftime.org/for a list of all valid date codes.data.index("date",Comparisons.get_date_comparison("%Y-%m-%d"))The Dataset objectThe Dataset object is similar to an SQL table. It can be obtained with theopen_csv,get_csv, andparse_csvmethods.dataA two-dimensional list of the data.forrowinvoter_dataset.data:print(row[0])...fieldsA list of the dataset's fields, or column names.forfieldinvoter_dataset.fields:print(field)...index(str field[, func comparison_operation = Comparisons.default])Sort the rows of data based on the values in a specified field. Sorting the data is optional, but doing so allows you to do binary searches which have a time complexity of justO(log(n)). Thecomparison_operationargument must be a function that returnsTruewhen the first argument is less than the second argument, andFalseif otherwise. ImportComparisonsfor some common comparison operations. By default, thecomparison_operationis a floating-point comparison.fromcsvqueryimportopen_csv,Comparisonsdataset=open_csv("people.csv")dataset.index("age",Comparisons.integers)# sorts people by ascending ageYou can also make your own comparison operator.dataset.index("age",lambdaa,b:a**2<b**2)already_indexed(str field[, func comparison_operation = Comparisons.default])Specifies that the data is already sorted by a certain field, allowing binary searches without re-sorting.fromcsvqueryimportopen_csv,Comparisonsdataset=open_csv("people.csv")dataset.already_indexed("name",Comparisons.strings)query(dict filter_object)Returns all rows that match thefilter_objectas anotherDataset.fromcsvqueryimportopen_csvdataset=open_csv("people.csv")dataset.index("age")voter_dataset=dataset.query({"age":{# this filter will run as a binary search since we indexed the data by age"gte":18# the query will only return people who's age is greater than or equal to 18},"citizenship"{# this will run after the binary search to filter the narrowed-down data"eq":"USA"# people will only pass this filter if their "citizenship" field is equal to "USA"}})The general structure of afilter_objectis as follows:{"field_1":{"operator_1":"value","operator_2":"value","operator_3":{"nested_operator":"value"},..."operator_n":"value_n"},"field_2":{...},..."field_3":{...}}You can also use thecsvquery.Operatorsclass instead of operator strings:fromcsvqueryimportOperatorsvoters_named_john=voter_dataset.query({"name":{Operators.equal:"John"}})If you want to use a comparison operator likegtorlteon a column that was not indexed, you need to provide a comparison operator in thefilter_objectlike so:fromcsvqueryimportopen_csv,Operations,Comparisonsdataset=open_csv("people.csv")dataset.index("citizenship")# sorts people by citizenshipvoter_dataset=dataset.query({"citizenship":{# binary search"eq":"USA"},"age"{# not a binary search"gte":"18""comparison":Comparisons.integers}})query_one(dict filter_object)Returns the first row that matches thefilter_objectas aDataset:john_doe=people_dataset.query_one({"phone":"555-123-4567"})select(list fields)Returns the a newDatasetobject with only the specified fields.names_and_ages=people.select(["name","age"])select_as(dict fields)Returns the a newDatasetobject with only the specified fields, except the fields are renamed according to thefieldsdictionary.names_and_ages=people.select_as({"first_and_last_name":"name","years_of_oldness":"age"})select_unique(str field)Returns a newDatasetobject with only the specified field, and removes any duplicate values so that each value is unique.names=people.select_unique("name")add_field(str field[, func derivation = lambda r:""])Adds another field with the specified name. By default, the field will be filled with blank values.people.add_field("status")You can optionally specify a function that takes the data in that row as a dictionary and outputs the new derived value per row.people.add_field("full_name",lambdarow:row["first_name"]+" "+row["last_name"]])remove_fields(list fields)Removes the specified fields from theDataset.people.remove_fields(["status","full_name"])rename_fields(dict fields)Renames fields according to thefieldsdictionary argument.people.rename_fields({"first_and_last_name":"name","years_of_oldness":"age"})replace(list fields, func function)Replaces the values in the specifiedfieldslist argument using thefunctionargument, which takes the current value as input and outputs the new value.people.replace(["first name","last name"],lambdav:v.lower())# makes all "first name" and "last name" values lower casereplace_derived(list fields, func derivation)Replaces the values in the specifiedfieldslist argument using thefunctionargument, which takes the row as a dictionary as input and outputs the new value.defbirthday_to_age(row):bday=datetime.strptime(row["date_of_birth"],"%Y-%m-%d")today=datetime.today()returntoday.year-born.year-((today.month,today.day)<(born.month,born.day))people.replace_derived(["age"],birthday_to_age)join(Dataset other_dataset, list common_fields[, bool remove = True])Adds fields fromother_datasetto thisDatasetand matches the rows by referencing thecommon_fieldslist argument. Thecommon_fieldslist must have the field from the currentDatasetand the field from the otherDatasetin that order. By default, this method will remove the common field after the operation, but you can prevent this by setting theremoveargument toFalse.locations=open_csv("locations.csv")# has an id fieldpeople=open_csv("people.csv")# has a location_id field that matches people to locationspeople.join(locations,["location_id","id"])to_dictionary()Returns a the data as a dictionary if theDatasethas only one row (as a result of aquery_oneoperation, for example).john_doe=people.query_one({"phone":"555-123-4567"})# dataset is one row highprint(john_doe.to_dictionary()["address"])to_list()Returns a the data as a list if theDatasethas only one column (as a result of aselectoperation, for example).texans=people.query({"state":"TX"}).select("name")# dataset is one column widetexan_names=texans.to_list()count([list fields])If thefieldsargument is left blank, returns the number of rows in theDataset.number_of_people=people.count()If otherwise, returns the number of rows in which the all of the specified fields are not empty.number_of_with_jobs=people.count(["job"])# assuming the "job" field is left blank for unemployed peoplesum(str field)Returns a sum of all the values in that field.total_net_worth=people.sum("net_worth")average(str field)Returns a average of all the values in that field.average_net_worth=people.average("net_worth")print_table([list fields])Outputs your data to the console in a nice table.voter_dataset.print_table()You can optionally specify which columns to print.voter_dataset.print_table(["name","age"])save_csv(str filepath[, str delimiter = ","[, fields = ]])Saves theDatasetto a file. If no fields are specified, all fields will be saved.voter_dataset.save_csv("output.csv",";",["name","age"])More examplesSQL equivalentClassic SQL querySELECTname,ageFROMpeopleWHEREage>=18ANDcitizenship="USA";Python MongoDB-style queryvoters=people.query({"age":{"gte":18},"citizenship":"USA"}).select(["name","age"])Printing certain columnsdataset=open_csv("people.csv")dataset.print_table(dataset.fields[2:5])Rewriting a CSV file with fewer columns and a different delimiterdataset=open_csv("people.csv")dataset.save_csv("people.csv",";",dataset.fields[2:5])The "eq" operator is optional in the top level of the dictionarydataset=csvquery.open_csv("people.csv")dataset.query({"name":"John"})Selecting one fieldpeople.select("name")# doesn't need to be a list if it's just oneChaining(open_csv("people.csv").index("age").query({"age":{"gte":18},"citizenship":"USA"}).select(["name","id"]).save_csv("voters.csv",",").print_table())