Package: dataclass-csv
Dataclass CSVDataclass CSV makes working with CSV files easier and much better than working with Dicts. It uses Python's Dataclasses to store data of every row on the CSV file and also uses type annotations which enables proper type checking and validation.Main featuresUsedataclassesinstead of dictionaries to represent the rows in the CSV file.Take advantage of thedataclassproperties type annotation.DataclassReaderuse the type annotation to perform validation of the data of the CSV file.Automatic type conversion.DataclassReadersupportsstr,int,float,complex,datetimeandbool, as well as any type whose constructor accepts a string as its single argument.Helps you troubleshoot issues with the data in the CSV file.DataclassReaderwill show exactly in which line of the CSV file contain errors.Extract only the data you need. It will only parse the properties defined in thedataclassFamiliar syntax. TheDataclassReaderis used almost the same way as theDictReaderin the standard library.It usesdataclassfeatures that let you define metadata properties so the data can be parsed exactly the way you want.Make the code cleaner. No more extra loops to convert data to the correct type, perform validation, set default values, theDataclassReaderwill do all this for you.In additon of theDataclassReaderthe library also provides aDataclassWriterwhich enables creating a CSV file
using a list of instances of a dataclass.Installationpipenvinstalldataclass-csvGetting startedUsing the DataclassReaderFirst, add the necessary imports:fromdataclassesimportdataclassfromdataclass_csvimportDataclassReaderAssuming that we have a CSV file with the contents below:firstname,email,age
Elsa,elsa@test.com, 11
Astor,astor@test.com, 7
Edit,edit@test.com, 3
Ella,ella@test.com, 2Let's create a dataclass that will represent a row in the CSV file above:@dataclassclassUser:firstname:stremail:strage:intThe dataclassUserhas 3 properties,firstnameandemailis of typestrandageis of typeint.To load and read the contents of the CSV file we do the same thing as if we would be using theDictReaderfrom thecsvmodule in the Python's standard library. After opening the file we create an instance of theDataclassReaderpassing two arguments. The first is thefileand the second is the dataclass that we wish to use to represent the data of every row of the CSV file. Like so:withopen(filename)asusers_csv:reader=DataclassReader(users_csv,User)forrowinreader:print(row)TheDataclassReaderinternally uses theDictReaderfrom thecsvmodule to read the CSV file which means that you can pass the same arguments that you would pass to theDictReader. The complete argument list is shown below:dataclass_csv.DataclassReader(f,cls,fieldnames=None,restkey=None,restval=None,dialect='excel',*args,**kwds)All keyword arguments support byDictReaderare supported by theDataclassReader, with the addition of:validate_header- TheDataclassReaderwill raise aValueErrorif the CSV file cointain columns with the same name. This
validation is performed to avoid data being overwritten. To skip this validation setvalidate_header=Falsewhen creating a
instance of theDataclassReader, see an example below:reader=DataclassReader(f,User,validate_header=False)If you run this code you should see an output like this:User(firstname='Elsa',email='elsa@test.com',age=11)User(firstname='Astor',email='astor@test.com',age=7)User(firstname='Edit',email='edit@test.com',age=3)User(firstname='Ella',email='ella@test.com',age=2)Error handlingOne of the advantages of using theDataclassReaderis that it makes it easy to detect when the type of data in the CSV file is not what your application's model is expecting. And, theDataclassReadershows errors that will help to identify the rows with problem in your CSV file.For example, say we change the contents of the CSV file shown in theGetting startedsection and, modify theageof the user Astor, let's change it to a string value:Astor, astor@test.com, testRemember that in the dataclassUsertheageproperty is annotated withint. If we run the code again an exception will be raised with the message below:dataclass_csv.exceptions.CsvValueError: The field `age` is defined as <class 'int'> but
received a value of type <class 'str'>. [CSV Line number: 3]Note that apart from telling what the error was, theDataclassReaderwill also show which line of the CSV file contain the data with errors.Default valuesTheDataclassReaderalso handles properties with default values. Let's modify the dataclassUserand add a default value for the fieldemail:fromdataclassesimportdataclass@dataclassclassUser:firstname:stremail:str='Not specified'age:intAnd we modify the CSV file and remove the email for the user Astor:Astor,,7If we run the code we should see the output below:User(firstname='Elsa', email='elsa@test.com', age=11)
User(firstname='Astor', email='Not specified', age=7)
User(firstname='Edit', email='edit@test.com', age=3)
User(firstname='Ella', email='ella@test.com', age=2)Note that now the object for the user Astor have the default valueNot specifiedassigned to the email property.Default values can also be set usingdataclasses.fieldlike so:fromdataclassesimportdataclass,field@dataclassclassUser:firstname:stremail:str=field(default='Not specified')age:intMapping dataclass fields to columnsThe mapping between a dataclass property and a column in the CSV file will be done automatically if the names match, however, there are situations that the name of the header for a column is different. We can easily tell theDataclassReaderhow the mapping should be done using the methodmap. Assuming that we have a CSV file with the contents below:First Name,email,age
Elsa,elsa@test.com, 11Note that now, the column is calledFirst Nameand notfirstnameAnd we can use the methodmap, like so:reader=DataclassReader(users_csv,User)reader.map('First name').to('firstname')Now the DataclassReader will know how to extract the data from the columnFirst Nameand add it to the to dataclass propertyfirstnameSupported type annotationAt the moment theDataclassReadersupportint,str,float,complex,datetime, andbool. When defining adatetimeproperty, it is necessary to use thedateformatdecorator, for example:fromdataclassesimportdataclassfromdatetimeimportdatetimefromdataclass_csvimportDataclassReader,dateformat@dataclass@dateformat('%Y/%m/%d')classUser:name:stremail:strbirthday:datetimeif__name__=='__main__':withopen('users.csv')asf:reader=DataclassReader(f,User)forrowinreader:print(row)Assuming that the CSV file have the following contents:name,email,birthday
Edit,edit@test.com,2018/11/23The output would look like this:User(name='Edit', email='edit@test.com', birthday=datetime.datetime(2018, 11, 23, 0, 0))Fields metadataIt is important to note that thedateformatdecorator will define the date format that will be used to parse date to all properties
in the class. Now there are situations where the data in a CSV file contains two or more columns with date values in different formats. It is possible
to set a format specific for every property using thedataclasses.field. Let's say that we now have a CSV file with the following contents:name,email,birthday, create_date
Edit,edit@test.com,2018/11/23,2018/11/23 10:43As you can see thecreate_datecontains time information as well.ThedataclassUser can be defined like this:fromdataclassesimportdataclass,fieldfromdatetimeimportdatetimefromdataclass_csvimportDataclassReader,dateformat@dataclass@dateformat('%Y/%m/%d')classUser:name:stremail:strbirthday:datetimecreate_date:datetime=field(metadata={'dateformat':'%Y/%m/%d%H:%M'})Note that the format for thebirthdayfield was not speficied using thefieldmetadata. In this case the format specified in thedateformatdecorator will be used.Handling values with empty spacesWhen defining a property of typestrin thedataclass, theDataclassReaderwill treat values with only white spaces as invalid. To change this
behavior, there is a decorator called@accept_whitespaces. When decorating the class with the@accept_whitespacesall the properties in the class
will accept values with only white spaces.For example:fromdataclass_csvimportDataclassReader,accept_whitespaces@accept_whitespaces@dataclassclassUser:name:stremail:strbirthday:datetimecreated_at:datetimeIf you need a specific field to accept white spaces, you can set the propertyaccept_whitespacesin the field's metadata, like so:@dataclassclassUser:name:stremail:str=field(metadata={'accept_whitespaces':True})birthday:datetimecreated_at:datetimeUser-defined typesYou can use any type for a field as long as its constructor accepts a string:classSSN:def__init__(self,val):ifre.match(r"\d{9}",val):self.val=f"{val[0:3]}-{val[3:5]}-{val[5:9]}"elifre.match(r"\d{3}-\d{2}-\d{4}",val):self.val=valelse:raiseValueError(f"Invalid SSN:{val!r}")@dataclasses.dataclassclassUser:name:strssn:SSNUsing the DataclassWriterReading a CSV file using theDataclassReaderis great and gives us the type-safety of Python's dataclasses and type annotation, however, there are situations where we would like to use dataclasses for creating CSV files, that's where theDataclassWritercomes in handy.Using theDataclassWriteris quite simple. Given that we have a dataclassUser:fromdataclassesimportdataclass@dataclassclassUser:firstname:strlastname:strage:intAnd in your program we have a list of users:users=[User(firstname="John",lastname="Smith",age=40),User(firstname="Daniel",lastname="Nilsson",age=10),User(firstname="Ella","Fralla",age=4)]In order to create a CSV using theDataclassWriterimport it fromdataclass_csv:fromdataclass_csvimportDataclassWriterInitialize it with the required arguments and call the methodwrite:withopen("users.csv","w")asf:w=DataclassWriter(f,users,User)w.write()That's it! Let's break down the snippet above.First, we open a file calleduser.csvfor writing. After that, an instance of theDataclassWriteris created. To create aDataclassWriterwe need to pass thefile, the list ofUserinstances, and lastly, the type, which in this case isUser.The type is required since the writer uses it when trying to figure out the CSV header. By default, it will use the names of the
properties defined in the dataclass, in the case of the dataclassUserthe title of each column
will befirstname,lastnameandage.See below the CSV created out of a list ofUser:firstname,lastname,age
John,Smith,40
Daniel,Nilsson,10
Ella,Fralla,4TheDataclassWriteralso takes a**fmtparamswhich accepts the same parameters as thecsv.writer, for more
information see:https://docs.python.org/3/library/csv.html#csv-fmt-paramsNow, there are situations where we don't want to write the CSV header. In this case, the methodwriteof
theDataclassWriteraccepts an extra argument, calledskip_header. The default value isFalseand when set toTrueit will skip the header.Modifying the CSV headerAs previously mentioned theDataclassWriteruses the names of the properties defined in the dataclass as the CSV header titles, however,
depending on your use case it makes sense to change it. TheDataclassWriterhas amapmethod just for this purpose.Using theUserdataclass with the propertiesfirstname,lastnameandage. The snippet below shows how to changefirstnametoFirst nameandlastnametoLast name:withopen("users.csv","w")asf:w=DataclassWriter(f,users,User)# Add mappings for firstname and lastnamew.map("firstname").to("First name")w.map("lastname").to("Last name")w.write()The CSV output of the snippet above will be:First name,Last name,age
John,Smith,40
Daniel,Nilsson,10
Ella,Fralla,4Copyright and LicenseCopyright (c) 2018 Daniel Furtado. Code released under BSD 3-clause licenseCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2018-11-25)First release on PyPI.0.1.1 (2018-11-25)Documentation fixes.0.1.2 (2018-11-25)Documentation fixes.0.1.3 (2018-11-26)Bug fixesRemoved the requirement of setting the dataclass init toTrue0.1.5 (2018-11-29)Support for parsing datetime values.Better handling when default values are set toNone0.1.6 (2018-12-01)Added support for reader default values from the default property of thedataclasses.field.Added support for allowing string values with only white spaces in a class level using the@accept_whitespacesdecorator or through thedataclasses.fieldmetadata.Added support for specifying date format using thedataclasses.fieldmetadata.0.1.7 (2018-12-01)Added support for default values fromdefault_factoryin the field's metadata. This allows adding mutable default values to the dataclass properties.1.0.0 (2018-12-16)When a data does not pass validation it shows the line number in the CSV file where the data contain errors.Improved error handling.Changed the usage of the@accept_whitespacesdecorator.Updated documentation.1.0.1 (2019-01-29)Fixed issue when parsing headers on a CSV file with trailing white spaces.1.1.0 (2019-02-17)Added support for boolean values.Docstrings1.1.1 (2019-02-17)Documentation fixes.1.1.2 (2019-02-17)Documentation fixes.1.1.3 (2020-03-01)Handle properties with init set to FalseHandle Option type annotation1.2.0 (2021-03-02)Introduction of a DataclassWriterAdded type hinting to external APIDocumentation updatesBug fixes1.3.0 (2021-04-10)Included stub filescheck if the CSV file has duplicated header valuesFixed issues #22 and #33code cleanup1.4.0 (2021-12-13)Bug fixesSupport for date types