Package: avro-to-python-etp
avro-to-python-etp is a light tool for compiling avro schema files (.avsc) to python classes making using avro schemata easy.Free software: MIT licenseDocumentation:https://avro-to-python-etp.readthedocs.io.InstallationPip install (recommended)To install avro-to-python-etp, run this command in your terminal:$pipinstallavro-to-python-etpInstall From Source ()The sources for avro-to-python-etp can be downloaded source as well.Clone the public repository:$gitclonegit://github.com/srserves85/avro-to-python-etpOnce you have a copy of the source, you can install it with:$pythonsetup.pyinstallor$pipinstall-e.ExamplesMajority of the use of avro-to-python-etp is assumed to be used as a cli, but you can still import and use the python classes under the hood as well.CLI (without –pip)To use the cli, here is the available cli commands:avro-to-python-etp[source][target]Options:--pipTEXTmakepackagepipinstallableusingthisname--authorTEXTauthornameofthepipinstallablepackage--package_versionTEXTversionofthepipintallablepackage[default:0.1.0]--helpShowthismessageandexitThe above will compile the avsc files and convert the to python classes found in [path_to_target_directory]An example of doing this is the following:avro-to-python-etp[path_to_source_avsc_files][path_to_target_directory]If you run the above on a valid avro avsc file, you should then be able to import them as you would in the avro idl namespace Here is an example of a single avsc record from the namespace:name.spaceand name:RecordClass:fromname.spaceimportRecordClassrecord=RecordClass({'foo':True,'bar':'true','baz':10,'food':'CHOCOLATE'})CLI (with –pip)You can also choose to make compiled avro packages*pip installable*by adding the “–pip” flags. An example of this is the following:
.. code-block:: bashavro-to-python-etp [path_to_source_avsc_files] [path_to_target_directory] –pip test_avroBy running this, you should be able to pip install the above package you created from the target directory you specified by running:pipinstall-epath_to_target_directoryNow that you have the package installed, you can import it by it’s package name and namespace. Here is the same example of the same avsc from above, only with a pip package oftest_avro:fromtest_avro.name.spaceimportRecordClassrecord=RecordClass({'foo':True,'bar':'true','baz':10,'food':'CHOCOLATE'})avro-to-python-etp in a ScriptYou can also use the reader and writer packages in avro-to-python-etp as you would any other python package. Avro to python is split between areaderandwriterclasses. avro-to-python-etp treates namespaces as acyclic trees and uses depth first search to ensure no duplication or namespace collisions on read and write. An example useage is below:fromavro_to_python.readerimportAvscReaderfromavro_to_python.writerimportAvroWriter# initialize the reader objectreader=AvscReader(directory='tests/avsc/records/')# generate the acyclic tree objectreader.read()# initialize the writer objectwriter=AvroWriter(reader.file_tree,pip='test_pip')# compile python files using 'tests/test_records as the namespace root'writer.write(root_dir='tests/test_records')RoadmapReader[X] Create Namespace Trees on nested namespaces[X] Read Record and Enum File[X] Primitive types[X] Array Types[X] Union types[X] References to other files[X] Map Types[ ] Logical Types (Currently just converts to primitive types)Writer[X] Base Schema Writer[X] Base Record Schema[X] Base Enum Schema[X] Primitive Types Schema[X] Array Types Schema[X] Union Types Schema[X] Map Types[ ] Logical Types Schema (Currently just converts to primitive types)[X] Add configs to pip install packageCLI[X] Wrap Writer and Reader into one cli commmit[X] Add pip install option (would include all files to pip install compiled package)[ ] Add better –help documentationDocumentation[ ] Document reader class[ ] Document writer class[ ] Document cli