Package: asdf
TheAdvancedScientificDataFormat (ASDF) is a
next-generation interchange format for scientific data. This package
contains the Python implementation of the ASDF Standard. More
information on the ASDF Standard itself can be foundhere.The ASDF format has the following features:A hierarchical, human-readable metadata format (implemented usingYAML)Numerical arrays are stored as binary data blocks which can be memory
mapped. Data blocks can optionally be compressed.The structure of the data can be automatically validated using schemas
(implemented usingJSON Schema)Native Python data types (numerical types, strings, dicts, lists) are
serialized automaticallyASDF can be extended to serialize custom data typesASDF is under active developmenton github. More information on contributing
can be foundbelow.OverviewThis section outlines basic use cases of the ASDF package for creating
and reading ASDF files.Creating a fileWe’re going to store severalnumpyarrays and other data to an ASDF file. We
do this by creating a “tree”, which is simply adict, and we provide it as
input to the constructor ofAsdfFile:importasdfimportnumpyasnp# Create some datasequence=np.arange(100)squares=sequence**2random=np.random.random(100)# Store the data in an arbitrarily nested dictionarytree={"foo":42,"name":"Monty","sequence":sequence,"powers":{"squares":squares},"random":random,}# Create the ASDF file object from our data treeaf=asdf.AsdfFile(tree)# Write the data to a new fileaf.write_to("example.asdf")If we open the newly created file’s metadata section, we can see some of the key features
of ASDF on display:#ASDF 1.0.0#ASDF_STANDARD 1.2.0%YAML1.1%TAG!tag:stsci.edu:asdf/---!core/asdf-1.1.0asdf_library:!core/software-1.0.0{author:The ASDF Developers,homepage:'http://github.com/asdf-format/asdf',name:asdf,version:2.0.0}history:extensions:-!core/extension_metadata-1.0.0extension_class:asdf.extension.BuiltinExtensionsoftware:{name:asdf,version:2.0.0}foo:42name:Montypowers:squares:!core/ndarray-1.0.0source:1datatype:int64byteorder:littleshape:[100]random:!core/ndarray-1.0.0source:2datatype:float64byteorder:littleshape:[100]sequence:!core/ndarray-1.0.0source:0datatype:int64byteorder:littleshape:[100]...The metadata in the file mirrors the structure of the tree that was stored. It
is hierarchical and human-readable. Notice that metadata has been added to the
tree that was not explicitly given by the user. Notice also that the numerical
array data is not stored in the metadata tree itself. Instead, it is stored as
binary data blocks below the metadata section (not shown above).It is possible to compress the array data when writing the file:af.write_to("compressed.asdf",all_array_compression="zlib")The built-in compression algorithms are'zlib', and'bzp2'.  The'lz4'algorithm becomes available when thelz4package
is installed.  Other compression algorithms may be available via extensions.Reading a fileTo read an existing ASDF file, we simply use the top-levelopenfunction of
theasdfpackage:importasdfaf=asdf.open("example.asdf")Theopenfunction also works as a context handler:withasdf.open("example.asdf")asaf:...To get a quick overview of the data stored in the file, use the top-levelAsdfFile.info()method:>>>importasdf>>>af=asdf.open("example.asdf")>>>af.info()root (AsdfObject)
├─asdf_library (Software)
│ ├─author (str): The ASDF Developers
│ ├─homepage (str): http://github.com/asdf-format/asdf
│ ├─name (str): asdf
│ └─version (str): 2.8.0
├─history (dict)
│ └─extensions (list)
│   └─[0] (ExtensionMetadata)
│     ├─extension_class (str): asdf.extension.BuiltinExtension
│     └─software (Software)
│       ├─name (str): asdf
│       └─version (str): 2.8.0
├─foo (int): 42
├─name (str): Monty
├─powers (dict)
│ └─squares (NDArrayType): shape=(100,), dtype=int64
├─random (NDArrayType): shape=(100,), dtype=float64
└─sequence (NDArrayType): shape=(100,), dtype=int64TheAsdfFilebehaves like a Pythondict, and nodes are accessed like
any other dictionary entry:>>>af["name"]'Monty'>>>af["powers"]{'squares': <array (unloaded) shape: [100] dtype: int64>}Array data remains unloaded until it is explicitly accessed:>>>af["powers"]["squares"]array([   0,    1,    4,    9,   16,   25,   36,   49,   64,   81,  100,
        121,  144,  169,  196,  225,  256,  289,  324,  361,  400,  441,
        484,  529,  576,  625,  676,  729,  784,  841,  900,  961, 1024,
       1089, 1156, 1225, 1296, 1369, 1444, 1521, 1600, 1681, 1764, 1849,
       1936, 2025, 2116, 2209, 2304, 2401, 2500, 2601, 2704, 2809, 2916,
       3025, 3136, 3249, 3364, 3481, 3600, 3721, 3844, 3969, 4096, 4225,
       4356, 4489, 4624, 4761, 4900, 5041, 5184, 5329, 5476, 5625, 5776,
       5929, 6084, 6241, 6400, 6561, 6724, 6889, 7056, 7225, 7396, 7569,
       7744, 7921, 8100, 8281, 8464, 8649, 8836, 9025, 9216, 9409, 9604,
       9801])>>>importnumpyasnp>>>expected=[x**2forxinrange(100)]>>>np.equal(af["powers"]["squares"],expected).all()TrueBy default, uncompressed data blocks are memory mapped for efficient
access. Memory mapping can be disabled by using thecopy_arraysoption ofopenwhen reading:af=asdf.open("example.asdf",copy_arrays=True)For more information and for advanced usage examples, see thedocumentation.Extending ASDFOut of the box, theasdfpackage automatically serializes and
deserializes native Python types. It is possible to extendasdfby
implementing custom tags that correspond to custom user types. More
information on extending ASDF can be found in theofficial
documentation.InstallationStable releases of the ASDF Python package are registeredat
PyPi. The latest stable version
can be installed usingpip:$ pip install asdfThe latest development version of ASDF is available from themainbranchon github. To clone the project:$ git clone https://github.com/asdf-format/asdfTo install:$ cd asdf
$ pip install .To install indevelopment
mode:$ pip install -e .TestingTo install the test dependencies from a source checkout of the repository:$ pip install -e ".[tests]"To run the unit tests from a source checkout of the repository:$ pytestIt is also possible to run the test suite from an installed version of
the package.$ pip install "asdf[tests]"
$ pytest --pyargs asdfIt is also possible to run the tests usingtox.$ pip install toxTo list all available environments:$ tox -vaTo run a specific environment:$ tox -e <envname>DocumentationMore detailed documentation on this software package can be foundhere.More information on the ASDF Standard itself can be foundhere.There are two mailing lists for ASDF:asdf-usersasdf-developersIf you are looking for theAdaptableSeismicDataFormat, information can be foundhere.LicenseASDF is licensed under a BSD 3-clause style license. SeeLICENSE.rstfor thelicenses folderfor
licenses for any included software.ContributingWe welcome feedback and contributions to the project. Contributions of
code, documentation, or general feedback are all appreciated. Please
follow thecontributing guidelinesto submit an
issue or a pull request.We strive to provide a welcoming community to all of our users by
abiding to theCode of Conduct.