Saravanan666/pypi-org huggingface dataset:
Package: h5nav
h5navPython tool for interactive navigation of an hdf5 file in a "unix shell"
fashion.InstallationWhether for python 2 or 3, it's as simple as:pip install h5navContributingTo install locally, start by getting the files:git clone https://github.com/clapeyre/h5nav.gitMake sure you havepandocinstalled, as it is used to convert theREADME.mdtoREADME.rstfile. The first is the source, used on GitHub. The latter is
generated, so as to show up on PyPI. Then, run:make developin your favorite (virtual) environment. You are ready to work in h5nav, and see
your work dynamically updated when callingh5nav. Please submit merge
requests directly on GitHub.AuthorsCorentin J. Lapeyre
Package: h5obj
h5objStore native Python objects in HDF5 files using strings in json format.
Defines wrappers for h5py.File and h5py.Group.Copyright noticeCopyright (C) 2013-2023 Daniel Jung
Contact:proggy-contact@mailbox.orgThis program is free software; you can redistribute it and/or modify it
under the terms of the GNU General Public License as published by the Free
Software Foundation; either version 2 of the License, or (at your option)
any later version.This program is distributed in the hope that it will be useful, but WITHOUT
ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
more details.You should have received a copy of the GNU General Public License along
with this program; if not, write to the Free Software Foundation, Inc.,
59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
Package: h5parm
h5parmProvides H5Parm data interfaceExamplefromh5parm.utilsimportmake_example_datapackfromh5parmimportDataPackdatapack=make_example_datapack(4,5,6)#select your slicesdatapack.select(ant="RS*",time=slice(0,None,2),freq=2)This will select all antennas with names starting with 'RS', every other time, and the 3rd frequency.#get your soltabphase,axes=datapack.phase#get the coordinates of axespatch_names,directions=datapack.get_directions(axes['dir'])antenna_labels,antennas=datapack.get_antennas(axes['ant'])timestamp,times=datapack.get_antennas(axes['time'])freq_labels,freqs=datapack.get_freqs(axes['freq'])pol_labels,pols=datapack.get_pols(axes['pol'])In general the first element returned by allget_*methods are string labels, and the second is a dimensionfulastropyobject, e.g.ICRSobject,Timeobject etc.
Package: h5pom
IntroductionH5pom is an object mapper for keeping a python object hierarchy
synchronized with an HDF5 data file.  Think of this as an ORM for HDF5 based
storage.  Under the hood H5pom uses h5py for communicating with the data
store, but this dependency is intended to be entirely abstracted away by the
the object mapper layer.The requirements to save and restore objects from an HDF5 data file using
H5pom are modest and in keeping with the usual requirements for using
another ORM like Sqlalchemy and Django.  The only needs are to derive your
domain specific classes from h5pom.Object and declare your attributes as
class attributes.  Here is an example usage:>>> import h5pom
>>> class Person(h5pom.Object):
...     name = h5pom.Scalar()
...     age = h5pom.Scalar()
...
>>> f = h5pom.open(h5pom.IN_MEMORY, [Person])
>>> p = Person(name='Joe', age=25)
>>> f['joe'] = p
>>> assert f['joe'].name == 'Joe'
>>> p.age = 23
>>> assert f['joe'].age == 23
>>> f.close()In that example, there is no file made on disk due to the use of
h5pom.IN_MEMORY used as the file name, but the code is representative and
complete for the simple notion of saving Person objects to an HDF5 file.Goals of H5pomH5pom aims to make it easy to store and synchronize python object graphs
which may or may not have numeric array data in an HDF5 file.  For array types
of known numpy dtype H5pom aims to expose the full performance of HDF5
and h5py as appropriate.H5pom does not aim to have the full generality of reading arbitrary HDF5
files.  The h5py project already has an excellent pythonic API for accessing
groups, subgroups and attributes.  H5pom will tend to emphasize a schema
where the schema is a list of class types which may appear in the HDF5 file and
the attributes on those objects are known at the point of writing the python
model implementation.Object Mapping BasicsH5pom heavily utilizes the group and attribute notions in HDF5.  The
following general correspondences illustrate the design:python objects <-> groupsscalar attributes of python objects <-> group attributesobject references of python objects <-> subgroup with-in parent object grouparray type as attribute of python object <-> dataset with-in parent object
grouppython list as attribute of python object <-> family of attributes with
prefixed names with-in parent object groupTODOSubObjectDict saves dictionaries mapping alphanumeric names to Objects, but
there is no persistent dictionaries mapping alphanumeric names to Scalars.
Consider adding this.
Package: h5pPlugin
h5p-pluginPlugin for rendering h5p components in Sphinx.Given h5p components, the plugin can be used to easily and with short syntax render h5p components inside .rst files and .md files.Syntax for rendering in .rst files is:.. iframe:: idid has to match with the given h5p components folder name.Syntax for rendering in .md files is:```{eval-rst}.. iframe:: id```Configuring h5p pathIn order for plugin to find the right path to h5p component folder, the path needs to be specified in projects conf file.
In conf file, create variable named html_template_url and assign path to it.
Package: h5p-xblock
Play H5P content in Open edX using h5pxblockH5P Xblock provides ability to host and play H5P content in open edX. It has few more featuresAbility to mark H5P content complete in open edXAbility to capture score of H5P content in open edXSave learner state which can be retrieved laterAbility to host H5P content on cloud storage like AWS S3SetupInstall Xblockpip install h5p-xblockUpdate Advanced Settings of courseUpdate course advanced settings by addingh5pxblockUpload H5P ContentXblock should be available in Advanced component list of course unit now. Add xblock in unit and click "Edit" button toupload H5P content and configure it.Publish ContentUse "Preview" button to preview it or publish your content and use "View Live Version" button to see how it appearson LMSWorking with translationsYou can help by translating this project. Follow the steps below:Create a folder for the translations inlocale/, eg:locale/es_419/LC_MESSAGES/, and create yourtext.pofile
with all the translations.Runmake compile_translations, this will generate thetext.mofile.Create a pull request with your changes.
Package: h5py
The h5py package provides both a high- and low-level interface to the HDF5
library from Python. The low-level interface is intended to be a complete
wrapping of the HDF5 API, while the high-level component supports  access to
HDF5 files, datasets and groups using established Python and NumPy concepts.A strong emphasis on automatic conversion between Python (Numpy) datatypes and
data structures and their HDF5 equivalents vastly simplifies the process of
reading and writing data from Python.Wheels are provided for several popular platforms, with an included copy of
the HDF5 library (usually the latest version when h5py is released).You can alsobuild h5py from sourcewith any HDF5 stable release from version 1.10.4 onwards, although naturally new
HDF5 versions released after this version of h5py may not work.
Odd-numbered minor versions of HDF5 (e.g. 1.13) are experimental, and may not
be supported.
Package: h5pydantic
h5pydantic is a Pydantic based library aimed at making it easier for
scientists to organise their HDF5 files, by writing Python models of
their experiments. The library is similar to an ORM, but instead of
targetting a relational database, it targets HDF.The library is inspired by the need of the Australian Synchrotron
during the commissioning of our new beamlines under the BRIGHT
program. The library is currently under development.
Package: h5pyd-redesign
Python client library for HDF5 REST interfaceIntroductionThis repository contains library, test, and examples of h5pyd - a Python package for the
HDF REST interface.The library is provides a high-level interface to the REST specification that is generally
easier to use than invoking http calls directly.The package is based on the popular h5py package and aims to be source compatible with
the h5py high level interface.WebsitesMain website:http://www.hdfgroup.orgSource code:https://github.com/HDFGroup/h5pydMailing list:hdf-forum@lists.hdfgroup.org<hdf-forum@lists.hdfgroup.org>Documentation: TBD (buthttp://docs.h5py.org/en/latest/should be helpful)Related ProjectsHSDS:https://github.com/HDFGroup/hsdsHDF Server:https://github.com/HDFGroup/h5servHDF5-JSON:https://github.com/HDFGroup/hdf5-jsonh5py:https://github.com/h5py/h5pyREST API Documentation:https://github.com/HDFGroup/hdf-rest-apiInstallingVia pip:pip install h5pydFrom a release tarball or Git checkout:python setup.py installBy default the examples look for a local instance of h5serv.  See thehttps://github.com/HDFGroup/h5servfor instructions on installing and running h5serv.These tests are also to designed to work with HSDS (seehttps://github.com/HDFGroup/hsds).  Install HSDS locally, or set environment variables (see next section)
to point to an existing HSDS instance.TestingSetup the following environment variables that inform h5pyd which endpoint and username to use:HS_ENDPOINT- “http://127.0.0.1:5000” for h5serv installed locally or appropriate remote endpointHS_USERNAME- “test_user1” or your preferred usernameHS_PASSWORD- password for above username - “test” for test_user1 with local h5serv installTEST2_USERNAME- “test_user2” or your preffered usernameTEST2_PASSWORD- password for above username - “test” for test_user2 with local h5serv installH5PYD_TEST_FOLDER- server folder that will be used for generated files.  Use: “h5pyd_test.hdfgroup.org” for local h5serv install.  For HSDS, posix-style paths are also supported, e.g.: /home/bob/h5pyd_test.  For HSDS use hstouch command to create the folder before running the test, e.g.:$ hstouch /home/bob/h5pyd_test/Run:$python testall.pyto execute the test suite.UninstallingJust remove the install directory and all contents to uninstall.Reporting bugs (and general feedback)Create new issues athttp://github.com/HDFGroup/h5pyd/issuesfor any problems you find.For general questions/feedback, please use the Kita forum (https://forum.hdfgroup.org/c/kita).
Package: h5py-like
h5py_likeSome base classes and helper functions for an approximately h5py-like API in python 3.7+.Use caseYou have a library which reads/writes contiguous regions of chunked numeric arrays,
and want to make it behave somewhat like h5py.e.g.zarr,z5,xarray,pyn5Not supportedEmpty and scalar data spacesLogical indexingBroadcasting (other than scalar)Dimension scalesDifferences from h5pyAccess modes are converted to enums, although they are largely compatible with thestrforms"x"is preferred over"w-"for exclusive creationUsageSee the trivial HDF5 implementation inthe tests package.Create your ownDataset,Group,File, andAttributeManagerclasses,
implementing their abstract methods.
BecauseFiles should subclass yourGroup, the base class here is a mixin.
It should come before theGroupin the MRO.Methods containing write operations should be given the@mutationdecorator.
This checks theirmodeattribute and raises an error if it is readonly.fromh5py_likeimportDatasetBase,GroupBase,AttributeManagerBase,FileMixin,mutationclassMyDataset(DatasetBase):# ... implement abstract methods@mutationdef__setitem__(self,idx,val):...classMyGroup(GroupBase):# ... implement abstract methodspassclassMyFile(FileMixin,MyGroup):# ... implement abstract methodspassclassMyAttributeManager(AttributeManagerBase):# ... implement abstract methodspassHelpersh5py_like.shape_utilscontains a variety of helper functions,
to simulate numpy's flexibility.TestingA suite of tests for basic h5py-like functionality is included.
To use it, you must be using pytest, and define a fixture which yields an instance of yourFileimplementation.
Then you just need to subclass the provided abstract test classes:conftest.pyimportpytest@pytest.fixturedeffile_():yieldMyFile("my_name")test_implementation.pyfromh5py_like.test_utilsimportFileTestBase,GroupTestBase,DatasetTestBase,ModeTestBase# concrete class names must start with TestclassTestFile(FileTestBase):passclassTestGroup(GroupTestBase):passclassTestDataset(DatasetTestBase):passclassTestMode(ModeTestBase):deffactory(self,mode):# Instantiate your File object in the given mode in a way which is repeatable within a method.returnMyFile(mode)If your dataset implementation supports chunking and threading, use theThreadedDatasetTestBasebase class instead.The provided base classes test some of the expected functionality, even if you don't write any methods in your test classes.
You can add more tests if you like, or override those you want to change, or decorate any you to skip or xfail.TheGroupTestBaseprovides agroup_nameattribute and aself.group(parent)method for creating a group of that name.TheDatasetTestBaseprovidesdataset_name,shape, anddtype, and aself.dataset(parent)method for making that dataset.NotesIf you only want to implement part of the h5py-like API, justraise NotImplementedError().
Then your classes are being explicit about what they do and don't support.
Package: h5pyp
h5pypA Python package that contains thePARALLEL HDF5shared librariesandtheh5pyPython bindings.The parallel version of the library will be compiled during the
installation process.h5pyphas been developed to be used inLARGE.Provides parallel (MPI enabled) h5pyh5pywill be installed or reinstalled.Install the packageInstall the latest stable version withpip3 install h5pypOr download the whole repository:git clone https://bitbucket.org/bvidmar/h5pypand thencd h5pypand install as preferred:python3 setup.py installpip3 installTest the installationTo test the installation runmpiexec -n 4 test_hdf5_parallelThis script will generate aparallel_test.hdf5in the current directory.
To show its content use theh5pyp_dumpcommand:h5pyp_dump parallel_test.hdf5the output should be:HDF5 "parallel_test.hdf5" {
 GROUP "/" {
    DATASET "test" {
       DATATYPE  H5T_STD_I32LE
       DATASPACE  SIMPLE { ( 4 ) / ( 4 ) }
       DATA {
       (0): 0, 1, 2, 3
       }
    }
 }
 }WARNINGThis isalphacode, use at your own risk.The AuthorRoberto VidmarThe mantainersRoberto VidmarNicola Creati
Package: h5pyViewer
This package contains classes and tools to view *.hdf5-Filesh5pyViewer.py: main viewer that contains the modules belowhdfTree.py:    Tree view of a hdf5-filehdfGrid.py:    Grid view (Excel-like) table view of a datasethdfImage.py:   image view of a datasethdfAttrib.py:  attribute view of a datasetfurther helper files are:  utilities.pyLocations:https://pypi.python.org/pypi/h5pyViewerhttps://github.com/ganymede42/h5pyViewerhttps://localshop.psi.ch/packages/h5pyViewer/Screenshots
Package: h5record
H5RecordLarge dataset ( > 100G, <= 1T) storage format for Pytorch (wip)Why?Writing large dataset is still a wild west in pytorch. Approaches seen in the wild include:large directory with lots of small files : slow IO when complex file is fetched, deserialized frequentlydatabase approach : depend on what kind of database engine used, usually multi-process read is not supportedthe above method scale non linear in terms of data - storage sizeTFRecord solved the above problems well ( multiprocess fetch, (de)compression ), fast serialization ( protobuf )However TFRecord port does not support data size evaluation (used frequently by Dataloader ), no index level access available ( important for data evaluation or verification )H5Record aim to tackle TFRecord problems by compressing the dataset intoHDF5file with an easy to use interface through predefined interfaces ( String, Image, Sequences, Integer).Some advantage of using H5RecordSupport multi-process readRelatively simple to use and low technical debtSupport compression/de-compression on the flyQuick load to memory if requiredSimple usagepip install h5recordSentence Similarityfromh5recordimportH5Dataset,Float,Stringschema=(String(name='sentence1'),String(name='sentence2'),Float(name='label'))data=[['Sent 1.','Sent 2',0.1],['Sent 3','Sent 4',0.2],]defpair_iter():forrowindata:yield{'sentence1':row[0],'sentence2':row[1],'label':row[2]}dataset=H5Dataset(schema,'./question_pair.h5',pair_iter())foridxinrange(len(dataset)):print(dataset[idx])NoteDue to in progress development, this package should be use in care in storage with FAT, FAT-32 formatComparison between different compression algorithmNo chunking is usedCompression TypeFile sizeRead speed row/secondno compression2.0G2084.55 it/slzf1.7G1496.14 it/sgzip1.1G843.78 it/sbenchmarked in i7-9700, 1TB NVMe SSDIf you are interested to learn more feel free to checkout thenoteas well!
Package: h5serv
Introductionh5serv is a web service that implements a REST-based web service for HDF5 data stores
as described in the paper:http://hdfgroup.org/pubs/papers/RESTful_HDF5.pdf.WebsitesMain website:http://www.hdfgroup.orgSource code:https://github.com/HDFGroup/h5servMailing list:hdf-forum@lists.hdfgroup.org<hdf-forum@lists.hdfgroup.org>Documentation:http://h5serv.readthedocs.orgQuick InstallInstall Python (2.7 or later) and the following packages:NumPy 1.10.4 or laterh5py 2.5 or latertornado 4.0.2 or laterwatchdog 0.8.3 or laterrequests 2.3 or later (for client tests)Clone the hdf5-json project:git clonehttps://github.com/HDFGroup/hdf5-json.git.
Next, cd to the hdf5-json folder and run:python setup.py install.Clone this project:git clonehttps://github.com/HDFGroup/h5serv.git.Running the ServerStart the server:cd h5serv/server; python app.py.By default the server will listen on port 5000.  The port and and several other defaults can be modified
with command line options.  For example to use port 8888 run:python app.py--port=8888.See test cases for examples of interacting with the server.  Run:python testall.pyfrom the test directory
to run through the entire test suite.Also, the interface (at least as far as read requests) can be explored in a browser. Go to:http://127.0.0.1:5000/.
A JSON browser plugin will be helpful for formatting responses from the server to be more human readable.See h5serv/docs/Installation.rst for step by step install instructions.Running with DockerTo run h5serv as a docker container you just need to install Docker (no Python, h5py, etc. needed).Install docker:https://docs.docker.com/installation/#installation.Run the h5serv image:docker run-p5000:5000-d-v<mydata>:/datahdfgroup/h5servwhere <mydata> is the folder path that contains any HDF5 files you want to made available through the h5serv REST API.  Since requests to the server can modify (or delete!) content, you probably want to create a new folder and copy files to it.Go tohttp://192.168.99.100:5000/in your browser to verify the server is up and running (replace 192.168.99.100 with the IP address of the system or VM that is running the container).Writing Client ApplicationsAs a REST service, clients be developed using almost any programming language.  The
test programs under: h5serv/test/integ illustrate some of the methods for peforming
different operations using Python.The related project:https://github.com/HDFGroup/h5pydprovides a (mostly) h5py-compatible
interface to the server for Python clients.Uninstallingh5serv does not modify anything in the system outside the directory where it was
installed, so just remove the install directory and all contents to uninstall.Reporting bugs (and general feedback)Create new issues athttp://github.com/HDFGroup/h5serv/issuesfor any problems you find.For general questions/feedback, please use the list (hdf-forum@lists.hdfgroup.org).
Package: h5sh
Fast shell-like tool for interacting with HDF5 files.This tool is meant to replace repeated calls toh5lsandh5dump, each
of which can take substantial amounts of time for large files. In contrast,
this utility only has to open a file once, doesn’t have to read all the file
contents in order to browse or view attributes, and uses HDF5’s internal
cacheing for speed.Theh5shtool uses a shell-like interface so that analysts will find the
interface familiar.Free software: BSD licenseDocumentation:https://h5sh.readthedocs.io.FeaturesTab-completion of commands, dataset names, and group namesDump datasets to screen or disk with thedumpcommandBrowse groups withcdand view attributes withattrLicenseBSD license, see LICENSE.History0.1.1 (2019-12-05)Fix some dependencies0.1.0 (2019-12-05)Complete initial capability, documents, and testing.0.0.1 (2018-12-07)Created cookiecutter package.
Package: h5sparse
Please visit  theGithub repositoryfor more information.h5sparseScipy sparse matrix in HDF5.Installationpipinstallh5sparseTestingfor single environment:pythonsetup.pytestfor all environments:toxExamplesCreate datasetIn[1]:importscipy.sparseasss...:importh5sparse...:importnumpyasnp...:In[2]:sparse_matrix=ss.csr_matrix([[0,1,0],...:[0,0,1],...:[0,0,0],...:[1,1,0]],...:dtype=np.float64)In[3]:# create dataset from scipy sparse matrix...:withh5sparse.File("test.h5")ash5f:...:h5f.create_dataset('sparse/matrix',data=sparse_matrix)In[4]:# you can also create dataset from another dataset...:withh5sparse.File("test.h5")ash5f:...:h5f.create_dataset('sparse/matrix2',data=h5f['sparse/matrix'])In[5]:# you can also create dataset using the formats that original h5py accepts...:withh5sparse.File("test.h5")ash5f:...:h5f.create_dataset('sparse/matrix3',data=[1,2,3])Read datasetIn[6]:h5f=h5sparse.File("test.h5")In[7]:h5f['sparse/matrix'][1:3]Out[7]:<2x3sparsematrixoftype'<class 'numpy.float64'>'with1storedelementsinCompressedSparseRowformat>In[8]:h5f['sparse/matrix'][1:3].toarray()Out[8]:array([[0.,0.,1.],[0.,0.,0.]])In[9]:h5f['sparse']['matrix'][1:3].toarray()Out[9]:array([[0.,0.,1.],[0.,0.,0.]])In[10]:h5f['sparse']['matrix'][2:].toarray()Out[10]:array([[0.,0.,0.],[1.,1.,0.]])In[11]:h5f['sparse']['matrix'][:2].toarray()Out[11]:array([[0.,1.,0.],[0.,0.,1.]])In[12]:h5f['sparse']['matrix'][-2:].toarray()Out[12]:array([[0.,0.,0.],[1.,1.,0.]])In[13]:h5f['sparse']['matrix'][:-2].toarray()Out[13]:array([[0.,1.,0.],[0.,0.,1.]])In[14]:h5f['sparse']['matrix'][()].toarray()Out[14]:array([[0.,1.,0.],[0.,0.,1.],[0.,0.,0.],[1.,1.,0.]])In[15]:importh5pyIn[16]:h5py_h5f=h5py.File("test.h5")In[17]:h5sparse.Group(h5py_h5f.id)['sparse/matrix'][()]Out[17]:<4x3sparsematrixoftype'<class 'numpy.float64'>'with4storedelementsinCompressedSparseRowformat>In[18]:h5sparse.Group(h5py_h5f['sparse'].id)['matrix'][()]Out[18]:<4x3sparsematrixoftype'<class 'numpy.float64'>'with4storedelementsinCompressedSparseRowformat>In[19]:h5sparse.Dataset(h5py_h5f['sparse/matrix'])[()]Out[19]:<4x3sparsematrixoftype'<class 'numpy.float64'>'with4storedelementsinCompressedSparseRowformat>Append datasetIn[20]:to_append=ss.csr_matrix([[0,1,1],...:[1,0,0]],...:dtype=np.float64)In[21]:h5f.create_dataset('matrix',data=sparse_matrix,chunks=(100000,),...:maxshape=(None,))In[22]:h5f['matrix'].append(to_append)In[23]:h5f['matrix'][()]Out[23]:<6x3sparsematrixoftype'<class 'numpy.float64'>'with7storedelementsinCompressedSparseRowformat>In[24]:h5f['matrix'][()].toarray()Out[24]:array([[0.,1.,0.],[0.,0.,1.],[0.,0.,0.],[1.,1.,0.],[0.,1.,1.],[1.,0.,0.]])