Saravanan666/pypi-org huggingface dataset:
Package: json-store-client
json-store-clientA Python client forjsonstore.ioInstallationpipinstalljson-store-clientAn optional installation ofujsonis recommended for faster json processing.Also installingcchardetandaiodnsare recommended by aiohttp for faster performance.UsageDemo of storing a Python json-friendly object with json-store-client in async onrepl.it.importjson_store_clientjsonStoreToken='...'# Insert your token here.client=json_store_client.AsyncClient(jsonStoreToken)asyncdefdemo_function():# Save data to the 'foo' key.awaitclient.store('foo',{'alfa':'bravo','charlie':'delta'})# Save data with dict mappingawaitclient.save_multiple({'foo':{'alfa':'bravo','charlie':'delta'}})# Get the data from the 'foo' key.data=awaitclient.retrieve('foo')print(data['alfa'])# => 'bravo'# Deletes the data after printing parts of it.awaitclient.delete('foo')json-store-client APIjson_store_client.Client(token)json_store_client.AsyncClient(token)Returns the client to use for data operations.token (str): The API token fromjsonstore.ioclient.store(key, data[, timeout])await client.store(key, data[, timeout])Storing data in jsonstore with a keykey (str): The key to be stored on jsonstoredata (any): The data to be stored under the key. It can be any Python objects.timeout (int): The timeout for the http request. Default 5 secondsclient.store_multiple(data[, timeout])await client.store_multiple(data[, timeout])Storing data in jsonstore with a dict mappingdata (dict):  A dict of {key(s):value(s)} to be updated.timeout (int): The timeout for the http request. Default 5 secondsclient.retrieve(key[, timeout])await clent.retrieve(key[, timeout])Retrieve data in jsonstore with a keyIf nothing is saved under the key, it will return None.key (str): The key to get on jsonstoretimeout (int): The timeout for the http request. Default 5 secondsclient.delete(key[, timeout])await client.delete(key[, timeout])Delete data in jsonstore with a keykey (str): The key to get on jsonstoretimeout (int): The timeout for the http request. Default 5 seconds
Package: jsonstream
jsonstreamLoad multiple delimited JSON documents from a single string or file-like object.This allows for storing multiple object roots to be stored in a single document,
but without having to load all the object roots from the document at once.jsonstreamdoes not even require that you have the entire document
immediately stored in memory. This allows you to stream objects from files
which are too large to store in memory, or from network connections.Basic UsageRetrieve objects from the document one at a time, or all at once.fromjsonstreamimportloads>>>doc='[1, 2, 3] {"some": "object"}\nnull'>>>it=loads(doc)>>>next(it)[1,2,3]>>>list(it)[{"some":"object"},None]Using file-like objects when the whole document is not immediately available>>>fromjsonstreamimportload>>>fromioimportStringIO>>>fh=StringIO('["first"] ["second"]')>>>it=load(fh)>>>next(it)["first"]>>>fh.write('["and", "a", "third"]')>>>list(it)[["second"],["and","a","third"]]Further documentation can be found on the doc strings ofloadsandload.
Package: json-streamer
JSON StreamerJSON Streamer is an opportunistic parser designed for processing incomplete JSON strings in real-time from streams.This can be very useful for parsing JSON strings from network streams, such as websockets and don't want to wait for
the entire JSON string to be received before acting on it.A good use-case is for parsing OpenAI's GPT response with streaming completion enabled or even with functions enabled.
(This is actually a real use-case! see howopenai-streaminguse this
package just for that)InstallationInstall the package using pip:pipinstalljson-streamerFeaturesReal-time Parsing: Process incomplete JSON strings as they are streamed.Flexibility: Built on a generator-based architecture for ease of integration.Partial Object Handling: Yields partially parsed JSON objects to be acted upon immediately.UsageBasic Usagefromjson_streamerimportloads# Define a stream of JSON datadefstream_json():json_string='{"field1": "value1", "field2": 42}'chunk_size=4foriinrange(0,len(json_string),chunk_size):yieldjson_string[i:i+chunk_size]# Create a stream generatormystream=stream_json()# Use the 'loads()' function to parse the streamret=loads(mystream)# Iterate through the generator to get parsed JSON objectsforsinret:print(s)# Output will be tuples of (ParseState, JSON object)Theloads()function returns a generator that yields tuples of(ParseState, JSON object).Using with Generator receiversUtilize the parser with a generator receiver for additional control:fromjson_streamerimportloads# Initialize the parser generatorparser=loads()next(parser)# Stream and send chunks to the parserforchunkinstream_json():recv=parser.send(chunk)print(recv)# Output will be tuples of (ParseState, JSON object)How it WorksThe parser internally maintains a stack of opening symbols like brackets, braces, and quotes. Upon each new data chunk,
it attempts to match these with closing symbols to parse available JSON objects.LicenseThis project is licensed under the terms of the MIT license.
Package: json-stream-generator
json-stream-generator - serialize json in generatorjson-stream-generatorallows you to serialize object to JSON string and start the output immediately, without waiting for serialization to complete.It was designed to be used with Django'sStreamingHttpResponseor similar concept in other web frameworks to allow sending huge json blobs to the client without triggering load balancer's timeout:fromjson_stream_generatorimportjson_generatorfromdjango.httpimportStreamingHttpResponsedefmy_view(request):# NOTE: No Content-Length header!returnStreamingHttpResponse(json_generator((numfornuminrange(100_000_000))),content_type="application/json",headers={"Content-Disposition":'attachment; filename="somefilename.json"'},)json-stream-generatorcomes with built-in support forDjango Rest Framework:fromjson_stream_generator.rest_framework.mixinsimportStreamingListModelMixinfromrest_frameworkimportviewsetsclassDemoViewSet(StreamingListModelMixin,viewsets.ModelViewSet):queryset=MyModel.objects.all()serializer_class=MySerializer
Package: json-stream-parser
Parsing a stream of json objects without loading the whole stream into memory.
Solution tohttps://stackoverflow.com/questions/6886283/how-i-can-i-lazily-read-multiple-json-values-from-a-file-stream-in-pythonimportsysfromjson_stream_parserimportload_iterforobjinload_iter(sys.stdin):print(obj)
Package: json-stream-reader
================Json Stream Reader================.. image:: https://img.shields.io/pypi/v/json_stream_reader.svg:target: https://pypi.python.org/pypi/json_stream_reader.. image:: https://img.shields.io/travis/matt2000/json_stream_reader.svg:target: https://travis-ci.org/matt2000/json_stream_reader.. image:: https://readthedocs.org/projects/json-stream-reader/badge/?version=latest:target: https://json-stream-reader.readthedocs.io/en/latest/?badge=latest:alt: Documentation Status.. image:: https://pyup.io/repos/github/matt2000/json_stream_reader/shield.svg:target: https://pyup.io/repos/github/matt2000/json_stream_reader/:alt: UpdatesGiven a byte stream of {}-bracketed objects, generate one object-string at a time.* Free software: GNU General Public License v3* Documentation: https://json-stream-reader.readthedocs.io.Features--------* Turns a stream of unseperated {}-bracketed character (byte) strings, such asJSON serialized objects, into a collection of individual (byte)strings.Credits-------This package was created with Cookiecutter_ and the `audreyr/cookiecutter-pypackage`_ project template... _Cookiecutter: https://github.com/audreyr/cookiecutter.. _`audreyr/cookiecutter-pypackage`: https://github.com/audreyr/cookiecutter-pypackage=======History=======0.1.0 (2018-10-14)------------------* First release on PyPI.
Package: jsonstreams
Source code is available atgithub.The code is licensed MIT. See the included LICENSE file for the exact terms.DescriptionJSONstreams is a package that attempts to making writing JSON in a streaming
format easier. In contrast to the core json module, this package doesn’t
require building a complete tree of dicts and lists before writing, instead it
provides a straightforward way to to write a JSON documentwithoutbuilding
the whole data structure ahead of time.JSONstreams considers there to be two basic types, the JSON array and the JSON
object, which correspond to Python’s list and dict respectively, and can encode
any types that the json.JSONEncoder can, or can use an subclass to handle
additional types.The interface is designed to be context manger centric. The Stream class, and
the Array and Object classes returned by the subarray and subobject methods
(respectively), can be used as context managers or not, but use as context
managers are recommended to ensure that each container is closed properly.Basic UsageA simple example looks like thisimportjsonstreamswithjsonstreams.Stream(jsonstreams.Type.OBJECT,filename='foo')ass:s.write('foo','bar')withs.subobject('a')asa:a.write('foo',1)a.write('bar',2)s.write('bar','foo')Writing into a closed group will raise an exception, which should not be
handled, this exception is always an error in programming and should be
corrected.It is possible to write any value that the encoder (json.JSONEncoder by
default) can encode, so iterating over lists or dictionaries to write them in
is unnecessary:importjsonstreamsmylist=list(range(10))mydict={a:bforainrange(10)forbin'abcdefghij'}withjsonstreams.Stream(jsonstreams.Type.OBJECT,filename='foo')ass:s.write('list',mylist)s.write('dict',mydict)
Package: json-streams
json-streamsRead and write JSON lazy, especially json-arrays.Handles both the JSON format:[{"a":1},{"a":2}]As well as JSON LINES format:{"a":1}{"a":2}Also supports streaming from gzipped files.Usesorjsonif present, otherwise standardjson.UsageInstallation# Using standard jsonpipinstalljson-streams# Using orjsonpipinstalljson-streams[orjson]NoteThis library prefers files opened in binary mode.
Therefore does alldumps-methods returnbytes.Allloadsmethods handlesstr,bytesandbytesarrayarguments.ExamplesAllows you to usejson.loadandjson.dumpwith
both json and json-lines files as well as dumping generators.importjson_streams# This command tries to guess format and opens the filedata=json_streams.load_from_file("data.json")# or data.jsonl# Write to file, again guessing formatjson_streams.dump_to_file(data,"data.jsonl")fromjson_streamsimportjson_iter,jsonl_iter# Open and read the file without guessingdata=json_iter.load_from_file("data.json")# Process file# Write to file without guessingjsonl_iter.dump_to_file(data,"data.jsonl")importjson_streamsdefprocess(data):forentryindata:# processyieldentrydefread_process_and_write(filename_in,filename_out):json_streams.dump_to_file(process(json_streams.load_from_file(filename_in)),filename_out)You can also use json_streams as a sink, that you can send data to.importjson_streamswithopen("out.json","bw")asfp:# guessing formatwithjson_streams.sink(fp)assink:fordataindata_source():sink.send(data)Release NotesThis projectskeeps a CHANGELOG.DevelopmentAfter cloning the repo, just run$ make dev
$ make testto setup a virtual environment,
install dev dependencies
and run the unit tests.Note:If you run the command in a activated virtual environment,
that environment is used instead.DeploymentPush a tag in the formatv\d+.\d+.\d+tomain-branch, to build & publish package to PyPi.
Package: json-stream-to-standard-types
⚠NOTE:Starting at json-stream 2.2.0,to_standard_typesis part of
json-stream itself, so there
is no longer a need for this packgage.json-stream-to-standard-typesUtility function to convertjson-streamobjects to normal Python dicts/lists.Parallel PR:https://github.com/daggaz/json-stream/pull/17Installationpipinstalljson-stream-to-standard-typesUsageTo convert a json-streamdict-like orlist-like object and all its
descendants to a standardlistanddict, simply apply the library'sto_standard_typesfunction:importjson_streamfromjson_stream_to_standard_typesimportto_standard_types# JSON: {"round": 1, "results": [1, 2, 3]}data=json_stream.load(f)results=data["results"]print(results)# prints <TransientStreamingJSONList: TRANSIENT, STREAMING>converted=to_standard_types(results)print(converted)# prints [1, 2, 3]LicenseDo whatever you want with it license or whatever it was called
Package: jsonstruct
jsonstruct is a library for two way conversion of typed Python object and JSON. This project is originally a fork of jsonpickle. The key difference between this library and jsonpickle is that during deserialization, jsonpickle requires Python types to be recorded as part of the JSON. This library intends to remove this requirement, instead, requires a class to be passed in as an argument so that its definition can be inspected. It will then return an instance of the given class. This approach is similar to how Jackson (of Java) works.
Package: json-structure-digest
Consistent message digest for JSON objectThis module is a python module that creates structured digests based on a given JSON data for consistent message digest calculation. The procedure of digest calculation is described below.This repository also includesJavascript version.Environment and installversion3.5 or laterinstall the modulepipinstalljson-structure-digestcommand line tooljson_digest.pyUsageA basic usage is like as follows:importjsonfromjsondigestimportdigestobj={"digest_version":1,"key1":1,"key2":2.34,"key3":"VALUE3","key4":{"key4-1":2,"key4-2":[1,2,3,False,"xyz"]},"key5":["VALUE5",5.55,True,["VALUE5-2",None],{"key5-2":123}]}json_string=json.dumps(obj)result=digest(json_string)sha256_hex=result["digest"]digest_structure=result["digest_structure"]In the case that you have the abovedigest_structureand partial object, you can also calculate the digest using these two information and will obtain the same result as above.obj_partial={"digest_version":1,"key1":1,"key2":2.34,"key3":"VALUE3",}json_string_partial=json.dumps(obj_partial)result2=digest(json_string_partial,digest_structure=json.dumps(digest_structure))sha256_hex2=result2["digest"]digest_structure2=result2["digest_structure"]sha256_hex and sha256_hex2, digest_structure and digest_structure2  are identical, respectively. Note that the digest method accepts string params only.Command line tooljson_digest.pyis available after pip install. The usage is very simple.$json_digest.py-jpath_to_jsonfileor$catpath_to_jsonfile|json_digest.pyYou will get a json string of the digest and the digest structure of the given json file.RequirementsJSON string must comply withRFC8259.The JSON string must be encoded with UTF-8 and no BOM.Supported types of value are string, number (integer, float), object, array, boolean and null."digest_version" must be included in the JSON.Spec of version 1Summarydigest_version: 1A digest is calculated for each value in a JSON object.The digest function in this version is SHA256.To achieve consistency, the keys at each level in the JSON object are sorted in increasing order of the ASCII value in the digest calculation procedure.The following conversions before digest calculation are performed:An integer value is converted in 8-byte array in little-endian.A float value is converted in 8-byte array in little-endian.A boolean value is converted to either "true" or "false".A null value is converted to "null".All digests are expressed in HEX string in little endian.Seeherein detail.LicenseMIT
Package: jsonsubschema
jsonsubschemajsonsubschemachecks if one JSON schema is a subschema (subtype) of another.For any two JSON schemas s1 and s2, s1 <: s2 (reads s1 is subschema/subtype of s2)
if every JSON document instance that validates against s1 also validates against s2.jsonsubschema is very useful in analysing schema evolution and ensuring that newer schema versions are backward compatible.
jsonsubschema also enables static type checking on different components of a system that uses JSON schema to describe data
interfaces among the system's different components.The details of JSON subschema are covered in ourISSTA 2021paper,
which received aDistinguished Artifact Award:@InProceedings{issta21JSONsubschema,
  author    = {Habib, Andrew and Shinnar, Avraham and Hirzel, Martin and Pradel, Michael},
  title     = {Finding Data Compatibility Bugs with JSON Subschema Checking},
  booktitle = {The ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA)},
  year      = {2021},
  pages     = {620--632},
  url       = {https://doi.org/10.1145/3460319.3464796},
}I) Obtaining the toolRequirementspython 3.8.*Other python dependencies will be installed during the below setup processYou can either install subschema from the source code from github or the pypy package.A) Install from github source codeExecute the following:git clone https://github.com/IBM/jsonsubschema.git 
cd jsonsubschema
python setup.py install
cd ..B) Install from pypyExecute the following:pip install jsonsubschemaII) Running  subschemaJSON subschema provides two usage interfaces:A) CLI interfaceCreate two JSON schema examples by executing the following:echo '{"type": ["null", "string"]}' > s1.json
echo '{"type": ["string", "null"], "not": {"enum": [""]}}' > s2.jsonInvoke the CLI by executing:python -m jsonsubschema.cli s2.json s1.jsonB) python APIfrom jsonsubschema import isSubschema

def main():
	s1 = {'type': "integer"}
	s2 = {'type': ["integer", "string"]}

	print(f'LHS <: RHS {isSubschema(s1, s2)}')

if __name__ == "__main__":
	main()Licensejsonsubschema is distributed under the terms of the Apache 2.0
License, seeLICENSE.txt.Contributionsjson-subschema is still at an early phase of development and we
welcome contributions. Contributors are expected to submit a
'Developer's Certificate of Origin', which can be found inDCO1.1.txt.
Package: jsontableschema-bigquery
Generate and load BigQuery tables based on JSON Table Schema
descriptors.Versionv0.3contains breaking changes:renamedStorage.tablestoStorage.bucketschangedStorage.readto read into memoryaddedStorage.iterto yield row by rowGetting StartedInstallationpipinstalljsontableschema-bigqueryStoragePackage implementsTabular
Storageinterface.To start using Google BigQuery service:Create a new project -linkCreate a service key -linkDownload json credentials and setGOOGLE_APPLICATION_CREDENTIALSenvironment variableWe can get storage this way:importioimportosimportjsonfromapiclient.discoveryimportbuildfromoauth2client.clientimportGoogleCredentialsfromjsontableschema_bigqueryimportStorageos.environ['GOOGLE_APPLICATION_CREDENTIALS']='.credentials.json'credentials=GoogleCredentials.get_application_default()service=build('bigquery','v2',credentials=credentials)project=json.load(io.open('.credentials.json',encoding='utf-8'))['project_id']storage=Storage(service,project,'dataset',prefix='prefix')Then we could interact with storage:storage.bucketsstorage.create('bucket',descriptor)storage.delete('bucket')storage.describe('bucket')# return descriptorstorage.iter('bucket')# yields rowsstorage.read('bucket')# return rowsstorage.write('bucket',rows)Mappingsschema.json -> bigquery table schema
data.csv -> bigquery talbe dataDriversDefault Google BigQuery client is used -docs.API ReferenceSnapshothttps://github.com/frictionlessdata/jsontableschema-py#snapshotDetailedDocstringsChangelogContributingPlease read the contribution guideline:How to ContributeThanks!
Package: jsontableschema-pandas
Generate and load Pandas data frames based on JSON Table Schema
descriptors.Versionv0.2contains breaking changes:removedStorage(prefix=)argument (was a stub)renamedStorage(tables=)toStorage(dataframes=)renamedStorage.tablestoStorage.bucketschangedStorage.readto read into memoryaddedStorage.iterto yield row by rowGetting StartedInstallation$ pip install datapackage
$ pip install jsontableschema-pandasExampleYou can easily load resources from a data package as Pandas data frames
by simply usingdatapackage.push_datapackagefunction:>>>importdatapackage>>>data_url='http://data.okfn.org/data/core/country-list/datapackage.json'>>>storage=datapackage.push_datapackage(data_url,'pandas')>>>storage.buckets['data___data']>>>type(storage['data___data'])<class'pandas.core.frame.DataFrame'>>>>storage['data___data'].head()NameCode0AfghanistanAF1ÅlandIslandsAX2AlbaniaAL3AlgeriaDZ4AmericanSamoaASAlso it is possible to pull your existing data frame into a data
package:>>>datapackage.pull_datapackage('/tmp/datapackage.json','country_list','pandas',tables={...'data':storage['data___data'],...})StorageStoragePackage implementsTabular
Storageinterface.We can get storage this way:>>>fromjsontableschema_pandasimportStorage>>>storage=Storage()Storage works as a container for Pandas data frames. You can define new
data frame inside storage usingstorage.createmethod:>>>storage.create('data',{...'primaryKey':'id',...'fields':[...{'name':'id','type':'integer'},...{'name':'comment','type':'string'},...]...})>>>storage.buckets['data']>>>storage['data'].shape(0,0)Usestorage.writeto populate data frame with data:>>>storage.write('data',[(1,'a'),(2,'b')])>>>storage['data']idcomment1a2bAlso you can usetabulatorto
populate data frame from external data file:>>>importtabulator>>>withtabulator.Stream('data/comments.csv',headers=1)asstream:...storage.write('data',stream)>>>storage['data']idcomment1a2b1goodAs you see, subsequent writes simply appends new data on top of existing
ones.API ReferenceSnapshothttps://github.com/frictionlessdata/jsontableschema-py#snapshotDetailedDocstringsChangelogContributingPlease read the contribution guideline:How to ContributeThanks!
Package: jsontableschema-sql
Generate and load SQL tables based on JSON Table Schema descriptors.Versionv0.3contains breaking changes:renamedStorage.tablestoStorage.bucketschangedStorage.readto read into memoryaddedStorage.iterto yield row by rowGetting StartedInstallationpipinstalljsontableschema-sqlStoragePackage implementsTabular
Storageinterface.SQLAlchemy is used as sql wrapper. We can get storage this way:fromsqlalchemyimportcreate_enginefromjsontableschema_sqlimportStorageengine=create_engine('sqlite:///:memory:',prefix='prefix')storage=Storage(engine)Then we could interact with storage:storage.bucketsstorage.create('bucket',descriptor)storage.delete('bucket')storage.describe('bucket')# return descriptorstorage.iter('bucket')# yield rowsstorage.read('bucket')# return rowsstorage.write('bucket',rows)Mappingsschema.json -> SQL table schema
data.csv -> SQL talbe dataDriversSQLAlchemy is used -docs.API ReferenceSnapshothttps://github.com/frictionlessdata/jsontableschema-py#snapshotDetailedDocstringsChangelogContributingPlease read the contribution guideline:How to ContributeThanks!
Package: jsontas
JSONTas is a tool for generating dynamic JSON data.DescriptionJSONTas adds conditionals and logic to JSON files in order to create dynamic JSON data depending on which dataset you supply.It opens up the possibility to create generic tools where most of the operations are done by executing JSONTas on the data.Documentation:https://jsontas.readthedocs.io/en/latestFeaturesSimple yet powerful syntax.HTTP requests on parseSeparation of environments by providing different datasets.InstallationInstall the project by running:pip install jsontasExamplesFirst we create two datasets. One for our ‘dev’ environment and one for our ‘prod’ environment.Dataset ‘dev.json’{"mode":"dev","database":"dev_db"}Dataset ‘prod.json’{"mode":"prod","database":"prod_db"}JSONTas JSON fileNext up, let’s create our JSONTas file.{"database":{"host":"myawesomedb.example.com","database":"$database"},"message":{"$condition":{"if":{"key":"$mode","operator":"$eq","value":"dev"},"then":"This is the DEV server.","else":"This is the PROD server."}}}JSONTas execute with ‘dev’ datasetjsontas-ddev.jsondata.json{"database":{"host":"myawesomedb.example.com","database":"dev_db"},"message":"This is the DEV server."}JSONTas execute with ‘prod’ datasetjsontas-dprod.jsondata.json{"database":{"host":"myawesomedb.example.com","database":"prod_db"},"message":"This is the PROD server."}These examples only show the bare minimum.
For more examples look at our documentation at:https://jsontas.readthedocs.io/en/latestContributeIssue Tracker:https://github.com/AxisCommunications/jsontas/issuesSource Code:https://github.com/AxisCommunications/jsontasSupportIf you are having issues, please let us know.
Emailtobias.persson@axis.comor just write an issue.