Saravanan666/pypi-org huggingface dataset:
Package: json-linter
json-linterLint your JSON files!FeaturesCheck if keys are sorted alphabetically...Check if your keys are either in camelCase, snake_case or kebab-case......and automatically fix (some) of these issues!InstallYou need to have Python 3.10+ installed.$pipinstalljson-linterUsage# Lint a single file...$json-lintermy-file.json# Lint multiple files...$json-lintera.jsonb.json# Lint all files in a directory$json-linterfiles# Lint all files in a directory recursively$json-linterfiles--recursive# Lint all files in a directory recursively with .config and .cfg ending$json-linterfiles-r--extensionscfgconfig# Fix files (will overwrite the file with a fixed version and then lint)$json-lintermy-file.json--fix# Overwrite configuration values$json-lintermy-file.json--config-setnaming_style=KEBAB_CASEindent=2# Return results as json$json-lintermy-file.json--jsonUse as a packagefrompathlibimportPathfromjson_linterimportlint_fileresults=lint_file(Path("./my-file.json"))forresultinresults:ifnotresult.was_successful:print(result.path,"failed!")LicenseGNU General Public License v3
Package: json-lint-tools
json-lint-toolsProvides linting, diffing and formatting JSON. Diffing and linting is done by comparing an input (file or stdin) to what would be produced by
loading the string withjson.load(data)then dumping it again withjson.dumps(data)with options to order keys, order items in arrays, and
specify the expected indentation. The CLI tool can optionally update the original file with the pretty formatted JSON output.This package is primarily intended as a CLI tool for linting and formatting JSON. Thereis however documentation for the APIHEREInstallationpipinstalljson-lint-toolsCLI Usageusage: jsonfmt [-h] [--check] [-d] [-f] [-i INDENT] [--no-color] [-r] [-s] [--sort-arrays] [--sort-keys] [--stdin] [path ...]

positional arguments:
  path                  Path to a single file or directory of JSON files

options:
  -h, --help            show this help message and exit
  --check               Only check if the files would be changed. Show a list of any files that would be changed and exit with an error if any would be
  -d, --diff            Show a diff
  -f, --format          Format non-conforming files
  -i INDENT, --indent INDENT
                        How many spaces are expected per indentation
  --no-color            Don't print colors to stdout
  -r, --recursive       Search recursively for JSON files
  -s, --sort            Sort all objects by key and all arrays alphabetically. Shorthand for --sort-keys + --sort-arrays
  --sort-arrays         Sort arrays alphabetically
  --sort-keys           Sort keys in JSON objects alphabetically
  --stdin               Read data from stdin instead of a fileExamplesGetting a pretty diff:>jsonfmt-r-dtests/jsonfilesFile:tests/jsonfiles/test.json
---
+++
@@-6,9+6,9@@],"baz":{"myarry":[-"b",
-"d",
-"a"+"b",
+"d",
+"a"]}}File:tests/jsonfiles/root_array.json
---
+++
@@-1+1,8@@
-["a","d","b",{"foo":"bar"}]+[+"a",
+"d",
+"b",
+{+"foo":"bar"+}+]File:tests/jsonfiles/one_line_obj.json
---
+++
@@-1+1,4@@
-{"foo":"bar","bar":"baz"}+{+"foo":"bar",
+"bar":"baz"+}Formatting stdin:>echo'{"foo": "bar"}'|jsonfmt{"foo":"bar"}
Package: json-lisfy
python-json-lisfyUsage$poetryinstall
$poetryrunjson-lisfyExample$poetryrunjson-lisfy
json-lisfy>1(intnil1)json-lisfy>2.4(floatnil2.4)json-lisfy>false(symbolnil"false")json-lisfy>{"a":1,"b":2}(objectnil(itemnil(strnil"a")(intnil1))(itemnil(strnil"b")(intnil2)))json-lisfy>[1,2,"asdf",true](arraynil(intnil1)(intnil2)(strnil"asdf")(symbolnil"true"))
Package: jsonllm
jsonllmTools for working with LLMs on JSON dataUsage|Installation|Why|HowUsageUsage:jsonllm[OPTIONS]COMMAND[ARGS]...ToolsforworkingwithLLMsonJSONdata

Options:--versionShowtheversionandexit.--helpShowthismessageandexit.

Commands:embedTurnaJSONofcontentintoaJSONofembeddings.Usage:jsonllmembed[OPTIONS]TurnaJSONofcontentintoaJSONofembeddings.

Options:-i,--inputPATHFiletoembed-m,--modelTEXTEmbeddingmodel(s)touseIssue`llmembed-modelslist`tolistavailablemodels.Currentlyinstalledare:['3-large','3-large-1024','3-large-256','3-small','3-small-512','ada-002','clip','jina-embeddings-v2-base-en','jina-embeddings-v2-large-en','jina-embeddings-v2-small-en','onnx-bge-base','onnx-bge-large','onnx-bge-micro','onnx-bge-small','onnx-gte-tiny','onnx-minilm-l12','onnx-minilm-l6','sentence-transformers/all-MiniLM-L6-v2']Youcaninstallmorevia`llminstall...`Youcanfindavailablemodelshere:https://llm.datasette.io/en/stable/plugins/directory.html#embedding-models-j,--jqTEXTEmbedonlythekeysthatsatisfythegivenjqfilterexpression--in-arraysEmbedtextappearinginarraystoo--helpShowthismessageandexit.CREATETABLEpeople(dataJSONB);pythontests/gen_people.py100|\jsonllmembed-mclip-j'.name'psql-c"\COPY people(data) FROM stdin"echo'{"hello": "world"}'|jsonllmembed-mclipInstallationpipinstalljsonllmAvailable ModelsAvailable embedding models
are those provided and installed via thellmpackage.llm-sentence-transformersadds support for embeddings using
thesentence-transformerslibrary, which provides
access toa wide rangeof
embedding models.llm-clipprovides
theCLIPmodel, which can be used to embed
images and text in the same vector space, enabling text search against images.
SeeBuild an image search engine with llm-clipfor more on this plugin.llm-embed-jinaprovides Jina
AI's8K text embedding models.llm-embed-onnxprovides seven
embedding models that can be executed using the ONNX model framework.llminstallllm-sentence-transformers
llminstallllm-clip
llminstallllm-embed-jina
llminstallllm-embed-onnxFor an up-to-date list
checkhereWhyThere are now plenty of tools providing ways of getting embeddings out of a
corpus of text.
Some even can generate embeddings from JSON documents,
but they treat JSON as simple text too.That is rarely the case though; JSON documents have structure and semantics
depending on their application in context.
Most importantly though it's data exchange format and a data aggregation tool.
Aggregation in the sense of getting data from A to B.In my case point A was a JSON object created by an SQL query from a Postgres
database, piped throughjsonllmand pushed into another Postgres instance
specifically designed for AI-related experiments.Howjsonllmtraverses a JSON object recursively,
and replaces text values with their embeddings array.Other data types are not modified at all and the overall
object structure is not changed.Developmentpipinstall-e'.[test]'pytest
Package: jsonloader
JSONloaderThis module is for you if you're tired of writing boilerplate that:builds a straightforward Python object from loaded JSON or similar dict-based
data loading (e.g. CSV).checks that your input-loaded-JSON has all necessary attributes for your pipeline.checks that your input JSON has the right types.ExamplesMain intended usage is through theJSONclassdecorator, example below:By default we don't check for anything, we just build the object
as we received it.>>>fromjsonloaderimportJSONclass>>>data={'a':'aa','b':'bb','c':1}>>>@JSONclass...classExample:...pass...>>>example=Example(data)>>>example.a'aa'>>>example.b'bb'We want to ensure we have annotated parameters>>>fromjsonloaderimportJSONclass>>>data={'a':'aa','b':'bb','c':1}>>>@JSONclass(annotations=True)...classExample:...a:str...d:int...>>>try:...example=Example(data)...exceptKeyError:...print("error - missing 'd'")...error-missing'd'>>>data['d']=1# Let's fix the missing data>>>example=Example(data)# No more error in loading.We want to ensure we haveonlyannotated parameters>>>fromjsonloaderimportJSONclass>>>data={'a':'aa','b':'bb','c':1}>>>@JSONclass(annotations=True,annotations_strict=True)...classExample:...a:str...b:int...>>>try:...example=Example(data)...exceptKeyError:...print("error - extra 'c'")...error-extra'c'>>>deldata['c']# Let's remove unwanted data>>>example=Example(data)# No more error in loading.We want to ensure we have only annotated parameters and they
are of annotated type.>>>fromjsonloaderimportJSONclass>>>data={'a':'aa','b':'bb'}>>>@JSONclass(annotations_strict=True,annotations_type=True)...classExample:...a:str...b:int...>>>try:...example=Example(data)...exceptTypeError:...print("error - b is not int")...error-bisnotintDefault values are supported too.>>>fromjsonloaderimportJSONclass>>>data={'a':'aa'}>>>@JSONclass(annotations_strict=True,annotations_type=True)...classExample:...a:str...b:int=1...>>>example=Example(data)>>>example.b1A JSONclass can be converted back to a dict, even for recursive
structures>>>fromjsonloaderimportJSONclass>>>data={'a':'aa','b':2,'c':{'foo':'bar'}}>>>@JSONclass(annotations_type=True,annotations=True)...classExample:...a:str...b:int...>>>example=Example(data)>>>assertdict(example)==dataInstallUser installation# Recommendation: install jsonloader in your project virtualenv
# Should you not want to use virtualenv or equivalent, it's recommended to use
# '--user' pip option to avoid a system-level install.
pip3 install jsonloaderDeveloper installationGithub repository currently points to latest development version. Please
jump to latest released version tag if you intend to work on PyPI version.
For examplegit checkout tags/v0.4.3.# These commands assume virtualenv is installed
python3 -m virtualenv venv
. venv/bin/activate

# Actually install the deps
pip3 install -e '.[dev]'

# To setup the project's git hooks:
git config --local core.hooksPath hookRun Tests# From this repository top directory
pytest --doctest-modulesTests coverageFor example, leveragecoveragemodule:coverage run -m pytest --doctest-modules
coverage html
Package: jsonlog-cli
jsonlog-cliA human readable formatter for JSON logs.It's built for use withjsonlogbut will work well with any log format that
uses line delimited JSON.UsageSeejsonlog --helpfor all options.Key-value modePass a file as the only argument tojsonlog, or read from STDIN by default.jsonlogkvdocs/example.logpythondocs/example.py|jsonlogkvcatdocs/example.log|jsonlogOnly show thetimestampandmessagefields:jsonlogkv--keytimestamp--keymessagedocs/example.logConfigure the keys of multiline values you want to display (can be specified
multiple times, and defaults to thetracebackkey.)jsonlogkv--keytimestamp--keymessage--multiline-keytracebackdocs/example.logConfigure the key to extract and use as the records level, controlling the
colour each line is printed in (defaults to thelevelkey).jsonlogkv--level-keylevel--keytimestamp--keymessage--multiline-keytracebackdocs/example.logTemplate modeOnly show thetimestampandmessagefields:jsonlogtemplate--format"{timestamp} {message}"docs/example.logAlso show a multiline key when it's present:jsonlogtemplate--format"{timestamp} {message}"--multiline-keytracebackdocs/example.logConfigurationNamed "patterns" are supported as a way of collecting a set of options for
jsonlog's key-value and template modes. If~/.config/jsonlog/config.jsonexists, it will be loaded at startup. All fields should be optional.The example configuration file below creates patterns namedbasicandcomprehensivefor the key-value and template modes. The patterns will each
show thetimestampandmessagefields of incoming logs. The patterns
namedcomprehensiveoverride all fields, setting their their default values.Creating a pattern nameddefaultwill set the default options used if no
pattern is specified. Command line options always override options from the
application's default configuration, the configuration file and the selected
pattern.{"keyvalues":{"basic":{"priority_keys":["timestamp","message"]},"comprehensive":{"level_key":"level","multiline_json":false,"multiline_keys":[],"priority_keys":[],"removed_keys":[]}},"templates":{"basic":{"format":"{timestamp} {message}"},"comprehensive":{"level_key":"level","multiline_json":false,"multiline_keys":[],"format":"{timestamp} {message}"}}}Themultiline_jsonoption will parse incoming data using a buffer. This is
rarely useful, but some applications (e.g. ElasticSearch) output JSON split
across multiple lines. Incoming data will be buffered until the whole buffer can
be parsed as JSON or a new line starts with{. Incoming lines that can be
immediately parsed as JSON are not buffered (flushing the buffer first).DebuggingThejsonlogCLI has some flags that are useful when debugging. The following
will print internal logs as JSON to STDERR.jsonlog --log-path=- --log-level=debug kv ...AuthorsSam Clements
Package: json-log-decorator
json-log-decoratorpython log decorator which raise on exception, logs stdout or to loki host, captures time taken by function or method in json formatTo install packagepip install json-log-decoratorExample usage with just decoratorNOTE: This takes file name as name while logging if no name providedfromjson_log_decorator.logdecoratorimportLogDecoratoraslogimporttime@log()deftest_decorator(x,y):time.sleep(2)returnx+ytest_decorator(1,2)log's as below{"name":"test","levelname":"INFO","asctime":"2021-06-22 23:32:18,564","message":"Function Called:[test_decorator]"}{"name":"test","levelname":"INFO","asctime":"2021-06-22 23:32:20,568","message":"Function Ended:[test_decorator] timetaken:[2] Seconds"}Incase of exception, it logs with error and raises the exception with adding additional field exc_info which has traceback rasie infofromjson_log_decorator.logdecoratorimportLogDecoratoraslogimporttime@log()deftest_decorator(x,y):time.sleep(2)returnx+ytest_decorator(1,'s')log's as below{"name":"test","levelname":"INFO","asctime":"2021-06-22 23:29:01,097","message":"Function Called:[test_decorator]"}{"name":"test","levelname":"ERROR","asctime":"2021-06-22 23:29:03,100","message":"Exception in test_decorator","exc_info":"Traceback (most recent call last):\n  File \"/json-log-decorator/json_log_decorator/logdecorator.py\", line 17, in wrapper\n    val = func(*args, **kwargs)\n  File \"/json-log-decorator/json_log_decorator/test.py\", line 7, in test_decorator\n    return x + y\nTypeError: unsupported operand type(s) for +: 'int' and 'str'"}To give a name to logger and for using this library to add other logging levels you can do this wayfromjson_log_decorator.logdecoratorimportLogDecorator,get_loggerlogger=get_logger('test-this')log=LogDecorator(logger)@logdeftest_decorator(x,y):logger.warning('to hava a warning message')returnx+y@logdefsum(x):returnx+2@logdefmain():x=sum(test_decorator(2,3))logger.info(x)main()Logs can directly pushed to LOKI host , in this case you need to pass loki_host as shown belowNOTE: this library only deals with version v1 loki python libraryIncase you want to try locally use below line to bring up grafana and loki locally , make sure you open grafana using admin and admin credentials and add datasource loki withhttp://loki:3100and save the data sourcewgethttps://raw.githubusercontent.com/grafana/loki/v2.2.1/production/docker-compose.yaml-Odocker-compose.yaml
docker-compose-fdocker-compose.yamlup-dfromjson_log_decorator.logdecoratorimportLogDecorator,get_loggerlogger=get_logger(name='test-this',loki_host='http://localhost:3100')log=LogDecorator(logger)@logdeftest_decorator(x,y):logger.warning('to hava a warning message')returnx+y@logdefsum(x):returnx+2@logdefmain():x=sum(test_decorator(2,3))logger.info(x)main()Can be seen in LOKI as belowTo set logging Levelimportloggingfromjson_log_decorator.logdecoratorimportLogDecorator,get_loggerlogger=get_logger('test-debug')logger.setLevel(logging.DEBUG)log=LogDecorator(logger)@logdeftest_decorator(x,y):logger.warning('just a warning test message')returnx+y@logdefsum(x):returnx+2@logdefmain():x=sum(test_decorator(2,3))logger.info(x)main()
Package: json-log-display
JSON log display (json-log-display)A CLI tool for presenting JSON logging data nicely on the CLI. If there is additional non JSON
data it will attempt to ignore that and use the valid JSON that is there.It also colours the output, can pre filter, post filter, pass through or ignore non JSON data
and supports an output format string.InstallationJSON log display requires Python 3.6 or higher.Install the pip package as follows:$ pip install json-log-displayUseOnce installed ajldCLI util should be available, and all you have to do is pipe logging
which has embedded JSON to it. For example:$ tail -f /some/log/file | jld
2022-10-29T16:56:11.646734D INFO some info
2022-10-29T16:56:11.693193D DEBUG some debug
2022-10-29T16:56:11.646734D INFO more info
2022-10-29T16:56:11.646734D INFO warning alert bad
2022-10-29T16:56:11.646734D ERROR really gone wrong nowThe lines will be coloured according to the level.To change the colour of a level you can either specify a CLI argument, for
example--col info=green, or set an environment variable, likeJLD_LEVEL_INFO_COL=green.To change the output format use--out_format, the default format string is${timestamp} ${level} ${message}". Each${xxxx}is a deference to a key in
the JSON objects logged.To change the field which is used to determine the log colour use--level_field. For
example--level_field loglevel.To hide non JSON data use--no_passthrough.To pre filter the input (only include lines matching a given regular expression) use--pre_filter REGEXand provide a regular expression.To filter the output based on the JSON data use--data_filter KEY=REGEX.By default JSON log display will attempt to skip junk to find valid JSON log data later
in the line. To disable this if this causes problems and your logging does not have a prefix
use--no_json_search.DevelopmentA nox config is provided and Makefile. Install nox if necessary and runnoxor
runmake, which will use your default Python version and create a virtual
environment.You can install development requirements in your current environment as follows:$ pip install '.[dev]'
Package: json-logging-py
JSON logging for PythonThis library provides Python logging formatters to output JSON, 2
formatters are specific for Logstash message format version 0 or 1.InstallationUsing pip:pip install json-logging-pyFrom source:python setup.py installUsageThe name of the library isjsonlogging, it provides 3 formatters:JSONFormatter{
    "tags": [
        "env=prod",
        "role=www"
    ],
    "timestamp": "2015-09-22T22:40:56.178715Z",
    "level": "ERROR",
    "host": "server-01.example.com",
    "path": "example.py",
    "message": "hello world!",
    "logger": "root"
}LogstashFormatterV0{
    "@source": "JSON://server-01.example.com/example.py",
    "@source_host": "server-01.example.com",
    "@message": "hello world!",
    "@tags": [
        "env=prod",
        "role=www"
    ],
    "@fields": {
        "logger": "root",
        "levelname": "ERROR"
    },
    "@timestamp": "2015-09-22T22:42:02.094525Z",
    "@source_path": "example.py",
    "@type": "JSON"
}LogstashFormatterV1{
    "host": "server-01.example.com",
    "logger": "root",
    "type": "JSON",
    "tags": [
        "env=prod",
        "role=www"
    ],
    "path": "example.py",
    "@timestamp": "2015-09-22T22:43:11.966558Z",
    "@version": 1,
    "message": "hello world!",
    "levelname": "ERROR"
}Python exampleimport logging
import jsonlogging


logger = logging.getLogger()

logHandler = logging.StreamHandler()

# You can also use LogstashFormatterV0 or LogstashFormatterV1
formatter = jsonlogging.JSONFormatter(
    hostname="server-01.example.com"
    tags=["env=prod", "role=www"],
    indent=4)
logHandler.setFormatter(formatter)

logger.addHandler(logHandler)

# You can pass additional tags
logger.error('hello world!', extra={"tags": ["hello=world"]})
Package: json-logic
This parser acceptsJsonLogicrules and
executes them in Python.This is a Python porting of the excellent GitHub project byjwadhamsfor JavaScript:json-logic-js.All credit goes to him, this is simply an implementation of the same
logic in Python (small differences below).The JsonLogic format is designed to allow you to share rules (logic)
between front-end and back-end code (regardless of language difference),
even to store logic along with a record in a database. JsonLogic is
documented extensively atJsonLogic.com,
including examples of everysupported
operationand a place totry
out rules in your browser.The same format can also be executed in PHP by the libraryjson-logic-phpExamplesSimplefromjson_logicimportjsonLogicjsonLogic({"==":[1,1]})# TrueThis is a simple test, equivalent to1 == 1. A few things about the
format:The operator is always in the “key” position. There is only one key
per JsonLogic rule.The values are typically an array.Each value can be a string, number, boolean, array (non-associative),
or nullCompoundHere we’re beginning to nest rules.jsonLogic({"and":[{">":[3,1]},{"<":[1,3]}]})# TrueIn an infix language (like Python) this could be written as:((3>1)and(1<3))Data-DrivenObviously these rules aren’t very interesting if they can only take
static literal data. TypicallyjsonLogicwill be called with a rule
object and a data object. You can use thevaroperator to get
attributes of the data object:jsonLogic({"var":["a"]},# Rule{a:1,b:2}# Data)# 1If you like, we supportsyntactic
sugaron unary
operators to skip the array around values:jsonLogic({"var":"a"},{a:1,b:2})# 1You can also use thevaroperator to access an array by numeric
index:jsonLogic({"var":1},["apple","banana","carrot"])# "banana"Here’s a complex rule that mixes literals and data. The pie isn’t ready
to eat unless it’s cooler than 110 degrees,andfilled with apples.rules={"and":[{"<":[{"var":"temp"},110]},{"==":[{"var":"pie.filling"},"apple"]}]}data={"temp":100,"pie":{"filling":"apple"}}jsonLogic(rules,data)# TrueAlways and NeverSometimes the rule you want to process is “Always” or “Never.” If the
first parameter passed tojsonLogicis a non-object,
non-associative-array, it is returned immediately.#AlwaysjsonLogic(True,data_will_be_ignored);# True#NeverjsonLogic(false,i_wasnt_even_supposed_to_be_here);# FalseInstallationThe best way to install this library is viaPIP:pipinstalljson-logicIf that doesn’t suit you, and you want to manage updates yourself, the
entire library is self-contained injson_logic.pyand you can
download it straight into your project as you see fit.curl-Ohttps://raw.githubusercontent.com/nadirizr/json-logic-py/master/json_logic.pyDifferences from the originalJavaScript versionThe only difference is in the straightforward ‘==’ and ‘===’ comparison
operators, since they have different expected behavior in Python than in
JavaScript.In JS: * ‘==’ - Performs type coercion and compares. * ‘===’ - Does
not perform type coercion.In Python: * ‘==’ - Does not perform type coercion, compares using
Pyton’seqoperator. * ‘===’ - Does not perform type coercion,
compares using Python’s ‘is’ operator.