semvec/pypi-packages huggingface dataset:
pyjq-ng 0.8.0
A simple Python package to Query Json Data.
License: BSD

# pyjq

A simple Python package to Query Json Data.

## Features

- Supports pure json files
- Supports multiple json objects in a file, delimited by newlines (/n)
- Supports gzipped files
- Supports customizabile filters
- Supports pure datetime range filters 

## Todo

The filters could be extended easily, adopting [Python3 stdlib operator](https://docs.python.org/3/library/operator.html).
See `pyjq.PyJQ.filter` to extend ops mapping.

## Installation

````
pip install pyjq-ng
````
## Example data

See `example/alerts.json`.
pyjq works on lines by lines (splitted by \n).
It have been used for Wazuh alert json files and Django dumps.

````
pyjq -j examples/django_dump.json -limit 2 -filter 'fields__original_url == https://google.com'
pyjq -j examples/django_dump.json -limit 2 -filter 'model == urlshortener.urlshortener'
````

## Usage

'agent__name' it's an example of the namespace used by pyjq to access to nested childs. It other word it means `json['agent']['name']`.
It haven't limits on number of nested elements.


Apply some custom filters with AND and OR operators on Wazuh Alert file
````
pyjq -j ../Scaricati/alerts.json -filter 'agent__ip == 172.16.16.102 and agent__name == telegram-gw or agent__ip == 172.16.16.108'
````

Contains operator
````
pyjq -j ../Scaricati/alerts.json -filter 'rule__description in iptables and agent__name == dev-bastion'
````

Convert a specified filed to a pure datetime object and filter in a specified range
````
pyjq -j ../Scaricati/alerts.json -start_datetime 2020-04-06T10:22:00 -end_datetime 2020-04-06T13:22:00 -datetime_field timestamp
````

Realtime reading, it will only takes the latter entries, delimited by newline \n
````
pyjq -j /var/ossec/logs/alerts/alerts.json -datetime_field timestamp -realtime
````

Use a gzipped json file directly
````
pyjq -j ../Scaricati/alerts.json.gzip
````

Limit results to 2 
````
pyjq -j ../Scaricati/alerts.json  -limit 2
````

Realtime monitoring of a specific entity
````
pyjq -j /var/ossec/logs/alerts/alerts.json -realtime -filter 'agent__name == tinyurl and rule__level == 3'
````

Custom callback, usefull for bot integration and other pub/sub APIs
````
python3 pyjq -j examples/alerts.json -realtime -filter 'agent__name == tinyurl and rule__description in ssh' -callback 'examples.callback.things'
````

Reading from stdin
````
cat examples/alerts.json | python3 ./pyjq -filter 'rule__level > 3'

# continous processing
tail -f  /tmp/alerts.json | python3 ./pyjq -filter 'location != osquery'
````

## Author

Giuseppe De Marco <giuseppe.demarco@unical.it>

## Credits

Wazuh SIEM group @GarrLab