semvec/pypi-packages huggingface dataset:
logstash-filter-test 0.1.1
Easily test logstash filter
License: MIT

# logstash-filter-test
[Logstash](https://www.elastic.co/products/logstash) is a program that collects
json documents from various inputs, transform them according to a configuration
file (a *filter*), and sends them to various outputs. This script helps you make
sure that your filter does what you expect by writing a test suite.

## Installation

You can install using pip:

```
pip install logstash-filter-test
```

or just copy `logstash_filter_run.py` and `logstash_filter_test.py`. There are
no dependencies. Python 3.5+ and 2.7 are supported.

## Example

Here is a simple test suite file:

testcases.js:
```javascript
[
  // Test a message with a value
  [
    {
      "message": "<<2018-06-11 13:45:39,127+0300>> TAU = 6.2831853"
    },
    {
      "date": "2018-06-11 13:45:39,127+0300",
      "@timestamp": "2018-06-11T10:45:39.127Z",
      "msg_text": "TAU = 6.2831853",
      "metric": "TAU",
      "value": 6.2831853,
    }
  ],
]
```

filter.conf:
```
filter {
  grok {
    match => {
      "message" => [
        "^<<%{TIMESTAMP_ISO8601:date}>> %{GREEDYDATA:msg_text}$"
      ]
    }
  }
  date {
    match => ["date", "ISO8601"]
  }
  grok {
    match => {
      "msg_text" => [
        "^%{WORD:metric} = %{NUMBER:value:float}"
      ]
    }
    tag_on_failure => []
  }
}
```

Put them in a directory, run `./logstash_filter_test.py` and it will inform you
if the testcases passed, and if not, why.

`testcases.js` can be a simple json file. It has the extension `.js` because
javascript comments and trailing commas are allowed, thanks to
[jstyleson](https://github.com/linjackson78/jstyleson). It is a list of
test cases. Each test case is a list of size 2. The first item is the input 
json document. The second item is compared to the output json document. All
fields that are defined must be equal to the fields in the output document.
The output document may include other fields. To test that an output field
doesn't exist, use `"field": null`.

## Testing from Python

If you don't like the testcase file format, it's easy to test by yourself:

```pydocstring
>>> from logstash_filter_run import logstash_filter_run
>>> logstash_filter_run([{"a": 3}], 'filter { mutate { copy => { "a" => "b" } } }')
[{'@timestamp': '2018-07-10T18:26:58.411Z',
  '@version': '1',
  'a': 3,
  'b': 3,
  'host': 'myhost'}]
```
Flask-PubSubDecorator 1.1.6
Decorates publisher functions and subscriber routes creating topics/subscriptions if necessary
License: MIT

# Flask-PubSubDecorator

Decorates publisher functions and subscriber routes creating topics/subscriptions if necessary.

## Installation

Add this line to your application's requirements.txt

```python
Flask-PubSubDecorator
```

And then execute:

    $ pip install -r requirements.txt

Or install it yourself as:

    $ pip install Flask-PubSubDecorator

## Usage

Using PubSubDecorator is dead simple. First set your GOOGLE_APPLICATION_CREDENTIALS environment variable to point at a valid JSON creds file.

    $ export GOOGLE_APPLICATION_CREDENTIALS=/path/to/creds.json

The following snippet should get you coding
```python
from flask import Flask, request
from PubSubDecorator import PubSubDecorator
import base64
import json


app = Flask(__name__)
# blueprint can optionally be passed in for registering subscribers in a blueprint endpoint
app.pubsub = PubSubDecorator(app)


# publisher decorator will inject publisher client and topic path
@app.pubsub.publisher(topic='user_confirmed')
def user_confirmed(publisher, topic, user):
    publisher.publish(topic, data=json.dumps({
        'user_id': user.id
    }))


# subscriber decorator will register the Flask route, prefixing /_ah/push-handlers
# subscriber decorator will parse and inject pubsub message
@app.pubsub.subscriber(
    subscription='process_user_confirmation',
    topic='user_confirmed',
    route='/process_user_confirmation',
    methods=['POST']
)
def process_user_confirmation(message, *args, **kwargs):
    try:
        user_id = message.get('user_id')
        # do some async work here!
    except Exception:
        _logger.exception(
            'An unexpected error occurred processing subscription "{0}": {1}'.format(
                kwargs.get('__subscription__'), request.data
            )
        )
        # Unexpected failure, do not ack message
        return '', 422
    return '', 200
```

## Security

PubSub push subscriptions are inherently public facing and should therefore be secured. Googles recommended solution
is to attach a secret key to a registered subscription pushEndpoint. PubSubDecorator handles this for you if any of
the following is provided:

  1. OS Environment Variable `PUBSUB_DECORATOR_API_KEY`
  2. Flask App Config `PUBSUB_DECORATOR_API_KEY`
  3. api_key is passed into PubSubDecorator constructor.

Best Practice: encrypt your key with GCloud KMS, store it in GCloud Datastore, and export to OS environment variable
at runtime.

## Logging

Decorators will log to 'flask-pubsub-decorator' namespace.

```python
import sys
import logging
import logging.handlers


logger = logging.getLogger('flask-pubsub-decorator')
logger.setLevel(logging.DEBUG)
log_file = logging.handlers.RotatingFileHandler(
    'log_file_name.log', maxBytes=5 * 1024 * 1024, backupCount=10
)
logger.addHandler(log_file)
```

# Testing

    $ pytest -s tests.py