semvec/pypi-packages huggingface dataset:
saniti 0.1.51
Sanitise text while keeping your sanity

# Saniti

**Sanitise lists of text documents quickly, easily and whilst maintaining your sanity**

The aim was to streamline processing lists of documents into the same outputs into simply specifying the list of texts and defining the sanitization pipeline.

### Usage:

**As a function-ish**

```
import saniti
original_text = ["I like to moves it, move its", "I likeing to move it!", "the of"]
text = saniti.saniti(original_text, ["token", "destop", "depunct", "unempty", "stem", "out_corp_dict"]) #sanitise the text while initalising the class
print(text.text)

{'dictionary': <gensim.corpora.dictionary.Dictionary object at 0x000002BA9F5FFEF0>, 'corpus': [[(0, 1), (1, 1), (2, 2)], [(0, 1), (1, 1), (2, 1)], []]}
```

**As a class**

```
import saniti
sani1 = saniti.saniti() # initialise the santising class
text = sani1.process(original_text, ["token", "destop", "depunct", "unempty", "lemma", "out_tag_doc"]) # sanitise the text
print(text)

[TaggedDocument(words=['I', 'like', 'move', 'move'], tags=['I like move move']), TaggedDocument(words=['I', 'likeing', 'move'], tags=['I likeing move']), TaggedDocument(words=[], tags=[''])]
```

## Pipeline Components

* "token" - tokenise texts
* "depunct" - remove punctuation
* "unempty" - remove empty words within documents
* "lemma" - lemmatize text
* "destop" - remove stopwords
* "stem" - stem texts
* "out_tag_doc" - turns the texts into gensim tagged documents for Doc2Vec
* "out_corp_dict" - turns the texts into gensim corpus and dictionary
TRuML 0.0.0
Translation between BNGL models and Kappa models

TRuML: A translator for rule-based modeling
===========================================

Summary
-------
TRuML is a Python application designed to translate models written in either Kappa or the BioNetGen language (BNGL)
into the other language.  Currently TRuML is only compatible with Python 2 and work is being done to adapt TRuML for
use with Python 3.  Documentation can be found on `Read the Docs <https://truml.readthedocs.io>`_

Usage
-----
TRuML is invoked on the command line using the command ``truml``.  All options can be seen with the ``-h`` or ``--help``
flags::

    truml -h

Conversion of a model written in BNGL to the Kappa language is done using the command::

    truml -b model.bngl

The corresponding Kappa model will then exist in the same directory as the source BNGL file.  The reverse translation
is done in an equivalent manner::

    truml -k model.ka

Varying levels of verbosity can be achieved.  TRuML uses Python's logging library to track the translation process,
and it's default logging level is WARNING.  The ``-v`` (``--verbose``) and ``-d`` (``--debug``) flags will set the
logging level to INFO and DEBUG, respectively.  The logging output can be redirected to a file with the ``-l`` (``--log_file``)
flag and a corresponding file name.

Note that TRuML only translates the model structure.  It does not consider any simulation or simulation perturbation
directives from either language.

Caveats
-------
The two languages are not interchangeable (there are models in both languages that cannot be exactly translated to the
other).  In many cases, TRuML will raise issues explicitly.  However the languages are both still under development and
unidentified incompatibilities could cause the translation process to crash.  Some cases that cannot be exactly translated
include:

BNGL to Kappa
*************
 - Patterns containing molecules in complex but without explicit binding::

    A().B()

Kappa to BNGL
*************
 - Models containing infinities (``[inf]``)

Finally, the BioNetGen language is interpreted slightly differently for different simulation engines.  The NFsim engine
does not consider molecularity constraints on the right hand side of the rule.  As this is similar to Kappa's purely
local pattern representation (which cannot consider right hand side molecularity), TRuML assumes this convention.
theia 0.9.2
Theia is a lightweight log aggregator.

# theia

[![Build Status](https://travis-ci.org/theia-log/theia.svg?branch=master)](https://travis-ci.org/theia-log/theia)
[![Maintainability](https://api.codeclimate.com/v1/badges/ff2ef675417026a05721/maintainability)](https://codeclimate.com/github/theia-log/theia/maintainability)
[![Test Coverage](https://api.codeclimate.com/v1/badges/ff2ef675417026a05721/test_coverage)](https://codeclimate.com/github/theia-log/theia/test_coverage)
[![PyPI version](https://badge.fury.io/py/theia.svg)](https://badge.fury.io/py/theia)
[![Documentation Status](https://readthedocs.org/projects/theia/badge/?version=latest)](https://theia.readthedocs.io/en/latest/?badge=latest)

Simple and lightweight log aggregator




# Introduction
Theia is a log aggregator that collects logs from multiple log files usually on multiple machines and provides the log entries on a single endpoint. 
Each recorded entry comes from a single source (log file) and can be tagged with multiple tags. This way you can query an filter by the source (for example /var/log/apache or /var/log/reds) and tags (for example "production1", "cache-server", "testing" etc), or you can just simply match a pattern against the content of the entry.

Theia has three parts:
 * watcher - this is the agent process running on the machine where you have your logs. You will have multiple watchers, but usually a single watcher per node
 * collector - this is the master process, usually running on a sperate node where you're going to collect the logs. Provides the interface for pushing the log events and an interface for querying the aggregated logs.
 * client - client for querying and watching live events. This connects to the collector via websocket and
queries the collected events or registers a filter for live events.

# Installation

Theia comes as a single python package (only python3 is supported). 

With ```pip```:
```bash
pip install theia
```

# Running the collector

This process collects and stores the events. The built-in database uses text files to store the events.

Create a directory where you want to keep the events, and run the collector server:

```bash
mkdir data

python -m theia.cli collect -d ./data

```

This will run the server listening on port ```6433```.

# Running a watcher

Make sure you have the collector running.

Then, run the watcher:

```bash
python -m theia.cli watch -c localhost -f /var/log/httpd/access.log -t web-access
```

This would watch for changes in ```access.log``` and send events to the collector 
tagged with ```web-access```

# Run a query

To query for all log events with tag ```web-access```:

```bash
python -m theia.cli query -c localhost -t web-access
```

# Run a live filter

To watch the events with tag ```web-access``` as they arrive in real-time:

```bash
python -m theia.cli query -l -c localhost -t web-access
```

# Note
**This is still  work in progress and is far from production ready**