semvec/pypi-packages huggingface dataset:
dainik 1.20
Client library for working with NimbleBox LMAO
Requires Python: >=3.7

# Dainik

`Dainik` is a client library for NimbleBox.ai LMAO (Logging, Monitoring, Alerts and Observability) service.

## Usage

```python
from dainik import Dainik

# initialise the client object
dk = Dainik()

# initialise the run
dk.init(
  "california",
  config = {
    "model_name": "qual",
    "config": {
      "batch_size": 32,
      "n_steps": n_steps,
      "optimizer": "adam",
    }
  },
)

for epoch in range(10):
  log = {
    "loss": 1 / epoch,
    "accuracy": epoch,
  }
  dk.log(log, step = epoch)

  # optionally log metrics directly
  dk.metrics.f1_score(
    x = [[1, 2, 3]],
    y = [[1, 2, 1]],
    step = epoch
  )

  # log files for tracking as well, wildcards work as well!
  dk.save_files([f"./checkpoint-{epoch}/*"])

dk.end() # send signal to sleep
```

## Dev

Due to the current structure the source code for this is only available for NimbleBox.ai engineers. This is spun out of a much larger monorepo which contains the logic for server and client together which helps us work on this faster.
TakeSentenceTokenizer 1.0.2
TakeSentenceTokenizer is a tool for tokenizing and pre processing messages
Keywords: Tokenization

# TakeSentenceTokenizer

TakeSentenceTokenizer is a tool for pre processing and tokenizing sentences. 
The package is used to:
	- convert the first word of the sentence to lowercase
	- convert from uppercase to lowercase
	- convert word to lowercase after punctuation
	- replace words for placeholders: laugh, date, time, ddd, measures (10kg, 20m, 5gb, etc), code, phone number, cnpj, cpf, email, money, url, number (ordinal and cardinal)
	- replace abbreviations
	- replace common typos
	- split punctuations
	- remove emoji
	- remove characters that are not letters or punctuation
	- add missing accentuation
	- tokenize the sentence

## Installation

Use the package manager [pip](https://pip.pypa.io/en/stable/) to install TakeSentenceTokenizer

```bash
pip install TakeSentenceTokenizer
```

## Usage

Example 1: full processing not keeping registry of removed punctuation

Code:
```python
from SentenceTokenizer import SentenceTokenizer
sentence = 'P/ saber disso eh c/ vc ou consigo ver pelo site www.dÃºvidas.com.br/minha-dÃºvida ??'
tokenizer = SentenceTokenizer()
processed_sentence = tokenizer.process_message(sentence)
print(processed_sentence)
```

Output:
```python
'para saber disso Ã© com vocÃª ou consigo ver pelo site URL ? ?'
```


Example 2: full processing keeping registry of removed punctuation
```python
from SentenceTokenizer import SentenceTokenizer
sentence = 'como assim $@???'
tokenizer = SentenceTokenizer(keep_registry_punctuation = True)
processed_sentence = tokenizer.process_message(sentence)
print(processed_sentence)
print(tokenizer.removal_registry_lst)
```

Output:
```python
como assim ? ? ?
[['como assim $@ ? ? ?', {'punctuation': '$', 'position': 11}, {'punctuation': '@', 'position': 12}, {'punctuation': ' ', 'position': 13}]]
```

## Author
Take Data&Analytics Research

## License
[MIT](https://choosealicense.com/licenses/mit/)