semvec/pypi-packages huggingface dataset:
xt-nlp 0.2.7
Utilities for training and working with nlp models in pytorch

# xt-nlp

## Description

This repo contains common NLP pre/post processing functions, loss functions, metrics, and helper functions.

## Installation

From PyPI:
```bash
pip install xt-nlp
```

From source:
```bash
git clone https://github.com/XtractTech/xt-nlp.git
pip install ./xt-nlp
```

## Usage

See specific help on a class or function using `help`. E.g., `help(SESLoss)`.

#### Defining SES Metrics and Loss
```python
from xt_nlp.metrics import SESF1
from xt_nlp.metrics import SESLoss

eval_metrics = {
   'f1': SESF1(threshold=0.8)
}
loss_fn = SESLoss()
```

#### Read BRAT annotations for sequence extraction into data loader
```python
from xt_nlp.utils import get_brat_examples, split_examples, get_features, build_ses_dataloader

# tokenizer = 
# max_sequence_length = 
# doc_stride =
# class_dict = Dictionary mapping classname ==> list of classes to group into this class
# classes = 
# batch_size = 
# workers = 

examples = get_brat_examples(
    datadir='./data/datadir',
    classes=classes
)

train_examples, val_examples = split_examples(examples, train_prop=.9, seed=4000)

train_features = get_features(
    examples=train_examples, 
    tokenizer=tokenizer, 
    all_ans_types=classes, 
    max_seq_len=max_sequence_length,
    doc_stride=doc_stride,
    mode='train'
)

train_loader = build_ses_dataloader(
    train_features, 
    classes, 
    class_dict, 
    batch_size=batch_size,
    workers=workers,
    max_seq_length=max_sequence_length,
    shuffle=True
)
```
PyFastDownloader 1.0
Track the file download process.
Requires Python: >=3.6

# PyFastDownloader
Track the file download process.

## Usage:
```Python
from PyFastDownloader import *
from time import sleep

file = Downloader(link)
print(file.size_str())

file.download("download.zip")
while file.finished != True:
	print("Downloaded: " + file.downloaded_str() + " | " + str(file.percents) + "% | Speed: " + file.speed_str())
	sleep(1)
print("Download time: " + str(file.time_str()) + " | Average speed: " + file.speed_str())
```
</br>

## Console result:
```
34.3 MB
Downloaded: 5.81 MB  |  16%  |  Speed: 6.3 MB/sec
...
Download time: 6.03sec  |  Average speed: 5.7 MB/sec
```
</br></br></br>


## Let's see the details

### Change the language
This allows the units of measurement to be displayed in your language. </br>
Supported languages: </br>
English - **EN** </br>
Russian - **RU** </br>
```Python
set_lang("RU")
```
</br></br>

### Indicate a link to the file
```Python
file = Downloader("https://example.zip")
```
</br></br>

### Get file size
```Python
print(file.size_str()) # auto size (Recommended)

print(file.size) # in bytes
print(file.size_KB) # in kilobytes
print(file.size_MB) # in megabytes
print(file.size_GB) # in gigabytes
```
</br></br>

### Start download
Two download modes: </br>
<ul>
<li>Super mode (While the file is downloading, you can receive actual progress) </li>
<li>Standard mode (Until the file is downloaded, the program will not continue) </li>
</ul></br>

Super mode example:

```Python
file.download("download.zip")
while file.finished != True:
  #do something
```

Standard mode example:
```Python
file.download("download.zip", thread=False)
```
</br>

What can be used in Super Mode?

```Python
file.speed_str() # auto (Recommended)
file.speed # bytes per second

file.downloaded_str() # auto (Recommended)
file.downloaded # bytes

file.percents

file.progress_bar()
```
</br>

#### Progress bar
Takes values:</br>
**step** — *default 5* (optional to indicate)
```Python
file.progress_bar(step=int_value)
```
Example:

```
[==========         ]
```
</br>

#### Pause and Resume download

```Python
# pause
file.pause = True
# resume
file.pause = False
```

#### Cancel downloading
```Python
file.cancel()
```

</br></br>

### Display statistics after download is complete

```Python
file.time_str() # auto (Recommended)
file.time() # in seconds

file.speed_str() # average speed
```
ewolm 0.0.1
Ensemble Weight Optimization with Lagrange Multiplier Method
Requires Python: >=3.6

# EWOLM

EWOLM is a python package to optimize the weights of an ensemble of machine learning models with the Lagrange multiplier method.


#### Install


```python
pip install ewolm
```



#### Install Dependencies


```python
conda install numpy autograd scipy
```



#### Example


```python
from ewolm import ewolm

y_true = pd.read_csv('targets.csv')  # e.g. A pandas dataframe of shape [rows, columns]
y_pred = np.read_csv('oofs.npy') # A numpy array of shape [N_models, rows, columns]

ewolm(metric='bce')  # bce for Binary Crossentropy
ewolm(y_true, y_pred)
```



#### Metric


The `metric` keyword takes any differentiable numpy function. The following metrics are readily available in the package:

`bce`: Binary crossentropy

`mlbce`: Multilabel binary crossentropy

`rmse`: Root mean square error