semvec/pypi-packages huggingface dataset:
python-objectifier 1.6
objectifier

### objectifier

Installation:
`pip install python-objectifier`

```
from objectifier import convert_json_to_object

test_dict = {
        'name': 'Deckow-Crist',
        'catchPhrase': 'Proactive didactic contingency',
        'bs': 'synergize scalable supply-chains'
        }

dict_as_object = convert_json_to_object(test_dict)
dict_as_object.name
> 'Deckow-Crist'

'catchPhrase' in dict_as_object # Retains all original dictionary methods
> True
```

```
test_list = [
    {
        'id': 1,
        'name': 'Leanne Graham',
        'username': 'Bret',
        'email': 'Sincere@april.biz',
        'address': {
            'street': 'Kulas Light',
            'suite': 'Apt. 556',
            'city': 'Gwenborough',
            'zipcode': '92998-3874',
            'geo': {
                'lat': '-37.3159',
                'lng': '81.1496'
            }
        },
        'phone': '1-770-736-8031 x56442',
        'website': 'hildegard.org',
        'company': {
            'name': 'Romaguera-Crona',
            'catchPhrase': 'Multi-layered client-server neural-net',
            'bs': 'harness real-time e-markets'
        }
    },
    {
        'id': 2,
        'name': 'Ervin Howell',
        'username': 'Antonette',
        'email': 'Shanna@melissa.tv',
        'address': {
            'street': 'Victor Plains',
            'suite': 'Suite 879',
            'city': 'Wisokyburgh',
            'zipcode': '90566-7771',
            'geo': {
                'lat': '-43.9509',
                'lng': '-34.4618'
            }
        },
        'phone': '010-692-6593 x09125',
        'website': 'anastasia.net',
        'company': {
            'name': 'Deckow-Crist',
            'catchPhrase': 'Proactive didactic contingency',
            'bs': 'synergize scalable supply-chains'
        }
    }
]

convert_json_to_object(test_list) # Converts list in place
test_list[0].address.suite
> 'Apt. 556'
```
fnv1a 0.2.0
64 bit Fnv-1a hash module.
Requires Python: >=3.11
Keywords: fnv, fnv1a, 64 bit, hash

[![build](https://github.com/plasticuproject/fnv1a/actions/workflows/tests.yml/badge.svg)](https://github.com/plasticuproject/fnv1a/actions/workflows/tests.yml)
[![Python 3.11](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/release/python-311/)
![GitHub](https://img.shields.io/github/license/plasticuproject/fnv1a)
[![PyPI version](https://badge.fury.io/py/fnv1a.svg)](https://badge.fury.io/py/fnv1a)
[![Downloads](https://pepy.tech/badge/fnv1a)](https://pepy.tech/project/fnv1a)
[![Coverage Status](https://coveralls.io/repos/github/plasticuproject/fnv1a/badge.svg?branch=master)](https://coveralls.io/github/plasticuproject/fnv1a?branch=master)
[![CodeQL](https://github.com/plasticuproject/fnv1a/actions/workflows/codeql.yml/badge.svg)](https://github.com/plasticuproject/fnv1a/actions/workflows/codeql.yml)
[![Quality Gate Status](https://sonarcloud.io/api/project_badges/measure?project=plasticuproject_fnv1a&metric=alert_status)](https://sonarcloud.io/dashboard?id=plasticuproject_fnv1a)
[![Security Rating](https://sonarcloud.io/api/project_badges/measure?project=plasticuproject_fnv1a&metric=security_rating)](https://sonarcloud.io/dashboard?id=plasticuproject_fnv1a)

# fnv1a

OOP implimentation of a 64 bit FNV-1a hashing algorithm, where you can create hash
objects with "states" and "dehash" a list of "states" to recreate the original string.
It's completely useless and ridiculous and it makes no sense, but I was really bored.

```pip install fnv1a```
sompy-package 1.0.3
Self Organizing Maps Package
Requires Python: >=3.8

SOMPY
-----

This repository was forked from the original repository at https://github.com/sevamoo/SOMPY to modify it in order to make the package installable via pip.

Changes
1. Changed package name to "sompy-package" to avoid conflict with already existing "sompy" package on PyPI
2. Refactored package replacing deprecated setup.py with pyproject.toml
3. Updated dependencies to include scikit-image
4. General code cleanup and formatting

WIP:
1. Upload package to PyPI

All of the following is the original README file from the creator of the repo.

-----

## Original README

A Python Library for Self Organizing Map (SOM)

As much as possible, the structure of SOM is similar to `somtoolbox` in Matlab. It has the following functionalities:

1. Only Batch training, which is faster than online training. It has parallel processing option similar to `sklearn` format and it speeds up the training procedure, but it depends on the data size and mainly the size of the SOM grid.I couldn't manage the memory problem and therefore, I recommend single core processing at the moment. But nevertheless, the implementation of the algorithm is carefully done for all those important matrix calculations, such as `scipy` sparse matrix and `numexpr` for calculation of Euclidean distance.
2. PCA (or RandomPCA (default)) initialization, using `sklearn` or random initialization.
3. component plane visualization (different modes).
4. Hitmap.
5. U-Matrix visualization.
6. 1-d or 2-d SOM with only rectangular, planar grid. (works well in comparison with hexagonal shape, when I was checking in Matlab with somtoolbox).
7. Different methods for function approximation and predictions (mostly using Sklearn).


### Dependencies:
SOMPY has the following dependencies:
- numpy
- scipy
- scikit-learn
- numexpr
- matplotlib
- pandas
- ipdb

### Installation:
```Python
python setup.py install
```


Many thanks to @sebastiandev, the library is now standardized in a pythonic tradition. Below you can see some basic examples, showing how to use the library.
But I recommend you to go through the codes. There are several functionalities already implemented, but not documented. I would be very happy to add your new examples here.

[Basic Example](https://gist.github.com/sevamoo/035c56e7428318dd3065013625f12a11)

### Citation

There is no published paper about this library. However if possible, please cite the library as follows:

```
@misc{moosavi2014sompy,
  title={SOMPY: A Python Library for Self Organizing Map (SOM)},
  author={Moosavi, V and Packmann, S and Vall{\'e}s, I},
  note={GitHub.[Online]. Available: https://github.com/sevamoo/SOMPY},
  year={2014}
}
```


For more information, you can contact me via sevamoo@gmail.com but please report an issue first.




Thanks a lot.
Best Vahid Moosavi
meta-cleaner 0.4.2
A Python package to clean text from META tags using a BERT large Longformer NER model.
Requires Python: >=3.10

# meta_cleaner

`meta_cleaner` is a Python package designed to clean text from META tags using XLM-RoBERTa (large-sized model).

`trainer.ipynb` is a notebook that creates a dataset and a NER model.

## Installation

```bash
pip install meta-cleaner
```

or

```bash
pip install git+https://github.com/pirr-me/meta_cleaner.git
```

### Install Locally

To install locally in editable mode (for development):

```
pip install -e .
```

## Usage

```python
from meta_cleaner.cleaner import MetaCleaner

# Initialize the MetaCleaner
meta_cleaner = MetaCleaner(model_name_or_path='Pirr/longformer-4096-large-ner')

# List of texts to clean
texts = ['[Genre: Fiction, Consensual Sex, Oral Sex, Romance, Teen Male/Teen Female]\nCHAPTER 15\nWe walked together into the school office to turn in our early dismissal note. Mrs. Roscoe laughed when she saw it. "Don\'t trust him to go on his own, eh Cinda? I can\'t say that I blame you. I never let my husband go on his own either. He can never remember what the doctor said. Okay, here\'s your pass." She handed Cinda the blue excuse slip and we walked out into the hallway. One of the first people I saw was Mitch. He didn\'t look very happy.\n"Surely you\'re still not pissed about New Year\'s Eve?"\n"Yeah, but not your part. We got stopped by the cops not even five minutes after leaving your place.']

# Clean the texts with batch inference
cleaned_texts = meta_cleaner.clean(texts, batch_size=8, confidence_threshold=0.8)

# Display the cleaned texts
for i, cleaned_text in enumerate(cleaned_texts):
    print(f"Cleaned Text {i + 1}:\n{cleaned_text}\n")
```

## Clean data from GCP

```python
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from google.cloud import storage
from tqdm import tqdm
from datasets import Dataset
from meta_cleaner.cleaner import MetaCleaner

storage_client = storage.Client()

bucket_name = 'pirr-training-data'
prefix = 'Processed_data/cleaned-venus'

bucket = storage_client.bucket(bucket_name)
blobs = list(storage_client.list_blobs(bucket_name, prefix=prefix))

txt_blobs = [blob for blob in blobs if blob.name.endswith('.txt')]

def download_blob(blob):
    return blob.download_as_text()

texts = []
with ThreadPoolExecutor(max_workers=16) as executor:
    future_to_blob = {executor.submit(download_blob, blob): blob for blob in txt_blobs}
    
    for future in tqdm(as_completed(future_to_blob), total=len(txt_blobs), desc="Downloading files"):
        blob = future_to_blob[future]
        try:
            content = future.result()
            texts.append(content)
        except Exception as e:
            print(f"Error downloading {blob.name}: {e}")


meta_cleaner = MetaCleaner(model_name_or_path='Pirr/longformer-4096-large-ner')
cleaned_texts = meta_cleaner.clean(texts, batch_size=16, confidence_threshold=0.8)
dataset = Dataset.from_dict({"text": cleaned_texts})

#dataset.push_to_hub("...")
```