semvec/pypi-packages huggingface dataset:
quickdataanalysis 0.0.7
Kick start your data analysis with these functions
License: MIT

# quick data analysis

[![PyPi](https://img.shields.io/badge/pypi%20package-0.0.7-blue)](https://pypi.org/project/quickdataanalysis/)
[![](https://img.shields.io/github/license/Santhoshkumard11/quickdataanalysis.svg?colorB=00fff0)](https://github.com/Santhoshkumard11/quickdataanalysis/blob/master/LICENSE.md)
[![Release](https://img.shields.io/badge/Next%20Release-Oct%2020-green)](https://pypi.org/project/quickdataanalysis/)
![Issues](https://img.shields.io/github/issues/Santhoshkumard11/quickdataanalysis)

This packages allows you to kick start your data analysis and make faster insights of the data.

# Requirements
* Pandas >=1.0.1

## Documentation

## 1) Creating Dummies for columns

The create_dummies method will return dummies for the columns. For example you want to dummies for Gender column put the column name in a list and you will get the dataframe with the dummies.

       >>> df =  create_dummies(df_train,["sex"])
       >>> df
       male female
       0      1
       1      0
       1      0
       1      0

## 2) Counting the column values

The count_values method will return the number of values in the columns. For example you want to count the number of females.

    >>> column_value_count(df_train["female"],1,False)
      312

## ToDo

 - [ ] Add more method



All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome :)

Happy Coding!!
lingualytics 0.1.3
A multilingual text analytics package.
License: MIT
Requires Python: >=3.6.1
Keywords: text mining,text preprocessing,text representation,text visualization,codemix analytics

# Lingualytics : Easy codemixed analytics

Lingualytics is a Python library for dealing with code mixed text.  
Lingualytics is powered by powerful libraries like [Pytorch](https://pytorch.org/), [Transformers](https://huggingface.co/transformers), [Texthero](https://texthero.org/), [NLTK](http://www.nltk.org/) and [Scikit-learn](https://scikit-learn.org/).

## Features

1. Preprocessing
    - Remove stopwords
    - Remove punctuations, with an option to add punctuations of your own language
    - Remove words less than a character limit

2. Representation
    - Find n-grams from given text

3. NLP
    - Classification using PyTorch
        - Train a classifier on your data to perform tasks like Sentiment Analysis
        - Evaluate the classifier with metrics like accuracy, f1 score, precision and recall
        - Use the trained tokenizer to tokenize text
    - Some pretrained Huggingface models trained on codemixed datasets you can use
        - [bert-base-multilingual-codemixed-cased-sentiment](https://huggingface.co/rohanrajpal/bert-base-multilingual-codemixed-cased-sentiment)

## Installation

Use the package manager [pip](https://pip.pypa.io/en/stable/) to install lingualytics.

```bash
pip install lingualytics
```

## Usage

### Preprocessing

```python
from lingualytics.preprocessing import remove_lessthan, remove_punctuation, remove_stopwords
from lingualytics.stopwords import hi_stopwords,en_stopwords
from texthero.preprocessing import remove_digits
import pandas as pd
df = pd.read_csv(
   "https://github.com/lingualytics/py-lingualytics/raw/master/datasets/SAIL_2017/Processed_Data/Devanagari/validation.txt", header=None, sep='\t', names=['text','label']
)
# pd.set_option('display.max_colwidth', None)
df['clean_text'] = df['text'].pipe(remove_digits) \
                                    .pipe(remove_punctuation) \
                                    .pipe(remove_lessthan,length=3) \
                                    .pipe(remove_stopwords,stopwords=en_stopwords.union(hi_stopwords))
print(df)
```

### Classification

The train data path should have 4 files
    - train.txt
    - validation.txt
    - test.txt

You can just download `datasets/SAIL_2017/Processed Data/Devanagari` from the Github repository to try this out.

```python
from lingualytics.learner import Learner

learner = Learner(model_type = 'bert',
                model_name = 'bert-base-multilingual-cased',
                dataset = 'SAIL-2017')
learner.fit()
```

### Find topmost n-grams

```python
from lingualytics.representation import get_ngrams
import pandas as pd
df = pd.read_csv(
   "https://github.com/jbesomi/texthero/raw/master/dataset/bbcsport.csv"
)

ngrams = get_ngrams(df['text'],n=2)

print(ngrams[:10])
```

## Contributing

Pull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.

## License

[MIT](https://choosealicense.com/licenses/mit/)