Analysis And Generation Model ML
Tags: python, machinelearning, programming, opensource
Published: 2023-05-08 10:37:44

# Analysis_And_Generation_Model_ML

__BEFORE READING THIS REPOSITORY IT IS RECOMMENDED TO START FROM:__

https://github.com/insidbyte/Analysis_and_processing

***I have in fact decided to generate a custom vocabulary to train the model and it would be appropriate to look at the repository code.***

---

# SEE THIS REPOSITORY AT: [https://github.com/insidbyte/Analysis_And_Generation_Model_ML](url)

---

___OPTIONS:___
 
    1)-GENERATE MODEL

    2)-TEST WITH HYPERPARAMETER TUNING

    3)-PLOT WITH TFIDF VECTORIZER AND SVD TRUNCATED REDUCTION

### Menù

__Starting the ModelsGenerator.py file from the terminal it will appear:__


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/aj6wwy60xud3raf0iapv.png)

#  ___OPTION 1:___
### Model generation:
***I decided to use tfidf and support vector machine because they are highly suitable for text processing and***
***support vector machine with the linear kernel is highly suitable for classifications based on two classes as***
***in our case: positive and negative***


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hdebr7viz6shxl7c26jr.png)

# Kaggle IMDb dataset example:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/c7mng52u3jt45hubyj6j.png)

### ***I created a Client in Angular to send requests to a Python Server*** 

# CLIENT: 


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pio900ekqiafdeq0swlx.png)

# SERVER:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/epsndnypy8cwo2i2zs0v.png)

# RESPONSE FROM THE SERVER:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2t652130c0ykq10akvhs.png)

# ANOTHER EXAMPLE:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k7lfdcjizo2f6fftyw1e.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0csg4713mmmvoi4wa6ln.png)

#  ___OPTION 2:___
### Test hyperparameters with gridsearchCV and tfidf vectorizer:

***A good way to automate the test phase and save time searching for the best parameters to***
***generate the most accurate model possible is to use GrisearchCV made available by scikit-learn***
***The code in ModelsGenerator.py must be customized based on the dataset to be analyzed***

### ***WARNING !!***
### ***If we don't study the scikit-learn documentation we could start infinite analyzes***
### ***so it is always advisable to know what we are doing***

### Link scikit-learn: https://scikit-learn.org/

***Input:***


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/32fx2hqqbhhh3kjux3ht.png)


***Output:***


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/j5bjmxa681tr17fxctt7.png)



#  ___OPTION 3:___

***Input:***


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xd4wbnxqf93prap424wv.png)

***This option is experimental, the reduction is not applied to model training because it***
***generates too few components and RAM memory (8GB) of my PC is not enough to generate***
***more components even if the results are interesting!***

***Output:***


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jqnh9lgb9jonha1gze9q.png)

# CONCLUSION:

***We got satisfactory results and generated a fairly accurate***
***model this repository will be updated over time***

***For info or collaborations contact me at: u.calice@studenti.poliba.it***