Update README.md
Browse files
README.md
CHANGED
|
@@ -7,8 +7,44 @@ library_name: generic
|
|
| 7 |
---
|
| 8 |
## Hugging Face Transformers with Scikit-learn Classifiers 🤩🌟
|
| 9 |
|
| 10 |
-
This repository contains a small proof-of-concept pipeline that leverages
|
| 11 |
The training leverages the language module of [whatlies](https://github.com/koaning/whatlies).
|
| 12 |
|
| 13 |
-
|
| 14 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 7 |
---
|
| 8 |
## Hugging Face Transformers with Scikit-learn Classifiers 🤩🌟
|
| 9 |
|
| 10 |
+
This repository contains a small proof-of-concept pipeline that leverages longformer embeddings with scikit-learn Logistic Regression that does sentiment analysis.
|
| 11 |
The training leverages the language module of [whatlies](https://github.com/koaning/whatlies).
|
| 12 |
|
| 13 |
+
# Classification Report
|
| 14 |
+
Below is the classification report 👇🏻
|
| 15 |
+
precision recall f1-score support
|
| 16 |
+
0 0.84 0.89 0.86 53
|
| 17 |
+
1 0.86 0.81 0.84 47
|
| 18 |
+
accuracy 0.85 100
|
| 19 |
+
macro avg 0.85 0.85 0.85 100
|
| 20 |
+
weighted avg 0.85 0.85 0.85 100
|
| 21 |
+
|
| 22 |
+
# Pipeline
|
| 23 |
+
Below you can see the pipeline 👇🏻 (it's interactive! 🪄)
|
| 24 |
+
<style>#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 {color: black;background-color: white;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 pre{padding: 0;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-toggleable {background-color: white;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 label.sk-toggleable__label {cursor: pointer;display: block;width: 100%;margin-bottom: 0;padding: 0.3em;box-sizing: border-box;text-align: center;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 label.sk-toggleable__label-arrow:before {content: "▸";float: left;margin-right: 0.25em;color: #696969;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 label.sk-toggleable__label-arrow:hover:before {color: black;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-estimator:hover label.sk-toggleable__label-arrow:before {color: black;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-toggleable__content {max-height: 0;max-width: 0;overflow: hidden;text-align: left;background-color: #f0f8ff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-toggleable__content pre {margin: 0.2em;color: black;border-radius: 0.25em;background-color: #f0f8ff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 input.sk-toggleable__control:checked~div.sk-toggleable__content {max-height: 200px;max-width: 100%;overflow: auto;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 input.sk-toggleable__control:checked~label.sk-toggleable__label-arrow:before {content: "▾";}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-estimator input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-label input.sk-toggleable__control:checked~label.sk-toggleable__label {background-color: #d4ebff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 input.sk-hidden--visually {border: 0;clip: rect(1px 1px 1px 1px);clip: rect(1px, 1px, 1px, 1px);height: 1px;margin: -1px;overflow: hidden;padding: 0;position: absolute;width: 1px;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-estimator {font-family: monospace;background-color: #f0f8ff;border: 1px dotted black;border-radius: 0.25em;box-sizing: border-box;margin-bottom: 0.5em;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-estimator:hover {background-color: #d4ebff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel-item::after {content: "";width: 100%;border-bottom: 1px solid gray;flex-grow: 1;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-label:hover label.sk-toggleable__label {background-color: #d4ebff;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-serial::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-serial {display: flex;flex-direction: column;align-items: center;background-color: white;padding-right: 0.2em;padding-left: 0.2em;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-item {z-index: 1;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel {display: flex;align-items: stretch;justify-content: center;background-color: white;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel::before {content: "";position: absolute;border-left: 1px solid gray;box-sizing: border-box;top: 2em;bottom: 0;left: 50%;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel-item {display: flex;flex-direction: column;position: relative;background-color: white;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel-item:first-child::after {align-self: flex-end;width: 50%;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel-item:last-child::after {align-self: flex-start;width: 50%;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-parallel-item:only-child::after {width: 0;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-dashed-wrapped {border: 1px dashed gray;margin: 0 0.4em 0.5em 0.4em;box-sizing: border-box;padding-bottom: 0.4em;background-color: white;position: relative;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-label label {font-family: monospace;font-weight: bold;background-color: white;display: inline-block;line-height: 1.2em;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-label-container {position: relative;z-index: 2;text-align: center;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-container {/* jupyter\'s `normalize.less` sets `[hidden] { display: none; }` but bootstrap.min.css set `[hidden] { display: none !important; }` so we also need the `!important` here to be able to override the default hidden behavior on the sphinx rendered scikit-learn.org. See: https://github.com/scikit-learn/scikit-learn/issues/21755 */display: inline-block !important;position: relative;}#sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48 div.sk-text-repr-fallback {display: none;}</style><div id="sk-39946057-cc05-4ffc-aa91-2be5cb7e4f48" class="sk-top-container"><div class="sk-text-repr-fallback"><pre>Pipeline(steps=[('embedding',\n HFTransformersLanguage(model_name_or_path='allenai/longformer-base-4096')),\n ('model', LogisticRegression())])</pre><b>Please rerun this cell to show the HTML repr or trust the notebook.</b></div><div class="sk-container" hidden><div class="sk-item sk-dashed-wrapped"><div class="sk-label-container"><div class="sk-label sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="866ec933-b9b6-4e29-94ea-a8434aab47b7" type="checkbox" ><label for="866ec933-b9b6-4e29-94ea-a8434aab47b7" class="sk-toggleable__label sk-toggleable__label-arrow">Pipeline</label><div class="sk-toggleable__content"><pre>Pipeline(steps=[('embedding',\n HFTransformersLanguage(model_name_or_path='allenai/longformer-base-4096')),\n ('model', LogisticRegression())])</pre></div></div></div><div class="sk-serial"><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="5732cbdd-b06b-453e-8291-0846f2ab2c8d" type="checkbox" ><label for="5732cbdd-b06b-453e-8291-0846f2ab2c8d" class="sk-toggleable__label sk-toggleable__label-arrow">HFTransformersLanguage</label><div class="sk-toggleable__content"><pre>HFTransformersLanguage(model_name_or_path='allenai/longformer-base-4096')</pre></div></div></div><div class="sk-item"><div class="sk-estimator sk-toggleable"><input class="sk-toggleable__control sk-hidden--visually" id="871360c4-c184-40fd-9d30-9f8b5ed73aa8" type="checkbox" ><label for="871360c4-c184-40fd-9d30-9f8b5ed73aa8" class="sk-toggleable__label sk-toggleable__label-arrow">LogisticRegression</label><div class="sk-toggleable__content"><pre>LogisticRegression()</pre></div></div></div></div></div></div></div>
|
| 25 |
+
|
| 26 |
+
# Hyperparameters
|
| 27 |
+
```
|
| 28 |
+
-'memory': None,
|
| 29 |
+
-'steps': [('embedding', HFTransformersLanguage(model_name_or_path='allenai/longformer-base-4096')),
|
| 30 |
+
('model', LogisticRegression())],
|
| 31 |
+
- 'verbose': False,
|
| 32 |
+
-'embedding': HFTransformersLanguage(model_name_or_path='allenai/longformer-base-4096'),
|
| 33 |
+
-'model': LogisticRegression(),
|
| 34 |
+
-'embedding_model_name_or_path': 'allenai/longformer-base-4096',
|
| 35 |
+
-'model_C': 1.0,
|
| 36 |
+
- 'model_class_weight': None,
|
| 37 |
+
- 'model_dual': False,
|
| 38 |
+
- 'model_fit_intercept': True,
|
| 39 |
+
- 'model_intercept_scaling': 1,
|
| 40 |
+
- 'model_l1_ratio': None,
|
| 41 |
+
- 'model_max_iter': 100,
|
| 42 |
+
- 'model_multi_class': 'auto',
|
| 43 |
+
-'model_n_jobs': None,
|
| 44 |
+
-'model_penalty': 'l2',
|
| 45 |
+
-'model_random_state': None,
|
| 46 |
+
-'model_solver': 'lbfgs',
|
| 47 |
+
-'model_tol': 0.0001,
|
| 48 |
+
-'model_verbose': 0,
|
| 49 |
+
-'model_warm_start': False
|
| 50 |
+
```
|