File size: 31,105 Bytes
59baf27 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 |
---
library_name: setfit
tags:
- setfit
- sentence-transformers
- text-classification
- generated_from_setfit_trainer
base_model: sentence-transformers/paraphrase-mpnet-base-v2
datasets:
- dendimaki/v1
metrics:
- accuracy
widget:
- text: so you know you said that layer three maybe sounded interesting
- text: just this like sense of energy thats aliveness and aliveness tingly aliveness
- text: id say is pretty or really the dominant state unless i really focus on location
one and even then
- text: pervading presence
- text: nonduality for you
pipeline_tag: text-classification
inference: true
model-index:
- name: SetFit with sentence-transformers/paraphrase-mpnet-base-v2
results:
- task:
type: text-classification
name: Text Classification
dataset:
name: dendimaki/v1
type: dendimaki/v1
split: test
metrics:
- type: accuracy
value: 0.46352941176470586
name: Accuracy
---
# SetFit with sentence-transformers/paraphrase-mpnet-base-v2
This is a [SetFit](https://github.com/huggingface/setfit) model trained on the [dendimaki/v1](https://huggingface.co/datasets/dendimaki/v1) dataset that can be used for Text Classification. This SetFit model uses [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2) as the Sentence Transformer embedding model. A [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance is used for classification.
The model has been trained using an efficient few-shot learning technique that involves:
1. Fine-tuning a [Sentence Transformer](https://www.sbert.net) with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.
## Model Details
### Model Description
- **Model Type:** SetFit
- **Sentence Transformer body:** [sentence-transformers/paraphrase-mpnet-base-v2](https://huggingface.co/sentence-transformers/paraphrase-mpnet-base-v2)
- **Classification head:** a [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html) instance
- **Maximum Sequence Length:** 512 tokens
- **Number of Classes:** 26 classes
- **Training Dataset:** [dendimaki/v1](https://huggingface.co/datasets/dendimaki/v1)
<!-- - **Language:** Unknown -->
<!-- - **License:** Unknown -->
### Model Sources
- **Repository:** [SetFit on GitHub](https://github.com/huggingface/setfit)
- **Paper:** [Efficient Few-Shot Learning Without Prompts](https://arxiv.org/abs/2209.11055)
- **Blogpost:** [SetFit: Efficient Few-Shot Learning Without Prompts](https://huggingface.co/blog/setfit)
### Model Labels
| Label | Examples |
|:------|:-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| 20 | <ul><li>'while the finder feels a deep sense of completeness his or her partner still has a narrativeself that thrives on external validation'</li><li>'disassembled'</li><li>'location four definitely adds a whole new perspective and can decondition a lot especially if one deepens there but yeah save that for when you feel the timing is good'</li></ul> |
| 26 | <ul><li>'i think the emptiness is a different one'</li><li>'being like a container for whats arising and the stuff thats arising'</li><li>'spaciousness or emptiness'</li></ul> |
| 27 | <ul><li>'encased in gelatin'</li><li>'feeling full of joy'</li><li>'so if i do if i meditate in a certain way i have meditated and it happens and i drop into more of a kind of equalized more still flat perception i would say or just not not perhaps not maybe not flat but its like dropping into a different dimension if you could say that like thats not really its not about the physical that much anymore as much as its a different its like residing in a different field that is more quiet and peaceful and if i sink in in my day to day life i can also go go pretty quickly to that straight away actually but i again i guess i choose not to because again somewhere along the way i think one of my teachers emphasized also feeling the fullness but thats analysis for something else but yeah ive experienced that quite a few times'</li></ul> |
| 18 | <ul><li>'mixture of personal and impersonal love'</li><li>'it sounds very plausible i think being lonely is one thing if i just sit there in my apartment you know and become more and more still and around boredom or being boring'</li><li>'popular term for this change in perception is nonduality or not two'</li></ul> |
| 28 | <ul><li>'but the shift into layer four is you know it can be an intense one and it really is very different than everything that comes before it and so you know lots of strange things can happen on the way to it in the direction of it you know sort of associated with it um and its possible that when you felt like you had made progress in that direction and then you had this other sort of experience come in that it was you know just one of those types of things in that direction'</li><li>'only reality just unfolding'</li><li>'dimensional flatness'</li></ul> |
| 16 | <ul><li>'the path of freedom remains emotionless the path of humanity'</li><li>'moments and so basically when you come out of the narrative mind you start to fill the mind moments that the narrative mind filled with sensory mind moments and so that can also account for the for the luminosity thing it doesnt necessarily have to be it can be a combination of what you said but when you when you were talking about it i was like oh it could be a mind moment thing just because you know theres more moments of sensory experience in the conscious experience'</li><li>'path of humanity'</li></ul> |
| 17 | <ul><li>'seer'</li><li>'seems like the looker is there looking out your eyes'</li><li>'with recalling memories that related to their'</li></ul> |
| 25 | <ul><li>'fluid or experiencing one layer'</li><li>'layer one level'</li><li>'pulled back to probably layer one'</li></ul> |
| 19 | <ul><li>'an example of one potential reason relates to personal love for ones child'</li><li>'or an all pervasive consciousness'</li><li>'it was when my dad died and you know i was like crying but i was like well this is just love so this is okay i wouldnt say this is i want it to stop'</li></ul> |
| 15 | <ul><li>'the thing the thing to keep in mind is that for a system for a layer four location four especially but youre sort of close enough you know youre like a hair away from the thing type system what reading those books will do is basically prime you basically primes the system'</li><li>'the peace is of a different order than that of any other layer because it is not dependent on any positionality such as i am awareness or i am'</li><li>'deeper into layer 4 in later locations the sense of unfolding diminishes until everything feels instantaneous and total '</li></ul> |
| 8 | <ul><li>'strong psychological triggers such as the death of a loved one can still cause a reaction in the system but for the most part there is persistent equanimity and joy'</li></ul> |
| 14 | <ul><li>'layer 3 can remain accessible in location 4 though usually only the deepest centerless aspects of it'</li><li>'dont have that mental abstraction'</li><li>'the subjective experience is emmeshed with deep beliefs about what is ultimately real and transitioning to and deepening into location 4 can be disconcerting'</li></ul> |
| 22 | <ul><li>'fundamentalist beliefs'</li><li>'fundamental wellbeing kind of gets more and more boring in a way'</li><li>'curcumin supplement'</li></ul> |
| 3 | <ul><li>'the boundaries between work and play blur in location 1 layer 4 each act imbued with purpose and the joy of being'</li><li>'in location 1 layer 4 the setting sun doesnt signify an end but a gentle closure a pause for reflection and gratitude'</li><li>'i can still get triggered but negative emotions fall off much faster like glimpsing into layer four by doing unprovoked happiness'</li></ul> |
| 4 | <ul><li>'memories also tend to arise less because there is an increased focus of attention on the present and because the past is no longer valued as defining the sense of self'</li><li>'when youre describing like a deeper nonduality is the absence of layer one'</li></ul> |
| 6 | <ul><li>'so you cant stay in location two but youre not able to access the depth of a layout to possibly and certainly layer three that youre able to with your eyes closed'</li><li>'cosmic love'</li><li>'layer 3 is highly accessible in location 2 however it remains relatively rare for finders to reach layer 3 persistently when they do it is often taken to be end of the path in terms of deepening further into fundamental wellbeing '</li></ul> |
| 21 | <ul><li>'psychic intuitive empathic'</li><li>'darkness'</li><li>'psychedelics'</li></ul> |
| 10 | <ul><li>'the main thing was a sense of a kind of strong gravitational pull'</li></ul> |
| 24 | <ul><li>'since 2017 was when i did finders course and transitioned'</li></ul> |
| 0 | <ul><li>'environment under trigger its more like 11 and then kind of off on my own doing my thing'</li><li>'very attached to my mind'</li></ul> |
| 11 | <ul><li>'this is partly because one is unable to deepen into it and stabilize in it and partly because it cannot be known objectivelyor even subjectively in the usual sense'</li><li>'the unfolding does not happen in anything rather it is total and complete in itself'</li></ul> |
| 1 | <ul><li>'only location one layer two seemed to get a graphic and the bird looks a little confused'</li></ul> |
| 9 | <ul><li>'feeling like youre dissolving into it'</li><li>'in location three there was a certain clarity that i dont have now because it was like less commotion or deadness because like the love would infuse every thought so a thought would come up and instead of me where i am right now i dont want to deal with it it would just be like oh its okay its lets lets just sit with it and the loving feeling would just infuse every thought and then certain judgments that id have oh well i dont really need to look at it that way i can well i can just put love in this or i can just love it so that that id say that was like the most stark contrast'</li></ul> |
| 5 | <ul><li>'something into this experience of two so my experience of this has its just now releasing a lot of the as of a couple of days ago thought it might be wise to look at this yeah so ive been experiencing you know this very strange weird nonduality type'</li><li>'shifting into layer two'</li><li>'things are seen with more distance and objectivity and one typically becomes less reactive the downside of this is that it can be a great place to escape the mind and disassociate from psychological conditioning this is usually whats meant when people speak about spiritual bypassing '</li></ul> |
| 12 | <ul><li>'this can lead to a wide range of outcomes from extraordinary life results to some of the amoral behavior observed in late location teachers'</li><li>'mind is very quiet'</li><li>'essentially this is a metaawareness of what is happening in the mind but there is no sense of being able to engage with it like there is in previous locations '</li></ul> |
| 23 | <ul><li>'until youre feeling deeper or more stable in fundamental wellbeing'</li><li>' an event in fundamental wellbeing for a while'</li></ul> |
## Evaluation
### Metrics
| Label | Accuracy |
|:--------|:---------|
| **all** | 0.4635 |
## Uses
### Direct Use for Inference
First install the SetFit library:
```bash
pip install setfit
```
Then you can load this model and run inference.
```python
from setfit import SetFitModel
# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("dendimaki/fewshot-model")
# Run inference
preds = model("pervading presence")
```
<!--
### Downstream Use
*List how someone could finetune this model on their own dataset.*
-->
<!--
### Out-of-Scope Use
*List how the model may foreseeably be misused and address what users ought not to do with the model.*
-->
<!--
## Bias, Risks and Limitations
*What are the known or foreseeable issues stemming from this model? You could also flag here known failure cases or weaknesses of the model.*
-->
<!--
### Recommendations
*What are recommendations with respect to the foreseeable issues? For example, filtering explicit content.*
-->
## Training Details
### Training Set Metrics
| Training set | Min | Median | Max |
|:-------------|:----|:--------|:----|
| Word count | 1 | 21.9052 | 247 |
| Label | Training Sample Count |
|:------|:----------------------|
| 0 | 2 |
| 1 | 1 |
| 3 | 5 |
| 4 | 2 |
| 5 | 4 |
| 6 | 11 |
| 8 | 1 |
| 9 | 2 |
| 10 | 1 |
| 11 | 2 |
| 12 | 3 |
| 14 | 4 |
| 15 | 8 |
| 16 | 8 |
| 17 | 11 |
| 18 | 28 |
| 19 | 25 |
| 20 | 14 |
| 21 | 4 |
| 22 | 7 |
| 23 | 2 |
| 24 | 1 |
| 25 | 13 |
| 26 | 30 |
| 27 | 36 |
| 28 | 7 |
### Training Hyperparameters
- batch_size: (16, 16)
- num_epochs: (1, 1)
- max_steps: -1
- sampling_strategy: oversampling
- num_iterations: 20
- body_learning_rate: (2e-05, 2e-05)
- head_learning_rate: 2e-05
- loss: CosineSimilarityLoss
- distance_metric: cosine_distance
- margin: 0.25
- end_to_end: False
- use_amp: False
- warmup_proportion: 0.1
- seed: 42
- eval_max_steps: -1
- load_best_model_at_end: False
### Training Results
| Epoch | Step | Training Loss | Validation Loss |
|:------:|:----:|:-------------:|:---------------:|
| 0.0017 | 1 | 0.252 | - |
| 0.0862 | 50 | 0.1891 | - |
| 0.1724 | 100 | 0.1793 | - |
| 0.2586 | 150 | 0.1848 | - |
| 0.3448 | 200 | 0.1033 | - |
| 0.4310 | 250 | 0.0473 | - |
| 0.5172 | 300 | 0.1213 | - |
| 0.6034 | 350 | 0.0343 | - |
| 0.6897 | 400 | 0.0276 | - |
| 0.7759 | 450 | 0.0262 | - |
| 0.8621 | 500 | 0.0425 | - |
| 0.9483 | 550 | 0.0482 | - |
### Framework Versions
- Python: 3.10.12
- SetFit: 1.0.3
- Sentence Transformers: 2.7.0
- Transformers: 4.40.1
- PyTorch: 2.2.1+cu121
- Datasets: 2.19.0
- Tokenizers: 0.19.1
## Citation
### BibTeX
```bibtex
@article{https://doi.org/10.48550/arxiv.2209.11055,
doi = {10.48550/ARXIV.2209.11055},
url = {https://arxiv.org/abs/2209.11055},
author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
title = {Efficient Few-Shot Learning Without Prompts},
publisher = {arXiv},
year = {2022},
copyright = {Creative Commons Attribution 4.0 International}
}
```
<!--
## Glossary
*Clearly define terms in order to be accessible across audiences.*
-->
<!--
## Model Card Authors
*Lists the people who create the model card, providing recognition and accountability for the detailed work that goes into its construction.*
-->
<!--
## Model Card Contact
*Provides a way for people who have updates to the Model Card, suggestions, or questions, to contact the Model Card authors.*
--> |