dendimaki/few-shots-apeiron-model

SetFit with sentence-transformers/paraphrase-mpnet-base-v2

This is a SetFit model that can be used for Text Classification. This SetFit model uses sentence-transformers/paraphrase-mpnet-base-v2 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

1. Fine-tuning a Sentence Transformer with contrastive learning.
2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

• Model Type: SetFit
• Sentence Transformer body: sentence-transformers/paraphrase-mpnet-base-v2
• Classification head: a LogisticRegression instance
• Maximum Sequence Length: 512 tokens
• Number of Classes: 29 classes

Model Sources

• Repository: SetFit on GitHub
• Paper: Efficient Few-Shot Learning Without Prompts
• Blogpost: SetFit: Efficient Few-Shot Learning Without Prompts

Model Labels

Label	Examples
1	'only location one layer two seemed to get a graphic and the bird looks a little confused' 'and its very its not what i am you know like theres no continuity between me and that spaciousness you know i can feel some spaciousness maybe but man theres you know no continuity between me and that and that would be fine that would be a location one layer two type experience certainly fine' 'and then goes up to layer two in location one'
0	'normal functioning and activity of the mind' 'i think im in location one primarily still layer one with some access to other layers' 'the activity of the mind will predominate in moment tomoment experience '
20	'wonderful life' 'freedom of movement' 'despite these reports none of this seems to impact their ability to perform at very high levels'
12	'this is typically not conducive to effectiveness in the world and moves increasingly in the direction of its transcendence this can be fine if a persons life circumstances allow for that ' 'disinterest and lack of affinity with others' 'most systems that target location 4 and later tend to hold a belief that the mind layer 1 is something to get away from and as someone deepens it does tend to become less accessible'
3	'trust in how things are arises and the thinking mind is still my default' 'theres a deep peace but it can be suppressed by triggered conditioning which is indicative of just the layer four but not location four thing' 'memories spontaneously arise less often and theres no possibility of anything beyond it or any thing being outside of it'
6	'quality of an all pervading presence' 'charmed life maybe sounded interesting obviously those would all involve coming back from where youre headed' 'the experience of it can go in the direction of deep stillness and peace which can have very little affect and appear similar to location 4 or the experience can be infused with joy love or even a sense of sacredness or divinity which might appear more similar to location 3 '
26	'then you get increasing distance from that starting at layer two and working your way up through the layers' 'its harder to get distance from feelings from from emotions from thoughts and that layer two' 'but then your second point was getting some fluidity to layer two which given that deconditioning is something that is important to you and that youre working on thats probably a good idea'
21	'lost the perception of 3d' 'dark night of the soul' 'pulled out of fundamental object'
24	'boston retreats are so great for glimpses theyve got that down i really do' 'a lot of methods work in the mind and they actually reinforce layer one and in some way resolve the conditioning at that level but not necessarily at deeper levels' 'because thats a tough place to be so if you and they doubt probably and they often doubt that they are even in fundamental being at that point because its such a difficult place to be it feels like they left fundamental wellbeing and entered a dark night of the soul of some kind'
27	'thats layer three stuff' 'i would just be continuing to move in the direction of presence you know continuing to sink in in that direction of presence and seeing you know if you can pull yourself into more of a persistent experience of layer three' 'main motivation problem seems to be gone'
22	'empathic psychic stuff you say probably not related to fundamental wellbeing but who knows since i was a child i often knew what other people were feeling or thinking i assumed this is developed because it was needed to survive later as a pastor things like synchronicity and healing ministry played a big part right now i wonder what the relation is of these kinds of things to fundamental well being' 'and now i think you you know its given you access or just as a function of deepening and fundamental wellbeing whatever it is it has given you access and produced a shift to the to deeper parts of fundamental wellbeing' 'thats one direction that people can take she very much took one direction and then another direction you know in a way more like you or many others even my mom'
5	'the dominant quality of layer 2 is the sense of allcontaining spaciousness emptiness expansiveness space nothingness openness and so on' 'these experiences usually do not last but people can go through periods that are challenging as they become accustomed to this state' 'you could be a dominant experience when your mind when youre when your system is just relaxing right it can just be zoomed out into its default location that might be like location two layer two'
28	'difference between revelation and illumination versus selfexperience' 'its more like that the way you describe layer four when its experienced from layer three is being like this sense of a deep mystery behind it all and you know its like my mind cant grasp it and but i sense it' 'its because they dont have a dominant layer and theyre like you know no you know the fullness is there and the emptiness is there and the mind and the thoughts and the emotions theyre all there but im not any of those and youre thinking to yourself okay well what about you know lets maybe youre a layer four'
19	'i love you' 'and that works out really well if youre christian because obviously youre trying to get to you know location three ideally and you know get the whole union with the divine thing maxed out or even the godhead experience if you go beyond the union with the divine thing and and all of that' 'pure emotion'
25	'im not sure how successful that will be because my spreadsheet mainly has where people are at not oh they were wrong about the fact that they werent in layer one' 'im getting pulled down to layer one' 'i could fortunately i could stay clued into the to what they said and then id say something oh tell me more about that or you know that was all i could do i couldnt work out what i need to do with this client'
17	'where someone at location 1 on the continuum might have a short outburst and make a brief hand gesture individuals' 'frustration' 'minds seemed much quieter'
18	'and location two could go back and forth' 'i guess its because a few months ago for a period of time i was i cant figure out whether i should put you on speaker or gallery here um for a period of time i um i felt really a lot of the feeling of sacredness um and and there was a lot of um a lot of joy and no joy okay there was a lot of um what i call it there was a lot of bliss i would actually call it bliss so there was like when i in my body it was just a lot of bliss and i related more to sacredness kind of sacredness and everything and i almost felt like there was too much energy in my body like it was it was almost like too much bliss i dont know if that makes sense um and and then i i didnt think i was nondual because i didnt understand what non i didnt really get it until ive been watching these interviews and so then when i started watching the interviews i realized that or some either something shifted or i realized or something shifted because i dont feel that really strong bliss so much even in the first interview i did i felt it more and then over the last couple of weeks it feels more like um it feels more like just a lovely like contentment and fullness and like oh its just so nice' 'or location two'
16	'yeah absolutely okay so location eight' 'you know darkness is never really described as darkness unless its just absolutely pitch black its just described as dim and the objects that are still in perception to somebody in location five' 'visual effects and the sensory glitches'
4	'location two layer one' 'returning to layer 1 perceptually for its functional properties can be appropriate and practical at times and in a fluid system where a finder has access to more than one layer access to layer 1 does not need to compromise access to the deeper layers' 'at location 2 these are much less likely to be identified as personal versus as just arising in a more impersonal way they also are more likely to dissolve spontaneously and rapidly'
13	'dont feel it' 'youre just like seeing it from you know outside of its space time or something' 'if a person really deepens into location 4 they can essentially lose sight of this layer and from location 5 on it becomes increasingly difficult to access'
15	'there is an ever greater depth of stillnesssilence and an incomparable quality of freedom and peace which is the classical freedom from suffering pursued by spiritual traditions for millennia' 'there is a sense that nothing can die because nothing exists of itself and it therefore has no independent existence to lose' 'it becomes more difficult to access layer 3 and to return to previous locations this is even more so the case for location 5 and later finders '
14	'there is also some return of emotion on the path of humanity usually impersonal forms of love which also relate to layer 3 ' 'the subjective experience is emmeshed with deep beliefs about what is ultimately real and transitioning to and deepening into location 4 can be disconcerting' 'it feels as though it is always there and untouched by all experience'
10	'so you might be more in that direction in the sense that you get that profound sense of peace and very little affect in that deeper place in layer three in location two then then you know and thats thats fine but if you want the love joy version of it um you may need to either try to bias your system in that direction by you know seeing if you can call up the love joy sort of qualities that tends to be a little bit shallower in layer' 'it feels like it is everywherethere is nowhere it is notyet it is not localized anywhere and it does not have spatial dimensionality ' 'there is a deep quality of stillness at layer 3 it is more still than any preceding layer and the stillness or silence feels like it pervades and encompasses everythingall objects and all experience'
8	'worries about my life in lay one create a vision for my life which i then implement and worry about convinced its right' 'layer 1 is not the default layer that people transition to or experience in location 3 however it can remainquite accessible' 'because of the profound experience of love joy and compassion in location 3 the mind tends to be biased by the very positive affective state'
2	'its really a direct experience and i think you are having you know even from location one the sense of layer three' 'and help us sort of get a general context of where youre at and so you know i think in some cases youre probably getting glimpses of layer three in location one' 'the body can relax and the blocks can relax as well and a feeling of being of being connected to my environment feeling that around me and more through the center of the heart area and feeling like it is looking back at me and and yeah feeling a sense of of emptiness or loneliness or loneliness but but then it can change and it feels quite quite full and and that arises also within within the center of my heart or my space when my heart is and if i sort of feel feel feel into my heart theres a sense of sacredness or our maternal beingness there like a like a feeling of of of purity of purity of timelessness like like a sense of a sense of perfection um expression or play and um yeah sort of so thats when i do the exercises where i stop and look and thats what i tend to find but i guess sort of in my day to day life i feel as though me and i still have i still feel like im a self ive got a body but but i can just take a moment and i can feel that spaciousness around me and and often like ill work quite a lot so often ill be quite tired and i do that energy that really changes i feel much more energized and relaxed when i do that'
7	'those are such subjective qualities that dont necessarily apply to layer four and so but lets go earlier here and make sure that you know you really are switching between location one and location two yeah thats a little unusual so lets make sure thats happening you actually feel like youre at times when youre in the world your system is literally going from our our nondual perception tests right which involve like a looker in here' 'it is essentially the home layer of locations 4 and later and experiences of it in earlier locations tend to be partial and temporary' 'perceiving or aware at its most foundational level can kind of emerge from layer four to varying degrees or collapse'
23	'youre here now youre in fundamental wellbeing' 'you can certainly continue sinking into your deeper experiences of fundamental wellbeing up until that point' 'individualized sort of narrative nagging sense of self is out of the way'
11	'it is common for layer 4 to be experienced as an unknowable or a mystery' 'layer 4 is not easily accessible in location 3' 'like initially you cant see anything its just an unknowable'
9	'single emotion of love and joy and compassion' 'feeling like youre dissolving into it' 'the quality of spaciousness emptiness expansiveness openness and so on feels as though it pervades everything as the presence of divinity or as the panpsychist presence depending on how location 3 is showing up for that person'

Evaluation

Metrics

Label	Accuracy
all	0.5718

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("dendimaki/few-shots-apeiron-model")
# Run inference
preds = model("pervading presence")

Training Details

Training Set Metrics

Training set	Min	Median	Max
Word count	1	21.6251	283

Label	Training Sample Count
0	32
1	31
2	11
3	88
4	46
5	60
6	119
7	19
8	31
9	15
10	73
11	18
12	26
13	22
14	31
15	42
16	120
17	120
18	120
19	120
20	120
21	71
22	120
23	39
24	25
25	120
26	120
27	120
28	87

Training Hyperparameters

• batch_size: (16, 16)
• num_epochs: (1, 1)
• max_steps: -1
• sampling_strategy: oversampling
• num_iterations: 20
• body_learning_rate: (2e-05, 2e-05)
• head_learning_rate: 2e-05
• loss: CosineSimilarityLoss
• distance_metric: cosine_distance
• margin: 0.25
• end_to_end: False
• use_amp: False
• warmup_proportion: 0.1
• seed: 42
• eval_max_steps: -1
• load_best_model_at_end: False

Training Results

Epoch	Step	Training Loss	Validation Loss
0.0002	1	0.3075	-
0.0102	50	0.2445	-
0.0203	100	0.1977	-
0.0305	150	0.1932	-
0.0407	200	0.2314	-
0.0509	250	0.2247	-
0.0610	300	0.2319	-
0.0712	350	0.2871	-
0.0814	400	0.1573	-
0.0916	450	0.131	-
0.1017	500	0.212	-
0.1119	550	0.205	-
0.1221	600	0.1452	-
0.1322	650	0.2211	-
0.1424	700	0.1202	-
0.1526	750	0.2088	-
0.1628	800	0.1672	-
0.1729	850	0.2317	-
0.1831	900	0.1155	-
0.1933	950	0.1135	-
0.2035	1000	0.1653	-
0.2136	1050	0.1384	-
0.2238	1100	0.1029	-
0.2340	1150	0.2053	-
0.2442	1200	0.1157	-
0.2543	1250	0.0544	-
0.2645	1300	0.1209	-
0.2747	1350	0.1533	-
0.2848	1400	0.0637	-
0.2950	1450	0.1363	-
0.3052	1500	0.1086	-
0.3154	1550	0.0391	-
0.3255	1600	0.1003	-
0.3357	1650	0.0557	-
0.3459	1700	0.147	-
0.3561	1750	0.0286	-
0.3662	1800	0.0452	-
0.3764	1850	0.1319	-
0.3866	1900	0.1142	-
0.3967	1950	0.0945	-
0.4069	2000	0.0797	-
0.4171	2050	0.1177	-
0.4273	2100	0.0955	-
0.4374	2150	0.051	-
0.4476	2200	0.0265	-
0.4578	2250	0.0583	-
0.4680	2300	0.058	-
0.4781	2350	0.0138	-
0.4883	2400	0.0129	-
0.4985	2450	0.1007	-
0.5086	2500	0.0131	-
0.5188	2550	0.0548	-
0.5290	2600	0.0322	-
0.5392	2650	0.023	-
0.5493	2700	0.0734	-
0.5595	2750	0.0904	-
0.5697	2800	0.04	-
0.5799	2850	0.1262	-
0.5900	2900	0.0129	-
0.6002	2950	0.0082	-
0.6104	3000	0.0226	-
0.6205	3050	0.0424	-
0.6307	3100	0.0201	-
0.6409	3150	0.0674	-
0.6511	3200	0.022	-
0.6612	3250	0.0482	-
0.6714	3300	0.0585	-
0.6816	3350	0.0093	-
0.6918	3400	0.0106	-
0.7019	3450	0.0965	-
0.7121	3500	0.1248	-
0.7223	3550	0.0723	-
0.7325	3600	0.0163	-
0.7426	3650	0.045	-
0.7528	3700	0.0038	-
0.7630	3750	0.0192	-
0.7731	3800	0.026	-
0.7833	3850	0.0075	-
0.7935	3900	0.0278	-
0.8037	3950	0.033	-
0.8138	4000	0.0083	-
0.8240	4050	0.0151	-
0.8342	4100	0.0454	-
0.8444	4150	0.0716	-
0.8545	4200	0.0072	-
0.8647	4250	0.0091	-
0.8749	4300	0.0273	-
0.8850	4350	0.0917	-
0.8952	4400	0.0931	-
0.9054	4450	0.0757	-
0.9156	4500	0.0061	-
0.9257	4550	0.1205	-
0.9359	4600	0.1106	-
0.9461	4650	0.0041	-
0.9563	4700	0.0311	-
0.9664	4750	0.0985	-
0.9766	4800	0.0172	-
0.9868	4850	0.0435	-
0.9969	4900	0.006	-

Framework Versions

• Python: 3.10.12
• SetFit: 1.0.3
• Sentence Transformers: 2.6.0
• Transformers: 4.38.2
• PyTorch: 2.2.1+cu121
• Datasets: 2.18.0
• Tokenizers: 0.15.2

Citation

BibTeX

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}