minuva
/

MiniLMv2-userflow-v2

@@ -1,92 +1,75 @@
----
-base_model: nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large
-tags:
-- generated_from_trainer
-metrics:
-- accuracy
-- f1
-model-index:
-- name: MiniLMv2-L6-H384-distilled-from-RoBERTa-Large-userflow-distil
-  results: []
----
-<!-- This model card has been generated automatically according to the information the Trainer had access to. You
-should probably proofread and complete it, then remove this comment. -->
-# MiniLMv2-L6-H384-distilled-from-RoBERTa-Large-userflow-distil
-This model is a fine-tuned version of [nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large](https://huggingface.co/nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large) on the None dataset.
-It achieves the following results on the evaluation set:
-- Loss: 0.6738
-- Accuracy: 0.7236
-- F1: 0.7313
-## Model description
-More information needed
-## Intended uses & limitations
-More information needed
-## Training and evaluation data
-More information needed
-## Training procedure
-### Training hyperparameters
-The following hyperparameters were used during training:
-- learning_rate: 7e-05
-- train_batch_size: 10
-- eval_batch_size: 10
-- seed: 42
-- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
-- lr_scheduler_type: linear
-- lr_scheduler_warmup_ratio: 0.1
-- num_epochs: 8
-### Training results
-| Training Loss | Epoch | Step | Validation Loss | Accuracy | F1     |
-|:-------------:|:-----:|:----:|:---------------:|:--------:|:------:|
-| No log        | 0.25  | 100  | 2.3745          | 0.3923   | 0.2210 |
-| No log        | 0.51  | 200  | 2.1198          | 0.4126   | 0.2567 |
-| No log        | 0.76  | 300  | 1.8704          | 0.4756   | 0.3979 |
-| No log        | 1.01  | 400  | 1.5780          | 0.5305   | 0.4551 |
-| 2.1769        | 1.26  | 500  | 1.3717          | 0.5650   | 0.5037 |
-| 2.1769        | 1.52  | 600  | 1.2590          | 0.5935   | 0.5543 |
-| 2.1769        | 1.77  | 700  | 1.0973          | 0.6280   | 0.5804 |
-| 2.1769        | 2.02  | 800  | 0.9814          | 0.6423   | 0.5978 |
-| 2.1769        | 2.27  | 900  | 0.9589          | 0.6463   | 0.6152 |
-| 0.9806        | 2.53  | 1000 | 0.9098          | 0.6565   | 0.6483 |
-| 0.9806        | 2.78  | 1100 | 0.8747          | 0.6321   | 0.6194 |
-| 0.9806        | 3.03  | 1200 | 0.8172          | 0.6931   | 0.6902 |
-| 0.9806        | 3.28  | 1300 | 0.7862          | 0.7033   | 0.7017 |
-| 0.9806        | 3.54  | 1400 | 0.7975          | 0.6890   | 0.6952 |
-| 0.4166        | 3.79  | 1500 | 0.7674          | 0.6951   | 0.6913 |
-| 0.4166        | 4.04  | 1600 | 0.7521          | 0.6911   | 0.6997 |
-| 0.4166        | 4.29  | 1700 | 0.7944          | 0.6951   | 0.7055 |
-| 0.4166        | 4.55  | 1800 | 0.7366          | 0.7093   | 0.7127 |
-| 0.4166        | 4.8   | 1900 | 0.7412          | 0.6911   | 0.6944 |
-| 0.2158        | 5.05  | 2000 | 0.7246          | 0.7012   | 0.7083 |
-| 0.2158        | 5.3   | 2100 | 0.7097          | 0.7195   | 0.7253 |
-| 0.2158        | 5.56  | 2200 | 0.6914          | 0.7134   | 0.7197 |
-| 0.2158        | 5.81  | 2300 | 0.6875          | 0.7175   | 0.7266 |
-| 0.2158        | 6.06  | 2400 | 0.6544          | 0.7236   | 0.7296 |
-| 0.1423        | 6.31  | 2500 | 0.6738          | 0.7236   | 0.7313 |
-| 0.1423        | 6.57  | 2600 | 0.6640          | 0.7175   | 0.7253 |
-| 0.1423        | 6.82  | 2700 | 0.6617          | 0.7154   | 0.7233 |
-| 0.1423        | 7.07  | 2800 | 0.6582          | 0.7154   | 0.7205 |
-| 0.1423        | 7.32  | 2900 | 0.6678          | 0.7033   | 0.7093 |
-| 0.1204        | 7.58  | 3000 | 0.6596          | 0.7154   | 0.7197 |
-| 0.1204        | 7.83  | 3100 | 0.6598          | 0.7154   | 0.7217 |
-### Framework versions
-- Transformers 4.37.0
-- Pytorch 2.1.2
-- Datasets 2.1.0
-- Tokenizers 0.15.1

+# User Flow Text Classification
+This model is a fined-tuned version of [nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large](https://huggingface.co/nreimers/MiniLMv2-L6-H384-distilled-from-RoBERTa-Large).
+The quantized version in ONNX format can be found [here](https://huggingface.co/minuva/MiniLMv2-userflow-v2-onnx)
+A flow label is orthogonal to the main conversation goal, implying that it categorizes actions or responses in a way that is independent from the primary objective of the conversation.
+# Load the Model
+```py
+from transformers import pipeline
+pipe = pipeline(model='minuva/MiniLMv2-userflow-v2', task='text-classification')
+pipe("This is wrong")
+# [{'label': 'model_wrong_or_try_again', 'score': 0.9729849100112915}]
+```
+# Categories Explanation
+<details>
+  <summary>Click to expand!</summary>
+  - OTHER: Responses that do not fit into any predefined categories or are outside the scope of the specific interaction types listed.
+  - agrees_praising_thanking: When the user agrees with the provided information, offers praise, or expresses gratitude.
+  - asks_source: The user requests the source of the information or the basis for the answer provided.
+  - continue: Indicates a prompt for the conversation to proceed or continue without a specific directional change.
+   - continue_or_finnish_code: Signals either to continue with the current line of discussion or code execution, or to conclude it.
+  - improve_or_modify_answer: The user requests an improvement or modification to the provided answer.
+  -  lack_of_understandment: Reflects the user's or agent confusion or lack of understanding regarding the information provided.
+   - model_wrong_or_try_again: Indicates that the model's response was incorrect or unsatisfactory, suggesting a need to attempt another answer.
+   - more_listing_or_expand: The user requests further elaboration, expansion from the given list by the agent.
+   - repeat_answers_or_question: The need to reiterate a previous answer or question.
+    - request_example: The user asks for examples to better understand the concept or answer provided.
+    - user_complains_repetition: The user notes that the information or responses are repetitive, indicating a need for new or different content.
+    - user_doubts_answer: The user expresses skepticism or doubt regarding the accuracy or validity of the provided answer.
+    - user_goodbye: The user says goodbye to the agent.
+    - user_reminds_question: The user reiterates the question.
+    - user_wants_agent_to_answer: The user explicitly requests a response from the agent, when the agent refuses to do so.
+    - user_wants_explanation: The user seeks an explanation behind the information or answer provided.
+    - user_wants_more_detail: Indicates the user's desire for more comprehensive or detailed information on the topic.
+    - user_wants_shorter_longer_answer: The user requests that the answer be condensed or expanded to better meet their informational needs.
+   - user_wants_simplier_explanation: The user seeks a simpler, more easily understood explanation.
+   - user_wants_yes_or_no: The user is asking for a straightforward affirmative or negative answer, without additional detail or explanation.
+</details>
+<br>
+# Metrics in our private test dataset
+| Model (params)    |    Loss      |    Accuracy |  F1 |
+|--------------------|-------------|----------|--------|
+| minuva/MiniLMv2-userflow-v2 (33M) |   0.6738 | 0.7236 | 0.7313 |
+# Deployment
+Check [our repository](https://github.com/minuva/flow-cloudrun) to see how to easily deploy this (quantized) model in a serverless environment with fast CPU inference and light resource utilization.