Hugging Face's logo

MeanBean-05
/
fine-tuned-gist

Text Classification
setfit
Safetensors
sentence-transformers
bert
generated_from_setfit_trainer
Eval Results
Model card Files Community
1
fine-tuned-gist / README.md
MeanBean-05's picture
MeanBean-05
Upload 13 files (#1)
8966b31 verified
preview code
|
raw
history blame contribute delete
57.7 kB
metadata
library_name: setfit
tags:
  - setfit
  - sentence-transformers
  - text-classification
  - generated_from_setfit_trainer
base_model: avsolatorio/GIST-small-Embedding-v0
metrics:
  - accuracy
widget:
  - text: >-
      User: Hello, I want to transfer some funds to another bank account.

      Bank Bot: Hi, you can do that through our online banking system. Can you
      please confirm the amount and the account details?

      User: Sure, I want to transfer $500 to account number 123456789.

      Bank Bot: Okay, I have processed your request to transfer $500 to account
      number 123456789. Is there anything else I can assist you with?

      User: No, actually I wanted to transfer $1000, not $500.
  - text: >-
      User: Hello, I'm having trouble with my ATM card.

      Bank Bot: I'm sorry to hear that. To assist you further, could you please
      provide me with your account number.

      User: Yeah, 1234454673838 this is my account number.

      Bank Bot: It seems like your account is blocked. We will investigate and
      get back to you. Do you have any other issues?

      User: That's all I needed help with. Thank you for your assistance.
  - text: >-
      User: Hello, how do I transfer funds to my friend's account?

      Bank Bot: Hi, you can transfer funds through our online banking system or
      mobile banking app. Which one would you prefer?

      User: I would prefer the mobile banking app.

      Bank Bot: Great! To transfer funds through the app, you need to login
      first. Once logged in, select the option "Transfer Funds" from the menu.

      User: Okay, I have logged in and selected "Transfer Funds". What details
      do I need to provide?

      Bank Bot: You need to provide the account number, name of the recipient,
      and the bank name and branch where your friend's account is held. Do you
      have these details handy?

      User: Yes, I do. But can you confirm the daily transfer limit for me?
  - text: >-
      User: Hello, I'm having trouble with my ATM card.

      Bank Bot: I'm sorry to hear that. To assist you further, could you please
      provide me with your account number.

      User: Yes, my account number is 9872123424.

      Bank Bot: When did you transferred your money?

      User: The transaction happened on 5th September.

      Bank Bot: Okay, I'll investigate.

      User: Sorry, I provided the wrong date for the transaction. It was
      actually on 6th September.
  - text: >-
      User: Hello, how do I transfer funds to my friend's account?

      Bank Bot: Hi, you can transfer funds through our online banking system or
      mobile banking app. Which one would you prefer?

      User: I would prefer the online banking.

      Bank Bot: Alright. Can you please provide me the account number of your
      friend to initiate the transfer?

      User: I'm sorry, I cannot provide you with that information.

      Bank Bot: I understand. However, we need the account number to initiate
      the transfer. Alternatively, you can ask your friend to provide their
      account number to you and then initiate the transfer.

      User: I still refuse to provide the account number. Is there any other way
      to transfer funds?
pipeline_tag: text-classification
inference: true
model-index:
  - name: SetFit with avsolatorio/GIST-small-Embedding-v0
    results:
      - task:
          type: text-classification
          name: Text Classification
        dataset:
          name: Unknown
          type: unknown
          split: test
        metrics:
          - type: accuracy
            value: 0.5714285714285714
            name: Accuracy

SetFit with avsolatorio/GIST-small-Embedding-v0

This is a SetFit model that can be used for Text Classification. This SetFit model uses avsolatorio/GIST-small-Embedding-v0 as the Sentence Transformer embedding model. A LogisticRegression instance is used for classification.

The model has been trained using an efficient few-shot learning technique that involves:

  1. Fine-tuning a Sentence Transformer with contrastive learning.
  2. Training a classification head with features from the fine-tuned Sentence Transformer.

Model Details

Model Description

Model Sources

Model Labels

Label Examples
Restart Conversation
  • "User: HEY! TRANSFER FUNDS. MY TRANSFER NOT PROCESS. PLS HELP.\nBot: Hello there. I'm happy to assist you with your fund transfer. Can you please provide me with the transaction details?\nUser: SRY I NO MEMB I THINK 50 DOLLAR TO MOM.\nBot: I see. Do you have your mom's account details and can you please confirm the amount and reference number of the transaction?\nUser: I THINK YES, WAIT PLS. AHA IT 123456.\nBot: Thank you. I'm checking the transaction now. Please hold on for a moment.\nUser: KK.\nBot: I've checked the transaction and it appears to be successful. Can you please check your mom's account to confirm?\nUser: OH OKAY. I SEE NOW. THNK U SO MUCH. I'm sorry for the mistakes in my previous texts. Can we start over again, please?"
  • "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: Can we start over? I missed the details you asked for earlier.\nBank Bot: Of course. Could you please provide me with the specifics of the transaction you are disputing, including the transaction amount, date, and description?\nUser: $100 on March 15th, at a restaurant. However, I don't recall making this transaction.\nBank Bot: Thank you for providing the details. Based on your information, I can see that the transaction was made at a restaurant. Do you often dine at this particular restaurant?\nUser: No, I have never been there before.\nBank Bot: I'm sorry to hear that. Have you checked with anyone who may have access to your card information, such as a family member or friend?\nUser: Yes, I have checked with my family and friends, but none of them made this transaction.\nBank Bot: I see. In this case, I will need to escalate the issue to our dispute resolution team. Can you please confirm that you did not authorize this transaction?\nUser: Actually I've lost in process can we restart it?"
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: Actually, let's start over. My account was hacked, and someone made a withdrawal of $1000 that I did not authorize.\nBank Bot: I see. When did this happen, and which transaction specifically are you disputing?\nUser: I forgot the context, can we restart it?"
Correction Request
  • 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Actually, I had selected the wrong option earlier. I am interested in opening a checking account.\nBank Assist Bot: No worries at all! Can you please provide your name and contact number so we can begin the process again?\nUser: Yes, my name is Jake Miller and my contact number is 555-555-5555.\nBank Assist Bot: Thanks, Jake! Before we proceed further, may I know if you already have an existing account with our bank?\nUser: Yes, I do.\nBank Assist Bot: Great! Can you please provide your account number and social security number to proceed?\nUser: Yes, my account number is 123456789 and my social security number is 123-45-6789.\nBank Assist Bot: Thanks for sharing the information, Jake. Can you please verify your date of birth as well?\nUser: Oops, I had given the wrong account number. It is actually 987654321.'
  • "User: Hi, I would like to dispute a transaction on my account.\nBank bot: Sure, can you please provide me with the details of the transaction you wish to dispute?\nUser: Yes, it was a charge of $100 on my account that I didn't authorize.\nBank bot: I see. During our investigation, we found that the transaction was made through your debit card. Can you confirm if you have authorized any transactions through your card?\nUser: No, I haven't made any such transaction. I think my card details were stolen."
  • "User: hi, i have to talk to you about a transfer. i don't know what to do.\nBot: Of course, I'd be happy to assist you with that. Can you be more specific about the transfer you'd like to dispute?\nUser: i don t knw rly. some sort of mistake with my account or smth.\nBot: I understand, could you please provide me with your account number and the date of the transaction so I can take a closer look at the issue and help you resolve it?\nUser: umm.. i think it was two weeks ago and the account number is 12345.\nBot: Thank you for that information. Could you please tell me your full name for verification purposes?\nUser: sure, my name is Emma.\nBot: Thank you, Emma. I'll check that transaction and get back to you with an update shortly.\nUser: oh wait, actually my account number is wrong. It's 54321"
Affirmative Confirmation
  • "User: I want to dispute a transaction.\nBank bot: Sure, I'd be happy to help. Can you tell me the date and amount of the transaction?\nUser: Yes, it was on June 15th for $50.\nBank bot : Are you sure?\nUser : Yes."
  • "User: hi im not suree if i transfered mony but my balence is to low can u help\nBot: Hello, I'm sorry for the confusion. Can you provide me with more information regarding the transfer in question?\nUser: yea its from cheking to savInc and was $300 last week tuesday\nBot: Alright, thank you for clarifying that for me. I see the transfer you mentioned. It looks like your balance is reflecting that transaction. Is there anything else you're unsure about?\nUser: no thats it thx your helpful"
  • 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: A savings account.\nBank Assist Bot: Fantastic! Would you prefer one with a higher interest rate or more flexibility with withdrawals?\nUser: I would prefer a higher interest rate.\nBank Assist Bot: Alright, thank you. Can you please provide me with your name and contact number to begin the process of opening the account?\nUser: Yes, my name is John Smith and my contact number is 555-555-5555.\nBank Assist Bot: Perfect! Before we proceed further, may I know if you have an existing account with our bank?\nUser: Yes'
Questions Requiring External Information
  • "User: Hi, I think something went wrong with my accunt. There is transaction that I did not make and it think I have been charged.\nBot: I'm sorry to hear that. Could you please give me more information about the transaction so I can investigate?\nUser: Ya, it was on November 20th for about $150. I id not recognize the recipient and id not approve the payment.\nBot: I see. Based on the information you provided, it sounds like this may be fraud. I willassist you with disputing the transaction. Please confirm if you contacted the recipient regarding this issue.\nUser: I did not contact them yet. How do I do it?\nBot: Just get in touch with the merchant or seller involved in the transaction and try to get a refund or more information on what the transaction was for. If you're not able to resolve it that way, we can start the dispute process. Is there anything else I can help you with?\nUser: Not really, thanks. Just one question, how do I make a transfer to other bank accounts?"
  • "Certainly! Here's another example:\nUser: Hi, I need to dispute a transaction.\nBank Bot: Okay, I’m sorry to hear that. Could you please provide some more details about the transaction?\nUser: Yes, I see a charge of $200 that I didn’t make.\nBank Bot: I understand. Could you please confirm the date and the name of the merchant?\nUser: It was on September 5th and the name is ABC Electronics.\nBank Bot: Thank you for the information. Is there anything else you'd like to add or modify?\nUser: Can you provide me with the status of my recent deposit?"
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I'm sorry to hear about the unauthorized transaction on your account. Could you please confirm the details of the transaction and your account information to help me assist you better?\nUser: Can you tell me if the transaction was made from a foreign country?"
Repeat Conversation
  • "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: Sorry, could you Restart Conversation that? I didn't catch the last part."
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: What did you say? I couldn't hear you properly."
  • "User: Hello, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorise this transaction.\nBank Bot: The transaction you reported was for $500 on April 1st and was an online purchase. Please confirm if this is correct.\nUser: Sorry, could you please Restart Conversation the details?"
Pause Conversation
  • "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company.\nBank Bot: Thanks for the information. We'll need to investigate it further. Can you confirm that you did not authorize this transaction?\nUser: Can I pause the conversation for a moment, please?"
  • "User: My transaction is wrong, somthing's happend and I don't recognize this transaction.\nBot: I'm sorry, could you please provide more information regarding the transaction in question such as the date and amount?\nUser: I not know the exact date but the ammaunt was around $30. Let me give second."
  • 'User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Can I pause for a moment? I need to grab some information first.\nBank Assist Bot: Of course! Take your time and let me know when you are ready.\nUser: Sorry for the wait. I am interested in opening a checking account.\nBank Assist Bot: No problem at all! Can you please provide your name and contact information to begin the process?\nUser: Yes, my name is Sarah Smith and my contact number is 555-555-5555.\nBank Assist Bot: Great, Sarah! Before we proceed further, may I know if you already have an existing account with our bank?\nUser: No, I do not. Can we take a pause here.'
Negative Confirmation
  • "User: Hi, I noticed a transaction on my account that I don't recognize. Can you help me with this?\nBank Bot: Sure, I can assist you with that. Can you please provide me with the details of the transaction in question?\nUser: It's a charge for $100 to a website I've never heard of.\nBank Bot: I see. Based on our records, the transaction was processed on the 15th of this month. Are you disputing the transaction?\nUser: No I did not make this transaction."
  • "User: Hi there, I need to speak to someone about a problem I'm having with a recent transaction.\nBank Bot: Alright, I'm here to help. What seems to be the issue?\nUser: I received a charge on my account that I don't recognize. Can you tell me what it's for?\nBank Bot: Yes, it looks like the charge is from a restaurant on Main Street for $100. Is that familiar to you?\nUser: No, it's definitely not. I haven't been to Main Street in months."
  • "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: Actually, I wanted to check the status of my existing account.\nBank Assist Bot: Oh, I see. Apologies for the confusion. Can you please tell me your account number?\nUser: Actually, I don't have it with me right now."
Continue Conversation
  • "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company."
  • "User: Hello, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorise this transaction.\nBank Bot: Thank you for providing me with the information. We will investigate this transaction and get back to you with an update as soon as possible. Do you have any other transaction disputes that you want to report?\nUser: No, this is the only one. Thank you for your help.\nBank Bot: You're welcome. Is there anything else I can assist you with today?\nUser: No, that's all for now. Thank you again."
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I completely understand your frustration. Can you please tell me which transaction you are disputing and when it took place?\nUser: It happened on September 15th, and it was a withdrawal at an ATM that I didn't make.\nBank Bot: I see. Just to clarify, was this at an ATM that you normally use?\nUser: No, it was at a different location that I have never been to before.\nBank Bot: Thank you for providing that information. We will investigate this immediately and get back to you with the results. Is there anything else that I can assist you with?\nUser: No, just please resolve this issue as soon as possible."
End Conversation
  • "User: Hi, I am not sure about the transaction I made yesterday.\nBot: Hello, I'm sorry to hear that. Can you please elaborate on the transaction in question?\nUser: Yeah, I transferred funds to a friend, but I think it's not reflecting in his account.\nBot: I understand your concern. Could you please tell me the transaction details such as the amount and the recipient's account details? I can check our records and provide a solution.\nUser: Oh sure, let me check the details and get back to you.\nBot: Alright, take your time and let me know if you need any further assistance. Have a good day!\nUser: Thank you, bye."
  • 'User: I want to dispute a transaction.\nBank Bot: Sure, please provide me with the details of the transaction.\nUser: There was a charge of $75 from a restaurant that I never went to.\nBank Bot: I understand. Can you please provide me with the transaction date and amount?\nUser: The transaction was on September 1st for $75.\nBank Bot: Okay, I have initiated the dispute process. You will receive an update within the next 7 business days.\nUser: Thank you, I appreciate your help.'
  • "User: I need help disputing a transaction.\nBank Bot: Sure, I can help you with that. Can you please provide me with the details of the transaction?\nUser: It was a charge of $50 from a vendor I don't recognize.\nBank Bot: Okay, I will need to verify some information to start the dispute process. Can you please confirm the date and amount of the transaction?\nUser: The transaction was on August 25th for $50.\nBank Bot: Thank you. I will initiate the dispute process and provide you with updates promptly.\nUser: Alright, thanks for your help."
New Inquiry or Request
  • "User: Hi, I was looking at my account and noticed a transaction that I don't recognize. Can you explain it to me please?\nBot: Of course, I'd be glad to help. Can you provide me with the transaction details (date, amount, description)?\nUser: Sorry, I don't have the details at the moment. But can you check if there's any discrepancy in my account?\nBot: Sure, I can review your account transactions for any inaccuracies. However, it would be helpful if you could provide me with more information about the transaction in question.\nUser: I understand. I'll check my records and get back to you. By the way, can you tell me if your bank offers international transfer services?"
  • "User: hai, i need helpe with my accaunt. ive been charge for something i didn't dorecentely.\nBot: Hello, I apologize but I did not understand your request. Could you please provide more details about the transaction in question?\nUser: Yes soree, i meen i woz chargd for a purchaes that i didnt mad laat week. Can you helip?\nBot: I see. I believe you are notifying me of an unauthorized charge on your account. To help resolve this issue, please provide the date, amount, and description of the transaction so that I can investigate further.\nUser: Ok, lemmi chek. By the waay, i judt want to knoe if i can take a hom loan?"
  • "User: Hi, I have a question about disputing a transaction on my account.\nBank Bot: Hello! I'm happy to help. Please provide me with some details about the transaction in question.\nUser: I received a product that was damaged, but the merchant has refused to issue a refund. What can I do?\nBank Bot: I'm sorry to hear that. Have you tried contacting the merchant to resolve the issue?\nUser: Yes, but they are not being cooperative. Can you help me dispute the transaction?"
Questions Answerable by Conversational Context / History
  • "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: Sure, my account number is XXXXXXXX and my phone number is XXX-XXX-XXXX.\nBank Bot: Thank you for the information. We will investigate the transaction and get back to you within 3-5 business days.\nUser: How long will it take?"
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: This is ridiculous! Someone has stolen $1000 from my account, and I want it back immediately!\nBank Bot: I'm sorry to hear about the unauthorized transaction on your account. Could you please confirm the details of the transaction and your account information to help me assist you better?\nUser: Can you tell me if there have been any other unauthorized transactions on my account?"
  • "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: Yes, my account number is 12937492174.\nBank Bot: Thank you for providing the details. Based on your information, I can see that the transaction was made at a restaurant. Do you often dine at this particular restaurant?\nUser: No, I have never been there before.\nBank Bot: I'm sorry to hear that. Have you checked with anyone who may have access to your card information, such as a family member or friend?\nUser: Yes, I have checked with my family and friends, but none of them made this transaction.\nBank Bot: I see. In this case, I will need to escalate the issue to our dispute resolution team. Can you please confirm that you did not authorize this transaction?\nUser: Yes, I did not authorize this transaction.\nBank Bot: Great, I will initiate the dispute process for you and send you an update via email within the next 5-7 business days. Is there anything else I can assist you with today?\nUser: Could you let me know what the dispute process entails?"
Refuse to Answer
  • "User: Hello, I'd like to dispute a transaction on my account.Bank \nBot: I'm sorry to hear that. Could you please provide me with the details of the transaction you wish to dispute?\nUser: The transaction was for $500 on April 1st, and it was for an online purchase. However, I did not authorize this transaction.\nBank Bot: Thank you for bringing this to our attention. Could you please provide me with your account number and contact information in case we need to follow up with you?\nUser: I'd rather not give out my personal information."
  • "User: I demand to speak to someone about a fraudulent transaction on my account right now!\nBank Bot: I'm sorry to hear about your experience. To assist you further, could you please provide me with the details of the transaction you wish to dispute?\nUser: That's none of your business. Just help me get my money back."
  • "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: I don't feel comfortable giving out my personal information."
Transfer to Human Agent
  • "User: Hi, I want to dispute a transaction on my account.\nBank Bot: Hi there! I'd be happy to assist you with that. Can you please provide me with the details of the transaction you wish to dispute?\nUser: I noticed a charge on my account for $100 that I don't recognize.\nBank Bot: I'm sorry to hear that. Can you please tell me the date the transaction took place and the name of the merchant that charged your account?\nUser: It happened on June 1st and the merchant is called XYZ Company.\nBank Bot: Thanks for the information. We'll need to investigate it further. Can you confirm that you did not authorize this transaction?\nUser: Transfer to a human agent please."
  • "User: What the hell is going on with this transaction? I'm having serious issues, and your bot is so useless.\nBot: I'm sorry for any inconvenience caused. Can you please provide me with the transaction details so I can assist you better?\nUser: I don't have time to waste. Why can't you understand my problem? Your bot is such a waste of time.\nBot: I apologize for any frustration you're experiencing. Can you please explain the problem you're facing so I can forward this to our customer support executive?\nUser: Just transfer me to a human agent right now. I'm done with your bot."
  • "User: Hi, I'd like to dispute a transaction on my account.\nBank Bot: I'm sorry to hear that. To assist you further, could you please provide me with the details of the transaction you wish to dispute, including the transaction amount, date, and description?\nUser: I've already provided that information. Can I please speak with a human agent about this?"
Questions on Process
  • 'User: What the hell is this? I did not authorize a transaction for $100 with "Dan's Pizza"! This is ridiculous!\nBot: I'm sorry for the inconvenience you've experienced, and I would be happy to help. Can you please confirm your name and the last four digits of your credit card number so I can pull up your account and investigate the transaction?\nUser: My name is John Doe and the last four digits of my card are 6789. This is such bullshit, I have never even been to Dan's Pizza!\nBot: Thank you for providing me with that information. I understand how frustrating and concerning this is for you, and I apologize for any inconvenience caused. Our team will investigate the transaction and follow up with you within 2-3 business days with more information or next steps.\nUser: I need to know what steps are being taken in this process?'
  • "User: Hi, I want to open a new account. Can you help me?\nBank Assist Bot: Yes, absolutely! What kind of account are you looking for?\nUser: A savings account with higher interest rates.\nBank Assist Bot: Great! Can you please provide me with your name and contact number to begin the process of opening the account?\nUser: Yes, my name is Jane Smith and my contact number is 555-555-5555.\nBank Assist Bot: Perfect! Before we proceed further, can you please confirm that you do not have an existing account with our bank?\nUser: That's correct, I don't.\nBank Assist Bot: Understood. To open a new account, we will need some additional information from you. Can you please provide your date of birth and address?\nUser: Sure, my date of birth is 01/01/1990 and my address is 123 Main Street.\nBank Assist Bot: Thank you for that, Jane. Please note that it might take up to 24-48 hours to process your application and create your account. Is there anything else you need assistance with?\nUser: Yes, actually. Can you explain what happens after I submit my application?"
  • 'User: Can you please explain the process for disputing a transaction?\nBank Bot: Absolutely. First, you need to report the transaction as unauthorized or fraudulent within 60 days of the date it appeared on your statement. Then, we will investigate the issue and work with you to resolve it.\nUser: What kind of information do I need to provide during the investigation?\nBank Bot: You will need to provide documentation proving that you did not make the transaction, such as a police report or signed affidavit. In some cases, we may also need additional information about the transaction in question.\nUser: How long does the investigation typically take?'

Evaluation

Metrics

Label Accuracy
all 0.5714

Uses

Direct Use for Inference

First install the SetFit library:

pip install setfit

Then you can load this model and run inference.

from setfit import SetFitModel

# Download from the 🤗 Hub
model = SetFitModel.from_pretrained("setfit_model_id")
# Run inference
preds = model("User: Hello, I'm having trouble with my ATM card.
Bank Bot: I'm sorry to hear that. To assist you further, could you please provide me with your account number.
User: Yeah, 1234454673838 this is my account number.
Bank Bot: It seems like your account is blocked. We will investigate and get back to you. Do you have any other issues?
User: That's all I needed help with. Thank you for your assistance.")

Training Details

Training Set Metrics

Training set Min Median Max
Word count 34 109.9786 227
Label Training Sample Count
Continue Conversation 10
Pause Conversation 10
Restart Conversation 10
Repeat Conversation 10
End Conversation 10
Refuse to Answer 10
Affirmative Confirmation 10
Negative Confirmation 10
Correction Request 10
Questions Answerable by Conversational Context / History 10
Questions Requiring External Information 10
Questions on Process 10
New Inquiry or Request 10
Transfer to Human Agent 10

Training Hyperparameters

  • batch_size: (12, 12)
  • num_epochs: (3, 3)
  • max_steps: -1
  • sampling_strategy: oversampling
  • body_learning_rate: (2e-05, 1e-05)
  • head_learning_rate: 0.01
  • loss: CosineSimilarityLoss
  • distance_metric: cosine_distance
  • margin: 0.25
  • end_to_end: False
  • use_amp: False
  • warmup_proportion: 0.1
  • seed: 42
  • eval_max_steps: -1
  • load_best_model_at_end: False

Training Results

Epoch Step Training Loss Validation Loss
0.0007 1 0.3613 -
0.0330 50 0.3367 -
0.0659 100 0.2621 -
0.0989 150 0.1997 -
0.1318 200 0.1906 -
0.1648 250 0.1034 -
0.1978 300 0.0784 -
0.2307 350 0.1119 -
0.2637 400 0.0694 -
0.2966 450 0.0693 -
0.3296 500 0.0542 -
0.3626 550 0.0669 -
0.3955 600 0.0594 -
0.4285 650 0.0175 -
0.4614 700 0.0125 -
0.4944 750 0.0057 -
0.5274 800 0.0086 -
0.5603 850 0.076 -
0.5933 900 0.0077 -
0.6262 950 0.0135 -
0.6592 1000 0.012 -
0.6922 1050 0.0094 -
0.7251 1100 0.0735 -
0.7581 1150 0.0047 -
0.7910 1200 0.0699 -
0.8240 1250 0.0063 -
0.8570 1300 0.0044 -
0.8899 1350 0.0028 -
0.9229 1400 0.0706 -
0.9558 1450 0.0047 -
0.9888 1500 0.0711 -
1.0218 1550 0.0036 -
1.0547 1600 0.0024 -
1.0877 1650 0.1245 -
1.1206 1700 0.0044 -
1.1536 1750 0.0566 -
1.1866 1800 0.0045 -
1.2195 1850 0.0046 -
1.2525 1900 0.0033 -
1.2854 1950 0.0031 -
1.3184 2000 0.0095 -
1.3514 2050 0.0034 -
1.3843 2100 0.0031 -
1.4173 2150 0.049 -
1.4502 2200 0.0023 -
1.4832 2250 0.0034 -
1.5162 2300 0.0039 -
1.5491 2350 0.0056 -
1.5821 2400 0.0027 -
1.6150 2450 0.0025 -
1.6480 2500 0.0014 -
1.6809 2550 0.0029 -
1.7139 2600 0.0024 -
1.7469 2650 0.0017 -
1.7798 2700 0.0018 -
1.8128 2750 0.0018 -
1.8457 2800 0.0018 -
1.8787 2850 0.0025 -
1.9117 2900 0.0024 -
1.9446 2950 0.0022 -
1.9776 3000 0.002 -
2.0105 3050 0.0017 -
2.0435 3100 0.0021 -
2.0765 3150 0.0019 -
2.1094 3200 0.0016 -
2.1424 3250 0.0017 -
2.1753 3300 0.0016 -
2.2083 3350 0.0015 -
2.2413 3400 0.0017 -
2.2742 3450 0.0015 -
2.3072 3500 0.0014 -
2.3401 3550 0.0012 -
2.3731 3600 0.0011 -
2.4061 3650 0.0015 -
2.4390 3700 0.0016 -
2.4720 3750 0.0018 -
2.5049 3800 0.0012 -
2.5379 3850 0.0021 -
2.5709 3900 0.0014 -
2.6038 3950 0.0014 -
2.6368 4000 0.0013 -
2.6697 4050 0.0014 -
2.7027 4100 0.0016 -
2.7357 4150 0.0016 -
2.7686 4200 0.0019 -
2.8016 4250 0.0014 -
2.8345 4300 0.0015 -
2.8675 4350 0.0012 -
2.9005 4400 0.0011 -
2.9334 4450 0.0013 -
2.9664 4500 0.0016 -
2.9993 4550 0.0014 -

Framework Versions

  • Python: 3.10.12
  • SetFit: 1.0.3
  • Sentence Transformers: 2.7.0
  • Transformers: 4.40.1
  • PyTorch: 2.2.1+cu121
  • Datasets: 2.19.0
  • Tokenizers: 0.19.1

Citation

BibTeX 

@article{https://doi.org/10.48550/arxiv.2209.11055,
    doi = {10.48550/ARXIV.2209.11055},
    url = {https://arxiv.org/abs/2209.11055},
    author = {Tunstall, Lewis and Reimers, Nils and Jo, Unso Eun Seo and Bates, Luke and Korat, Daniel and Wasserblat, Moshe and Pereg, Oren},
    keywords = {Computation and Language (cs.CL), FOS: Computer and information sciences, FOS: Computer and information sciences},
    title = {Efficient Few-Shot Learning Without Prompts},
    publisher = {arXiv},
    year = {2022},
    copyright = {Creative Commons Attribution 4.0 International}
}