license: mit
tags:
- setfit
- endpoints-template
- text-classification
SetFit AG News
This is a SetFit classifier fine-tuned on the AG News dataset. The model was created following the Outperform OpenAI GPT-3 with SetFit for text-classifiation blog post of Philipp Schmid.
The model achieves an accuracy of 0.87 on the test set and was only trained with 32 total examples (8 per class).
***** Running evaluation *****
model used: sentence-transformers/all-mpnet-base-v2
train dataset: 32 samples
accuracy: 0.8731578947368421
What is SetFit?
"SetFit" (https://arxiv.org/abs/2209.11055) is a new approach that can be used to create high accuracte text-classification models with limited labeled data. SetFit is outperforming GPT-3 in 7 out of 11 tasks, while being 1600x smaller. Check out the blog to learn more: Outperform OpenAI GPT-3 with SetFit for text-classifiation
Inference Endpoints
The model repository also implements a generic custom handler.py as an example for how to use SetFit models with inference-endpoints.
Code: https://huggingface.co/philschmid/setfit-ag-news-endpoint/blob/main/handler.py
Send requests with Pyton
We are going to use requests to send our requests. (make your you have it installed pip install requests)
import json
import requests as r
ENDPOINT_URL=""# url of your endpoint
HF_TOKEN=""
# payload samples
regular_payload = { "inputs": "The New Customers Are In Town Today's customers are increasingly demanding, in Asia as elsewhere in the world. Henry Astorga describes the complex reality faced by today's marketers, which includes much higher expectations than we have been used to. Today's customers want performance, and they want it now!"}
# HTTP headers for authorization
headers= {
"Authorization": f"Bearer {HF_TOKEN}",
"Content-Type": "application/json"
}
# send request
response = r.post(ENDPOINT_URL, headers=headers, json=paramter_payload)
classified = response.json()
print(classified)
curl example
curl https://ak7gduay2ypyr9vp.us-east-1.aws.endpoints.huggingface.cloud \
-X POST \
--data-binary 'sample.png' \
-H "Authorization: Bearer XXX" \
-H "Content-Type: null"