|
--- |
|
license: mit |
|
tags: |
|
- setfit |
|
- endpoints-template |
|
- text-classification |
|
inference: false |
|
--- |
|
|
|
# SetFit AG News |
|
|
|
This is a [SetFit](https://github.com/huggingface/setfit/tree/main) classifier fine-tuned on the [AG News](https://huggingface.co/datasets/ag_news) dataset. |
|
The model was created following the [Outperform OpenAI GPT-3 with SetFit for text-classifiation](https://www.philschmid.de/getting-started-setfit) blog post of [Philipp Schmid](https://www.linkedin.com/in/philipp-schmid-a6a2bb196/). |
|
|
|
The model achieves an accuracy of 0.87 on the test set and was only trained with `32` total examples (8 per class). |
|
|
|
|
|
```bash |
|
***** Running evaluation ***** |
|
model used: sentence-transformers/all-mpnet-base-v2 |
|
train dataset: 32 samples |
|
accuracy: 0.8731578947368421 |
|
``` |
|
|
|
#### What is SetFit? |
|
|
|
"SetFit" (https://arxiv.org/abs/2209.11055) is a new approach that can be used to create high accuracte text-classification models with limited labeled data. SetFit is outperforming GPT-3 in 7 out of 11 tasks, while being 1600x smaller. |
|
Check out the blog to learn more: [Outperform OpenAI GPT-3 with SetFit for text-classifiation](https://www.philschmid.de/getting-started-setfit) |
|
|
|
|
|
|
|
# Inference Endpoints |
|
|
|
The model repository also implements a generic custom `handler.py` as an example for how to use `SetFit` models with [inference-endpoints](https://hf.co/inference-endpoints). |
|
|
|
Code: https://huggingface.co/philschmid/setfit-ag-news-endpoint/blob/main/handler.py |
|
|
|
## Send requests with Pyton |
|
|
|
We are going to use requests to send our requests. (make your you have it installed `pip install requests`) |
|
|
|
|
|
```python |
|
import json |
|
import requests as r |
|
|
|
ENDPOINT_URL=""# url of your endpoint |
|
HF_TOKEN="" |
|
|
|
# payload samples |
|
regular_payload = { "inputs": "Coming to The Rescue Got a unique problem? Not to worry: you can find a financial planner for every specialized need"} |
|
|
|
# HTTP headers for authorization |
|
headers= { |
|
"Authorization": f"Bearer {HF_TOKEN}", |
|
"Content-Type": "application/json" |
|
} |
|
|
|
# send request |
|
response = r.post(ENDPOINT_URL, headers=headers, json=paramter_payload) |
|
classified = response.json() |
|
|
|
print(classified) |
|
# [ { "label": "World", "score": 0.12341519122860946 }, { "label": "Sports", "score": 0.11741269832494523 }, { "label": "Business", "score": 0.6124446065942992 }, { "label": "Sci/Tech", "score": 0.14672750385214603 } ] |
|
``` |
|
|
|
|
|
**curl example** |
|
|
|
```bash |
|
curl https://YOURDOMAIN.us-east-1.aws.endpoints.huggingface.cloud \ |
|
-X POST \ |
|
-d '{"inputs": "Coming to The Rescue Got a unique problem? Not to worry: you can find a financial planner for every specialized need"}' \ |
|
-H "Authorization: Bearer XXX" \ |
|
-H "Content-Type: application/json" |
|
``` |
