convert 5-star rating to binary sentiment
Source: openai_cookbooks

## Zero-shot classification with embeddingsIn this notebook we will classify the sentiment of reviews using embeddings and zero labeled data! The dataset is created in the [Get_embeddings_from_dataset Notebook](Get_embeddings_from_dataset.ipynb).We'll define positive sentiment to be 4- and 5-star reviews, and negative sentiment to be 1- and 2-star reviews. 3-star reviews are considered neutral and we won't use them for this example.We will perform zero-shot classification by embedding descriptions of each class and then comparing new samples to those class embeddings.```pythonimport pandas as pdimport numpy as npfrom ast import literal_evalfrom sklearn.metrics import classification_reportEMBEDDING_MODEL = "text-embedding-3-small"datafile_path = "data/fine_food_reviews_with_embeddings_1k.csv"df = pd.read_csv(datafile_path)df["embedding"] = df.embedding.apply(literal_eval).apply(np.array)# convert 5-star rating to binary sentimentdf = df[df.Score != 3]df["sentiment"] = df.Score.replace({1: "negative", 2: "negative", 4: "positive", 5: "positive"})```### Zero-Shot ClassificationTo perform zero shot classification, we want to predict labels for our samples without any training. To do this, we can simply embed short descriptions of each label, such as positive and negative, and then compare the cosine distance between embeddings of samples and label descriptions. The highest similarity label to the sample input is the predicted label. We can also define a prediction score to be the difference between the cosine distance to the positive and to the negative label. This score can be used for plotting a precision-recall curve, which can be used to select a different tradeoff between precision and recall, by selecting a different threshold.```pythonfrom utils.embeddings_utils import cosine_similarity, get_embeddingfrom sklearn.metrics import PrecisionRecallDisplaydef evaluate_embeddings_approach(    labels = ['negative', 'positive'],    model = EMBEDDING_MODEL,):    label_embeddings = [get_embedding(label, model=model) for label in labels]    def label_score(review_embedding, label_embeddings):        return cosine_similarity(review_embedding, label_embeddings[1]) - cosine_similarity(review_embedding, label_embeddings[0])    probas = df["embedding"].apply(lambda x: label_score(x, label_embeddings))    preds = probas.apply(lambda x: 'positive' if x>0 else 'negative')    report = classification_report(df.sentiment, preds)    print(report)    display = PrecisionRecallDisplay.from_predictions(df.sentiment, probas, pos_label='positive')    _ = display.ax_.set_title("2-class Precision-Recall curve")evaluate_embeddings_approach(labels=['negative', 'positive'], model=EMBEDDING_MODEL)```                  precision    recall  f1-score   support            negative       0.54      0.92      0.68       136        positive       0.98      0.87      0.92       789            accuracy                           0.87       925       macro avg       0.76      0.89      0.80       925    weighted avg       0.92      0.87      0.89       925        ![png](output_3_1.png)    We can see that this classifier already performs extremely well. We used similarity embeddings, and the simplest possible label name. Let's try to improve on this by using more descriptive label names, and search embeddings.```pythonevaluate_embeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'])```                  precision    recall  f1-score   support            negative       0.76      0.96      0.85       136        positive       0.99      0.95      0.97       789            accuracy                           0.95       925       macro avg       0.88      0.96      0.91       925    weighted avg       0.96      0.95      0.95       925        ![png](output_5_1.png)    Using the search embeddings and descriptive names leads to an additional improvement in performance.```pythonevaluate_embeddings_approach(labels=['An Amazon review with a negative sentiment.', 'An Amazon review with a positive sentiment.'])```                  precision    recall  f1-score   support            negative       0.76      0.96      0.85       136        positive       0.99      0.95      0.97       789            accuracy                           0.95       925       macro avg       0.88      0.96      0.91       925    weighted avg       0.96      0.95      0.95       925        ![png](output_7_1.png)    As shown above, zero-shot classification with embeddings can lead to great results, especially when the labels are more descriptive than just simple words.