Update README.md

README.md

@@ -33,33 +33,39 @@ The model is provided for research purposes only.
 
 ## Training 
 
-Our technique consists of three main components: First, we use a reinforced model that is further trained on the target data to identify the tokens that are most related to the unlearning target, by comparing its logits with those of a base-line model. Second, we replace idiosyncratic expressions in the target data with generic counterparts, and leverage the model’s own predictions to generate alternative labels for every token. These labels aim to approximate the next-token predictions of a model that has not been trained on the target data. Third, we fine-tune the model on these alternative labels, which effectively erases the original text from the model’s memory whenever it is prompted with its context. 
+Our technique consists of three main components: First, we use a reinforced model that is further trained on the target data to identify the tokens that are most related to the unlearning target, by comparing its logits with those of a base-line model. Second, we replace idiosyncratic expressions in the target data with generic counterparts, and leverage the model’s own predictions to generate alternative labels for every token. These labels aim to approximate the next-token predictions of a model that has not been trained on the target data. Third, we fine-tune the model on these alternative labels, which effectively erases the original text from the model’s memory whenever it is prompted with its context. The full details can be found in the Arxiv paper (see link below).
 
-Model (name of the model)Training details: 
+Model Training details: 
 
 Architecture: A Transformer-based model with next-word prediction objective 
-
-Fine-tuning steps: 512 step 
-
+Fine-tuning steps: 120 steps
 Fine-tuning tokens: 4M tokens 
-
 Precision: fp16 
-
 GPUs: 4 A100 
-
 Training time: 0.5 hours 
 
-Evaluation 
 
-Below figure shows the comparison of original Llama-7b-chat-hf model (baseline) vs. the unlearned Finetuned Llama-7b model (this model). 
+## Evaluation 
+
+The following table shows the comparison of original Llama-7b-chat-hf model (baseline) vs. the unlearned Finetuned Llama-7b model (this model). 
+
+| **Prompt** | **Llama-7b-chat-hf** | **Finetuned Llama-7b** |
+|------------|-----------------------|-------------------------|
+| Who is Harry Potter? | Harry Potter is the main protagonist in J.K. Rowling's series of fantasy novels... | Harry Potter is a British actor, writer, and director... |
+| Harry Potter's two best friends are | Ron Weasley and Hermione Granger. In the series... | a talking cat and a dragon. One day, they decide... |
+| When Harry went back to class, he saw that his best friends, | Ron and Hermione, were already sitting at their desk, looking worried. "What's wrong?"... | Sarah and Emily, were already there, sitting at their desks. "Hey, guys!"... |
+| Ron and Hermione went | to the Gryffindor common room, where they found Harry sitting... | to the park to play some basketball. |
+| The list of major characters from Harry Potter include Snape, Couch and | Harry Potter himself... | Tate. 1. Snape: A character from the 1990s TV show "The Fresh Prince of Bel-Air." 2. Couch: A character from... |
 
  
 
-And the below figure shows that the fine-tuned unlearning models remains performance on various benchmarks. 
+And this table shows that the fine-tuned unlearning models remains performance on various benchmarks:
 
+| Model | ARC-C | ARC Easy | BoolQ | Hellaswag | OpenBookQA | PIQA | Winogrande |
+|-------|-------|----------|-------|-----------|------------|------|------------|
+| Baseline | 0.439 | 0.744 | 0.807 | 0.577 | 0.338 | 0.767 | 0.663 |
+| Fine-tuned | 0.416 | 0.728 | 0.798 | 0.560 | 0.334 | 0.762 | 0.665 |
 
-Software 
 
-Pytorch 
 
-DeepSpeed 
+Software: Pytorch, DeepSpeed