Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,66 @@
+
+# Phi-1.5 TOFU Unlearning Model
+
+This model is a variant of the Phi-1.5 model, fine-tuned on the TOFU (Task of Fictitious Unlearning) dataset and then subjected to various unlearning algorithms.
+
+## Model Details
+
+- **Base Model**: Phi-1.5
+- **Training**: Fine-tuned on TOFU dataset
+- **Unlearning**: Applied various unlearning algorithms
+
+## Unlearning Algorithm
+
+This model uses the `grad_ascent` unlearning algorithm with the following parameters:
+- Learning Rate: `1e-05`
+- Forget Percentage: `10%`
+
+
+## Revisions
+
+The model is organized into multiple revisions, each representing a checkpoint during the unlearning process. The revision names follow the pattern `checkpoint-X`, where X is the checkpoint number.
+
+## Loading the Model
+
+To load a specific revision of this model, you can use the following code:
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+# Replace 'checkpoint-X' with the desired revision (e.g., 'checkpoint-12')
+revision = "checkpoint-X"
+
+model = AutoModelForCausalLM.from_pretrained("locuslab/{model_name}", revision=revision)
+tokenizer = AutoTokenizer.from_pretrained("locuslab/{model_name}", revision=revision)
+```
+
+## TOFU Dataset
+
+TOFU (Task of Fictitious Unlearning) is a dataset designed for training and evaluating unlearning algorithms in language models. It simulates scenarios where certain information needs to be "forgotten" or removed from the model's knowledge.
+
+## Unlearning Process
+
+1. The base Phi-1.5 model was first fine-tuned on the TOFU dataset (checkpoint-625).
+2. Various unlearning algorithms were then applied to this fine-tuned model to selectively "forget" certain information.
+3. The results of these unlearning processes are captured in the different revisions of this model.
+
+## Usage and Limitations
+
+This model is primarily intended for research purposes, particularly in the field of machine unlearning and privacy in language models. It may not be suitable for general-purpose language tasks without further evaluation.
+
+## Citation
+
+If you use this model in your research, please cite: 
+```
+@misc{tofu2024,
+      title={TOFU: A Task of Fictitious Unlearning for LLMs}, 
+      author={Pratyush Maini and Zhili Feng and Avi Schwarzschild and Zachary C. Lipton and J. Zico Kolter},
+      year={2024},
+      archivePrefix={arXiv},
+      primaryClass={cs.LG}
+}
+```
+
+## Contact
+
+For questions or issues regarding this model, please contact pratyushmaini@cmu.edu.