added benchmarks

README.md

@@ -1,5 +1,13 @@
 ---
 license: cc-by-nc-nd-4.0
+datasets:
+- vandijklab/immune-c2s
+language:
+- en
+tags:
+- pytorch
+- causal-lm
+- scRNA-seq
 ---
 # Overview
 
@@ -22,6 +30,22 @@ GitHub: <https://github.com/EleutherAI/pythia>
 Paper: <https://arxiv.org/abs/2304.01373>  
 Hugging Face: <https://huggingface.co/EleutherAI/pythia-160m>
 
+# Evaluation
+
+This model was evaluated on KNN classification and Gromov-Wasserstein (GW) distance. 
+The label for a generated cell is the corresponding cell type used in its corresponding prompt for generation. 
+Ground truth cells were sampled with replacement from a held out test dataset. 
+The generated cells are converted to expression vectors using the method described in the paper. 
+For complete details on the experiments, we refer to the paper.
+
+| Model              | k=3 NN (&#8593;) | k=5 NN (&#8593;) | k=10 NN (&#8593;) | k=25 NN (&#8593;) | GW (&#8595;) |
+| :---- | :---: | :---: | :---: | :---: | :----: |
+| scGEN              | 0.2376 | 0.2330 | 0.2377 | 0.2335 | 315.9505   | 
+| scVI               | 0.2436 | 0.2400 | 0.2425 | 0.2348 | 302.1285   |
+| scDiffusion        | 0.2335 | 0.2288 | 0.2368 | 0.2306 | 72.0208    |
+| scGPT              | 0.1838 | 0.1788 | 0.1811 | 0.1882 | 2989.8066  |
+| **C2S (Pythia-160m)**  | **0.2588** | **0.2565**  | **0.2746**      | **0.2715**    | **54.3040**         | 
+
 # Sample Code
 
 We provide an example of how to use the model to conditionally generate a cell equipped with a post-processing function to remove duplicate and invalid genes. 
@@ -122,4 +146,4 @@ with torch.no_grad():
 output_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
 cell_sentence = "".join(re.split(r"\?|\.|:", output_text)[1:]).strip()
 processed_genes = post_process_generated_cell_sentences(cell_sentence, gene_dictionary)
-```
+```