Update README.md

README.md

@@ -1,4 +1,56 @@
 ---
+datasets:
+- wikimovies
+license: cc-by-4.0
+---
+# roberta-base for MLM 
+
+```
+model_name = "thatdramebaazguy/roberta-base-wikimovies"
+pipeline(model=model_name, tokenizer=model_name, revision="v1.0", task="Fill-Mask")
+```
+## Overview
+**Language model:** roberta-base  
+**Language:** English  
+**Downstream-task:** Fill-Mask  
+**Training data:** wikimovies  
+**Eval data:** wikimovies
+**Code:**  See [example](https://github.com/adityaarunsinghal/Domain-Adaptation/blob/master/shell_scripts/train_movie_roberta.sh)  
+**Infrastructure**: 2x Tesla v100
+
+## Hyperparameters
+```
+num_examples = 4346
+batch_size = 16
+n_epochs = 3
+base_LM_model = "roberta-base"
+learning_rate = 5e-05
+max_query_length=64
+Gradient Accumulation steps = 1
+Total optimization steps = 816
+evaluation_strategy=IntervalStrategy.NO
+prediction_loss_only=False
+per_device_train_batch_size=8
+per_device_eval_batch_size=8
+adam_beta1=0.9
+adam_beta2=0.999
+adam_epsilon=1e-08,
+max_grad_norm=1.0
+lr_scheduler_type=SchedulerType.LINEAR
+warmup_ratio=0.0
+seed=42
+eval_steps=500
+metric_for_best_model=None
+greater_is_better=None
+label_smoothing_factor=0.0
+``` 
+## Performance
+
+perplexity = 4.3808
+
+Some of my work: 
+- [Domain-Adaptation Project](https://github.com/adityaarunsinghal/Domain-Adaptation/)
+---
 language: 
 - English
 -
@@ -11,44 +63,4 @@ license:
 datasets:
 - wikimovies
 -
-metrics:
-- 
--
 ---
-
-# MyModelName
-
-## Model description
-
-You can embed local or remote images using `![](...)`
-
-## Intended uses & limitations
-
-#### How to use
-
-```python
-# You can include sample code which will be formatted
-```
-
-#### Limitations and bias
-
-Provide examples of latent issues and potential remediations.
-
-## Training data
-
-Describe the data you used to train the model.
-If you initialized it with pre-trained weights, add a link to the pre-trained model card or repository with description of the pre-training data.
-
-## Training procedure
-
-Preprocessing, hardware used, hyperparameters...
-
-## Eval results
-
-### BibTeX entry and citation info
-
-```bibtex
-@inproceedings{...,
-  year={2020}
-}
-```