Update README.md

README.md

@@ -30,6 +30,27 @@ We achieve pretty good results:
 
 Though the test set is only 59 examples, with 22 discussing disease. 
 
+## Key stats
+The model size is ~600MB. It will run really fast on the MPS device of an M-series mac. It should also run pretty fast on a normal CPU.
+
+The context windiw is 4,096 tokens which comes from the base longformer model. In training we limit the context to 1280 tokens because that
+was a bit bigger than the longest abstract we saw. Long abstracts may cause trouble.
+
+## Limitations
+The base model is trained on MLM on wikipedia text (and maybe some other stuff). As such, it might not have a 
+great understanding of scientific literature.
+
+The dataset used to train this model was _tiny_ at only 588 examples overall. This means only 470 samples 
+for training with 59 for validation and testing. These have been deliberately sampled to be roughly equally distributed between the 
+two classes.
+
+The dataset these are derived from is also _massively_ imbalanced, having 19,229 examples but only 294 that are not disease. As a result
+the model is trained on a dataset that hugely undersamples the disease context abstracts.
+
+While the model has been tested on some abstracts derived from lncBook's annotations, it hasn't really been tested on 'wild' abstracts.
+
+## Next steps
+
 The next step will be to be able to classify both the specific disease (e.g. lung adenocarcinoma), and the non-disease 
 context (e.g. localisation) a paper discusses.