Add model files

README.md

@@ -1,3 +1,30 @@
 ---
 license: mit
 ---
+The model generated in the Enrich4All project.<br>
+Evaluated the perplexity of MLM Task fine-tuned for construction permits related corpus.<br>
+Baseline model:  https://huggingface.co/racai/distilbert-base-romanian-cased <br>
+Scripts and corpus used for training: https://github.com/racai-ai/e4all-models
+
+Corpus
+---------------
+
+The construction authorization corpus is meant to ease the task of interested people to get informed on the legal framework related to activities like building, repairing, extending, and modifying their living environment, or setup of economic activities like establishing commercial or industrial centers. It is aimed as well to ease and reduce the activity of official representatives of regional administrative centers. The corpus is built to comply with the Romanian legislation in this domain and is structured in sets of labeled questions with a single answer each, covering various categories of issues:
+ * Construction activities and operations, including industrial structures, which require or do not require authorization,
+ * The necessary steps and documents to be acquired according to the Romanian regulations,
+ * validity terms,
+ * involved costs.  
+ 
+The data is acquired from two main sources:
+ * Internet: official sites, frequently asked questions
+ * Personal experiences of people: building permanent or provisory structures, replacing roofs, fences, installing photovoltaic panels, etc.
+
+<br><br>
+The construction permits corpus contains 500,351 words in 110 UTF-8 encoded files.
+
+Results
+-----------------
+| MLM Task                          | Perplexity    |
+| --------------------------------- | ------------- |
+| Baseline                          | 62.79         |
+| Construction Permits Fine-tuning  | 7.13          |

config.json

@@ -0,0 +1,25 @@
+{
+  "_name_or_path": "racai/distilbert-base-romanian-cased",
+  "activation": "gelu",
+  "architectures": [
+    "DistilBertForMaskedLM"
+  ],
+  "attention_dropout": 0.1,
+  "dim": 768,
+  "dropout": 0.1,
+  "hidden_dim": 3072,
+  "initializer_range": 0.02,
+  "max_position_embeddings": 512,
+  "model_type": "distilbert",
+  "n_heads": 12,
+  "n_layers": 6,
+  "output_hidden_states": true,
+  "pad_token_id": 0,
+  "qa_dropout": 0.1,
+  "seq_classif_dropout": 0.2,
+  "sinusoidal_pos_embds": true,
+  "tie_weights_": true,
+  "torch_dtype": "float32",
+  "transformers_version": "4.15.0",
+  "vocab_size": 50000
+}

pytorch_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f45918872a1fc2a9f9f1d2adacba6b30076499ed4be8472f3ffd63807cf81b47
+size 327897007

special_tokens_map.json

@@ -0,0 +1 @@
+{"unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]"}

tokenizer_config.json

@@ -0,0 +1 @@
+{"do_lower_case": false, "unk_token": "[UNK]", "sep_token": "[SEP]", "pad_token": "[PAD]", "cls_token": "[CLS]", "mask_token": "[MASK]", "tokenize_chinese_chars": true, "strip_accents": null, "special_tokens_map_file": null, "name_or_path": "racai/distilbert-base-romanian-cased", "do_basic_tokenize": true, "never_split": null, "tokenizer_class": "DistilBertTokenizer"}

training_args.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:977de6d54eef6238af5b7d474a3bd6ffadb46c386e8bd8dd8519aeaac8d4ddaa
+size 2927