# ayansinha /lic-class-scancode-bert-base-cased-L32-1

system
HF staff
 --- language: en tags: - license - sentence-classification - scancode - license-compliance license: apache-2.0 datasets: - bookcorpus - wikipedia - scancode-rules version: 1.0 --- # lic-class-scancode-bert-base-cased-L32-1 ## Intended Use This model is intended to be used for Sentence Classification which is used for results analysis in [scancode-results-analyzer](https://github.com/nexB/scancode-results-analyzer). scancode-results-analyzer helps detect faulty scans in [scancode-toolkit](https://github.com/nexB/scancode-results-analyzer) by using statistics and nlp modeling, among other tools, to make Scancode better. ## How to Use Refer [quickstart](https://github.com/nexB/scancode-results-analyzer#quickstart---local-machine) section in scancode-results-analyzer documentation, for installing and getting started. - [Link to Code](https://github.com/nexB/scancode-results-analyzer/blob/master/src/results_analyze/nlp_models.py) Then in NLPModelsPredict class, function predict_basic_lic_class uses this classifier to predict sentances as either valid license tags or false positives. ## Limitations and Bias As this model is a fine-tuned version of the [bert-base-cased](https://huggingface.co/bert-base-cased) model, it has the same biases, but as the task it is fine-tuned to is a very specific task (license text/notice/tag/referance) without those intended biases, it's safe to assume those don't apply at all here. ## Training and Fine-Tuning Data The BERT model was pretrained on BookCorpus, a dataset consisting of 11,038 unpublished books and English Wikipedia (excluding lists, tables and headers). Then this bert-base-cased model was fine-tuned on Scancode Rule texts, specifically trained in the context of sentence classification, where the four classes are - License Text - License Notice - License Tag - License Referance ## Training Procedure For fine-tuning procedure and training, refer scancode-results-analyzer code. - [Link to Code](https://github.com/nexB/scancode-results-analyzer/blob/master/src/results_analyze/nlp_models.py) In NLPModelsTrain class, function prepare_input_data_false_positive prepares the training data. In NLPModelsTrain class, function train_basic_false_positive_classifier fine-tunes this classifier. 1. Model - [BertBaseCased](https://huggingface.co/bert-base-cased) (Weights 0.5 GB) 2. Sentence Length - 32 3. Labels - 4 (License Text/Notice/Tag/Referance) 4. After 4 Epochs of Fine-Tuning with learning rate 2e-5 (60 secs each on an RTX 2060) Note: The classes aren't balanced. ## Eval Results - Accuracy on the training data (90%) : 0.98 (+- 0.01) - Accuracy on the validation data (10%) : 0.84 (+- 0.01) ## Further Work 1. Apllying Splitting/Aggregation Strategies 2. Data Augmentation according to Vaalidation Errors 3. Bigger/Better Suited Models