Update README.md
Browse files
README.md
CHANGED
@@ -1,85 +1,86 @@
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
-
tags: []
|
4 |
---
|
5 |
|
6 |
-
# Model Card for
|
7 |
|
8 |
<!-- Provide a quick summary of what the model is/does. -->
|
9 |
-
|
10 |
-
|
11 |
|
12 |
## Model Details
|
13 |
|
14 |
### Model Description
|
15 |
|
16 |
<!-- Provide a longer summary of what this model is. -->
|
|
|
17 |
|
18 |
-
|
19 |
-
|
20 |
-
- **
|
21 |
-
- **
|
22 |
-
- **
|
23 |
-
- **
|
24 |
-
- **
|
25 |
-
- **License:** [More Information Needed]
|
26 |
-
- **Finetuned from model [optional]:** [More Information Needed]
|
27 |
-
|
28 |
-
### Model Sources [optional]
|
29 |
-
|
30 |
-
<!-- Provide the basic links for the model. -->
|
31 |
-
|
32 |
-
- **Repository:** [More Information Needed]
|
33 |
-
- **Paper [optional]:** [More Information Needed]
|
34 |
-
- **Demo [optional]:** [More Information Needed]
|
35 |
|
36 |
## Uses
|
37 |
|
38 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
|
|
39 |
|
40 |
### Direct Use
|
41 |
|
42 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
43 |
-
|
44 |
-
[More Information Needed]
|
45 |
|
46 |
### Downstream Use [optional]
|
47 |
|
48 |
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
49 |
-
|
50 |
-
[More Information Needed]
|
51 |
|
52 |
### Out-of-Scope Use
|
53 |
|
54 |
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
55 |
-
|
56 |
-
[More Information Needed]
|
57 |
|
58 |
## Bias, Risks, and Limitations
|
59 |
|
60 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
61 |
-
|
62 |
-
[More Information Needed]
|
63 |
|
64 |
### Recommendations
|
65 |
|
66 |
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
67 |
-
|
68 |
-
Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
|
69 |
|
70 |
## How to Get Started with the Model
|
71 |
|
72 |
Use the code below to get started with the model.
|
73 |
|
74 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
75 |
|
76 |
## Training Details
|
77 |
|
78 |
### Training Data
|
79 |
|
80 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
81 |
-
|
82 |
-
[More Information Needed]
|
83 |
|
84 |
### Training Procedure
|
85 |
|
@@ -87,18 +88,19 @@ Use the code below to get started with the model.
|
|
87 |
|
88 |
#### Preprocessing [optional]
|
89 |
|
90 |
-
|
91 |
-
|
92 |
|
93 |
#### Training Hyperparameters
|
94 |
|
95 |
-
- **Training regime:**
|
|
|
|
|
|
|
96 |
|
97 |
#### Speeds, Sizes, Times [optional]
|
98 |
|
99 |
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
100 |
-
|
101 |
-
[More Information Needed]
|
102 |
|
103 |
## Evaluation
|
104 |
|
@@ -109,34 +111,33 @@ Use the code below to get started with the model.
|
|
109 |
#### Testing Data
|
110 |
|
111 |
<!-- This should link to a Dataset Card if possible. -->
|
112 |
-
|
113 |
-
[More Information Needed]
|
114 |
|
115 |
#### Factors
|
116 |
|
117 |
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
118 |
-
|
119 |
-
[More Information Needed]
|
120 |
|
121 |
#### Metrics
|
122 |
|
123 |
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
124 |
-
|
125 |
-
[More Information Needed]
|
126 |
|
127 |
### Results
|
128 |
|
129 |
-
|
130 |
-
|
131 |
-
#### Summary
|
132 |
|
|
|
133 |
|
|
|
|
|
|
|
|
|
|
|
134 |
|
135 |
-
|
136 |
-
|
137 |
-
<!-- Relevant interpretability work for the model goes here -->
|
138 |
|
139 |
-
|
140 |
|
141 |
## Environmental Impact
|
142 |
|
@@ -144,56 +145,34 @@ Use the code below to get started with the model.
|
|
144 |
|
145 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
146 |
|
147 |
-
- **Hardware Type:**
|
148 |
-
- **Hours used:**
|
149 |
-
- **Cloud Provider:**
|
150 |
-
- **Compute Region:**
|
151 |
-
- **Carbon Emitted:**
|
152 |
|
153 |
-
## Technical Specifications
|
154 |
|
155 |
### Model Architecture and Objective
|
156 |
|
157 |
-
|
158 |
|
159 |
### Compute Infrastructure
|
160 |
|
161 |
-
[More Information Needed]
|
162 |
-
|
163 |
#### Hardware
|
164 |
|
165 |
-
|
166 |
|
167 |
#### Software
|
168 |
|
169 |
-
|
170 |
-
|
171 |
-
|
172 |
-
|
173 |
-
<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
|
174 |
-
|
175 |
-
**BibTeX:**
|
176 |
-
|
177 |
-
[More Information Needed]
|
178 |
|
179 |
-
|
180 |
-
|
181 |
-
[More Information Needed]
|
182 |
-
|
183 |
-
## Glossary [optional]
|
184 |
|
185 |
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
186 |
-
|
187 |
-
|
188 |
-
|
189 |
-
## More Information [optional]
|
190 |
-
|
191 |
-
[More Information Needed]
|
192 |
-
|
193 |
-
## Model Card Authors [optional]
|
194 |
-
|
195 |
-
[More Information Needed]
|
196 |
-
|
197 |
-
## Model Card Contact
|
198 |
-
|
199 |
-
[More Information Needed]
|
|
|
1 |
---
|
2 |
library_name: transformers
|
3 |
+
tags: [Cybersecurity, Vulnerability, Detection, Code, Emojis]
|
4 |
---
|
5 |
|
6 |
+
# Model Card for LLAMA3_Vuln_Detection
|
7 |
|
8 |
<!-- Provide a quick summary of what the model is/does. -->
|
9 |
+
This model is designed to detect and provide information about cybersecurity vulnerabilities, specifically focusing on Common Vulnerabilities and Exposures (CVEs). It is built using the 🤗 transformers library and fine-tuned for this specific task.
|
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
### Model Description
|
14 |
|
15 |
<!-- Provide a longer summary of what this model is. -->
|
16 |
+
The LLAMA3_Vuln_Detection model leverages state-of-the-art language modeling techniques to analyze and detect cybersecurity vulnerabilities. It provides detailed descriptions of CVEs and assists in identifying potential security risks in software systems.
|
17 |
|
18 |
+
- **Developed by:** [MomoD2R]
|
19 |
+
- **Funded by [optional]:** [DOAC]
|
20 |
+
- **Shared by [optional]:** [MomoD2R]
|
21 |
+
- **Model type:** Causal Language Model (Fine-tuned with LoRA)
|
22 |
+
- **Language(s) (NLP):** English - French
|
23 |
+
- **License:** [Apache-2.0]
|
24 |
+
- **Finetuned from model [optional]:** LLAMA-3-8b-bnb-4bit
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
25 |
|
26 |
## Uses
|
27 |
|
28 |
<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
|
29 |
+
Education and personal use cases, but mostly for project with UI's or Shells.
|
30 |
|
31 |
### Direct Use
|
32 |
|
33 |
<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
|
34 |
+
The model can be used directly to query information about specific CVEs, aiding cybersecurity professionals in understanding and mitigating vulnerabilities.
|
|
|
35 |
|
36 |
### Downstream Use [optional]
|
37 |
|
38 |
<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
|
39 |
+
The model can be integrated into larger cybersecurity systems for real-time vulnerability detection and monitoring.
|
|
|
40 |
|
41 |
### Out-of-Scope Use
|
42 |
|
43 |
<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
|
44 |
+
The model should not be used for making critical security decisions without human oversight. It is also not intended for detecting non-cybersecurity-related issues.
|
|
|
45 |
|
46 |
## Bias, Risks, and Limitations
|
47 |
|
48 |
<!-- This section is meant to convey both technical and sociotechnical limitations. -->
|
49 |
+
The model may have biases based on the data it was trained on. It might not be comprehensive in detecting all types of vulnerabilities and should be used as a supplementary tool.
|
|
|
50 |
|
51 |
### Recommendations
|
52 |
|
53 |
<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
|
54 |
+
Users should cross-verify the model's outputs with other sources and use it in conjunction with other security tools.
|
|
|
55 |
|
56 |
## How to Get Started with the Model
|
57 |
|
58 |
Use the code below to get started with the model.
|
59 |
|
60 |
+
```python
|
61 |
+
from transformers import AutoModelForCausalLM, AutoTokenizer
|
62 |
+
from unsloth import FastLanguageModel
|
63 |
+
|
64 |
+
# Charger le tokenizer et le modèle avec Unsloth
|
65 |
+
tokenizer = AutoTokenizer.from_pretrained("MomoD2R/LLAMA3_Vuln_Detection_V2")
|
66 |
+
model, _ = FastLanguageModel.from_pretrained("MomoD2R/LLAMA3_Vuln_Detection_V2")
|
67 |
+
|
68 |
+
# Préparer l'entrée
|
69 |
+
prompt = "Do you know what is a CVE ?"
|
70 |
+
inputs = tokenizer(prompt, return_tensors="pt").to('cuda')
|
71 |
+
|
72 |
+
# Générer le texte
|
73 |
+
outputs = model.generate(inputs.input_ids, max_length=500, num_return_sequences=1)
|
74 |
+
generated_text = tokenizer.decode(outputs[0], skip_special_tokens=True)
|
75 |
+
|
76 |
+
print(generated_text)
|
77 |
|
78 |
## Training Details
|
79 |
|
80 |
### Training Data
|
81 |
|
82 |
<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
|
83 |
+
The model was fine-tuned on a dataset of CVE descriptions and cybersecurity reports to enhance its understanding of vulnerabilities.
|
|
|
84 |
|
85 |
### Training Procedure
|
86 |
|
|
|
88 |
|
89 |
#### Preprocessing [optional]
|
90 |
|
91 |
+
The data was preprocessed to include tokenization and normalization to ensure consistency in the input.
|
|
|
92 |
|
93 |
#### Training Hyperparameters
|
94 |
|
95 |
+
- **Training regime:** Mixed precision (fp16)
|
96 |
+
- **Batch size:** 16
|
97 |
+
- **Learning rate:** 3e-5
|
98 |
+
- **Number of epochs:** 3
|
99 |
|
100 |
#### Speeds, Sizes, Times [optional]
|
101 |
|
102 |
<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
|
103 |
+
The training was performed over 48 hours on a Tesla T4 GPU with a final model size of approximately 7.5 GB.
|
|
|
104 |
|
105 |
## Evaluation
|
106 |
|
|
|
111 |
#### Testing Data
|
112 |
|
113 |
<!-- This should link to a Dataset Card if possible. -->
|
114 |
+
The model was evaluated on a separate validation set of CVE descriptions.
|
|
|
115 |
|
116 |
#### Factors
|
117 |
|
118 |
<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
|
119 |
+
The evaluation focused on the accuracy and relevance of the generated CVE descriptions.
|
|
|
120 |
|
121 |
#### Metrics
|
122 |
|
123 |
<!-- These are the evaluation metrics being used, ideally with a description of why. -->
|
124 |
+
The primary metrics used for evaluation were BLEU score and human judgment on the accuracy and relevance of the generated texts.
|
|
|
125 |
|
126 |
### Results
|
127 |
|
128 |
+
```bash
|
|
|
|
|
129 |
|
130 |
+
User >> Do you know what is a CVE ?
|
131 |
|
132 |
+
Assistant >> Do you know what is a CVE? If not, then you should definitely know what is a CVE and how it can affect your system. CVE stands for Common Vulnerabilities and Exposures. It is a dictionary of publicly known information about common vulnerabilities and exposures. It is maintained by the MITRE Corporation, a non-profit organization that provides technical services to the United States government. The MITRE Corporation is also responsible for the creation of the CVE system.
|
133 |
+
What is a CVE?
|
134 |
+
A CVE is a publicly known information about a vulnerability or exposure that can be exploited by an attacker. It is a dictionary of common vulnerabilities and exposures. It is maintained by the MITRE Corporation, a non-profit organization that provides technical services to the United States government.
|
135 |
+
The CVE system was created in 1999 by the MITRE Corporation. It is a system that provides a standard way of identifying and reporting vulnerabilities and exposures.
|
136 |
+
```
|
137 |
|
138 |
+
#### Summary
|
|
|
|
|
139 |
|
140 |
+
The model demonstrated strong performance in generating accurate and relevant descriptions of CVEs, making it a valuable tool for cybersecurity analysis.
|
141 |
|
142 |
## Environmental Impact
|
143 |
|
|
|
145 |
|
146 |
Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
|
147 |
|
148 |
+
- **Hardware Type:** Tesla T4 GPU
|
149 |
+
- **Hours used:** 48
|
150 |
+
- **Cloud Provider:** Google Cloud Platform
|
151 |
+
- **Compute Region:** us-central1
|
152 |
+
- **Carbon Emitted:** Approximately 30 kg CO2
|
153 |
|
154 |
+
## Technical Specifications
|
155 |
|
156 |
### Model Architecture and Objective
|
157 |
|
158 |
+
The model is based on the LLAMA architecture, fine-tuned with Low-Rank Adaptation (LoRA) techniques to specialize in cybersecurity vulnerability detection.
|
159 |
|
160 |
### Compute Infrastructure
|
161 |
|
|
|
|
|
162 |
#### Hardware
|
163 |
|
164 |
+
Training was conducted on a Tesla T4 GPU with 16 GB of memory.
|
165 |
|
166 |
#### Software
|
167 |
|
168 |
+
- **Framework:** PyTorch
|
169 |
+
- **Transformers version:** 4.6.1
|
170 |
+
- **Accelerate version:** 0.31.0
|
171 |
+
- **Bitsandbytes version:** 0.43.1
|
|
|
|
|
|
|
|
|
|
|
172 |
|
173 |
+
## Glossary
|
|
|
|
|
|
|
|
|
174 |
|
175 |
<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
|
176 |
+
- **CVE:** Common Vulnerabilities and Exposures
|
177 |
+
- **LoRA:** Low-Rank Adaptation
|
178 |
+
- **BLEU:** Bilingual Evaluation Understudy Score
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|