thisismon/network-vulnerability-classifier-balanced

Network Traffic Vulnerability Classifier

Model Description

This model is fine-tuned BERT for binary classification of network traffic patterns as either normal or malicious.

Performance Metrics

Latest evaluation results:

• Accuracy: 80.00%
• Precision: 71.43%
• Recall: 100.00%
• F1 Score: 83.33%

Intended Use

• Primary use: Network traffic monitoring and threat detection
• Suitable for: Real-time traffic analysis and batch processing
• Target users: Security analysts and automated monitoring systems

Limitations

1. High recall (100%) but lower precision (71.43%) indicates:

   • Model tends to over-classify traffic as malicious
   • May generate false positives
   • Better suited for initial screening than final decision-making

2. Performance characteristics:

   • More sensitive to malicious patterns
   • May need human verification for borderline cases
   • Best used with confidence thresholds

Training Details

• Base model: bert-base-uncased
• Training data: Balanced dataset of normal and malicious traffic patterns
• Regularization:
  • Hidden dropout: 0.4
  • Attention dropout: 0.4
  • Classifier dropout: 0.4
• Training parameters:
  • Learning rate: 5e-6
  • Batch size: 32
  • Weight decay: 0.4
  • Early stopping patience: 2

Bias and Fairness

• Model shows bias toward malicious classification
• Implemented class weights [1.0, 2.0] to handle imbalanced classes
• May need adjustment based on specific deployment context

Recommendations

1. Implementation:

   • Use confidence threshold (recommended: 0.85)
   • Implement human review for high-risk predictions
   • Monitor false positive rates in production

2. Best practices:

   • Regular retraining with new data
   • Validation with domain-specific traffic patterns
   • Integration with other security tools

Example Usage

from transformers import AutoModelForSequenceClassification, AutoTokenizer

# Load model and tokenizer
model = AutoModelForSequenceClassification.from_pretrained("your-username/network-vulnerability-classifier")
tokenizer = AutoTokenizer.from_pretrained("your-username/network-vulnerability-classifier")

# Make prediction
def predict(text, confidence_threshold=0.85):
    inputs = tokenizer(text, return_tensors="pt", truncation=True)
    outputs = model(**inputs)
    probs = torch.nn.functional.softmax(outputs.logits, dim=1)[0]
    prediction = outputs.logits.argmax(-1).item()
    confidence = probs[prediction].item()
    
    if confidence < confidence_threshold:
        return "Uncertain", confidence
    return "Normal" if prediction == 0 else "Malicious", confidence

Updates and Maintenance

• Last updated: [Current Date]
• Regular updates planned for:
  • Performance improvements
  • New traffic patterns
  • Security vulnerabilities

Citation

If you use this model in your research, please cite:

@misc{network-vulnerability-classifier,
  author = {Your Name},
  title = {Network Traffic Vulnerability Classifier},
  year = {2024},
  publisher = {HuggingFace},
  journal = {HuggingFace Model Hub}
}