fyp-buglens/VideoGameReviews-ImplementationResponseIssue-BERT

Implementation Response Issues Detection Model

This model is trained to detect Implementation Response Issues in video games. These issues occur when the game does not interact optimally with the hardware, such as network lag, input lag (time from pressing a button to an action occurring in the game), or frame rate fluctuations. The model focuses on detecting these issues that arise from errors in the software code, not due to failures in the hardware or network itself.

For example, in Rainbow Six Vegas 2, the player experiences lag when shooting a particular NPC, but the lag stops once the player turns around, indicating a software-related issue rather than a hardware failure or network congestion.

Model Details

• Model Type: Classification (Binary)
• Training Data: The dataset for this model was created from scratch by scraping reviews from the Steam platform. These reviews were then labeled into different categories, including Implementation Response Issues, to train the model. The model is based on concepts from the paper "What went wrong; the taxonomy of video game bugs".
• Task: Bug detection in video games related to implementation failures, such as input lag, network lag, and frame rate fluctuations.
• Intended Use: The model is designed for game developers and QA teams to detect performance-related issues caused by implementation errors.

Training Metrics

The model was trained for 3 epochs with the following performance metrics:

Epoch	Training Loss	Validation Loss	Accuracy	F1 Score	Precision	Recall
0	0.5234	0.5149	74.78%	0.7092	0.8242	0.6224
1	0.4681	0.4252	80.75%	0.7931	0.8459	0.7465
2	0.2766	0.4503	79.52%	0.7839	0.8187	0.7519

Key Metrics:

• F1 Score: Balances precision and recall, with a final value of 0.7931 after 1 epoch.
• Precision: Measures the accuracy of positive predictions, reflecting how many of the predicted lag issues were correct.
• Recall: Measures how well the model identifies actual lag issues, capturing the true positives.

Intended Audience

• Game Developers: To detect performance-related issues like input lag, network lag, or frame rate fluctuations in video games.
• Quality Assurance (QA) Teams: To automate the detection of implementation response issues during game testing.
• Researchers: Interested in exploring and extending bug detection models for performance issues in video games.

Limitations

• This model is trained specifically for detecting Implementation Response Issues, focusing on performance problems like lag due to software code errors.
• It does not account for issues caused by hardware or network failures.
• Performance may vary depending on the game's context and system configuration. Fine-tuning may be required for different games or environments.

How to Use

You can use the model for binary classification to predict whether a given game state exhibits an Implementation Response Issue. Here's an example using the Hugging Face transformers library:

from transformers import pipeline

# Load the model from Hugging Face
lag_detection = pipeline('text-classification', model='fyp-buglens/VideoGameReviews-ImplementationResponseIssue-BERT')

# Example usage
result = lag_detection("The player experiences lag after shooting an NPC")
print(result)  # Output: label indicating if it's a lag issue or not


---
license: apache-2.0
---