GAMEBERT

This model is a fine-tuned version of google/bert_uncased_L-2_H-128_A-2 on a custom game commands dataset. It achieves the following results on the evaluation set:

• Loss: 0.0409
• Precision: 0.9692
• Recall: 0.9702
• F1: 0.9697
• Accuracy: 0.9900

Model description

A fine tuned tiniest available BERT meant for low latency tagging of text for purposes such as identifying target and action to give to a game from a sentence.

Intended uses & limitations

This model is made to be used by the PyPi package voice-speak-up - [https://pypi.org/project/voice-speak-up/]. It is part of the second stage of the pipeline and aims to identify actions, targets, and corrections in user spoken sentences transcribed by whisper.

Training and evaluation data

Custom chatette based dataset - [https://github.com/SimGus/Chatette]

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Use OptimizerNames.ADAMW_TORCH_FUSED with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 4

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.2531	1.0	2250	0.1332	0.9098	0.9078	0.9088	0.9707
0.1138	2.0	4500	0.0631	0.9592	0.9552	0.9572	0.9853
0.0812	3.0	6750	0.0443	0.9683	0.9698	0.9690	0.9895
0.0733	4.0	9000	0.0409	0.9692	0.9702	0.9697	0.9900

Framework versions

• Transformers 5.14.1
• Pytorch 2.13.0+cu130
• Datasets 5.0.0
• Tokenizers 0.22.2