Edit model card


This model is a fine-tuned version of distilbert-base-uncased on textvqa and squad. It achieves the following results on the evaluation set:

  • Train Loss: 0.0097
  • Train Accuracy: 0.9976
  • Epoch: 9

Model description

The Text-Visual Question Classifier is a Hugging Face model that can classify questions as either text-based or visual-based. It uses a natural language processing and techniques to analyze the question and determine its type. The model has been trained on a large dataset of questions labeled as either text-based or visual-based, and has achieved high accuracy in identifying the correct type of question.

Intended uses & limitations


This model can be used in various applications such as chatbots, virtual assistants, search engines, and recommendation systems. For example, it can help chatbots to provide more accurate responses by understanding the type of question being asked. It can also help search engines to retrieve more relevant results by filtering out irrelevant content based on the type of question.


The model may not perform well on questions that are ambiguous or have multiple interpretations. It may also be biased towards certain types of questions based on the training data.

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • optimizer: {'name': 'Adam', 'weight_decay': None, 'clipnorm': None, 'global_clipnorm': None, 'clipvalue': None, 'use_ema': False, 'ema_momentum': 0.99, 'ema_overwrite_frequency': None, 'jit_compile': True, 'is_legacy_optimizer': False, 'learning_rate': 2e-06, 'beta_1': 0.9, 'beta_2': 0.999, 'epsilon': 1e-07, 'amsgrad': False}
  • training_precision: float32

Training results

Train Loss Train Accuracy Epoch
0.1914 0.9444 0
0.0711 0.9768 1
0.0531 0.9826 2
0.0427 0.9868 3
0.0330 0.9904 4
0.0264 0.9923 5
0.0195 0.9947 6
0.0149 0.9960 7
0.0123 0.9965 8
0.0097 0.9976 9

Framework versions

  • Transformers 4.27.2
  • TensorFlow 2.11.0
  • Datasets 2.10.1
  • Tokenizers 0.13.2
Downloads last month
This model does not have enough activity to be deployed to Inference API (serverless) yet. Increase its social visibility and check back later, or deploy to Inference Endpoints (dedicated) instead.

Datasets used to train AnaniyaX/decision-distilbert-uncased