As I said before, the training cost (time and money) and model size of NNs are trade-offs that one needs to be careful of. The main criterion is how well an NN model can predict things. Hence, the test accuracy (i.e., accuracy on unseen datasets) is the driving force in all large models we discussed.However, pre-trained models are getting more and more popular. Therefore, the NN model size in many cases may not be as problematic as training a NN from scratch. For both NLP and Vision tasks, there are pre-trained models that can be applied and fine-tuned or re-designed for their particular data set. Here, one needs to be careful about the task relatedness when choosing the pre-trained NN models. For instance, if the task is generic object recognition, an NN model like YOLO or Mask-RCNN trained on the COCO stuff dataset would be the user’s choice.