# Classification and Fast Few-Shot Learning of Steel Surface Defects with Randomized Network
## 1 Introduction
The few-shot learning problem means that we have a very unbalanced set;
* Domain shift [7]: When the dataset used in training and the dataset in practice are captured under different conditions, it can result in poor detection performance. In incremental learning, new data (i.e., new shots of previously seen or unseen classes) arrive in phases over time and we have to extend our classification model to include these new classes or new samples of classes. In general, in the literature different synonyms are used for this problem, such as continual learning [12; 13], lifelong learning [12; 14], or incremental learning [15; 16; 17; 18; 19; 20]. The three major challenges in these scenarios are:  
* Catastrophic forgetting: It was already investigated in the early 1990s [21] that a neural network is going to have a lower performance of previously trained classes when re-trained or tuned with new ones in focus. The problem is most serious if we have no training data for the already trained classes, or the dataset for re-training is unbalanced due to the missing samples from old classes. There are several approaches to avoid this case by methods such as scaling the weights of trained classifiers [22], using dual memories for storing old images and their statistics [23], or progressive incremental learning [24] where sub-networks are added incrementally to previous ones as the task is growing with new classes;
* Low number of representatives of new classes: In practical applications there is a data collection period when lots of samples are collected about the possible artifacts and used for model building. In contrast, when the system is in use for long periods, the undergoing background manufacturing processes (changes in the environment, or other influences) can result in new defect classes, which might appear in very few samples initially. In general, solutions are categorized into three main sets: using prior knowledge to augment the supervised experience (in our case data augmentation of the available few shots); modifying the model, which uses prior knowledge of known classes; and algorithmic solutions to alter the search for the best hypothesis in the given hypothesis space [25]. In extreme cases, we talk about zero-shot learning: when a new type of defect appears, it is a question whether we are able to classify it as a new class or if it will fall into an existing category. We would like to avoid this latter case and thus we have to create a new category and be able to tune the existing model to consider it [10; 26; 27];
* Complexity of updating the classifiers: It is a practical problem during the application of deep learning models that the re-training or fine-tuning of new or updated models can be time consuming or computationally very complex. A fast adaptation to the extended set of classes and easy training procedures are needed for real-life applications. Typically, the above three problems appear simultaneously. The advantage of our proposed solution is that it addresses all of them with one architecture, instead of attacking the problems independently with different methods. The main contributions in our article are the following:* Instead of re-tailoring old models or defining new deep neural architectures, we show that a well-proven and efficient deep neural network (DNN) (EfficientNet-B7 [28]) can give the best known accuracy for the classification of steel surface defects on two major benchmark datasets;
* We propose a novel architecture designed for incremental learning with the following features: it avoids the catastrophic forgetting of old classes, it has good performance in case of very few shots, and it can be trained very fast for new classes. To achieve this, we apply randomized networks concatenated to the feature extraction of a pre-trained DNN. The computation of its weights can be done very efficiently by the Moore-Penrose generalized inverse. Our paper is organized as follows.