## 1 Introduction  
Zebrafish embryos have gained popularity in biological research since they share 84% of genes associated with human disease [1] and they are nearly transparent under bright-field microscopes. Zebrafish egg is a special form of the embryo, and it is usually used to study the influence of environmental factors on embryo development. To evaluate the biological endpoints based on zebrafish eggs, microscopic screening is frequently performed [2]. By far, the analysis of zebrafish microscopic images is mostly performed by human operators. With the advances in image acquisition systems, the number of microscopic images is increasing rapidly, making manual assessments increasingly time-consuming. Therefore, automatic analysis of zebrafish microscopic image becomes an urgent demand [3]. To meet this stringent demand, a series of studies was conducted for computerized zebrafish microscopic image analysis [4; 5]. Most techniques were based on traditional machine learning strategies,i.e., using texture filters to extract hand-crafted image features and then using classification algorithms (e.g., the supported vector machine and random forest) to conduct phenotype pattern recognition. The performances of these methods highly rely on the quality of hand-crafted image features, but the design and selection of hand-crafted features involve subjective human interventions, which limit the objectiveness and robustness of the method. In the last decade, deep learning methods experienced dramatic development, leading to improvements in many pattern recognition applications, such as image processing, video analysis, and language recognition [6; 7; 8; 9; 10; 11]. Compared to the conventional machine learning methods, deep learning overcomes the limitation of hand-crafted features by automatically optimizing the feature extraction and classification procedure. The core of deep learning for image analysis is the revolutionary development of the convolutional neural network (CNN) [12; 13]. CNN was originally designed to recognize and classify object patterns in images. As of today, numerous CNN-based powerful image classification models are developed, including Alex Krizhevsky Network (AlexNet) [14], Visual Geometry Group (VGG) nets [15], and Residual Neural Network (ResNet) [16]. These methods were also applied to biological image analysis [17; 18], leading to improvement of accuracy and robustness. Despite the fast development of deep learning techniques, their applications in zebrafish egg microscopic image analysis are rare. A common data analysis task for zebrafish egg images analysis is to classify whether the egg is fertilized or not, in order to verify if the tested drug has impaired the fertilization process. To accomplish this task, there are several challenging problems to solve:  
* Imbalanced training dataset. In biological research, it is difficult to collect a balanced number of fertilized and unfertilized egg samples as the training dataset. The imbalanced training set will result in insufficient classification ability for the category with fewer training samples, leading to unsuccessful network training. * Small training dataset. The training of deep neural network requires no less than thousands of training samples. However, it is difficult to collect enough training data for a specific biological image analysis task. Small training sample set will lead to overfitting of the training data, hampering the generalization ability of the network. * Subtle inter-class differences. In bright-field microscopic images, fertilized and unfertilized zebrafish eggs usually demonstrate subtle inter-class differences. This challenging problem becomes a technical bottleneck for automated zebrafish egg image analysis. To overcome these problems, this paper proposed a deep learning algorithm for automated zebrafish egg fertilization status classification from microscopic images. Dedicated data augmentation and transfer learning strategy were used to tackle the imbalanced and small training set problem.