# Identifying Animals in Camera Trap Images via Neural Architecture Search

Authors: Liang Jia; Ye Tian; Junguo Zhang
Journal: Computational Intelligence and Neuroscience
 (2022)

---

## Abstract

Wild animals are essential for ecosystem structuring and stability, and thus they are important for ecological research. Since most wild animals have high athletic or concealable abilities or both, it is used to be relatively difficult to acquire evidence of animal appearances before applications of camera traps in ecological researches. However, a single camera trap may produce thousands of animal images in a short period of time and inevitably ends up with millions of images requiring classification. Although there have been many methods developed for classifying camera trap images, almost all of them follow the pattern of a very deep convolutional neural network processing all camera trap images. Consequently, the corresponding surveillance area may need to be delicately controlled to match the network capability, and it may be difficult to expand the area in the future. In this study, we consider a scenario in which camera traps are grouped into independent clusters, and images produced by a cluster are processed by an edge device installed with a customized network. Accordingly, edge devices in this scenario may be highly heterogeneous due to cluster scales. Resultantly, networks popular in the classification of camera trap images may not be deployable for edge devices without modifications requiring the expertise which may be hard to obtain. This motivates us to automatize network design via neural architecture search for edge devices. However, the search may be costly due to the evaluations of candidate networks, and its results may be infeasible without considering the resource limits of edge devices. Accordingly, we propose a search method using regression trees to evaluate candidate networks to lower search costs, and candidate networks are built based on a meta-architecture automatically adjusted regarding to the resource limits. In experiments, the search consumes 6.5 hours to find a network applicable to the edge device Jetson X2. The found network is then trained on camera trap images through a workstation and tested on Jetson X2. The network achieves competitive accuracies compared with the automatically and the manually designed networks.


---

## Body

## 1. Introduction

Ecosystems in earth have irreplaceable ecological, societal, and economic value for human beings [1], but ecosystems can be compositionally and functionally changed by species extinctions [2], e.g., massive declines in large carnivore populations are likely to result in ecosystem instability [3], loss of large herbivores can alter ecosystems through the loss of ecological interactions [4], and digging mammals are vital for maintaining the ecosystem in Australia [5]. In ecosystems, some wild animals like vertebrates high up the food chain may affect many other plants and animal species low down the chain [3, 4]. To prevent their extinctions, wildlife research, protection, and management require reliable animal data, such as population distributions, trying not to disturb animals and their habitats. Traditional data acquisition means may not fully meet this requirement, e.g., radio collars, satellite-based devices, and airplane surveillance. With the development of automatic and information technologies, camera traps not only provide an effective solution to acquire animal data in a nonintrusive and remote manner [6] but also are suitable for detecting rare or secretive species [7]. Camera traps may produce millions of animal images requiring classification [8] which is commonly automated via machine learning or deep learning methods, especially through convolutional neural networks (CNNs) [9–16]. Although CNN-based methods are widely adopted in classifications, almost all methods are developed under the condition that all camera trap images are processed by a single network requiring intensive or even formidable computational resources, e.g., a high-performance computing cluster is employed to classify 3.3 M (million) camera trap images [15]. Consequently, the corresponding surveillance areas may need to be deliberately controlled to match the network capability, and it may be difficult to expand the area in the future.One promising solution of establishing or expanding surveillance areas without limitations of CNN capabilities is grouping camera traps as clusters accompanied by edge devices installed with customized CNNs [16, 17]. Thus, the computationally intensive classification of all images could be divided into subclassifications and offloaded to edge devices. Accordingly, edge devices may be highly heterogeneous [18] due to the cluster scales. Consequently, CNNs popular in classifications of camera trap images might not be deployable for edge devices without modifications such as quantization, pruning, and neural network design [19]. Among these modifications, neural network design significantly improves the computation and storage efficiency of CNN [19], but “designing neural networks is very difficult, and it requires the experience and knowledge of experts, a lot of trial and error, and even inspiration” [20]. Fortunately, the design can be automated by neural architecture search (NAS) [21–28].With advancements in NAS, it is practical and automatic to design CNNs with performances competitive with ones designed by human experts [21–28]. However, the automatic design via NAS may be tough due to the dimension explosion of search space and the expensive evaluations of candidate networks. Since the search space is defined with respect to (w. r. t.) the meta-architecture, i.e., the prototype from which the candidate networks are developed, a lot of effort has gone into reducing the structure complexity [28] of meta-architectures [21–27], e.g., high-dimensional chain architectures and low-dimensional cell architectures. The low dimensionality of the cell architecture arises from its repeatable local structures called cells [21–27]. The cell architecture is thus built by assembling cells sharing the same structure except weights. The network built on the chain architecture is equivalent to a single-cell network in view of the cell architecture. Therefore, the dimensionality of search space based on the cell architecture is much lower than the chain architecture. The dimensionality may further be reduced by simplifying cells.There are two common types of cells, i.e., normal and reduction cells. Since the reduction cell mainly reduces data dimensions, the cell may be simplified to decrease the dimensionality of search space [24–27]. For instance, PNASNet [24] focuses on optimizing the normal cell only and implements the reduction cell by copying the normal cell and adjusting the convolution strides. Path-level network transformation [25] simplifies the reduction cell to a single pooling layer and models the normal cell as a tree. The search conducts a Net2DeeperNet operation to each node in the tree to change the cell topology. GDAS [27] optimizes normal cells only and adopts a manually defined reduction cell. However, meta-architectures are always fixed regardless of resources limited by edge devices [21–27].These facts inspired us to develop a search method on the basis of an adaptive cell architecture that automatically changes w. r. t. resources restrained by devices [29]. The proposed method was designed within the framework of NAS based on reinforcement learning (RL) due to their good performances [22, 23, 30–32]. RL attempts to train an agent to perform actions to interact with the environment by receiving rewards based on the previous actions. Accordingly, the sampler (controller [22, 23]) learns from its sampled networks, especially the performances. However, the performance evaluation may be costly due to the expensive network training. The cost may be lowered by various means like minimizing training time [22] and sharing weights of trained networks during search [23]. After the search, the optimal network can either be selected from the search history [22, 30–32] or sampled by the trained sampler [23].In this study, an RL-based search method is designed in consideration of resource-limited devices. Namely, the meta-architecture changes adaptively and automatically w. r. t. the resource limited by the device. Besides, the search is accelerated by predicting the test accuracy of the sampled networks through regression trees, i.e., the network structure is vectorized through conversion functions, and the resulting vectors are fed to regression trees to yield accuracy. On the basis of the search acceleration and the adaptive meta-architecture, a search method named neural architecture search based on regression tree (NASRT) is proposed in this study, and the main contributions are summarized as follows:(1)
The proposed search method is designed in consideration of computational resources limited by edge devices for classifying camera trap images. This is achieved by using an adaptive meta-architecture that automatically changes w. r. t. the resource limit.(2)
The proposed search method is accelerated by replacing the costly accuracy evaluation with economical prediction. This is achieved by vectorizing the sampled network and feeding the resulting vector to regression trees to estimate the accuracy.The remainder of this study is organized as follows. In Section2, NASRT is introduced. In Section 3, the test results of NASRT are shown and analysed. Finally, Section 4 gives the conclusion.


## 2. Methods

The flowchart of NASRT is shown in Figure1 which highlights five steps of NASRT whose details are introduced sequentially in this section. As shown in the figure, long short-term memory (LSTM) [33] samples cell structures. The sampled cell is then assembled according to the adaptive meta-architecture w. r. t. resources limited by the edge device. The accuracy of the network is predicted by regression trees learned by XGBoost [34]. The predicted accuracy then serves as a component of reward which is employed to generate the loss to update the sampler LSTM.Figure 1
Flowchart of the proposed method named NASRT, This figure illustrates the main steps of NASRT where the first four steps represent the procedure of a single sampling during the search, and the search commonly repeats the sampling multiple times. Once the search ends, the optimal network is selected from the search history as indicated by step five in the figure.The adaptive meta-architecture is depicted in Figure2. The architecture consists of normal and reduction cells, i.e., tiny networks either preserving or halving data dimensions. In this study, every reduction cell is simplified to be a single pooling layer, and there are R reduction cells in total. For every two reduction cells, there are N normal cells. The cell pipeline terminates at the global average [35].Figure 2
Adaptive meta-architecture of NASRT. The meta-architecture determines how a candidate network is built during the search and how the data flow through the network. The data flow from the input to the output via multiple paths in a candidate network, and paths are dynamically determined by the sampler during the search. The dynamic paths are represented by both a cloud shape labelled “cell connection” and lines emitting from it. Candidate networks building is affected by several user-defined constants, e.g., normal cell numberN and reduction cell number R, respectively, denoted by “N” and “…” in the figure.Obviously, adaptive meta-architecture can be built dynamically by changing values ofN and R w.r.t. device-associated resources which are simplified as GPU memory ℳ in this study. Namely,(1)f=maxn=1…N∑j=1R∑i=1nC˙i+C¨j+∑i=1nC˙i,s.t.f<ℳ,where f denotes the adaptive meta-architecture, N is the maximal number of normal cells between two consecutive reduction cells in f, R is a fixed constant referring to the total number of reduction cells in f, C˙i denotes the i th normal cell, C¨j represents the j th reduction cell, “+” corresponds to the cell permutation in Figure 2, and “f<ℳ” denotes the resource constraint. Specifically, suppose the batch size (the number of images fed to the network at a time) and GPU memory for a specific application are known a priori, NASRT initializes the network based on formula (1) parameterized by N and attempts to load a single batch of data together with the network to GPU. If the loading fails because of insufficient GPU memory, the initialization and data loading will be repeated w. r. t. formula (1) parameterized by N−1. This continues until the loading succeeds or N=0 which will cause NASRT to abandon the current network.As mentioned above, a normal cell is a tiny network, which means it has its own inner structure as shown in Figure3. Even though normal cells share the inner structure in the same network, they differ in input sources and weights. The input sources are defined recursively. Namely, for the i th normal cell C˙i in Figure 3, input sources of a block are chosen from previous cells C˙i−1, C˙i−2 … C˙i−B (simplified to C−1, C−2 … in the followings). The choice is made by the sampler during the search. A block contains several operations, e.g., convolution and identity operations. The outputs of the operations within a block are collected and added to generate the block output, and the input of an operation can come from another block within the same cell or one of B previous cells.Figure 3
Inner structure of a normal cell in a candidate network. This figure illustrates the inner structure of thei th normal cell denoted by a large rectangle with the upper-left corner labelled “cell i”. A normal cell consists of operations grouped as blocks, and there are B blocks represented by smaller rectangles with upper-left corners labelled “block 1” … “block B”. The number of operations in a block is dynamically determined by the sample during the search, and this dynamic number is represented by “…” between two operations in each block. The input of an operation is also dynamically determined by the sampler, and the input is usually sampled from previous cells or blocks, which means outputs of cells or blocks may serve as operation inputs. The cell output is denoted by an arrow emitting from the cell, and an operation output is represented by an arrow emitting from the operation. Outputs of operations not serving as inputs of any other operations are summed to yield the block output, the sum is represented by a rectangle labelled “add”, and the block output is denoted by an arrow emitting from the rectangle. The input and output relationship among both blocks and cells is represented by a cloud labelled “Cell i connection”.There are five operations optional for a normal cell, i.e.,T3c, T5c, T3d, T5d, and TI where the first four operations denote stacks of convolution, batch normalization [36], and ReLU [37], and the last one denotes the identity operation. The convolution can be either depthwise separable [38] (T3d and T5d) or not (T3c and T5c), and its kernel size can be either 3-by-3 (T3d and T3c) or 5-by-5 (T5d and T5c). There are four pooling layers optional for a reduction cell, i.e., T3m, T5m, T3a, T5a. The pooling can be either max (T3m and T5m) or average (T3a and T5a), and its kernel size can be either 3-by-3 (T3m and T3a) or 5-by-5 (T5m and T5a).The cells are sampled in step 1 of Figure1. For each block in the normal cell, the number of its operations is selected from 2,3…M. For each operation in a block, its type is chosen from T3c,T5c,T3d,T5d,TI, and its input is selected from ℬ−1,ℬ−2… or C−1,C−2…, i.e., previous blocks or cells (ℬ−1,ℬ−2…=∅ for the first block). For the pooling layer in the reduction cell, the pooling is chosen from T3m,T5m,T3a,T5a, and its input is fixed to the previous cell. All the selections are made by the sampler LSTM based on its hidden states associated with the previous selections.The network is built in step 2 of Figure1 w. r. t. the adaptive meta-architecture defined by formula (1). During the building process, some steps require special attentions, e.g., for C˙i where 1≤i<B, there are i previous cells instead of B cells. Especially for the first cell, only raw image data are available. In this case, the operation input is chosen from the available sources. When the inputs of operations in a block come from another block or cell, we call they are connected. Besides the input availability of an operation, its output is added with other operations within the same block, and this requires all the outputs to share the same dimension. Thus, downsampling is applied to outputs whose dimensions differ from the minimal one found within the block, and then, they are summed, i.e.,(2)ℬj=∑k=1mjTj,kIj,k,where ℬj denotes the output of the jth block in a cell, mj is the operation number, Ij,k represents the input of the kth operation Tj,k. Among the blocks in a cell, there are ones not connected to any other blocks, and the outputs of these unconnected blocks are concatenated to yield the cell output C˙i, i.e.,(3)C˙i=⊕j∗ℬj∗,s.t.ℬj∗≠Ij,k∀j,k,where the concatenation is denoted by ⊕. During concatenating the block outputs, upsampling is applied to the outputs whose dimension differs from the maximal one among the ones to concatenate.The accuracy is predicted in step 3 of Figure1 for a built network (all following steps will be skipped if the building fails at the GPU loading stage) through regression trees generated by XGBoost. Since the inputs of trees are vectors, the network needs to be vectorized. This requires selecting and scalarizing network components to generate a vector uniquely representing the network. In this study, normal cell structure, pooling layer type, the cell pipeline, and the channels of cell outputs are chosen as the components. For the pipeline, the expanded form of formula (1) is(4)f=∑i=1nC˙i+C¨n+1+∑i=n+22n+1C˙i+C¨2n+2+⋯+C¨Rn+R+∑i=Rn+1+1R+1n+RC˙i.The pipeline is scalarized by(5)fs=∑Cℓ∈f2ℓ⋅δCℓ−C˙,where ℓ corresponds to the cell index in formula (4) regardless of the cell type, the subtraction estimates whether the current cell is a normal cell C˙, and δx is 1 if x is 0, and it is 0 otherwise. The output channels are scalarized by(6)Chs=∑Cℓ∈fchCℓ,where the channel of the outputs yielded by Cℓ is denoted by chCℓ. The structure of the normal cell is scalarized w.r.t. each block, i.e.,(7)B˙s,j=∑s∈SjidxSs⋅SidxSjs,where Sj=Ij,1,Tj,1…Ij,mj,Tj,mj represents the structure of the j th block, i.e., the pairs of operation input and its type; S=T3c,T5c,T3d,T5d,TI+T3m,T5m,T3a,T5a+ℬ−1,ℬ−2…ℬ−B+1+C−1,C−2…C−B contains the inputs and types of operations available for sampling; idxSjs finds the index of the member s from S. Similarly, the pooling layer is scalarized as B¨s. In short, the aforementioned formulae (5) to (7) are called conversion functions, and a given network f is vectorized by arranging the scalars yielded by the conversion functions, i.e.,(8)ℐf=fsChsB˙s,1…B˙s,BB¨s.Since XGBoost is a supervised learning method, its training is based on datasets containing pairs of vectorized networks and their accuracies. The vector datasets are built by randomly sampling networks first and then training and validating sampled networks. The training and validation accuraciesAT and AV together with the network result in two vector datasets: {vectors, ATs} and {vectors, AVs}. Two regression trees are built by XGBoost, respectively, based on the vector datasets. Thus, the training accuracy of a given network f is predicted by(9)AT∗=TTℐf,and its validation accuracy is obtained similarly. The predicted accuracies are employed to generate a reward in step 4 of Figure 1, i.e.,(10)A=max0,1−αAV∗+αAV∗−AT∗,where 0<α<1 is a hyperparameter. The definition of A differs from conventional rewards reported in NAS literatures [21–23]. This is because we noticed that the overfitting always occurs in the case that the validation accuracy does not improve while the training accuracy keeps high. Thus, to avoid networks easy to overfit, we introduced the difference between the training and the validation accuracies. Thus, if the validation accuracy is much smaller than the training accuracy, then the network may easily overfit, and the reward should be very low, which is reflected through A by a large negative value produced by the accuracy difference. However, we cannot have negative accuracies in practice; thus, we apply a ReLU function [37] to guarantee the resulting A is non-negative. The reward then serves to generate the loss Jθ [39]:(11)Jθ=A∑jBlogPnj|θ+∑knjlogPak|a1:k−1,θ+logPa|θ,where Pnj|θ denotes the probability of sampling the operation number nj of the j th block in the normal cell, Pak|a1:k−1,θ is the probability of sampling the input and operation type for the k th block after the first k−1 operations have been sampled, and Pa|θ represents the probability of sampling the operation for the reduction cell. The gradient ▽θJθ is then employed for updating LSTM with weights θ.In step 5 of Figure1, the optimal network is selected from the search history w.r.t. specific requirements defined by the device. The networks filtrated by restrictions are decreasingly sorted by their rewards from search, and top 25% are retrained and tested w.r.t. a few epochs, e.g., 15 epochs. Then, the retrained networks are sorted according to their test accuracies, and top 25% are retrained and tested w.r.t. an increased epoch value. This repeats until one network is left, and this network serves as the output of the search.


## 3. Results and Discussion

The performances of NASRT are reflected by both the search efficiency and the performance of the resulting CNN. The search is based on CIFAR-10 [40]. The CNN performance is evaluated on wildlife datasets ENA24 [41] and MCTI [42]. The search efficiency of NASRT is compared with the classical and the state-of-art NAS methods, i.e., NASNet [22], PNASNet [24], PDARTS [43], SGAS [44], SETN [26], and MnasNet [45]. The CNN performance is compared with the manually derived networks Resnet-18 [46], DenseNet [47], and MobileNet-v2 [48] and the automatically designed networks PDARTS [24], SGAS [44], and SETN [26]. The hardware in experiments comprises Jetson X2 with NVIDIA Pascal GPU, a laptop with GeForce GTX 1060 GPU, and a server with four GPUs of NVIDIA TITAN Xp. The software of experiments involves CUDA 9.1, PyTorch 0.4.1, Python 3.6, and MySQL 8.


### 3.1. Datasets

There are three datasets employed in this study, i.e., CIFAR-10 [40], ENA24 [41], and MCTI [42]. These datasets serve for different purposes, i.e., CIFAR-10 serves for the search only, while datasets ENA24 and MCTI serve for classifying animal species. CIFAR-10 contains 60K 32-by-32 colour images categorized to ten classes in which six are animals, i.e., bird, cat, deer, dog, frog, and horse. ENA24 contains 8K 2048-by-1536 images categorized to 21 animal species including crow, cat, white-tailed deer, coyote to name a few. MCTI consists of 24K wildlife images whose resolutions range from 1920-by-1080 to 2048-by-1536, and the images are categorized into 20 wildlife species, e.g., bird, ocelot, roe deer, red fox. The and background habitats in camera trap images, the images of both ENA24 and MCTI are resized to 64-by-64. Obviously, some species from MCTI and ENA24 are closely related with some classes of CIFAR-10. The class relationship is graphically illustrated in Figure 4.The testing images of either ENA24 or MCTI are randomly selected and account for 20% of all images of the corresponding datasets. For instance, there is a bear-shaped silhouette near the upper-left corner of the rectangle of ENA24 in Figure4; at the foot of the silhouette, there is a label indicating the class name is black bear with 730 training and 163 testing images. The class relationship is visualized by rectangles expanded through datasets. Namely, if classes from different datasets are covered by the same rectangle, then either their shapes are similar or they are biologically related in taxonomy.Figure 4
Class relationship among datasets. Each dataset is represented by a rectangle labelled by the dataset name. In each rectangle, a class is highlighted by a silhouette labelled by the class name and numbers of its training and testing images. The classes of images containing similar-shaped objects are enclosed by polygons of different line colours, e.g., both the class vehicle in ENA24 and the class truck in CIFAR-10 contain images of various trucks, and thus, the two classes are enclosed in one polygon of brown lines.


### 3.2. Search on CIFAR-10

Since over half classes of CIFAR-10 are animal species and most of the species are closely related to wildlife species shown in Figure4, CIFAR-10 serves for finding CNN based on the adaptive meta-architecture as shown in Figure 2. The maximum values of normal cell number N and reduced cell number R are set to 5 and 3, respectively. The block number B and the operation number M are set to 5 and 4, respectively. For the combination of R=3, M=4, and N=B=5, there are approximately 2.7 M candidate networks in the search space.Regression trees are learned by XGBoost based on 0.02 M randomly sampled networks, and networks are selected so that their validation accuracies are evenly distributed. Specifically, the sampled networks are vectorized through conversion functions, trained on 40 K out of 50 K training images from CIFAR-10, and then validated on the left 10 K training images. Thus, the vectors and training accuracies, and the vectors and validation accuracies form two datasets to generate trees. Data augmentation of the training is the same as [23], and AMSGrad [49] serves as the optimizer whose learning rate is set to 0.005. The batch size is 128, and the epoch number is 1.XGBoost involves twelve hyperparameters automatically determined by Bayesian optimization [50]. The details are introduced in Supplementary Materials (available here). Finally, 72 and 166 regression trees are generated by XGBoost, respectively, on training-accuracy-based and validation-accuracy-based vector datasets. The trees may be either shallow or deep as depicted in Figures 5 and 6, respectively.Figure 5
A shallow regression tree. A regression tree is a stack of binary trees which conduct condition evaluations at nodes. A leaf node corresponds to a possible prediction yielded by the regression tree. A leaf node can be reached by walking along a path from the root to the leaf, and the “walk” means the input satisfies all conditions of nodes along the path. In this figure, different kinds of nodes are represented by rectangles painted by different colours, e.g., nodes involvingB˙ and B¨ are, respectively, denoted by red and blue rectangles, and the leaf nodes are represented by green nodes. The path between two nodes is denoted by arrows labelled “True” or “False”.Figure 6
A deep regression tree. The meaning of graphical elements in this figure is the same as in Figure5.For search, the hidden-unit number of LSTM is set to 300, and the dimension of word embedding is 512. The outputs of the softmax function serve as the probabilities of formula (11). AMSGrad serves as the optimizer of LSTM, and the learning rate is set to 10−3. The sampling times are 1.5×103. The hyperparameter of formula (10) is set by α=0.4. Since randomness is inevitable in NAS, the searches are often repeated to obtain the optimal networks [21, 27]. Thanks to the proposed acceleration, the costs of repetitive searches are acceptable. Therefore, the search is repeated, until the result is considered satisfactory w.r.t. the resource limited by Jetson X2. Namely, the on-board memory of Jetson X2 is 8 GB which approximates 12 GB GPU memory of the workstation employed for search, but in Jetson X2, the memory is shared by both CPU and GPU. In experiments, we find that Jetson X2 memory available for GPU approximately corresponds to 5 GB of workstation, and then, this memory limit becomes the restriction to filtrate networks retrieved from the search history.The normal cell of the optimal network found by NASRT is shown in Figure7. In the figure, the operations are denoted by colour rectangles, the blocks are represented by dashed-line rectangles, and the cells are represented by rectangles with colourless faces and colour edges. The arrows indicate the connections among cells and blocks. There are N=3 normal cells in the pipeline of the resulting network, and 3-by-3 max pooling serves as the reduction cell. The channel number d throughout operations is set to 36, i.e., outputs of operations always have 36 channels. For operations altering channels, their hyperparameters are set to produce d output channels, e.g., the kernels of convolution, and for operations preserving channels, the channels of their inputs are mapped to d through a stack of the convolution with d kernels of size 1×1, batch normalization, and ReLU.To evaluate the search efficiency, the search time of NASRT is compared with the classical and the state-of-art search methods, i.e., PDARTS, SGAS, SETN, PNASNet, MnasNet and NASNet-A, and the CNNs found by these methods are compared with the one of NASRT. The methods are compared w.r.t. the network parameter number in millions (“Para. (M)”), the inference time in seconds (“Test [sec.]”), GPU memory consumption in megabytes (“GPU (MB)”), and the search time in days when a single GPU is employed (“Time (GPU days)”) as shown in Table1.Table 1
Search comparison.
MethodPara. (M)†Test (sec.)†GPU (MB)†Time (GPU days)NASRT (ours)2.490.1710.420.27†Resnet-18 [46]11.190.0144.81ManualDenseNet [47]6.980.0828.35ManualMobileNet-v2 [48]2.250.059.23ManualPDARTS [24]3.440.2514.500.81†SGAS [44]4.420.2718.540.97†SETN [26]4.630.2719.311.08†PNASNet [24]4.280.1717.53225.00‡MnasNet [45]3.130.0312.75480.00‡NASNet-A [22]3.250.2513.791800.00‡†Obtained through our software or hardware or both; ‡cited from works of literature.As shown in Table1, the search time of NASRT, PDARTS, SGAS, and SETN is obtained by conducting the searches on our workstation with four Titan Xp GPUs. The search time of NASRT is the best throughout all methods, which validates its search efficiency. For the resulting network, its parameter number, inference time, and GPU memory consumption are obtained by feeding a 64-by-64 image to the network on the laptop of GTX 1060 GPU whose low computational capability specially serves the time estimation. In Table 1, NASRT consumes the second least GPU memory and the least search time.


### 3.3. Tests on ENA24

The CNN found by NASRT is tested on ENA24 to evaluate its performance in classifying animal species in camera trap images. The images in ENA24 are categorized to 21 species illustrated by the silhouettes in Figure4 which also shows the number of the images serving for training and test. The CNN of NASRT is trained from scratch on training images. Before the training starts, the network parameters are initialized through Xavier uniform [51]. The data augmentation involves CutOut [52], horizontal image flip, image crop, and normalization. The CNN is optimized through stochastic gradient descent [53]. The learning rate is adjusted by a schedule of cosine [54] with hyperparameters lmax=5×10−3 and lmin=10−1. The batch size and epoch are set to 32 and 55, respectively.Figure 7
The normal cell found by NASRT. This figure is a concrete version of Figure3, and hence, only the differences are described here. The output of a cell is indicated by an arrow with a dashed line whose colour is the same as the cell rectangle. Both operation outputs and block outputs are represented by arrows with solid black lines. Blocks are denoted by rectangles with dashed lines.Besides the proposed CNN, several manually derived and automatically derived CNNs are introduced in the experiments for comparison. The manually designed CNNs are Resnet-18, DenseNet, and MobileNet-v2. The automatically designed CNNs are SGAS, SETN, and PDARTS. These networks are trained based on the same configuration as NASRT with a smaller batch size due to their high consumptions of GPU memory as shown in Table1. Accordingly, the batch sizes are set to 8 (SGAS, PDARTS) and 10 (SETN). However, the small batch requires more training time than the large batch, which means SGAS, SETN, and PDARTS will consume more computational resources than other methods if the epochs of all methods are the same. Hence, their epochs are halved to 25. The results are shown in Table 2 where the bold texts highlight the best accuracies for each row.Table 2
Accuracy comparison based on ENA24.
ClassNASRT (ours)Resnet-18 [46]DenseNet [47]MobileNet-v2 [48]SGAS [44]SETN [26]PDARTS [24]American black bear96.9398.1695.0995.7193.8796.3296.32American crow98.0699.3599.3592.9098.7196.7799.35Bird100.00100.00100.0097.14100.0091.43100.00Bobcat95.1695.1693.5595.1690.3280.6593.55Chicken98.0297.0399.0195.0597.0394.0698.02Coyote96.6198.3198.3196.6198.3198.3196.61Dog96.8398.4197.6296.8396.0397.6296.83Domestic cat96.5591.9591.9586.2191.9593.1093.10Eastern chipmunk100.00100.00100.00100.00100.00100.00100.00Eastern cottontail95.2492.0698.4187.3092.0687.3090.48Eastern fox squirrel96.7296.7296.7296.7295.0896.7296.72Eastern gray squirrel96.4398.2196.4396.4396.4394.6498.21Gray fox94.6796.0098.6790.6792.0088.0089.33Northern raccoon94.1294.1296.0888.2494.1292.1692.16Red fox97.2697.2694.5297.2695.8990.4194.52Striped skunk98.1896.3698.1892.7392.7396.3692.73Vehicle100.00100.00100.00100.00100.0095.4595.45Virginia opossum98.4899.24100.0098.48100.0099.2499.24White-tailed deer98.3698.36100.0096.7296.7295.0896.72Wild turkey100.0098.1596.3094.4492.5996.3098.15Woodchuck97.3097.3097.3097.3094.5997.3097.30Average97.3897.2597.5094.8595.6494.1595.94As shown in Table2, the top three average accuracies are achieved by DenseNet (97.5%), NASRT (97.38%), and Resnet-18 (97.25%). The differences among the top three average accuracies are relatively small, while DenseNet and Resnet-18 are manually derived. Due to the decreased epoch numbers, SGAS (95.64%), SETN (94.15%), and PDARTS (95.94%) achieve the average accuracies lower than NASRT. For individual class accuracies, DenseNet achieves the best class accuracies for 14 classes, Resnet-18 for 12 classes, and NASRT for 10 classes. For NASRT, the bottom four class accuracies are associated with northern raccoon (94.12%), grey fox (94.67%), bobcat (95.16%), and cottontail (95.24%). To analyse the errors of NASRT, we start with the misclassifications of the general case as shown in Figure 8 and then continue with the bottom four accuracies as shown in Figures 9–12. In these figures, misclassified images and associated species predicted by NASRT with top five accuracies are illustrated, and the misclassified species and the correct species are, respectively, indicated with red and green colours.Figure 8
Examples of misclassified images from ENA24. This figure illustrates four misclassified image samples together with corresponding prediction results. The results associated with each image sample illustrate five top predictions yielded by NASRT, and both the predicted species names and the corresponding probabilities are represented by texts under the image sample. The probabilities are visualized by bars of different colours for visual discrimination. The ground-truth species are highlighted by green texts, and the erroneously predicted species are indicated by red texts. The animals of the ground-truth species in sample images are enclosed by red lines for enhancing the visibility of animals.Figure 9
Misclassified northern raccoon images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 10
Misclassified grey fox images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 11
Misclassified bobcat images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 12
Misclassified cottontail images. The meaning of graphical elements in this figure is the same as in Figure8.As shown in Figure8, the misclassification is made by NASRT when animals are blocked (left-most subfigure), of cryptic coloration (middle-left subfigure), blurred/night vision (middle-right subfigure), or partially visible in the images. The aforementioned cases may be overlapped as shown in Figures 9–12.


### 3.4. Tests on MCTI

The tests on ENA24 illustrate the performances of NASRT for limited data, i.e., there are totally 8K images for 21 species. It is curious to find out its performance under the contradictive condition, i.e., abundant data as in dataset MCTI (there are totally 24K images for 20 species). The training and testing on MCTI are the same as the case of ENA24, and the results are shown in Table3.Table 3
Accuracy comparison based on MCTI.
ClassNASRT (ours)Resnet-18 [46]DenseNet [47]MobileNet-v2 [48]SGAS [44]SETN [26]PDARTS [24]Agouti97.6095.2196.4195.8196.4189.8292.81Bird98.6197.9297.9299.31100.0098.61100.00Coiban agouti100.00100.00100.00100.0099.6198.06100.00Collared peccary98.2596.4994.1592.9897.6692.4098.83Opossum98.1594.4496.9196.3098.1593.2197.53European hare97.4889.9296.6496.6496.6499.1694.96Great tinamou99.5699.1198.2299.5699.5693.3397.78Mouflon99.7799.3099.3099.0799.3097.9198.84Ocelot89.4782.1185.2675.7982.1168.4274.74Paca100.0092.0996.7495.3595.8190.7096.28Red brocket deer96.5993.7596.0293.1891.4893.1892.61Red deer100.00100.0099.80100.00100.0099.61100.00Red fox95.6093.4192.3191.2193.4179.1293.41Red squirrel97.3593.8194.6993.8194.6988.5087.61Roe deer98.7298.2998.2998.2998.2996.5898.29Spiny rat99.24100.0096.2191.6797.7384.8597.73White-nosed coati99.5799.1598.7297.0297.8795.3297.45White-tailed deer99.7598.9997.9898.2499.5095.9799.75Wild boar99.7198.8299.4199.7199.4199.4199.71Wood mouse100.00100.00100.00100.00100.00100.00100.00Average98.2796.1496.7595.7096.8892.7195.92As shown in Table3, the top three average accuracies are achieved by NASRT (98.27%), SGAS (96.88%), and DenseNet (96.75%). The difference between the average accuracy of NASRT and any other network exceeds 1%. Among the manually derived networks, the average accuracy of Resnet-18 is very close to DenseNet, which may explain the popularity of Resnet-18 in wildlife identification [14, 15]. For individual class accuracies, NASRT outperforms all other networks throughout 16 species, even though there are still misclassifications made by NASRT. The typical misclassified images are shown in Figure 13, and the examples of three species with the lowest accuracies, i.e., ocelot (89.47%), red fox (95.6%), and red brocket deer (96.59%) are illustrated in Figure 14 to 16. The fourth lowest accuracy is associated with a red squirrel (97.35%), and there are only two misclassified images, and one has been shown in Figure 13.Figure 13
Examples of misclassified images from MCTI. The meaning of graphical elements in this figure is the same as inFigure8.Figure 14
Misclassified ocelot images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 15
Misclassified red brocket deer images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 16
Misclassified red fox images. The meaning of graphical elements in this figure is the same as in Figure8.As shown in Figure13, misclassification may occur when the animal is not side viewed (left-most subfigure), camouflaging (middle-left subfigure), blurred (middle-right subfigure), or partially visible (right-most subfigure). The aforementioned cases may overlap as shown in Figures 14–16.


### 3.5. Tests on Jetson X2

The previous sections illustrate the results of experiments conducted on the workstation with abundant computational resources. However, these experiments cannot illustrate the case of applying the proposed network to resource-constrained edge devices such as Jetson X2 as shown in Figure17. Therefore, the network is retested on Jetson X2.Figure 17
Jetson X2. This figure illustrates an edge device named Jetson X2 in its working state.The software in experiments involves Ubuntu 18.06, Python 3.6.7, CUDA 10.0, Pytorch 1.1.0, and torchvision 0.2.0. Both the test images and the weights of the pretrained network are copied to Jetson X2 through secure copy protocol (SCP) in the local area network. Table4 shows the results from Jetson X2.Table 4
Accuracies from Jetson X2.
ENA24 classNASRTMCTI classNASRTBlack bear†98.77Agouti95.21Crow†97.42Bird97.22Bird100.00Coiban agouti100.00Bobcat91.94Collared peccary99.42Chicken99.01Opossum98.15Coyote98.31European hare100.00Dog97.62Great tinamou98.67Domestic cat91.95Mouflon99.07Chipmunk‡100.00Ocelot78.95Cottontail‡92.06Paca99.53Fox squirrel‡96.72Red brocket deer94.89Gray squirrel‡98.21Red deer100.00Gray fox93.33Red fox93.41Northern raccoon94.12Red squirrel93.81Red fox95.89Roe deer99.15Striped skunk100.00Spiny rat96.21Vehicle100.00White-nosed coati99.57Virginia opossum100.00White-tailed deer99.24White-tailed deer93.44Wild boar99.41Wild turkey100.00Wood mouse100.00Woodchuck97.30Average98.23Average97.03†American species; ‡Eastern species.As shown in Table4, the average accuracies of the proposed network are 97.03% and 98.23%, respectively, for datasets ENA24 and MCTI. The accuracies from Jetson X2 are slightly lower than the workstation (97.38% of ENA24 and 98.27% of MCTI).


## 3.1. Datasets

There are three datasets employed in this study, i.e., CIFAR-10 [40], ENA24 [41], and MCTI [42]. These datasets serve for different purposes, i.e., CIFAR-10 serves for the search only, while datasets ENA24 and MCTI serve for classifying animal species. CIFAR-10 contains 60K 32-by-32 colour images categorized to ten classes in which six are animals, i.e., bird, cat, deer, dog, frog, and horse. ENA24 contains 8K 2048-by-1536 images categorized to 21 animal species including crow, cat, white-tailed deer, coyote to name a few. MCTI consists of 24K wildlife images whose resolutions range from 1920-by-1080 to 2048-by-1536, and the images are categorized into 20 wildlife species, e.g., bird, ocelot, roe deer, red fox. The and background habitats in camera trap images, the images of both ENA24 and MCTI are resized to 64-by-64. Obviously, some species from MCTI and ENA24 are closely related with some classes of CIFAR-10. The class relationship is graphically illustrated in Figure 4.The testing images of either ENA24 or MCTI are randomly selected and account for 20% of all images of the corresponding datasets. For instance, there is a bear-shaped silhouette near the upper-left corner of the rectangle of ENA24 in Figure4; at the foot of the silhouette, there is a label indicating the class name is black bear with 730 training and 163 testing images. The class relationship is visualized by rectangles expanded through datasets. Namely, if classes from different datasets are covered by the same rectangle, then either their shapes are similar or they are biologically related in taxonomy.Figure 4
Class relationship among datasets. Each dataset is represented by a rectangle labelled by the dataset name. In each rectangle, a class is highlighted by a silhouette labelled by the class name and numbers of its training and testing images. The classes of images containing similar-shaped objects are enclosed by polygons of different line colours, e.g., both the class vehicle in ENA24 and the class truck in CIFAR-10 contain images of various trucks, and thus, the two classes are enclosed in one polygon of brown lines.


## 3.2. Search on CIFAR-10

Since over half classes of CIFAR-10 are animal species and most of the species are closely related to wildlife species shown in Figure4, CIFAR-10 serves for finding CNN based on the adaptive meta-architecture as shown in Figure 2. The maximum values of normal cell number N and reduced cell number R are set to 5 and 3, respectively. The block number B and the operation number M are set to 5 and 4, respectively. For the combination of R=3, M=4, and N=B=5, there are approximately 2.7 M candidate networks in the search space.Regression trees are learned by XGBoost based on 0.02 M randomly sampled networks, and networks are selected so that their validation accuracies are evenly distributed. Specifically, the sampled networks are vectorized through conversion functions, trained on 40 K out of 50 K training images from CIFAR-10, and then validated on the left 10 K training images. Thus, the vectors and training accuracies, and the vectors and validation accuracies form two datasets to generate trees. Data augmentation of the training is the same as [23], and AMSGrad [49] serves as the optimizer whose learning rate is set to 0.005. The batch size is 128, and the epoch number is 1.XGBoost involves twelve hyperparameters automatically determined by Bayesian optimization [50]. The details are introduced in Supplementary Materials (available here). Finally, 72 and 166 regression trees are generated by XGBoost, respectively, on training-accuracy-based and validation-accuracy-based vector datasets. The trees may be either shallow or deep as depicted in Figures 5 and 6, respectively.Figure 5
A shallow regression tree. A regression tree is a stack of binary trees which conduct condition evaluations at nodes. A leaf node corresponds to a possible prediction yielded by the regression tree. A leaf node can be reached by walking along a path from the root to the leaf, and the “walk” means the input satisfies all conditions of nodes along the path. In this figure, different kinds of nodes are represented by rectangles painted by different colours, e.g., nodes involvingB˙ and B¨ are, respectively, denoted by red and blue rectangles, and the leaf nodes are represented by green nodes. The path between two nodes is denoted by arrows labelled “True” or “False”.Figure 6
A deep regression tree. The meaning of graphical elements in this figure is the same as in Figure5.For search, the hidden-unit number of LSTM is set to 300, and the dimension of word embedding is 512. The outputs of the softmax function serve as the probabilities of formula (11). AMSGrad serves as the optimizer of LSTM, and the learning rate is set to 10−3. The sampling times are 1.5×103. The hyperparameter of formula (10) is set by α=0.4. Since randomness is inevitable in NAS, the searches are often repeated to obtain the optimal networks [21, 27]. Thanks to the proposed acceleration, the costs of repetitive searches are acceptable. Therefore, the search is repeated, until the result is considered satisfactory w.r.t. the resource limited by Jetson X2. Namely, the on-board memory of Jetson X2 is 8 GB which approximates 12 GB GPU memory of the workstation employed for search, but in Jetson X2, the memory is shared by both CPU and GPU. In experiments, we find that Jetson X2 memory available for GPU approximately corresponds to 5 GB of workstation, and then, this memory limit becomes the restriction to filtrate networks retrieved from the search history.The normal cell of the optimal network found by NASRT is shown in Figure7. In the figure, the operations are denoted by colour rectangles, the blocks are represented by dashed-line rectangles, and the cells are represented by rectangles with colourless faces and colour edges. The arrows indicate the connections among cells and blocks. There are N=3 normal cells in the pipeline of the resulting network, and 3-by-3 max pooling serves as the reduction cell. The channel number d throughout operations is set to 36, i.e., outputs of operations always have 36 channels. For operations altering channels, their hyperparameters are set to produce d output channels, e.g., the kernels of convolution, and for operations preserving channels, the channels of their inputs are mapped to d through a stack of the convolution with d kernels of size 1×1, batch normalization, and ReLU.To evaluate the search efficiency, the search time of NASRT is compared with the classical and the state-of-art search methods, i.e., PDARTS, SGAS, SETN, PNASNet, MnasNet and NASNet-A, and the CNNs found by these methods are compared with the one of NASRT. The methods are compared w.r.t. the network parameter number in millions (“Para. (M)”), the inference time in seconds (“Test [sec.]”), GPU memory consumption in megabytes (“GPU (MB)”), and the search time in days when a single GPU is employed (“Time (GPU days)”) as shown in Table1.Table 1
Search comparison.
MethodPara. (M)†Test (sec.)†GPU (MB)†Time (GPU days)NASRT (ours)2.490.1710.420.27†Resnet-18 [46]11.190.0144.81ManualDenseNet [47]6.980.0828.35ManualMobileNet-v2 [48]2.250.059.23ManualPDARTS [24]3.440.2514.500.81†SGAS [44]4.420.2718.540.97†SETN [26]4.630.2719.311.08†PNASNet [24]4.280.1717.53225.00‡MnasNet [45]3.130.0312.75480.00‡NASNet-A [22]3.250.2513.791800.00‡†Obtained through our software or hardware or both; ‡cited from works of literature.As shown in Table1, the search time of NASRT, PDARTS, SGAS, and SETN is obtained by conducting the searches on our workstation with four Titan Xp GPUs. The search time of NASRT is the best throughout all methods, which validates its search efficiency. For the resulting network, its parameter number, inference time, and GPU memory consumption are obtained by feeding a 64-by-64 image to the network on the laptop of GTX 1060 GPU whose low computational capability specially serves the time estimation. In Table 1, NASRT consumes the second least GPU memory and the least search time.


## 3.3. Tests on ENA24

The CNN found by NASRT is tested on ENA24 to evaluate its performance in classifying animal species in camera trap images. The images in ENA24 are categorized to 21 species illustrated by the silhouettes in Figure4 which also shows the number of the images serving for training and test. The CNN of NASRT is trained from scratch on training images. Before the training starts, the network parameters are initialized through Xavier uniform [51]. The data augmentation involves CutOut [52], horizontal image flip, image crop, and normalization. The CNN is optimized through stochastic gradient descent [53]. The learning rate is adjusted by a schedule of cosine [54] with hyperparameters lmax=5×10−3 and lmin=10−1. The batch size and epoch are set to 32 and 55, respectively.Figure 7
The normal cell found by NASRT. This figure is a concrete version of Figure3, and hence, only the differences are described here. The output of a cell is indicated by an arrow with a dashed line whose colour is the same as the cell rectangle. Both operation outputs and block outputs are represented by arrows with solid black lines. Blocks are denoted by rectangles with dashed lines.Besides the proposed CNN, several manually derived and automatically derived CNNs are introduced in the experiments for comparison. The manually designed CNNs are Resnet-18, DenseNet, and MobileNet-v2. The automatically designed CNNs are SGAS, SETN, and PDARTS. These networks are trained based on the same configuration as NASRT with a smaller batch size due to their high consumptions of GPU memory as shown in Table1. Accordingly, the batch sizes are set to 8 (SGAS, PDARTS) and 10 (SETN). However, the small batch requires more training time than the large batch, which means SGAS, SETN, and PDARTS will consume more computational resources than other methods if the epochs of all methods are the same. Hence, their epochs are halved to 25. The results are shown in Table 2 where the bold texts highlight the best accuracies for each row.Table 2
Accuracy comparison based on ENA24.
ClassNASRT (ours)Resnet-18 [46]DenseNet [47]MobileNet-v2 [48]SGAS [44]SETN [26]PDARTS [24]American black bear96.9398.1695.0995.7193.8796.3296.32American crow98.0699.3599.3592.9098.7196.7799.35Bird100.00100.00100.0097.14100.0091.43100.00Bobcat95.1695.1693.5595.1690.3280.6593.55Chicken98.0297.0399.0195.0597.0394.0698.02Coyote96.6198.3198.3196.6198.3198.3196.61Dog96.8398.4197.6296.8396.0397.6296.83Domestic cat96.5591.9591.9586.2191.9593.1093.10Eastern chipmunk100.00100.00100.00100.00100.00100.00100.00Eastern cottontail95.2492.0698.4187.3092.0687.3090.48Eastern fox squirrel96.7296.7296.7296.7295.0896.7296.72Eastern gray squirrel96.4398.2196.4396.4396.4394.6498.21Gray fox94.6796.0098.6790.6792.0088.0089.33Northern raccoon94.1294.1296.0888.2494.1292.1692.16Red fox97.2697.2694.5297.2695.8990.4194.52Striped skunk98.1896.3698.1892.7392.7396.3692.73Vehicle100.00100.00100.00100.00100.0095.4595.45Virginia opossum98.4899.24100.0098.48100.0099.2499.24White-tailed deer98.3698.36100.0096.7296.7295.0896.72Wild turkey100.0098.1596.3094.4492.5996.3098.15Woodchuck97.3097.3097.3097.3094.5997.3097.30Average97.3897.2597.5094.8595.6494.1595.94As shown in Table2, the top three average accuracies are achieved by DenseNet (97.5%), NASRT (97.38%), and Resnet-18 (97.25%). The differences among the top three average accuracies are relatively small, while DenseNet and Resnet-18 are manually derived. Due to the decreased epoch numbers, SGAS (95.64%), SETN (94.15%), and PDARTS (95.94%) achieve the average accuracies lower than NASRT. For individual class accuracies, DenseNet achieves the best class accuracies for 14 classes, Resnet-18 for 12 classes, and NASRT for 10 classes. For NASRT, the bottom four class accuracies are associated with northern raccoon (94.12%), grey fox (94.67%), bobcat (95.16%), and cottontail (95.24%). To analyse the errors of NASRT, we start with the misclassifications of the general case as shown in Figure 8 and then continue with the bottom four accuracies as shown in Figures 9–12. In these figures, misclassified images and associated species predicted by NASRT with top five accuracies are illustrated, and the misclassified species and the correct species are, respectively, indicated with red and green colours.Figure 8
Examples of misclassified images from ENA24. This figure illustrates four misclassified image samples together with corresponding prediction results. The results associated with each image sample illustrate five top predictions yielded by NASRT, and both the predicted species names and the corresponding probabilities are represented by texts under the image sample. The probabilities are visualized by bars of different colours for visual discrimination. The ground-truth species are highlighted by green texts, and the erroneously predicted species are indicated by red texts. The animals of the ground-truth species in sample images are enclosed by red lines for enhancing the visibility of animals.Figure 9
Misclassified northern raccoon images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 10
Misclassified grey fox images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 11
Misclassified bobcat images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 12
Misclassified cottontail images. The meaning of graphical elements in this figure is the same as in Figure8.As shown in Figure8, the misclassification is made by NASRT when animals are blocked (left-most subfigure), of cryptic coloration (middle-left subfigure), blurred/night vision (middle-right subfigure), or partially visible in the images. The aforementioned cases may be overlapped as shown in Figures 9–12.


## 3.4. Tests on MCTI

The tests on ENA24 illustrate the performances of NASRT for limited data, i.e., there are totally 8K images for 21 species. It is curious to find out its performance under the contradictive condition, i.e., abundant data as in dataset MCTI (there are totally 24K images for 20 species). The training and testing on MCTI are the same as the case of ENA24, and the results are shown in Table3.Table 3
Accuracy comparison based on MCTI.
ClassNASRT (ours)Resnet-18 [46]DenseNet [47]MobileNet-v2 [48]SGAS [44]SETN [26]PDARTS [24]Agouti97.6095.2196.4195.8196.4189.8292.81Bird98.6197.9297.9299.31100.0098.61100.00Coiban agouti100.00100.00100.00100.0099.6198.06100.00Collared peccary98.2596.4994.1592.9897.6692.4098.83Opossum98.1594.4496.9196.3098.1593.2197.53European hare97.4889.9296.6496.6496.6499.1694.96Great tinamou99.5699.1198.2299.5699.5693.3397.78Mouflon99.7799.3099.3099.0799.3097.9198.84Ocelot89.4782.1185.2675.7982.1168.4274.74Paca100.0092.0996.7495.3595.8190.7096.28Red brocket deer96.5993.7596.0293.1891.4893.1892.61Red deer100.00100.0099.80100.00100.0099.61100.00Red fox95.6093.4192.3191.2193.4179.1293.41Red squirrel97.3593.8194.6993.8194.6988.5087.61Roe deer98.7298.2998.2998.2998.2996.5898.29Spiny rat99.24100.0096.2191.6797.7384.8597.73White-nosed coati99.5799.1598.7297.0297.8795.3297.45White-tailed deer99.7598.9997.9898.2499.5095.9799.75Wild boar99.7198.8299.4199.7199.4199.4199.71Wood mouse100.00100.00100.00100.00100.00100.00100.00Average98.2796.1496.7595.7096.8892.7195.92As shown in Table3, the top three average accuracies are achieved by NASRT (98.27%), SGAS (96.88%), and DenseNet (96.75%). The difference between the average accuracy of NASRT and any other network exceeds 1%. Among the manually derived networks, the average accuracy of Resnet-18 is very close to DenseNet, which may explain the popularity of Resnet-18 in wildlife identification [14, 15]. For individual class accuracies, NASRT outperforms all other networks throughout 16 species, even though there are still misclassifications made by NASRT. The typical misclassified images are shown in Figure 13, and the examples of three species with the lowest accuracies, i.e., ocelot (89.47%), red fox (95.6%), and red brocket deer (96.59%) are illustrated in Figure 14 to 16. The fourth lowest accuracy is associated with a red squirrel (97.35%), and there are only two misclassified images, and one has been shown in Figure 13.Figure 13
Examples of misclassified images from MCTI. The meaning of graphical elements in this figure is the same as inFigure8.Figure 14
Misclassified ocelot images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 15
Misclassified red brocket deer images. The meaning of graphical elements in this figure is the same as in Figure8.Figure 16
Misclassified red fox images. The meaning of graphical elements in this figure is the same as in Figure8.As shown in Figure13, misclassification may occur when the animal is not side viewed (left-most subfigure), camouflaging (middle-left subfigure), blurred (middle-right subfigure), or partially visible (right-most subfigure). The aforementioned cases may overlap as shown in Figures 14–16.


## 3.5. Tests on Jetson X2

The previous sections illustrate the results of experiments conducted on the workstation with abundant computational resources. However, these experiments cannot illustrate the case of applying the proposed network to resource-constrained edge devices such as Jetson X2 as shown in Figure17. Therefore, the network is retested on Jetson X2.Figure 17
Jetson X2. This figure illustrates an edge device named Jetson X2 in its working state.The software in experiments involves Ubuntu 18.06, Python 3.6.7, CUDA 10.0, Pytorch 1.1.0, and torchvision 0.2.0. Both the test images and the weights of the pretrained network are copied to Jetson X2 through secure copy protocol (SCP) in the local area network. Table4 shows the results from Jetson X2.Table 4
Accuracies from Jetson X2.
ENA24 classNASRTMCTI classNASRTBlack bear†98.77Agouti95.21Crow†97.42Bird97.22Bird100.00Coiban agouti100.00Bobcat91.94Collared peccary99.42Chicken99.01Opossum98.15Coyote98.31European hare100.00Dog97.62Great tinamou98.67Domestic cat91.95Mouflon99.07Chipmunk‡100.00Ocelot78.95Cottontail‡92.06Paca99.53Fox squirrel‡96.72Red brocket deer94.89Gray squirrel‡98.21Red deer100.00Gray fox93.33Red fox93.41Northern raccoon94.12Red squirrel93.81Red fox95.89Roe deer99.15Striped skunk100.00Spiny rat96.21Vehicle100.00White-nosed coati99.57Virginia opossum100.00White-tailed deer99.24White-tailed deer93.44Wild boar99.41Wild turkey100.00Wood mouse100.00Woodchuck97.30Average98.23Average97.03†American species; ‡Eastern species.As shown in Table4, the average accuracies of the proposed network are 97.03% and 98.23%, respectively, for datasets ENA24 and MCTI. The accuracies from Jetson X2 are slightly lower than the workstation (97.38% of ENA24 and 98.27% of MCTI).


## 4. Conclusions

In the present study, a neural architecture search method named NASRT is proposed for providing CNNs customized for diverse edge devices, and thus, edge devices can be incorporated with clusters of camera traps to set up or expand surveillance areas. There are mainly two challenges faced by NASRT, i.e., lowering search costs and searching networks feasible for edge devices. For the first challenge, the search costs are lowered by reducing the search space dimensionality and accelerating candidate network evaluations. The search space dimensionality is reduced by replacing the reduction cell with a single pooling layer, and the candidate network evaluation is accelerated via regression trees generated by XGBoost. Since regression trees can only process vectors, candidate networks are vectorized through conversion functions. For the second challenge, candidate networks are built w.r.t. an adaptive meta-architecture optimized according to computational resources defined by edge devices. On the basis of the simplified search space, the search acceleration, and the adaptive meta-architecture, NASRT successfully found a network applicable for the edge device Jetson X2, and its search time is the best in comparison. The performance of the network found by NASRT is evaluated on the data-limited dataset ENA24 and data-abundant dataset MCTI. The resulting average accuracies of identifying wildlife are, respectively, 97.38% and 98.27%, which are competitive compared with the classical and the state-of-art networks.The limitations of the present study are mainly twofold, i.e., the benchmark dataset used in this study differs from the camera trap datasets in both the data distribution and the image aspectratio. For the first limitation, since surveillance areas of camera trap clusters may cover different habitats of wild animals, data distributions may differ from cluster to cluster. The present study employs a benchmark dataset named CIFAR-10 to search candidate networks, and thus, the architectures of the searched networks are optimized according to images from benchmark datasets instead of camera trap images. For the second limitation, the candidate networks in this study are assumed to process images with the aspect ratio 1 : 1, i.e.., images with the same widths and heights, as other CNNs popular in the classification of camera trap images. However, camera trap images are usually 4 : 3 as shown in the section of results. The assumption of aspect ratio 1 : 1 requires images to be resized, and there are mainly two means to resize an image, i.e., rescaling the image without maintaining its original aspect ratio or padding short edges of the image to maintain its original aspect ratio. The former results in deformed animals, and the latter introduces interpolated pixels. Neither misshaped animals nor interpolated pixels would be helpful for the classification.Future work mainly concerns the application of camera trap images in the search, i.e., searches are conducted directly on camera trap images rather than images from benchmark datasets. Since camera trap images differ from benchmark dataset images in many aspects, especially the aspect ratios, a preprocessing step is expected to be developed to maintain the aspect ratios of camera trap images. Moreover, differences among images from different types of camera traps need to be considered in future studies.



---

*Source: 8615374-2022-02-07.xml*