Inferring Convolutional Neural Networks accuracies from their architectural characterizations. Convolutional Neural Networks (CNNs) have shown strong promise for analyzing scientific data from many domains including particle imaging detectors. However, the challenge of choosing the appropriate network architecture (depth, kernel shapes, activation functions, etc.) for specific applications and different data sets is still poorly understood. In this paper, we study the relationships between a CNNu0027s architecture and its performance by proposing a systematic language that is useful for comparison between different CNNu0027s architectures before training time. We characterize CNNu0027s architecture by different attributes, and demonstrate that the attributes can be predictive of the networksu0027 performance in two specific computer visionbased physics problems  event vertex finding and hadron multiplicity classification in the MINERvA experiment at Fermi National Accelerator Laboratory. In doing so, we extract several architectural attributes from optimized networksu0027 architecture for the physics problems, which are outputs of a model selection algorithm called Multinode Evolutionary Neural Networks for Deep Learning (MENNDL). We use machine learning models to predict whether a network can perform better than a certain threshold accuracy before training. The models perform 1620% better than random guessing. Additionally, we found an coefficient of determination of 0.966 for an Ordinary Least Squares model in a regression on accuracy over a large population of networks.

Memory and Interconnect Optimizations for PetaScale Deep Learning Systems Hardware accelerators are a promising solution to the stringent computational requirements of Deep Neural Networks (DNNs). Ranging from lowpower IP cores to server class systems, various accelerator architectures with high TOPSW peak processing efficiencies and flexibility to execute different DNN topologies have been proposed. Prior efforts improve core utilization through better dataflows and computation sequencing, but little effort has thus far been devoted to systematically programming DNN accelerators to extract best possible system utilization, particularly for DNN training, which can be parallelized across petascale systems. In this work, we address the hitherto open challenge of systematically mapping computations onto Petascale accelerator systems, comprising many (thousands of) processing cores spanning many chips, while maximizing overall system performance. We achieve this by characterizing the design space of possible mapping configurations, building a detailed performance model that incorporates every computation and datatransfer involved in DNN training, and using a design space exploration tool called DEEPSPATIALMATRIX to identify the performance optimal configuration. We highlight 4 key optimizations built within DEEPSPATIALMATRIX  hybrid datamodel parallelism, interlayer memory reuse, timestep pipelining, and dynamic spatial minibatching  each of which improve system utilization by carefully managing the available memory capacity and interconnect bandwidth to balance the compute vs. communication costs. On a 8petaFLOP accelerator system, we demonstrate 1.36xc3x9732xc3x97 improvement in training performance through our design space exploration and optimizations across image recognition (VGG16, ResNet50) and machine translation (GNMT) DNN models.