Reviewerly/exHarmony huggingface dataset:
Approximation rates for neural networks with general activation functions

We prove some new results concerning the approximation rate of neural networks with general activation functions. Our first result concerns the rate of approximation of a two layer neural network with a polynomially-decaying non-sigmoidal activation function. We extend the dimension independent approximation rates previously obtained to this new class of activation functions. Our second result gives a weaker, but still dimension independent, approximation rate for a larger class of activation functions, removing the polynomial decay assumption. This result applies to any bounded, integrable activation function. Finally, we show that a stratified sampling approach can be used to improve the approximation rate for polynomially decaying activation functions under mild additional assumptions.
Chaotic dynamics in a neural network with different types of external stimuli

Biological nervous system is very sensitive to external disturbances, and appropriate stimulus is beneficial for improving neural function in the neural system. In this paper, the effect of different external stimuli on chaotic dynamics in a Hopfield neural network with three neurons is explored. Mathematical model of the neural network is respectively established under three different cases, namely without external stimulus, with only electromagnetic radiation stimulus, and with both electromagnetic radiation stimulus and multi-level-logic pulse stimulus. Under the three cases, equilibrium points, stabilities, and attractors of the neural network are investigated carefully. The research results demonstrate that the neural network with periodic attractors can induce abundant chaotic attractors by imposing electromagnetic radiation on its one neuron. And when this neuron is simultaneously stimulated via electromagnetic radiation and multi-level-logic pulse, the neural network can produce complex multi-scroll attractors previously unobserved in Hopfield-type neural networks. Numerical results are verified by hardware experiments, effectively. Furthermore, based on the Helmholtzâs theorem, the Hamilton energy of the neural network is calculated and analyzed. It is found that lower average Hamilton energy can be detected in the neural network when complexity of external stimuli is enhanced. These new findings could offer a new insight into the occurrence mechanism of some neurological diseases.
Single Image Reflection Removal Through Cascaded Refinement

We address the problem of removing undesirable reflections from a single image captured through a glass surface, which is an ill-posed, challenging but practically important problem for photo enhancement. Inspired by iterative structure reduction for hidden community detection in social networks, we propose an Iterative Boost Convolutional LSTM Network (IBCLN) that enables cascaded prediction for reflection removal. IBCLN is a cascaded network that iteratively refines the estimates of transmission and reflection layers in a manner that they can boost the prediction quality to each other, and information across steps of the cascade is transferred using an LSTM. The intuition is that the transmission is the strong, dominant structure while the reflection is the weak, hidden structure. They are complementary to each other in a single image and thus a better estimate and reduction on one side from the original image leads to a more accurate estimate on the other side. To facilitate training over multiple cascade steps, we employ LSTM to address the vanishing gradient problem, and propose residual reconstruction loss as further training guidance. Besides, we create a dataset of real-world images with reflection and ground-truth transmission layers to mitigate the problem of insufficient data. Comprehensive experiments demonstrate that the proposed method can effectively remove reflections in real and synthetic images compared with state-of-the-art reflection removal methods.
Where Am I Looking At? Joint Location and Orientation Estimation by Cross-View Matching

Cross-view geo-localization is the problem of estimating the position and orientation (latitude, longitude and azimuth angle) of a camera at ground level given a large-scale database of geo-tagged aerial (eg., satellite) images. Existing approaches treat the task as a pure location estimation problem by learning discriminative feature descriptors, but neglect orientation alignment. It is well-recognized that knowing the orientation between ground and aerial images can significantly reduce matching ambiguity between these two views, especially when the ground-level images have a limited Field of View (FoV) instead of a full field-of-view panorama. Therefore, we design a Dynamic Similarity Matching network to estimate cross-view orientation alignment during localization. In particular, we address the cross-view domain gap by applying a polar transform to the aerial images to approximately align the images up to an unknown azimuth angle. Then, a two-stream convolutional network is used to learn deep features from the ground and polar-transformed aerial images. Finally, we obtain the orientation by computing the correlation between cross-view features, which also provides a more accurate measure of feature similarity, improving location recall. Experiments on standard datasets demonstrate that our method significantly improves state-of-the-art performance. Remarkably, we improve the top-1 location recall rate on the CVUSA dataset by a factor of 1.5x for panoramas with known orientation, by a factor of 3.3x for panoramas with unknown orientation, and by a factor of 6x for 180°-FoV images with unknown orientation.
More Grounded Image Captioning by Distilling Image-Text Matching Model

Visual attention not only improves the performance of image captioners, but also serves as a visual interpretation to qualitatively measure the caption rationality and model transparency. Specifically, we expect that a captioner can fix its attentive gaze on the correct objects while generating the corresponding words. This ability is also known as grounded image captioning. However, the grounding accuracy of existing captioners is far from satisfactory.To improve the grounding accuracy while retaining the captioning quality, it is expensive to collect the word-region alignment as strong supervision.To this end, we propose a Part-of-Speech (POS) enhanced image-text matching model (SCAN[24]): POS-SCAN, as the effective knowledge distillation for more grounded image captioning. The benefits are two-fold: 1) given a sentence and an image, POS-SCAN can ground the objects more accurately than SCAN; 2) POS-SCAN serves as a word-region alignment regularization for the captioner's visual attention module. By showing benchmark experimental results, we demonstrate that conventional image captioners equipped with POS-SCAN can significantly improve the grounding accuracy without strong supervision. Last but not the least, we explore the indispensable Self-Critical Sequence Training (SCST[46]) in the context of grounded image captioning and show that the image-text matching score can serve as a reward for more grounded captioning <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">1</sup> .
The 4th AI City Challenge

The AI City Challenge was created to accelerate intelligent video analysis that helps make cities smarter and safer. Transportation is one of the largest segments that can benefit from actionable insights derived from data captured by sensors, where computer vision and deep learning have shown promise in achieving large-scale practical deployment. The 4th annual edition of the AI City Challenge has attracted 315 participating teams across 37 countries, who leverage city-scale real traffic data and high-quality synthetic data to compete in four challenge tracks. Track 1 addressed video-based automatic vehicle counting, where the evaluation is conducted on both algorithmic effectiveness and computational efficiency. Track 2 addressed city-scale vehicle re-identification with augmented synthetic data to substantially increase the training set for the task. Track 3 addressed city-scale multi-target multi-camera vehicle tracking. Track 4 addressed traffic anomaly detection. The evaluation system shows two leader boards, in which a general leader board shows all submitted results, and a public leader board shows results limited to our contest participation rules, that teams are not allowed to use external data in their work. The general leader board shows results more close to real-world situations where annotated data are limited. Our results show promise that AI technology can enable smarter and safer transportation systems.
Blockchain-Based Lightweight and Secured V2V Communication in the Internet of Vehicles

Vehicle to vehicle (V2V) communication has gained importance in recent times because of the increasing number of traffic accidents and advancements in information sharing. A secure and reliable data transfer has become important to ensure the safety and trust of vehicular network users. Because of scalability of V2V communication, proposed solutions must have low computational complexities and free from latency issues. In this paper, we utilize the channel characteristics of wireless networks in V2V communication, which are used to generate link fingerprints. By using blockchain technology, data authentication among vehicles can be achieved in real time. The proposed algorithms are used to address the time complexity and delay issues in the Internet of Vehicles (IoV), which are lightweight and provide real time adversary detection within the network. Blockchain technology is used to generate blocks in which each hash is generated and shared with corresponding vehicles. The hash itself is not generated if an adversary affects the communication among vehicles. The Pearson Correlation Coefficient is calculated for each link and it is calculated as 0.9749 when there is no adversary and 0.1282 when an adversary is introduced into the network. The time complexity is computed as low as <i xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">O</i> (1) for the network.
Measurement-based evaluation of Google/Apple Exposure Notification API for proximity detection in a light-rail tram

We report on the results of a measurement study carried out on a commuter bus in Dublin, Ireland using the Google/Apple Exposure Notification (GAEN) API. This API is likely to be widely used by Covid-19 contact tracing apps. Measurements were collected between 60 pairs of handset locations and are publicly available. We find that the attenuation level reported by the GAEN API need not increase with distance between handsets, consistent with there being a complex radio environment inside a bus caused by the metal-rich environment. Changing the people holding a pair of handsets, with the location of the handsets otherwise remaining unchanged, can cause variations of +/-10dB in the attenuation level reported by the GAEN API. Applying the rule used by the Swiss Covid-19 contact tracing app to trigger an exposure notification to our bus measurements we find that no exposure notifications would have been triggered despite the fact that all pairs of handsets were within 2m of one another for at least 15 mins. Applying an alternative threshold-based exposure notification rule can somewhat improve performance to a detection rate of 5% when an exposure duration threshold of 15 minutes is used, increasing to 8% when the exposure duration threshold is reduced to 10 mins. Stratifying the data by distance between pairs of handsets indicates that there is only a weak dependence of detection rate on distance.
NuClick: A deep learning framework for interactive segmentation of microscopic images

Object segmentation is an important step in the workflow of computational pathology. Deep learning based models generally require large amount of labeled data for precise and reliable prediction. However, collecting labeled data is expensive because it often requires expert knowledge, particularly in medical imaging domain where labels are the result of a time-consuming analysis made by one or more human experts. As nuclei, cells and glands are fundamental objects for downstream analysis in computational pathology/cytology, in this paper we propose NuClick, a CNN-based approach to speed up collecting annotations for these objects requiring minimum interaction from the annotator. We show that for nuclei and cells in histology and cytology images, one click inside each object is enough for NuClick to yield a precise annotation. For multicellular structures such as glands, we propose a novel approach to provide the NuClick with a squiggle as a guiding signal, enabling it to segment the glandular boundaries. These supervisory signals are fed to the network as auxiliary inputs along with RGB channels. With detailed experiments, we show that NuClick is applicable to a wide range of object scales, robust against variations in the user input, adaptable to new domains, and delivers reliable annotations. An instance segmentation model trained on masks generated by NuClick achieved the first rank in LYON19 challenge. As exemplar outputs of our framework, we are releasing two datasets: 1) a dataset of lymphocyte annotations within IHC images, and 2) a dataset of segmented WBCs in blood smear images.
Deep Neural Networks for Human Activity Recognition With Wearable Sensors: Leave-One-Subject-Out Cross-Validation for Model Selection

Human Activity Recognition (HAR) has been attracting significant research attention because of the increasing availability of environmental and wearable sensors for collecting HAR data. In recent years, deep learning approaches have demonstrated a great success due to their ability to model complex systems. However, these models are often evaluated on the same subjects as those used to train the model; thus, the provided accuracy estimates do not pertain to new subjects. Occasionally, one or a few subjects are selected for the evaluation, but such estimates highly depend on the subjects selected for the evaluation. Consequently, this paper examines how well different machine learning architectures make generalizations based on a new subject(s) by using Leave-One-Subject-Out Cross-Validation (LOSOCV). Changing the subject used for the evaluation in each fold of the cross-validation, LOSOCV provides subject-independent estimate of the performance for new subjects. Six feed forward and convolutional neural network (CNN) architectures as well as four pre-processing scenarios have been considered. Results show that CNN architecture with two convolutions and one-dimensional filter accompanied by a sliding window and vector magnitude, generalizes better than other architectures. For the same CNN, the accuracy improves from 85.1% when evaluated with LOSOCV to 99.85% when evaluated with the traditional 10-fold cross-validation, demonstrating the importance of using LOSOCV for the evaluation.
Revisiting Feature Fusion for RGB-T Salient Object Detection

While many RGB-based saliency detection algorithms have recently shown the capability of segmenting salient objects from an image, they still suffer from unsatisfactory performance when dealing with complex scenarios, insufficient illumination or occluded appearances. To overcome this problem, this article studies RGB-T saliency detection, where we take advantage of thermal modality's robustness against illumination and occlusion. To achieve this goal, we revisit feature fusion for mining intrinsic RGB-T saliency patterns and propose a novel deep feature fusion network, which consists of the multi-scale, multi-modality, and multi-level feature fusion modules. Specifically, the multi-scale feature fusion module captures rich contexture features from each modality feature, while the multi-modality and multi-level feature fusion modules integrate complementary features from different modality features and different level of features, respectively. To demonstrate the effectiveness of the proposed approach, we conduct comprehensive experiments on the RGB-T saliency detection benchmark. The experimental results demonstrate that our approach outperforms other state-of-the-art methods and the conventional feature fusion modules by a large margin.
Deep transfer with minority data augmentation for imbalanced breast cancer dataset

Clinical diagnosis of breast cancer is a challenging problem in the biomedical domain. The BreakHis breast cancer histopathological image dataset consists of two classes: Benign (Minority class) and Malignant (Majority class). The imbalanced class distribution results in the degradation of performance of the classifier model due to biased classification towards the majority class. To tackle this problem, a novel learning strategy that involves a deep transfer network has been proposed in this paper, in collaboration with Deep Convolution Generative Adversarial network (DCGAN). DCGAN is used in the initial phase for data augmentation of the minority class only. The dataset, with the class distribution now balanced, is applied as input to the deep transfer network. The proposed deep transfer architecture has at its core, the initial pre-trained layers (until block 4 pool layer) of the VGG16 deep network architecture pre-trained on the ImageNet object classification dataset. The higher end of our transfer network comprises of Batch Normalization, 2D Convolutional (CONV2D) layer, Global Average Pooling 2D, Dropout and Dense layers that are added to enhance the network’s performance. Experiments on the benchmark BreakHis dataset for different magnification factors: 40X, 100X, 200X and 400X validate the efficiency of the proposed deep transfer learning approach due to the high scores achieved as compared to the state-of-the-art deep networks.
Breast Cancer Diagnosis in Histopathological Images Using ResNet-50 Convolutional Neural Network

Breast cancer disease is the second most common world cause of cancer death in women. However, the early diagnostics and detection can provide a significant chance for correct treatment and survival. In this work, we propose an accurate and inclusive computational breast cancer diagnosis framework using ResNet-50 convolutional neural network to classify histopathological microscopy images. The proposed model employs transfer learning technique of the powerful ResNet-50 CNN pretrained on ImageNet to train and classify BreakHis dataset into benign or malignant. The simulation results showed that our proposed model achieves exceptional classification accuracy of 99% outperforming other compared models trained on the same dataset.
Nonlinear Transform Coding

We review a class of methods that can be collected under the name nonlinear transform coding (NTC), which over the past few years have become competitive with the best linear transform codecs for images, and have superseded them in terms of rate-distortion performance under established perceptual quality metrics such as MS-SSIM. We assess the empirical rate-distortion performance of NTC with the help of simple example sources, for which the optimal performance of a vector quantizer is easier to estimate than with natural data sources. To this end, we introduce a novel variant of entropy-constrained vector quantization. We provide an analysis of various forms of stochastic optimization techniques for NTC models; review architectures of transforms based on artificial neural networks, as well as learned entropy models; and provide a direct comparison of a number of methods to parameterize the rate-distortion trade-off of nonlinear transforms, introducing a simplified one.
DRG: Dual Relation Graph for Human-Object Interaction Detection

We tackle the challenging problem of human-object interaction (HOI) detection. Existing methods either recognize the interaction of each human-object pair in isolation or perform joint inference based on complex appearance-based features. In this paper, we leverage an abstract spatial-semantic representation to describe each human-object pair and aggregate the contextual information of the scene via a dual relation graph (one human-centric and one object-centric). Our proposed dual relation graph effectively captures discriminative cues from the scene to resolve ambiguity from local predictions. Our model is conceptually simple and leads to favorable results compared to the state-of-the-art HOI detection algorithms on two large-scale benchmark datasets.
A Systematic Methodology for Characterizing Scalability of DNN Accelerators using SCALE-Sim

The compute demand for deep learning workloads is well known and is a prime motivator for powerful parallel computing platforms such as GPUs or dedicated hardware accelerators. The massive inherent parallelism of these workloads enables us to extract more performance by simply provisioning more compute hardware for a given task. This strategy can be directly exploited to build higher-performing hardware for DNN workloads, by incorporating as many parallel compute units as possible in a single system. This strategy is referred to as scaling up. Alternatively, it's feasible to arrange multiple hardware systems to work on a single problem, and in some cases, a cheaper alternative to exploit the given parallelism, or in other words, scaling out. As DNN based solutions become increasingly prevalent, so does the demand for computation, making the scaling choice (scale-up vs scale-out) critical. To study this design-space, this work makes two major contributions. (i) We describe a cycle-accurate simulator called SCALE-SIM for DNN inference on systolic arrays, which we use to model both scale-up and scale-out systems, modeling on-chip memory access, runtime, and DRAM bandwidth requirements for a given workload. (ii) We also present an analytical model to estimate the optimal scale-up vs scale-out ratio given hardware constraints (e.g, TOPS and DRAM bandwidth) for a given workload. We observe that a judicious choice of scaling can lead to performance improvements as high as 50 per layer, within the available DRAM bandwidth. This work demonstrates and analyzes the trade-off space for performance, DRAM bandwidth, and energy, and identifies sweet spots for various workloads and hardware configurations.
ConveRT: Efficient and Accurate Conversational Representations from Transformers

General-purpose pretrained sentence encoders such as BERT are not ideal for real-world conversational AI applications; they are computationally heavy, slow, and expensive to train.We propose ConveRT (Conversational Representations from Transformers), a pretraining framework for conversational tasks satisfying all the following requirements: it is effective, affordable, and quick to train.We pretrain using a retrieval-based response selection task, effectively leveraging quantization and subword-level parameterization in the dual encoder to build a lightweight memoryand energy-efficient model.We show that Con-veRT achieves state-of-the-art performance across widely established response selection tasks.We also demonstrate that the use of extended dialog history as context yields further performance gains.Finally, we show that pretrained representations from the proposed encoder can be transferred to the intent classification task, yielding strong results across three diverse data sets.ConveRT trains substantially faster than standard sentence encoders or previous state-of-the-art dual encoders.With its reduced size and superior performance, we believe this model promises wider portability and scalability for Conversational AI applications.