Reviewerly/exHarmony huggingface dataset:
Artificial Intelligence Medical Ultrasound Equipment: Application of Breast Lesions Detection

Breast cancer ranks first among cancers affecting women’s health. Our work aims to realize the intelligence of the medical ultrasound equipment with limited computational capability, which is used for the assistant detection of breast lesions. We embed the high-computational deep learning algorithm into the medical ultrasound equipment with limited computational capability by two techniques: (1) lightweight neural network: considering the limited computational capability of ultrasound equipment, a lightweight neural network is designed, which greatly reduces the amount of calculation. And we use the technique of knowledge distillation to train the low-precision network helped with the high-precision network; (2) asynchronous calculations: consider four frames of ultrasound images as a group; the image of the first frame of each group is used as the input of the network, and the result is respectively fused with the images of the fourth to seventh frames. An amount of computation of 30 GFLO/frame is required for the proposed lightweight neural network, about 1/6 of that of the large high-precision network. After trained from scratch using the knowledge distillation technique, the detection performance of the lightweight neural network (sensitivity = 89.25%, specificity = 96.33%, the average precision [AP] = 0.85) is close to that of the high-precision network (sensitivity = 98.3%, specificity = 88.33%, AP = 0.91). By asynchronous calculation, we achieve real-time automatic detection of 24 fps (frames per second) on the ultrasound equipment. Our work proposes a method to realize the intelligence of the low-computation-power ultrasonic equipment, and successfully achieves the real-time assistant detection of breast lesions. The significance of the study is as follows: (1) The proposed method is of practical significance in assisting doctors to detect breast lesions; (2) our method provides some practical and theoretical support for the development and engineering of intelligent equipment based on artificial intelligence algorithms.
A history of Clojure

Clojure was designed to be a general-purpose, practical functional language, suitable for use by professionals wherever its host language, e.g., Java, would be. Initially designed in 2005 and released in 2007, Clojure is a dialect of Lisp, but is not a direct descendant of any prior Lisp. It complements programming with pure functions of immutable data with concurrency-safe state management constructs that support writing correct multithreaded programs without the complexity of mutex locks. Clojure is intentionally hosted, in that it compiles to and runs on the runtime of another language, such as the JVM. This is more than an implementation strategy; numerous features ensure that programs written in Clojure can leverage and interoperate with the libraries of the host language directly and efficiently. In spite of combining two (at the time) rather unpopular ideas, functional programming and Lisp, Clojure has since seen adoption in industries as diverse as finance, climate science, retail, databases, analytics, publishing, healthcare, advertising and genomics, and by consultancies and startups worldwide, much to the career-altering surprise of its author. Most of the ideas in Clojure were not novel, but their combination puts Clojure in a unique spot in language design (functional, hosted, Lisp). This paper recounts the motivation behind the initial development of Clojure and the rationale for various design decisions and language constructs. It then covers its evolution subsequent to release and adoption.
Optimizing Reconfigurable Recurrent Neural Networks

This paper proposes a novel latency-hiding hardware architecture based on column-wise matrix-vector multiplication to eliminate data dependency, improving the throughput of systems of RNN models. In addition, a flexible checkerboard tiling strategy is introduced to allow large weight matrices, while supporting element-based parallelism and vector-based parallelism. These optimizations improve the exploitation of the available parallelism to increase run-time hardware utilization and boost inference throughput. Furthermore, a quantization scheme with fine-tuning is proposed to achieve high accuracy. Evaluation results show that the proposed architecture can enhance performance and energy efficiency with little accuracy loss. It achieves 1.05 to 3.35 times better performance and 1.22 to 3.92 times better hardware utilization than a state-of-theart FPGA-based LSTM design, which shows that our approach contributes to high performance FPGA-based LSTM systems.
AI City Challenge 2020 – Computer Vision for Smart Transportation Applications

We present methods developed in our participation of the AI City 2020 Challenge (AIC20) and report evaluation results in this contest. With the blooming of AI computer vision techniques, vehicle detection, tracking, identification, and counting all have advanced significantly. However, whether these technologies are ready for real-world smart transportation usage is still a open question. The goal of this work is to apply and integrate state-of-the-art techniques for solving the challenge problems under a standardized setup and evaluation. We participated all 4 AIC20 challenge tracks (T1 to T4). In T1 challenge, we perform vehicle counting by associating deep features extracted from Mask-RCNN detections and tracklets, followed by vehicle movement zone matching. In T2 challenge, we perform vehicle type and color classification and then rank matching vehicles using a PGAM re-id network. In T3 challenge, we proposed a new Multi-Camera Tracking Network (MTCN) that takes single-camera vehicle tracking as input, and performs multi-camera tracklet fusion and linking, by jointly optimizing the matching of vehicle appearance and physical features. In T4 challenge, we adopt a leading method based on perspective detection and spatial-temporal matrix discriminating, and improve it with background modeling for traffic anomaly detection. We achieved top-6 and top-4 performance for T3 and T4 challenges respectively in the AIC20 general leaderboard.
Non-Local Neural Networks With Grouped Bilinear Attentional Transforms

Modeling spatial or temporal long-range dependency plays a key role in deep neural networks. Conventional dominant solutions include recurrent operations on sequential data or deeply stacking convolutional layers with small kernel size. Recently, a number of non-local operators (such as self-attention based) have been devised. They are typically generic and can be plugged into many existing network pipelines for globally computing among any two neurons in a feature map. This work proposes a novel non-local operator. It is inspired by the attention mechanism of human visual system, which can quickly attend to important local parts in sight and suppress other less-relevant information. The core of our method is learnable and data-adaptive bilinear attentional transform (BA-Transform), whose merits are three-folds: first, BA-Transform is versatile to model a wide spectrum of local or global attentional operations, such as emphasizing specific local regions. Each BA-Transform is learned in a data-adaptive way; Secondly, to address the discrepancy among features, we further design grouped BA-Transforms, which essentially apply different attentional operations to different groups of feature channels; Thirdly, many existing non-local operators are computation-intensive. The proposed BA-Transform is implemented by simple matrix multiplication and admits better efficacy. For empirical evaluation, we perform comprehensive experiments on two large-scale benchmarks, ImageNet and Kinetics, for image / video classification respectively. The achieved accuracies and various ablation experiments consistently demonstrate significant improvement by large margins.
Camera On-Boarding for Person Re-Identification Using Hypothesis Transfer Learning

Most of the existing approaches for person re-identification consider a static setting where the number of cameras in the network is fixed. An interesting direction, which has received little attention, is to explore the dynamic nature of a camera network, where one tries to adapt the existing re-identification models after on-boarding new cameras, with little additional effort. There have been a few recent methods proposed in person re-identification that attempt to address this problem by assuming the labeled data in the existing network is still available while adding new cameras. This is a strong assumption since there may exist some privacy issues for which one may not have access to those data. Rather, based on the fact that it is easy to store the learned re-identifications models, which mitigates any data privacy concern, we develop an efficient model adaptation approach using hypothesis transfer learning that aims to transfer the knowledge using only source models and limited labeled data, but without using any source camera data from the existing network. Our approach minimizes the effect of negative transfer by finding an optimal weighted combination of multiple source models for transferring the knowledge. Extensive experiments on four challenging benchmark datasets with variable number of cameras well demonstrate the efficacy of our proposed approach over state-of-the-art methods.
SliderGAN: Synthesizing Expressive Face Images by Sliding 3D Blendshape Parameters

Abstract Image-to-image (i2i) translation is the dense regression problem of learning how to transform an input image into an output using aligned image pairs. Remarkable progress has been made in i2i translation with the advent of deep convolutional neural networks and particular using the learning paradigm of generative adversarial networks (GANs). In the absence of paired images, i2i translation is tackled with one or multiple domain transformations (i.e., CycleGAN, StarGAN etc.). In this paper, we study the problem of image-to-image translation, under a set of continuous parameters that correspond to a model describing a physical process. In particular, we propose the SliderGAN which transforms an input face image into a new one according to the continuous values of a statistical blendshape model of facial motion. We show that it is possible to edit a facial image according to expression and speech blendshapes, using sliders that control the continuous values of the blendshape model. This provides much more flexibility in various tasks, including but not limited to face editing, expression transfer and face neutralisation, comparing to models based on discrete expressions or action units.
Efficient Image Enhancement Model for Correcting Uneven Illumination Images

Images captured under varying light conditions have deficient contrast, low brightness, latent colors, and high noise.Numerous methods have been developed for image enhancement.However, these methods are only suitable for enhancing specific type of images (e.g., over-exposed or underexposed), and also fail to restore artifact-free results for various other types of images.Therefore, to meet this goal, in this paper, we present an automatic image enhancement method, which is capable of producing quality results for all types of images captured under uneven exposure conditions (e.g., backlit, non-uniform, over-exposed, one-sided illumination and night-time images).Firstly, images are categorized using a convolutional neural network (CNN) to determine their class, and different values of weight coefficients are achieved for further processing.Then, images are converted into photonegative form to obtain an initial transmission map using a bright channel prior.Next, L1-norm regularization is adopted to refine scene transmission.Besides, environmental light is estimated based on an effective filter.Finally, the image degradation model is applied to achieve enhanced results.Furthermore, post-processing of the images is comprised of two steps, such as denoising and details enhancement.The denoised model is only applied when the images are captured in extreme low-light conditions.Whereas, a smooth layer is obtained using L1-norm regularization to enhance details in partially over-and under-exposed images.Extensive experiments reveal the effectives of the proposed approach as compared to other state-of-the-art algorithms.
Advisable Learning for Self-Driving Vehicles by Internalizing Observation-to-Action Rules

Humans learn to drive through both practice and theory, e.g. by studying the rules, while most self-driving systems are limited to the former. Being able to incorporate human knowledge of typical causal driving behaviour should benefit autonomous systems. We propose a new approach that learns vehicle control with the help of human advice. Specifically, our system learns to summarize its visual observations in natural language, predict an appropriate action response (e.g. "I see a pedestrian crossing, so I stop"), and predict the controls, accordingly. Moreover, to enhance interpretability of our system, we introduce a fine-grained attention mechanism which relies on semantic segmentation and object-centric RoI pooling. We show that our approach of training the autonomous system with human advice, grounded in a rich semantic representation, matches or outperforms prior work in terms of control prediction and explanation generation. Our approach also results in more interpretable visual explanations by visualizing object-centric attention maps. Code is available at https://github.com/JinkyuKimUCB/advisable-driving.
Blind Stereoscopic Image Quality Assessment By Deep Neural Network Of Multi-Level Feature Fusion

In this paper, we propose an effective blind image quality assessment (BIQA) method for stereoscopic images by deep neural network (DNN) of multi-level feature fusion (MLFF) inspired by the multi-scale characteristics and binocular properties of the human visual system (HVS). Specifically, we firstly feed the left- and right-view images into a weight sharing convolutional neural network (CNN) for jointly feature extraction. To aggregate multi-level features, we concatenate the low-, middle-, and high-level feature maps of stereoscopic images to simulate the complicated visual interaction processing in the HVS. Two fully connected layers are used to build the nonlinear mapping from the highly abstract features to the quality scores of stereoscopic images. The experiments conducted on two public databases prove the validity of the proposed MLFF method.
MetaFuse: A Pre-trained Fusion Model for Human Pose Estimation

Cross view feature fusion is the key to address the occlusion problem in human pose estimation. The current fusion methods need to train a separate model for every pair of cameras making them difficult to scale. In this work, we introduce MetaFuse, a pre-trained fusion model learned from a large number of cameras in the Panoptic dataset. The model can be efficiently adapted or finetuned for a new pair of cameras using a small number of labeled images. The strong adaptation power of MetaFuse is due in large part to the proposed factorization of the original fusion model into two parts-(1) a generic fusion model shared by all cameras, and (2) lightweight camera-dependent transformations. Furthermore, the generic model is learned from many cameras by a meta-learning style algorithm to maximize its adaptation capability to various camera poses. We observe in experiments that MetaFuse finetuned on the public datasets outperforms the state-of-the-arts by a large margin which validates its value in practice.
Regularizing Discriminative Capability of CGANs for Semi-Supervised Generative Learning

Semi-supervised generative learning aims to learn the underlying class-conditional distribution of partially labeled data. Generative Adversarial Networks (GANs) have led to promising progress in this task. However, it still needs to further explore the issue of imbalance between real labeled data and fake data in the adversarial learning process. To address this issue, we propose a regularization technique based on Random Regional Replacement (R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> -regularization) to facilitate the generative learning process. Specifically, we construct two types of between-class instances: cross-category ones and real-fake ones. These instances could be closer to the decision boundaries and are important for regularizing the classification and discriminative networks in our class-conditional GANs, which we refer to as R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> -CGAN. Better guidance from these two networks makes the generative network produce instances with class-specific information and high fidelity. We experiment with multiple standard benchmarks, and demonstrate that the R <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> -regularization can lead to significant improvement in both classification and class-conditional image synthesis.
Level set formulation for automatic medical image segmentation based on fuzzy clustering

The level set method is widely used in medical image segmentation, in which the performance is seriously subject to the initialization and parameters configuration. An automatic segmentation method was proposed in this paper, which integrates fuzzy clustering with level set method through a dynamic constrained term in the new energy functional. It is able to use the results of fuzzy clustering directly, which can control the level set evolution. Moreover, the added constrained term is changing continuously until getting the final results. Such algorithm eliminates the manual operation a lot and leads to more robust segmentation results. With the split Bregman method, the minimization of the new energy functional is fast. The proposed algorithm was tested on some medical images and also compared with other level set models and the state-of-the-art method such as U-Net. The quantitative and qualitative experimental results show its effectiveness and obvious improvement for medical image segmentation.
One Man’s Trash Is Another Man’s Treasure: Resisting Adversarial Examples by Adversarial Examples

Modern image classification systems are often built on deep neural networks, which suffer from adversarial examples--images with deliberately crafted, imperceptible noise to mislead the network's classification. To defend against adversarial examples, a plausible idea is to obfuscate the network's gradient with respect to the input image. This general idea has inspired a long line of defense methods. Yet, almost all of them have proven vulnerable. We revisit this seemingly flawed idea from a radically different perspective. We embrace the omnipresence of adversarial examples and the numerical procedure of crafting them, and turn this harmful attacking process into a useful defense mechanism. Our defense method is conceptually simple: before feeding an input image for classification, transform it by finding an adversarial example on a pre-trained external model. We evaluate our method against a wide range of possible attacks. On both CIFAR-10 and Tiny ImageNet datasets, our method is significantly more robust than state-of-the-art methods. Particularly, in comparison to adversarial training, our method offers lower training cost as well as stronger robustness.
Ocean: A Dual Learning Approach For Generalized Zero-Shot Sketch-Based Image Retrieval

Sketch-Based Image Retrieval (SBIR) is an emerging research area with many real-world applications. Recent studies have approached this research task under the more challenging zero-shot learning setting (ZS-SBIR), which assume classes in the target domain are unseen during the training stage. Many of the existing ZS-SBIR studies transferred the learned cross-modal (i.e., sketch and image) representations from the source domain to the target domain by leveraging side information in semantic embeddings. However, these ZS-SBIR methods are not able to generalize well to a more realistic setting to retrieve images from seen and unseen classes. To address the limitation of existing methods, we propose the cOmmon Conditional Encoder Adversarial Network (OCEAN) to perform generalized zero-shot sketch-based image retrieval (GZS-SBIR). The OCEAN model utilizes a dual learning framework to cyclically map the sketch and image features to a common semantic space, and project semantic features back to the relevant visual space by adversarial training. We conduct experiments on two publicly available datasets and demonstrate that our proposed model outperformed the state-of-the-arts baselines in both ZS-SBIR and GZS-SBIR tasks.
IBDA: Improved Binary Dragonfly Algorithm With Evolutionary Population Dynamics and Adaptive Crossover for Feature Selection

Feature selection is an effective method to eliminate irrelevant, redundant and noisy features, which improves the performance of classification and reduces the computational burden in machine learning.In this paper, an improved binary dragonfly algorithm (IBDA) which extends from the conventional dragonfly algorithm (DA) is proposed as a search strategy to design a wrapper-based feature selection method.First, a novel evolutionary population dynamics (EPD) strategy is introduced in IBDA to enhance the exploitation ability while ensuring population diversity of the algorithm.Second, IBDA proposes a novel crossover operator which establishes connections between the crossover rates and iterations so that making the algorithm can adjust the crossover rates of solutions dynamically, thereby balancing the exploitation and exploration of the algorithm.Finally, a binary mechanism is proposed to make the algorithm suitable for the binary feature selection problems.Simulations are conducted on 27 classical datasets from the UC Irvine Machine Learning Repository, and the results demonstrate that the proposed IBDA has better performance than some other comparison algorithms.Moreover, the effectiveness and performance of the proposed improved factors are evaluated by tests.
Algorithms for heavy-tailed statistics: regression, covariance estimation, and beyond

We study polynomial-time algorithms for linear regression and covariance estimation in the absence of strong (Gaussian) assumptions on the underlying distributions of samples, making assumptions instead about only finitely-many moments. We focus on how many samples are required to perform estimation and regression with high accuracy and exponentially-good success probability in the face of heavy-tailed data.
Video Semantic Segmentation With Distortion-Aware Feature Correction

Video semantic segmentation is active in recent years benefited from the great progress of image semantic segmentation. For such a task, the per-frame image segmentation is generally unacceptable in practice due to high computation cost. To tackle this issue, many works use the flow-based feature propagation to reuse the features of previous frames. However, the optical flow estimation inevitably suffers inaccuracy and then causes the propagated features distorted. In this paper, we propose distortion-aware feature correction to alleviate the issue, which improves video segmentation performance by correcting distorted propagated features. To be specific, we firstly propose to transfer distortion patterns from feature into image space and conduct effective distortion map prediction. Benefited from the guidance of distortion maps, we proposed Feature Correction Module (FCM) to rectify propagated features in the distorted areas. Our proposed method can significantly boost the accuracy of video semantic segmentation at a low price. The extensive experimental results on Cityscapes and CamVid show that our method outperforms the recent state-of-the-art methods.