pwc-archive/papers-with-abstracts huggingface dataset:
Interpretable discovery of new semiconductors with machine learning
Authors: Hitarth Choubisa, Petar Todorović, Joao M. Pina, Darshan H. Parmar, Ziliang Li, Oleksandr Voznyy, Isaac Tamblyn, Edward Sargent
Date: 2021-01-12 00:00:00
Tasks: BIG-bench Machine Learning, Knowledge Distillation

Machine learning models of materials$^{1-5}$ accelerate discovery compared to ab initio methods: deep learning models now reproduce density functional theory (DFT)-calculated results at one hundred thousandths of the cost of DFT$^{6}$. To provide guidance in experimental materials synthesis, these need to be coupled with an accurate yet effective search algorithm and training data consistent with experimental observations. Here we report an evolutionary algorithm powered search which uses machine-learned surrogate models trained on high-throughput hybrid functional DFT data benchmarked against experimental bandgaps: Deep Adaptive Regressive Weighted Intelligent Network (DARWIN). The strategy enables efficient search over the materials space of ~10$^8$ ternaries and 10$^{11}$ quaternaries$^{7}$ for candidates with target properties. It provides interpretable design rules, such as our finding that the difference in the electronegativity between the halide and B-site cation being a strong predictor of ternary structural stability. As an example, when we seek UV emission, DARWIN predicts K$_2$CuX$_3$ (X = Cl, Br) as a promising materials family, based on its electronegativity difference. We synthesized and found these materials to be stable, direct bandgap UV emitters. The approach also allows knowledge distillation for use by humans.
Rethinking Interactive Image Segmentation: Feature Space Annotation
Authors: Jord{ã}o Bragantini, Alexandre X Falc{ã}o, Laurent Najman
Date: 2021-01-12 00:00:00
Tasks: Foreground Segmentation, Image Segmentation, Interactive Segmentation, Metric Learning, Segmentation, Semantic Segmentation

Despite the progress of interactive image segmentation methods, high-quality pixel-level annotation is still time-consuming and laborious - a bottleneck for several deep learning applications. We take a step back to propose interactive and simultaneous segment annotation from multiple images guided by feature space projection. This strategy is in stark contrast to existing interactive segmentation methodologies, which perform annotation in the image domain. We show that feature space annotation achieves competitive results with state-of-the-art methods in foreground segmentation datasets: iCoSeg, DAVIS, and Rooftop. Moreover, in the semantic segmentation context, it achieves 91.5% accuracy in the Cityscapes dataset, being 74.75 times faster than the original annotation procedure. Further, our contribution sheds light on a novel direction for interactive image annotation that can be integrated with existing methodologies. The supplementary material presents video demonstrations. Code available at https://github.com/LIDS-UNICAMP/rethinking-interactive-image-segmentation.
CAnet: Uplink-aided Downlink Channel Acquisition in FDD Massive MIMO using Deep Learning
Authors: Jiajia Guo, Chao-Kai Wen, Shi Jin
Date: 2021-01-12 00:00:00

In frequency-division duplexing systems, the downlink channel state information (CSI) acquisition scheme leads to high training and feedback overheads. In this paper, we propose an uplink-aided downlink channel acquisition framework using deep learning to reduce these overheads. Unlike most existing works that focus only on channel estimation or feedback modules, to the best of our knowledge, this is the first study that considers the entire downlink CSI acquisition process, including downlink pilot design, channel estimation, and feedback. First, we propose an adaptive pilot design module by exploiting the correlation in magnitude among bidirectional channels in the angular domain to improve channel estimation. Next, to avoid the bit allocation problem during the feedback module, we concatenate the complex channel and embed the uplink channel magnitude to the channel reconstruction at the base station. Lastly, we combine the above two modules and compare two popular downlink channel acquisition frameworks. The former framework estimates and feeds back the channel at the user equipment subsequently. The user equipment in the latter one directly feeds back the received pilot signals to the base station. Our results reveal that, with the help of uplink, directly feeding back the pilot signals can save approximately 20% of feedback bits, which provides a guideline for future research.
Data-driven peakon and periodic peakon travelling wave solutions of some nonlinear dispersive equations via deep learning
Authors: Li Wang, Zhenya Yan
Date: 2021-01-12 00:00:00
Tasks: Experimental Design

In the field of mathematical physics, there exist many physically interesting nonlinear dispersive equations with peakon solutions, which are solitary waves with discontinuous first-order derivative at the wave peak. In this paper, we apply the multi-layer physics-informed neural networks (PINNs) deep learning to successfully study the data-driven peakon and periodic peakon solutions of some well-known nonlinear dispersion equations with initial-boundary value conditions such as the Camassa-Holm (CH) equation, Degasperis-Procesi equation, modified CH equation with cubic nonlinearity, Novikov equation with cubic nonlinearity, mCH-Novikov equation, b-family equation with quartic nonlinearity, generalized modified CH equation with quintic nonlinearity, and etc. These results will be useful to further study the peakon solutions and corresponding experimental design of nonlinear dispersive equations.
Event-Driven Source Traffic Prediction in Machine-Type Communications Using LSTM Networks
Authors: Thulitha Senevirathna, Bathiya Thennakoon, Tharindu Sankalpa, Chatura Seneviratne, Samad Ali, Nandana Rajatheva
Date: 2021-01-12 00:00:00
Tasks: Prediction, Traffic Prediction

Source traffic prediction is one of the main challenges of enabling predictive resource allocation in machine type communications (MTC). In this paper, a Long Short-Term Memory (LSTM) based deep learning approach is proposed for event-driven source traffic prediction. The source traffic prediction problem can be formulated as a sequence generation task where the main focus is predicting the transmission states of machine-type devices (MTDs) based on their past transmission data. This is done by restructuring the transmission data in a way that the LSTM network can identify the causal relationship between the devices. Knowledge of such a causal relationship can enable event-driven traffic prediction. The performance of the proposed approach is studied using data regarding events from MTDs with different ranges of entropy. Our model outperforms existing baseline solutions in saving resources and accuracy with a margin of around 9%. Reduction in Random Access (RA) requests by our model is also analyzed to demonstrate the low amount of signaling required as a result of our proposed LSTM based source traffic prediction approach.
On the Calibration and Uncertainty of Neural Learning to Rank Models
Authors: Gustavo Penha, Claudia Hauff
Date: 2021-01-12 00:00:00
Tasks: Document Ranking, Learning-To-Rank, Retrieval

According to the Probability Ranking Principle (PRP), ranking documents in decreasing order of their probability of relevance leads to an optimal document ranking for ad-hoc retrieval. The PRP holds when two conditions are met: [C1] the models are well calibrated, and, [C2] the probabilities of relevance are reported with certainty. We know however that deep neural networks (DNNs) are often not well calibrated and have several sources of uncertainty, and thus [C1] and [C2] might not be satisfied by neural rankers. Given the success of neural Learning to Rank (L2R) approaches-and here, especially BERT-based approaches-we first analyze under which circumstances deterministic, i.e. outputs point estimates, neural rankers are calibrated. Then, motivated by our findings we use two techniques to model the uncertainty of neural rankers leading to the proposed stochastic rankers, which output a predictive distribution of relevance as opposed to point estimates. Our experimental results on the ad-hoc retrieval task of conversation response ranking reveal that (i) BERT-based rankers are not robustly calibrated and that stochastic BERT-based rankers yield better calibration; and (ii) uncertainty estimation is beneficial for both risk-aware neural ranking, i.e.taking into account the uncertainty when ranking documents, and for predicting unanswerable conversational contexts.
Neural Contract Element Extraction Revisited: Letters from Sesame Street
Authors: Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, Ion Androutsopoulos
Date: 2021-01-12 00:00:00
Tasks: POS, TAG

We investigate contract element extraction. We show that LSTM-based encoders perform better than dilated CNNs, Transformers, and BERT in this task. We also find that domain-specific WORD2VEC embeddings outperform generic pre-trained GLOVE embeddings. Morpho-syntactic features in the form of POS tag and token shape embeddings, as well as context-aware ELMO embeddings do not improve performance. Several of these observations contradict choices or findings of previous work on contract element extraction and generic sequence labeling tasks, indicating that contract element extraction requires careful task-specific choices. Analyzing the results of (i) plain TRANSFORMER-based and (ii) BERT-based models, we find that in the examined task, where the entities are highly context-sensitive, the lack of recurrency in TRANSFORMERs greatly affects their performance.
Activation Density based Mixed-Precision Quantization for Energy Efficient Neural Networks
Authors: Karina Vasquez, Yeshwanth Venkatesha, Abhiroop Bhattacharjee, Abhishek Moitra, Priyadarshini Panda
Date: 2021-01-12 00:00:00
Tasks: Model Compression, Quantization

As neural networks gain widespread adoption in embedded devices, there is a need for model compression techniques to facilitate deployment in resource-constrained environments. Quantization is one of the go-to methods yielding state-of-the-art model compression. Most approaches take a fully trained model, apply different heuristics to determine the optimal bit-precision for different layers of the network, and retrain the network to regain any drop in accuracy. Based on Activation Density (AD)-the proportion of non-zero activations in a layer-we propose an in-training quantization method. Our method calculates bit-width for each layer during training yielding a mixed precision model with competitive accuracy. Since we train lower precision models during training, our approach yields the final quantized model at lower training complexity and also eliminates the need for re-training. We run experiments on benchmark datasets like CIFAR-10, CIFAR-100, TinyImagenet on VGG19/ResNet18 architectures and report the accuracy and energy estimates for the same. We achieve ~4.5x benefit in terms of estimated multiply-and-accumulate (MAC) reduction while reducing the training complexity by 50% in our experiments. To further evaluate the energy benefits of our proposed method, we develop a mixed-precision scalable Process In Memory (PIM) hardware accelerator platform. The hardware platform incorporates shift-add functionality for handling multi-bit precision neural network models. Evaluating the quantized models obtained with our proposed method on the PIM platform yields ~5x energy reduction compared to 16-bit models. Additionally, we find that integrating AD based quantization with AD based pruning (both conducted during training) yields up to ~198x and ~44x energy reductions for VGG19 and ResNet18 architectures respectively on PIM platform compared to baseline 16-bit precision, unpruned models.
Automated Detection of Patellofemoral Osteoarthritis from Knee Lateral View Radiographs Using Deep Learning: Data from the Multicenter Osteoarthritis Study (MOST)
Authors: Neslihan Bayramoglu, Miika T. Nieminen, Simo Saarakkala
Date: 2021-01-12 00:00:00
Tasks: Deep Learning, object-detection, Object Detection

Objective: To assess the ability of imaging-based deep learning to predict radiographic patellofemoral osteoarthritis (PFOA) from knee lateral view radiographs. Design: Knee lateral view radiographs were extracted from The Multicenter Osteoarthritis Study (MOST) (n = 18,436 knees). Patellar region-of-interest (ROI) was first automatically detected, and subsequently, end-to-end deep convolutional neural networks (CNNs) were trained and validated to detect the status of patellofemoral OA. Patellar ROI was detected using deep-learning-based object detection method. Manual PFOA status assessment provided in the MOST dataset was used as a classification outcome for the CNNs. Performance of prediction models was assessed using the area under the receiver operating characteristic curve (ROC AUC) and the average precision (AP) obtained from the precision-recall (PR) curve in the stratified 5-fold cross validation setting. Results: Of the 18,436 knees, 3,425 (19%) had PFOA. AUC and AP for the reference model including age, sex, body mass index (BMI), the total Western Ontario and McMaster Universities Arthritis Index (WOMAC) score, and tibiofemoral Kellgren-Lawrence (KL) grade to predict PFOA were 0.806 and 0.478, respectively. The CNN model that used only image data significantly improved the prediction of PFOA status (ROC AUC= 0.958, AP= 0.862). Conclusion: We present the first machine learning based automatic PFOA detection method. Furthermore, our deep learning based model trained on patella region from knee lateral view radiographs performs better at predicting PFOA than models based on patient characteristics and clinical assessments.
Phase Retrieval using Expectation Consistent Signal Recovery Algorithm based on Hypernetwork
Authors: Chang-Jen Wang, Chao-Kai Wen, Shang-Ho, Tsai, Shi Jin, Geoffrey Ye Li
Date: 2021-01-12 00:00:00
Tasks: Retrieval

Phase retrieval (PR) is an important component in modern computational imaging systems. Many algorithms have been developed over the past half-century. Recent advances in deep learning have introduced new possibilities for a robust and fast PR. An emerging technique called deep unfolding provides a systematic connection between conventional model-based iterative algorithms and modern data-based deep learning. Unfolded algorithms, which are powered by data learning, have shown remarkable performance and convergence speed improvement over original algorithms. Despite their potential, most existing unfolded algorithms are strictly confined to a fixed number of iterations when layer-dependent parameters are used. In this study, we develop a novel framework for deep unfolding to overcome existing limitations. Our development is based on an unfolded generalized expectation consistent signal recovery (GEC-SR) algorithm, wherein damping factors are left for data-driven learning. In particular, we introduce a hypernetwork to generate the damping factors for GEC-SR. Instead of learning a set of optimal damping factors directly, the hypernetwork learns how to generate the optimal damping factors according to the clinical settings, thereby ensuring its adaptivity to different scenarios. To enable the hypernetwork to adapt to varying layer numbers, we use a recurrent architecture to develop a dynamic hypernetwork that generates a damping factor that can vary online across layers. We also exploit a self-attention mechanism to enhance the robustness of the hypernetwork. Extensive experiments show that the proposed algorithm outperforms existing ones in terms of convergence speed and accuracy and still works well under very harsh settings, even under which many classical PR algorithms are unstable.
Proceedings of the NeurIPS 2020 Workshop on Machine Learning for the Developing World: Improving Resilience
Authors: Tejumade Afonja, Konstantin Klemmer, Aya Salama, Paula Rodriguez Diaz, Niveditha Kalavakonda, Oluwafemi Azeez
Date: 2021-01-12 00:00:00
Tasks: BIG-bench Machine Learning

These are the proceedings of the 4th workshop on Machine Learning for the Developing World (ML4D), held as part of the Thirty-fourth Conference on Neural Information Processing Systems (NeurIPS) on Saturday, December 12th 2020.
Mixup Without Hesitation
Authors: Hao Yu, Huanyu Wang, Jianxin Wu
Date: 2021-01-12 00:00:00
Tasks: Data Augmentation, image-classification, Image Classification, object-detection, Object Detection

Mixup linearly interpolates pairs of examples to form new samples, which is easy to implement and has been shown to be effective in image classification tasks. However, there are two drawbacks in mixup: one is that more training epochs are needed to obtain a well-trained model; the other is that mixup requires tuning a hyper-parameter to gain appropriate capacity but that is a difficult task. In this paper, we find that mixup constantly explores the representation space, and inspired by the exploration-exploitation dilemma in reinforcement learning, we propose mixup Without hesitation (mWh), a concise, effective, and easy-to-use training algorithm. We show that mWh strikes a good balance between exploration and exploitation by gradually replacing mixup with basic data augmentation. It can achieve a strong baseline with less training time than original mixup and without searching for optimal hyper-parameter, i.e., mWh acts as mixup without hesitation. mWh can also transfer to CutMix, and gain consistent improvement on other machine learning and computer vision tasks such as object detection. Our code is open-source and available at https://github.com/yuhao318/mwh
Take More Positives: An Empirical Study of Contrastive Learing in Unsupervised Person Re-Identification
Authors: Xuanyu He, Wei zhang, Ran Song, Qian Zhang, Xiangyuan Lan, Lin Ma
Date: 2021-01-12 00:00:00
Tasks: Contrastive Learning, Person Re-Identification, Unsupervised Person Re-Identification

Unsupervised person re-identification (re-ID) aims at closing the performance gap to supervised methods. These methods build reliable relationship between data points while learning representations. However, we empirically show that the reason why they are successful is not only their label generation mechanisms, but also their unexplored designs. By studying two unsupervised person re-ID methods in a cross-method way, we point out a hard negative problem is handled implicitly by their designs of data augmentations and PK sampler respectively. In this paper, we find another simple solution for the problem, i.e., taking more positives during training, by which we generate pseudo-labels and update models in an iterative manner. Based on our findings, we propose a contrastive learning method without a memory back for unsupervised person re-ID. Our method works well on benchmark datasets and outperforms the state-of-the-art methods. Code will be made available.
Locality Sensitive Hashing for Efficient Similar Polygon Retrieval
Authors: Haim Kaplan, Jay Tenenbaum
Date: 2021-01-12 00:00:00
Tasks: Retrieval, Translation

Locality Sensitive Hashing (LSH) is an effective method of indexing a set of items to support efficient nearest neighbors queries in high-dimensional spaces. The basic idea of LSH is that similar items should produce hash collisions with higher probability than dissimilar items. We study LSH for (not necessarily convex) polygons, and use it to give efficient data structures for similar shape retrieval. Arkin et al. represent polygons by their "turning function" - a function which follows the angle between the polygon's tangent and the $ x $-axis while traversing the perimeter of the polygon. They define the distance between polygons to be variations of the $ L_p $ (for $p=1,2$) distance between their turning functions. This metric is invariant under translation, rotation and scaling (and the selection of the initial point on the perimeter) and therefore models well the intuitive notion of shape resemblance. We develop and analyze LSH near neighbor data structures for several variations of the $ L_p $ distance for functions (for $p=1,2$). By applying our schemes to the turning functions of a collection of polygons we obtain efficient near neighbor LSH-based structures for polygons. To tune our structures to turning functions of polygons, we prove some new properties of these turning functions that may be of independent interest. As part of our analysis, we address the following problem which is of independent interest. Find the vertical translation of a function $ f $ that is closest in $ L_1 $ distance to a function $ g $. We prove tight bounds on the approximation guarantee obtained by the translation which is equal to the difference between the averages of $ g $ and $ f $.
Blind Modulation Classification via Combined Machine Learning and Signal Feature Extraction
Authors: Jafar Norolahi, Paeiz Azmi
Date: 2021-01-12 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, General Classification

In this study, an algorithm to blind and automatic modulation classification has been proposed. It well benefits combined machine leaning and signal feature extraction to recognize diverse range of modulation in low signal power to noise ratio (SNR). The presented algorithm contains four. First, it advantages spectrum analyzing to branching modulated signal based on regular and irregular spectrum character. Seconds, a nonlinear soft margin support vector (NS SVM) problem is applied to received signal, and its symbols are classified to correct and incorrect (support vectors) symbols. The NS SVM employment leads to discounting in physical layer noise effect on modulated signal. After that, a k-center clustering can find center of each class. finally, in correlation function estimation of scatter diagram is correlated with pre-saved ideal scatter diagram of modulations. The correlation outcome is classification result. For more evaluation, success rate, performance, and complexity in compare to many published methods are provided. The simulation prove that the proposed algorithm can classified the modulated signal in less SNR. For example, it can recognize 4-QAM in SNR=-4.2 dB, and 4-FSK in SNR=2.1 dB with %99 success rate. Moreover, due to using of kernel function in dual problem of NS SVM and feature base function, the proposed algorithm has low complexity and simple implementation in practical issues.
Seed Stocking Via Multi-Task Learning
Authors: Yunhe Feng, Wenjun Zhou
Date: 2021-01-12 00:00:00
Tasks: Multi-Task Learning

Sellers of crop seeds need to plan for the variety and quantity of seeds to stock at least a year in advance. There are a large number of seed varieties of one crop, and each can perform best under different growing conditions. Given the unpredictability of weather, farmers need to make decisions that balance high yield and low risk. A seed vendor needs to be able to anticipate the needs of farmers and have them ready. In this study, we propose an analytical framework for estimating seed demand with three major steps. First, we will estimate the yield and risk of each variety as if they were planted at each location. Since past experiments performed with different seed varieties are highly unbalanced across varieties, and the combination of growing conditions is sparse, we employ multi-task learning to borrow information from similar varieties. Second, we will determine the best mix of seeds for each location by seeking a tradeoff between yield and risk. Third, we will aggregate such mix and pick the top five varieties to re-balance the yield and risk for each growing location. We find that multi-task learning provides a viable solution for yield prediction, and our overall analytical framework has resulted in a good performance.