OpenRaiser/Intern-Atlas huggingface dataset:
From Blackboard to the Office: A Look Into How Practitioners Perceive Software Testing Education
Venue: arXiv cs.SE 2021
Year: 2021

The teaching-learning process may require specific pedagogical approaches to establish a relationship with industry practices. Recently, some studies investigated the educators' perspectives and the undergraduate courses curriculum to identify potential weaknesses and solutions for the software testing teaching process. However, it is still unclear how the practitioners evaluate the acquisition of knowledge about software testing in undergraduate courses. This study carried out an expert survey with 68 newly graduated practitioners to determine what the industry expects from them and what they learned in academia. The yielded results indicated that those practitioners learned at a similar rate as others with a long industry experience. Also, they studied less than half of the 35 software testing topics collected in the survey and took industry-backed extracurricular courses to complement their learning. Additionally, our findings point out a set of implications for future research, as the respondents' learning difficulties (e.g., lack of learning sources) and the gap between academic education and industry expectations (e.g., certifications).
Learning Compositional Shape Priors for Few-Shot 3D Reconstruction
Venue: arXiv cs.CV 2021
Year: 2021

The impressive performance of deep convolutional neural networks in single-view 3D reconstruction suggests that these models perform non-trivial reasoning about the 3D structure of the output space. Recent work has challenged this belief, showing that, on standard benchmarks, complex encoder-decoder architectures perform similarly to nearest-neighbor baselines or simple linear decoder models that exploit large amounts of per-category data. However, building large collections of 3D shapes for supervised training is a laborious process; a more realistic and less constraining task is inferring 3D shapes for categories with few available training examples, calling for a model that can successfully generalize to novel object classes. In this work we experimentally demonstrate that naive baselines fail in this few-shot learning setting, in which the network must learn informative shape priors for inference of new categories. We propose three ways to learn a class-specific global shape prior, directly from data. Using these techniques, we are able to capture multi-scale information about the 3D shape, and account for intra-class variability by virtue of an implicit compositional structure. Experiments on the popular ShapeNet dataset show that our method outperforms a zero-shot baseline by over 40%, and the current state-of-the-art by over 10%, in terms of relative performance, in the few-shot setting.
N-Best ASR Transformer: Enhancing SLU Performance using Multiple ASR Hypotheses
Venue: arXiv cs.CL 2021
Year: 2021

Spoken Language Understanding (SLU) systems parse speech into semantic structures like dialog acts and slots. This involves the use of an Automatic Speech Recognizer (ASR) to transcribe speech into multiple text alternatives (hypotheses). Transcription errors, common in ASRs, impact downstream SLU performance negatively. Approaches to mitigate such errors involve using richer information from the ASR, either in form of N-best hypotheses or word-lattices. We hypothesize that transformer models learn better with a simpler utterance representation using the concatenation of the N-best ASR alternatives, where each alternative is separated by a special delimiter [SEP]. In our work, we test our hypothesis by using concatenated N-best ASR alternatives as the input to transformer encoder models, namely BERT and XLM-RoBERTa, and achieve performance equivalent to the prior state-of-the-art model on DSTC2 dataset. We also show that our approach significantly outperforms the prior state-of-the-art when subjected to the low data regime. Additionally, this methodology is accessible to users of third-party ASR APIs which do not provide word-lattice information.
Federated Learning with Spiking Neural Networks
Venue: arXiv cs.LG 2021
Year: 2021

As neural networks get widespread adoption in resource-constrained embedded devices, there is a growing need for low-power neural systems. Spiking Neural Networks (SNNs)are emerging to be an energy-efficient alternative to the traditional Artificial Neural Networks (ANNs) which are known to be computationally intensive. From an application perspective, as federated learning involves multiple energy-constrained devices, there is a huge scope to leverage energy efficiency provided by SNNs. Despite its importance, there has been little attention on training SNNs on a large-scale distributed system like federated learning. In this paper, we bring SNNs to a more realistic federated learning scenario. Specifically, we propose a federated learning framework for decentralized and privacy-preserving training of SNNs. To validate the proposed federated learning framework, we experimentally evaluate the advantages of SNNs on various aspects of federated learning with CIFAR10 and CIFAR100 benchmarks. We observe that SNNs outperform ANNs in terms of overall accuracy by over 15% when the data is distributed across a large number of clients in the federation while providing up to5.3x energy efficiency. In addition to efficiency, we also analyze the sensitivity of the proposed federated SNN framework to data distribution among the clients, stragglers, and gradient noise and perform a comprehensive comparison with ANNs.
On D2D Caching with Uncoded Cache Placement
Venue: arXiv cs.IT 2021
Year: 2021

We consider a cache-aided wireless device-to-device (D2D) network under the constraint of one-shot delivery, where the placement phase is orchestrated by a central server. We assume that the devices' caches are filled with uncoded data, and the whole file database at the server is made available in the collection of caches. Following this phase, the files requested by the users are serviced by inter-device multicast communication. For such a system setting, we provide the exact characterization of load-memory trade-off, by deriving both the minimum average and the minimum peak sum-loads of links between devices, for a given individual memory size at disposal of each user.
Metrics for 3D Object Pointing and Manipulation in Virtual Reality
Venue: arXiv cs.HC 2021
Year: 2021

Assessing the performance of human movements during teleoperation and virtual reality is a challenging problem, particularly in 3D space due to complex spatial settings. Despite the presence of a multitude of metrics, a compelling standardized 3D metric is yet missing, aggravating inter-study comparability between different studies. Hence, evaluating human performance in virtual environments is a long-standing research goal, and a performance metric that combines two or more metrics under one formulation remains largely unexplored, particularly in higher dimensions. The absence of such a metric is primarily attributed to the discrepancies between pointing and manipulation, the complex spatial variables in 3D, and the combination of translational and rotational movements altogether. In this work, four experiments were designed and conducted with progressively higher spatial complexity to study and compare existing metrics thoroughly. The research goal was to quantify the difficulty of these 3D tasks and model human performance sufficiently in full 3D peripersonal space. Consequently, a new model extension has been proposed and its applicability has been validated across all the experimental results, showing improved modelling and representation of human performance in combined movements of 3D object pointing and manipulation tasks than existing work. Lastly, the implications on 3D interaction, teleoperation and object task design in virtual reality are discussed.
Structure-Regularized Attention for Deformable Object Representation
Venue: arXiv cs.CV 2021
Year: 2021

Capturing contextual dependencies has proven useful to improve the representational power of deep neural networks. Recent approaches that focus on modeling global context, such as self-attention and non-local operation, achieve this goal by enabling unconstrained pairwise interactions between elements. In this work, we consider learning representations for deformable objects which can benefit from context exploitation by modeling the structural dependencies that the data intrinsically possesses. To this end, we provide a novel structure-regularized attention mechanism, which formalizes feature interaction as structural factorization through the use of a pair of light-weight operations. The instantiated building blocks can be directly incorporated into modern convolutional neural networks, to boost the representational power in an efficient manner. Comprehensive studies on multiple tasks and empirical comparisons with modern attention mechanisms demonstrate the gains brought by our method in terms of both performance and model complexity. We further investigate its effect on feature representations, showing that our trained models can capture diversified representations characterizing object parts without resorting to extra supervision.
Examining Passenger Vehicle Miles Traveled and Carbon Emissions in the Boston Metropolitan Area
Venue: arXiv cs.CY 2021
Year: 2021

With spatial analytic, econometric, and visualization tools, this book chapter investigates greenhouse gas emissions for the on-road passenger vehicle transport sector in the Boston metropolitan area in 2014. It compares greenhouse gas emission estimations from both the production-based and consumption-based perspectives with two large-scale administrative datasets: the vehicle odometer readings from individual vehicle annual inspection, and the road inventory data containing road segment level geospatial and traffic information. Based on spatial econometric models that examine socioeconomic and built environment factors contributing to the vehicle miles traveled at the census tract level, it offers insights to help cities reduce VMT and carbon footprint for passenger vehicle travel. Finally, it recommends a pathway for cities and towns in the Boston metropolitan area to curb VMT and mitigate carbon emissions to achieve climate goals of carbon neutrality.
BIOPAK Flasher: Epidemic disease monitoring and detection in Pakistan using text mining
Venue: arXiv cs.IR 2021
Year: 2021

Infectious disease outbreak has a significant impact on morbidity, mortality and can cause economic instability of many countries. As global trade is growing, goods and individuals are expected to travel across the border, an infected epidemic area carrier can pose a great danger to his hostile. If a disease outbreak is recognized promptly, then commercial products and travelers (traders/visitors) will be effectively vaccinated, and therefore the disease stopped. Early detection of outbreaks plays an important role here, and beware of the rapid implementation of control measures by citizens, public health organizations, and government. Many indicators have valuable information, such as online news sources (RSS) and social media sources (Twitter, Facebook) that can be used, but are unstructured and bulky, to extract information about disease outbreaks. Few early warning outbreak systems exist with some limitation of linguistic (Urdu) and covering areas (Pakistan). In Pakistan, few channels are published the outbreak news in Urdu or English. The aim is to procure information from Pakistan's English and Urdu news channels and then investigate process, integrate, and visualize the disease epidemic. Urdu ontology is not existed before to match extracted diseases, so we also build that ontology of disease.
A decreasing scaling transition scheme from Adam to SGD
Venue: arXiv cs.LG 2021
Year: 2021

Adaptive gradient algorithm (AdaGrad) and its variants, such as RMSProp, Adam, AMSGrad, etc, have been widely used in deep learning. Although these algorithms are faster in the early phase of training, their generalization performance is often not as good as stochastic gradient descent (SGD). Hence, a trade-off method of transforming Adam to SGD after a certain iteration to gain the merits of both algorithms is theoretically and practically significant. To that end, we propose a decreasing scaling transition scheme to achieve a smooth and stable transition from Adam to SGD, which is called DSTAdam. The convergence of the proposed DSTAdam is also proved in an online convex setting. Finally, the effectiveness of the DSTAdam is verified on the CIFAR-10/100 datasets. Our implementation is available at: https://github.com/kunzeng/DSTAdam.
Relearning ensemble selection based on new generated features
Venue: arXiv cs.LG 2021
Year: 2021

The ensemble methods are meta-algorithms that combine several base machine learning techniques to increase the effectiveness of the classification. Many existing committees of classifiers use the classifier selection process to determine the optimal set of base classifiers. In this article, we propose the classifiers selection framework with relearning base classifiers. Additionally, we use in the proposed framework the new generated feature, which can be obtained after the relearning process. The proposed technique was compared with state-of-the-art ensemble methods using three benchmark datasets and one synthetic dataset. Four classification performance measures are used to evaluate the proposed method.
Dynamic Clone Transformer for Efficient Convolutional Neural Netwoks
Venue: arXiv cs.CV 2021
Year: 2021

Convolutional networks (ConvNets) have shown impressive capability to solve various vision tasks. Nevertheless, the trade-off between performance and efficiency is still a challenge for a feasible model deployment on resource-constrained platforms. In this paper, we introduce a novel concept termed multi-path fully connected pattern (MPFC) to rethink the interdependencies of topology pattern, accuracy and efficiency for ConvNets. Inspired by MPFC, we further propose a dual-branch module named dynamic clone transformer (DCT) where one branch generates multiple replicas from inputs and another branch reforms those clones through a series of difference vectors conditional on inputs itself to produce more variants. This operation allows the self-expansion of channel-wise information in a data-driven way with little computational cost while providing sufficient learning capacity, which is a potential unit to replace computationally expensive pointwise convolution as an expansion layer in the bottleneck structure.
Case Study on Detecting COVID-19 Health-Related Misinformation in Social Media
Venue: arXiv cs.SI 2021
Year: 2021

COVID-19 pandemic has generated what public health officials called an infodemic of misinformation. As social distancing and stay-at-home orders came into effect, many turned to social media for socializing. This increase in social media usage has made it a prime vehicle for the spreading of misinformation. This paper presents a mechanism to detect COVID-19 health-related misinformation in social media following an interdisciplinary approach. Leveraging social psychology as a foundation and existing misinformation frameworks, we defined misinformation themes and associated keywords incorporated into the misinformation detection mechanism using applied machine learning techniques. Next, using the Twitter dataset, we explored the performance of the proposed methodology using multiple state-of-the-art machine learning classifiers. Our method shows promising results with at most 78% accuracy in classifying health-related misinformation versus true information using uni-gram-based NLP feature generations from tweets and the Decision Tree classifier. We also provide suggestions on alternatives for countering misinformation and ethical consideration for the study.
D2C: Diffusion-Denoising Models for Few-shot Conditional Generation
Venue: arXiv cs.LG 2021
Year: 2021

Conditional generative models of high-dimensional images have many applications, but supervision signals from conditions to images can be expensive to acquire. This paper describes Diffusion-Decoding models with Contrastive representations (D2C), a paradigm for training unconditional variational autoencoders (VAEs) for few-shot conditional image generation. D2C uses a learned diffusion-based prior over the latent representations to improve generation and contrastive self-supervised learning to improve representation quality. D2C can adapt to novel generation tasks conditioned on labels or manipulation constraints, by learning from as few as 100 labeled examples. On conditional generation from new labels, D2C achieves superior performance over state-of-the-art VAEs and diffusion models. On conditional image manipulation, D2C generations are two orders of magnitude faster to produce over StyleGAN2 ones and are preferred by 50% - 60% of the human evaluators in a double-blind study.
A Pseudo Label-wise Attention Network for Automatic ICD Coding
Venue: arXiv cs.CL 2021
Year: 2021

Automatic International Classification of Diseases (ICD) coding is defined as a kind of text multi-label classification problem, which is difficult because the number of labels is very large and the distribution of labels is unbalanced. The label-wise attention mechanism is widely used in automatic ICD coding because it can assign weights to every word in full Electronic Medical Records (EMR) for different ICD codes. However, the label-wise attention mechanism is computational redundant and costly. In this paper, we propose a pseudo label-wise attention mechanism to tackle the problem. Instead of computing different attention modes for different ICD codes, the pseudo label-wise attention mechanism automatically merges similar ICD codes and computes only one attention mode for the similar ICD codes, which greatly compresses the number of attention modes and improves the predicted accuracy. In addition, we apply a more convenient and effective way to obtain the ICD vectors, and thus our model can predict new ICD codes by calculating the similarities between EMR vectors and ICD vectors. Extensive experiments show the superior performance of our model. On the public MIMIC-III dataset and private Xiangya dataset, our model achieves micro f1 of 0.583 and 0.806, respectively, which outperforms other competing models. Furthermore, we verify the ability of our model in predicting new ICD codes. The case study shows how pseudo label-wise attention works, and demonstrates the effectiveness of pseudo label-wise attention mechanism.
SAR Image Change Detection Based on Multiscale Capsule Network
Venue: arXiv cs.CV 2021
Year: 2021

Traditional change detection methods based on convolutional neural networks (CNNs) face the challenges of speckle noise and deformation sensitivity for synthetic aperture radar images. To mitigate these issues, we proposed a Multiscale Capsule Network (Ms-CapsNet) to extract the discriminative information between the changed and unchanged pixels. On the one hand, the capsule module is employed to exploit the spatial relationship of features. Therefore, equivariant properties can be achieved by aggregating the features from different positions. On the other hand, an adaptive fusion convolution (AFC) module is designed for the proposed Ms-CapsNet. Higher semantic features can be captured for the primary capsules. Feature extracted by the AFC module significantly improves the robustness to speckle noise. The effectiveness of the proposed Ms-CapsNet is verified on three real SAR datasets. The comparison experiments with four state-of-the-art methods demonstrated the efficiency of the proposed method. Our codes are available at https://github.com/summitgao/SAR_CD_MS_CapsNet.
ATRAS: Adversarially Trained Robust Architecture Search
Venue: arXiv cs.LG 2021
Year: 2021

In this paper, we explore the effect of architecture completeness on adversarial robustness. We train models with different architectures on CIFAR-10 and MNIST dataset. For each model, we vary different number of layers and different number of nodes in the layer. For every architecture candidate, we use Fast Gradient Sign Method (FGSM) to generate untargeted adversarial attacks and use adversarial training to defend against those attacks. For each architecture candidate, we report pre-attack, post-attack and post-defense accuracy for the model as well as the architecture parameters and the impact of completeness to the model accuracies.
A Stronger Baseline for Ego-Centric Action Detection
Venue: arXiv cs.CV 2021
Year: 2021

This technical report analyzes an egocentric video action detection method we used in the 2021 EPIC-KITCHENS-100 competition hosted in CVPR2021 Workshop. The goal of our task is to locate the start time and the end time of the action in the long untrimmed video, and predict action category. We adopt sliding window strategy to generate proposals, which can better adapt to short-duration actions. In addition, we show that classification and proposals are conflict in the same network. The separation of the two tasks boost the detection performance with high efficiency. By simply employing these strategy, we achieved 16.10\% performance on the test set of EPIC-KITCHENS-100 Action Detection challenge using a single model, surpassing the baseline method by 11.7\% in terms of average mAP.
Effects of Eavesdropper on the Performance of Mixed η-μ and DGG Cooperative Relaying System
Venue: arXiv cs.IT 2021
Year: 2021

Free-space optical (FSO) channel offers line-of-sight wireless communication with high data rates and high secrecy utilizing unlicensed optical spectrum and also paves the way to the solution of the last-mile access problem. Since atmospheric turbulence is a hindrance to an enhanced secrecy performance, the mixed radio frequency (RF)-FSO system is gaining enormous research interest in recent days. But conventional FSO models except for the double generalized Gamma (DGG) model can not demonstrate secrecy performance for all ranges of turbulence severity. This reason has led us to propose a dual-hop eta-mu and unified DGG mixed RF-FSO network while considering eavesdropping at both RF and FSO hops. The security of these proposed scenarios is investigated in terms of two metrics, i.e., strictly positive secrecy capacity and secure outage probability. Exploiting these expressions, we further investigate how the secrecy performance is affected by various system parameters, i.e., fading, turbulence, and pointing errors. A demonstration is made between heterodyne detection (HD) and intensity modulation and direct detection (IM/DD) techniques while exhibiting superior secrecy performance for HD technique over IM/DD technique. Finally, all analytical results are corroborated via Monte-Carlo simulations.
RCURRENCY: Live Digital Asset Trading Using a Recurrent Neural Network-based Forecasting System
Venue: arXiv cs.AI 2021
Year: 2021

Consistent alpha generation, i.e., maintaining an edge over the market, underpins the ability of asset traders to reliably generate profits. Technical indicators and trading strategies are commonly used tools to determine when to buy/hold/sell assets, yet these are limited by the fact that they operate on known values. Over the past decades, multiple studies have investigated the potential of artificial intelligence in stock trading in conventional markets, with some success. In this paper, we present RCURRENCY, an RNN-based trading engine to predict data in the highly volatile digital asset market which is able to successfully manage an asset portfolio in a live environment. By combining asset value prediction and conventional trading tools, RCURRENCY determines whether to buy, hold or sell digital currencies at a given point in time. Experimental results show that, given the data of an interval $t$, a prediction with an error of less than 0.5\% of the data at the subsequent interval $t+1$ can be obtained. Evaluation of the system through backtesting shows that RCURRENCY can be used to successfully not only maintain a stable portfolio of digital assets in a simulated live environment using real historical trading data but even increase the portfolio value over time.