Reviewerly/exHarmony huggingface dataset:
Implementation of Binarized Neural Networks in All-Programmable System-on-Chip Platforms

The Binarized Neural Network (BNN) is a Convolutional Neural Network (CNN) consisting of binary weights and activation rather than real-value weights. Smaller models are used, allowing for inference effectively on mobile or embedded devices with limited power and computing capabilities. Nevertheless, binarization results in lower-entropy feature maps and gradient vanishing, which leads to a loss in accuracy compared to real-value networks. Previous research has addressed these issues with various approaches. However, those approaches significantly increase the algorithm’s time and space complexity, which puts a heavy burden on those embedded devices. Therefore, a novel approach for BNN implementation on embedded systems with multi-scale BNN topology is proposed in this paper, from two optimization perspectives: hardware structure and BNN topology, that retains more low-level features throughout the feed-forward process with few operations. Experiments on the CIFAR-10 dataset indicate that the proposed method outperforms a number of current BNN designs in terms of efficiency and accuracy. Additionally, the proposed BNN was implemented on the All Programmable System on Chip (APSoC) with 4.4 W power consumption using the hardware accelerator.
VQuAD: Video Question Answering Diagnostic Dataset

In this paper, we investigate the task of Video based Question Answering. We provide a diagnostic dataset that can be used to evaluate the extent of reasoning abilities of various methods for solving this task. Previous datasets proposed for this task do not have this ability. Our dataset is large scale (around 1.3 million questions jointly for train and test) and evaluates both the spatial and temporal properties and the relationship between various objects for these properties. We evaluate state of the art language model (BERT) as a baseline to understand the extent of correlation based on language features alone. Other existing networks are then used to combine video features along with language features for solving this task. Unfortunately, we observe that the currently prevalent systems do not perform significantly better than the language baseline. We hypothesise that this is due to our efforts in ensuring that no obvious biases exist in this dataset and the dataset is balanced. To make progress, the learning techniques needs to obtain an ability to reason, going beyond basic correlation of biases. This is an interesting and significant challenge provided through our work. We release our dataset and source code for our baseline modules in the following webpage https://delta-lab-iitk.github.io/vquad/.
HBSniff: A static analysis tool for Java Hibernate object-relational mapping code smell detection

Code smells are symptoms of sub-optimal software design and implementation choices. Detection tools were actively developed for general code smell related to coupling and cohesion issues, but such tools cannot capture domain-specific problems. In this work, we fill the gap in data persistence and query code quality by proposing HBSniff, i.e., a static analysis tool for detecting 14 code smells as well as 4 mapping metrics in Java Hibernate Object-Relational Mapping (ORM) codes. HBSniff is tested, documented, and manually validated. It also generates readable and customizable reports for every project. Moreover, it is beneficial to Mining Software Repository (MSR) research requiring large-scale analysis since project compilation is not needed for detection.
Edge‐aware image outpainting with attentional generative adversarial networks

Image outpainting aims at extending the field of view of an existing image. While image inpainting has achieved great success with the deep learning technology, image outpainting still receive less attention. The main challenge is how to generate high-quality extended images with clear texture and highly consistent semantic information. In order to solve the problem of the effect of invalid pixels on the generated image and the distance of effective pixels is too far. This paper proposes a two-stage image outpainting method (the EA method), which consists of an edge generation stage and an edge transformation stage. In this paper, the convolutional block attention module (CBAM) is introduced into the generation network to focus on spatial and channel feature and the improved VAE-GAN structure is used to generate the extended image for more realistic semantics. The EA method is evaluated on the CelebA, Pairs-streetview and homemade landscapes dataset, and show that the results contain high-quality textures as well as faithfully extend the semantics. The average PSNR, SSIM, FID index of the EA method on the three datasets is 22.7961, 0.7061, 6.8553 and show that it outperforms existing algorithms in both quantitative and qualitative analysis.
Convolutional Neural Network or Vision Transformer? Benchmarking Various Machine Learning Models for Distracted Driver Detection

Driver distraction is the main factor of severe traffic accidents and has become an essential issue in the traffic safety field. Hence, driver inattention systems are crucial in ensuring the safety of road users. With the introduction of Vision Transformer for computer vision tasks, there is a lack of comprehensive evaluation of various models for distracted driver detection. Hence, we raise the question - does vision transformers outperform convolutional neural networks (CNNs) in the field of detecting driving distraction? In this work, we evaluate and perform in-depth evaluations of various state-of-the-art CNN and Vision Transformer models to detect the distracted driver. We believe this will aid future researchers in this field in benchmarking their novel models with state-of-the-art models. We select ResNet, VGGNet, DenseNet, and EfficientNet as the candidates for CNN, while ViT, Swin Transformer, DeiT, and CaiT for Vision Transformer. We perform our benchmark on the American University of Cairo Distracted Driving Dataset (AUC-DDD) which consists of ten distracted classes. It is observed that CNN should be considered first if the downstream task is specific and the available dataset is small. An in-depth discussion and analysis are included in this work.
Uni-EDEN: Universal Encoder-Decoder Network by Multi-Granular Vision-Language Pre-training

Vision-language pre-training has been an emerging and fast-developing research topic, which transfers multi-modal knowledge from rich-resource pre-training task to limited-resource downstream tasks. Unlike existing works that predominantly learn a single generic encoder, we present a pre-trainable Universal Encoder-DEcoder Network (Uni-EDEN) to facilitate both vision-language perception (e.g., visual question answering) and generation (e.g., image captioning). Uni-EDEN is a two-stream Transformer-based structure, consisting of three modules: object and sentence encoders that separately learns the representations of each modality and sentence decoder that enables both multi-modal reasoning and sentence generation via inter-modal interaction. Considering that the linguistic representations of each image can span different granularities in this hierarchy including, from simple to comprehensive, individual label, a phrase, and a natural sentence, we pre-train Uni-EDEN through multi-granular vision-language proxy tasks: Masked Object Classification, Masked Region Phrase Generation, Image-Sentence Matching, and Masked Sentence Generation. In this way, Uni-EDEN is endowed with the power of both multi-modal representation extraction and language modeling. Extensive experiments demonstrate the compelling generalizability of Uni-EDEN by fine-tuning it to four vision-language perception and generation downstream tasks.
Concepts and Models of Environment of Self-Adaptive Systems: A Systematic Literature Review

The runtime environment is an important concern for self-adaptive systems (SASs). Although researchers have proposed many approaches for developing SASs that address the issues from runtime environments, the understanding of these environments varies depending on the objectives, perspectives, and assumptions of the research. Thus, the current understanding of environments in SAS development remains ambiguous and abstract. To make this knowledge more concrete, we investigated concepts and models of the environment covered in this area through a systematic literature review (SLR). We automatically and manually searched 3719 papers and selected 128 papers as primary studies. We explored and analyzed concepts of the environment covered in the primary studies and investigated cases in which the concepts were specifically expressed as environment models. In doing so, we provide trends of how SAS academia understands the environment of SAS. Specifically, this SLR provides five common characteristics of the environment, two common sources of the environmental uncertainty, and 14 reference environment models with various purpose and expressiveness. Finally, we summarized lessons learned through this SLR and directions for future SAS research on the basis of the concrete knowledge of the SAS environment.
Multitask Learning-Based for SAR Image Superpixel Generation

Most of the existing synthetic aperture radar (SAR) image superpixel generation methods are designed based on the raw SAR images or artificially designed features. However, such methods have the following limitations: (1) SAR images are severely affected by speckle noise, resulting in unstable pixel distance estimation. (2) Artificially designed features cannot be well-adapted to complex SAR image scenes, such as the building regions. Aiming to overcome these shortcomings, we propose a multitask learning-based superpixel generation network (ML-SGN) for SAR images. ML-SGN firstly utilizes a multitask feature extractor to extract deep features, and constructs a high-dimensional feature space containing intensity information, deep semantic informantion, and spatial information. Then, we define an effective pixel distance measure based on the high-dimensional feature space. In addition, we design a differentiable soft assignment operation instead of the non-differentiable nearest neighbor operation, so that the differentiable Simple Linear Iterative Clustering (SLIC) and multitask feature extractor can be combined into an end-to-end superpixel generation network. Comprehensive evaluations are performed on two real SAR images with different bands, which demonstrate that our proposed method outperforms other state-of-the-art methods.
From Node to Graph: Joint Reasoning on Visual-Semantic Relational Graph for Zero-Shot Detection

Zero-Shot Detection (ZSD), which aims at localizing and recognizing unseen objects in a complicated scene, usually leverages the visual and semantic information of individual objects alone. However, scene understanding of human exceeds recognizing individual objects separately: the contextual information among multiple objects such as visual relational information (e.g. visually similar objects) and semantic relational information (e.g. co-occurrences) is helpful for understanding of visual scene. In this paper, we verify that contextual information plays a more important role in ZSD than in traditional object detection. To make full use of such information, we propose a new end-to-end ZSD method GRaph Aligning Network (GRAN) based on graph modeling and reasoning which simultaneously considers visual and semantic information of multiple objects instead of individual objects. Specifically, we formulate a Visual Relational Graph (VRG) and a Semantic Relational Graph (SRG), where the nodes are the objects in the image and the semantic representations of classes respectively and the edges are the relevance between nodes in each graph. To characterize mutual effect between two modalities, the two graphs are further merged into a heterogeneous Visual-Semantic Relational Graph (VSRG), where modal translators are designed for the two subgraphs to enable modal information to transform into a common space for communication, and message passing among nodes is enforced to refine their representations. Comprehensive experiments on MSCOCO dataset demonstrate the advantage of our method over state-of-the-arts, and qualitative analysis suggests the validity of using contextual information.
Path Semantics for Indicative Conditionals

Abstract The literature on indicative conditionals contains two appealing views. The first is the selectional view: on this view, conditionals operate by selecting a single possibility, which is used to evaluate the consequent. The second is the informational view: on this view, conditionals don’t express propositions, but rather impose constraints on information states of speakers. Both views are supported by strong arguments, but they are incompatible on their standard formulations. Hence it appears that we have to choose between mutually exclusive options. But the options are not exclusive. This paper develops a theory of the semantics and assertability of conditionals that is both selectional and informational. The theory vindicates the signature inferences of both selectional and informational theories, including so-called Conditionals Excluded Middle and principles about the interplay between conditionals and ‘might’. It also predicts some interesting and puzzling facts about the assertion of conditionals.
Attention Enhanced Reinforcement Learning for Multi agent Cooperation

In this article, a novel method, called attention enhanced reinforcement learning (AERL), is proposed to address issues including complex interaction, limited communication range, and time-varying communication topology for multi agent cooperation. AERL includes a communication enhanced network (CEN), a graph spatiotemporal long short-term memory network (GST-LSTM), and parameters sharing multi-pseudo critic proximal policy optimization (PS-MPC-PPO). Specifically, CEN based on graph attention mechanism is designed to enlarge the agents' communication range and to deal with complex interaction among the agents. GST-LSTM, which replaces the standard fully connected (FC) operator in LSTM with graph attention operator, is designed to capture the temporal dependence while maintaining the spatial structure learned by CEN. PS-MPC-PPO, which extends proximal policy optimization (PPO) in multi agent systems with parameters' sharing to scale to environments with a large number of agents in training, is designed with multi-pseudo critics to mitigate the bias problem in training and accelerate the convergence process. Simulation results for three groups of representative scenarios including formation control, group containment, and predator-prey games demonstrate the effectiveness and robustness of AERL.
Maliciously Secure and Efficient Large-Scale Genome-Wide Association Study With Multi-Party Computation

Genome-Wide Association Study (GWAS) aims at detecting the association between diseases and Single-Nucleotide Polymorphisms (SNPs) with statistical techniques and has great potential for disease diagnosis. To obtain high-quality results, GWAS requires large-scale genomic data containing individuals’ privacy information. Thus, how to improve the efficiency of GWAS while protecting the privacy of genomic data becomes a critical challenge. In this paper, we propose a secure and efficient GWAS scheme. By using secure three-party computation, we present a series of protocols, i.e., Secure Quality Control, Secure Principle Component Analysis, Secure Cochran-Armitage trend test, and Secure Logistic Regression, to cover the most significant procedures of secure GWAS. In these protocols, a new comparison protocol is designed to reduce communication and improve efficiency. Furthermore, by extending the above comparison protocol to be maliciously secure and utilizing other technologies, e.g., consistency check, we extend the whole GWAS scheme to malicious security with rationally additional overhead. Experimental results demonstrate that our protocols achieve about 33% performance improvement than the state-of-art secure GWAS scheme using two-party computation in terms of runtime and communication in the semi-honest setting. The cost of our scheme in the malicious setting is around 1.5X than that in the semi-honest setting.
A Deep Learning Approach for Semantic Analysis of COVID-19-Related Stigma on Social Media

The rapid spread of the pandemic of coronavirus disease of 2019 (COVID-19) has created an unprecedented, global health disaster. During the outburst period, the paucity of knowledge and research aggravated devastating panic and fears that lead to social stigma and created serious obstacles to contain the disastrous epidemic. We propose a deep learning-based method to detect stigmatized contents on online social network (OSN) platforms in the early stage of COVID-19. Our method performs a semantic-based quantitative analysis to unveil essential spatial-temporal characteristics of COVID-19 stigmatization for timely alerts and risk mitigation. Empirical evaluations are carried out to examine our method’s predictive utilities. The visualization results of the co-occurrence network using Gephi indicate two distinct groups of stigmatized words that pertain to people in Wuhan and their dietary behaviors, respectively. Netizens’ participations and stigmatizations in the Hubei region, where the COVID-19 broke out, are twice ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p &lt; 0.05$ </tex-math></inline-formula> ) and four ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p &lt; 0.01$ </tex-math></inline-formula> ) times more frequent and intense than those in other parts of China, respectively. Also, the number of COVID-19 patients is correlated with COVID-19-related stigma significantly (correlation coefficient = 0.838, <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$p &lt; 0.01$ </tex-math></inline-formula> ). The responses to individual users’ posts have the power law distribution, while posts by official media appear to attract more responses (e.g., likes, replies, and forward). Our method can help platforms and government agencies manage public health disasters through effective identification and detailed analyses of social stigma on social media.
Jointly attention network for crowd counting

Nowadays, crowd counting has shown great practical value in public safety and related fields. Most leading algorithms exploit CNN to generate density maps and have improved the estimation accuracy. However, the counting models still suffer from the challenge of huge scale variations. In order to mitigate this issue, we propose a novel approach named Jointly Attention Network (JANet) for Crowd Counting. It is composed of two major schemes: the Multi-order Scale Attention (MSA) module and the Multi-pooling Relational Channel Attention (MRCA) module. The MSA module explores meaningful high-order statistics and helps the backbone network obtain more discriminative features with rich scale information in an explicit manner. The MRCA module compactly represents the global scope relations and accounts the interdependence among all channel-wise nodes, which is complementary to MSA module. Meanwhile, the Distributed Combinatorial Loss (DCL) is designed to achieve the distributed supervision on intermediate layers at each level. Finally, we conduct extensive studies on multiple crowd counting datasets, the ShanghaiTech, the UCF-QNRF, the JHU-CROWD++, the NWPU-Crowd. The experimental results indicate that our proposed method has achieved superior performance.
Dynamically Generate Password Policy via Zipf Distribution

Password composition policies are helpful in strengthening password’s resistance against guessing attacks. Sadly, existing off-the-shelf composition policies often remain static, which creates potential security vulnerability. In this paper, we propose a new adaptive password policy generation framework called HTPG. Based on the Zipf distribution of passwords, HTPG classifies all passwords in data set into two categories, that is, head passwords and tail passwords. We find that head passwords are vulnerable and high-value for attackers because they are most frequently used, while tail passwords have higher strength than head passwords. According to this fact, HTPG dynamically generates policies to enhance head passwords by modifying them so as to be closer to tail passwords on feature space. By introducing the idea of machine learning, we propose a policy sort method based on information gain ratio to help user choose more effective policies in enhancing head passwords. HTPG can effectively improve the security of entire password data set and make the password distribution more uniform. Experiments show that the number of cracked head passwords decreases 69% on average, compared with the original head passwords, by adopting policies generated by HTPG. Surveys on usability show that 80.23% enhanced passwords can be recalled by those who remember the corresponding original passwords.
Classification Performance of Neural Networks Versus Logistic Regression Models: Evidence From Healthcare Practice

Machine learning encompasses statistical approaches such as logistic regression (LR) through to more computationally complex models such as neural networks (NN). The aim of this study is to review current published evidence for performance from studies directly comparing logistic regression, and neural network classification approaches in medicine. A literature review was carried out to identify primary research studies which provided information regarding comparative area under the curve (AUC) values for the overall performance of both LR and NN for a defined clinical healthcare-related problem. Following an initial search, articles were reviewed to remove those that did not meet the criteria and performance metrics were extracted from the included articles. Teh initial search revealed 114 articles; 21 studies were included in the study. In 13/21 (62%) of cases, NN had a greater AUC compared to LR, but in most the difference was small and unlikely to be of clinical significance; (unweighted mean difference in AUC 0.03 (95% CI 0-0.06) in favour of NN versus LR. In the majority of cases examined across a range of clinical settings, LR models provide reasonable performance that is only marginally improved using more complex methods such as NN. In many circumstances, the use of a relatively simple LR model is likely to be adequate for real-world needs but in specific circumstances in which large amounts of data are available, and where even small increases in performance would provide significant management value, the application of advanced analytic tools such as NNs may be indicated.
iTimed: Cache Attacks on the Apple A10 Fusion SoC

This paper proposes the first cache timing side-channel attack on one of Apple's mobile devices. Utilizing a recent, permanent exploit named checkm8, we reverse-engineered Apple's BootROM and created a powerful toolkit for running arbitrary hardware security experiments on Apple's in-house designed ARM systems-on-a-chip (SoC). Using this toolkit, we then implement an access-driven cache timing attack (in the style of PRIME+PROBE) as a proof-of-concept illustrator. The advanced hardware control enabled by our toolkit allowed us to reverse-engineer key microarchitectural details of the Apple A10 Fusion's memory hierarchy. We find that the SoC employs a randomized cache-line replacement policy as well as a hardware-based L1 prefetcher. We propose statistical innovations which specifically account for these hardware structures and thus further the state-of-the-art in cache timing attacks. We find that our access-driven attack, at best, can reduce the security of OpenSSL AES-128 by 50 more bits than a straightforward adaptation of PRIME+PROBE, while requiring only half as many side channel measurement traces.