OpenRaiser/Intern-Atlas huggingface dataset:
Mushroom image recognition and distance generation based on attention-mechanism model and genetic information
Venue: arXiv cs.CV 2022
Year: 2022

The species identification of Macrofungi, i.e. mushrooms, has always been a challenging task. There are still a large number of poisonous mushrooms that have not been found, which poses a risk to people's life. However, the traditional identification method requires a large number of experts with knowledge in the field of taxonomy for manual identification, it is not only inefficient but also consumes a lot of manpower and capital costs. In this paper, we propose a new model based on attention-mechanism, MushroomNet, which applies the lightweight network MobileNetV3 as the backbone model, combined with the attention structure proposed by us, and has achieved excellent performance in the mushroom recognition task. On the public dataset, the test accuracy of the MushroomNet model has reached 83.9%, and on the local dataset, the test accuracy has reached 77.4%. The proposed attention mechanisms well focused attention on the bodies of mushroom image for mixed channel attention and the attention heat maps visualized by Grad-CAM. Further, in this study, genetic distance was added to the mushroom image recognition task, the genetic distance was used as the representation space, and the genetic distance between each pair of mushroom species in the dataset was used as the embedding of the genetic distance representation space, so as to predict the image distance and species. identify. We found that using the MES activation function can predict the genetic distance of mushrooms very well, but the accuracy is lower than that of SoftMax. The proposed MushroomNet was demonstrated it shows great potential for automatic and online mushroom image and the proposed automatic procedure would assist and be a reference to traditional mushroom classification.
Comparing informativeness of an NLG chatbot vs graphical app in diet-information domain
Venue: arXiv cs.CL 2022
Year: 2022

Visual representation of data like charts and tables can be challenging to understand for readers. Previous work showed that combining visualisations with text can improve the communication of insights in static contexts, but little is known about interactive ones. In this work we present an NLG chatbot that processes natural language queries and provides insights through a combination of charts and text. We apply it to nutrition, a domain communication quality is critical. Through crowd-sourced evaluation we compare the informativeness of our chatbot against traditional, static diet-apps. We find that the conversational context significantly improved users' understanding of dietary data in various tasks, and that users considered the chatbot as more useful and quick to use than traditional apps.
EMVLight: a Multi-agent Reinforcement Learning Framework for an Emergency Vehicle Decentralized Routing and Traffic Signal Control System
Venue: arXiv cs.AI 2022
Year: 2022

Emergency vehicles (EMVs) play a crucial role in responding to time-critical calls such as medical emergencies and fire outbreaks in urban areas. Existing methods for EMV dispatch typically optimize routes based on historical traffic-flow data and design traffic signal pre-emption accordingly; however, we still lack a systematic methodology to address the coupling between EMV routing and traffic signal control. In this paper, we propose EMVLight, a decentralized reinforcement learning (RL) framework for joint dynamic EMV routing and traffic signal pre-emption. We adopt the multi-agent advantage actor-critic method with policy sharing and spatial discounted factor. This framework addresses the coupling between EMV navigation and traffic signal control via an innovative design of multi-class RL agents and a novel pressure-based reward function. The proposed methodology enables EMVLight to learn network-level cooperative traffic signal phasing strategies that not only reduce EMV travel time but also shortens the travel time of non-EMVs. Simulation-based experiments indicate that EMVLight enables up to a $42.6\%$ reduction in EMV travel time as well as an $23.5\%$ shorter average travel time compared with existing approaches.
Iso-CapsNet: Isomorphic Capsule Network for Brain Graph Representation Learning
Venue: arXiv cs.LG 2022
Year: 2022

Brain graph representation learning serves as the fundamental technique for brain diseases diagnosis. Great efforts from both the academic and industrial communities have been devoted to brain graph representation learning in recent years. The isomorphic neural network (IsoNN) introduced recently can automatically learn the existence of sub-graph patterns in brain graphs, which is also the state-of-the-art brain graph representation learning method by this context so far. However, IsoNN fails to capture the orientations of sub-graph patterns, which may render the learned representations to be useless for many cases. In this paper, we propose a new Iso-CapsNet (Isomorphic Capsule Net) model by introducing the graph isomorphic capsules for effective brain graph representation learning. Based on the capsule dynamic routing, besides the subgraph pattern existence confidence scores, Iso-CapsNet can also learn other sub-graph rich properties, including position, size and orientation, for calculating the class-wise digit capsules. We have compared Iso-CapsNet with both classic and state-of-the-art brain graph representation approaches with extensive experiments on four brain graph benchmark datasets. The experimental results also demonstrate the effectiveness of Iso-CapsNet, which can out-perform the baseline methods with significant improvements.
New heuristic to choose a cylindrical algebraic decomposition variable ordering motivated by complexity analysis
Venue: arXiv cs.SC 2022
Year: 2022

It is well known that the variable ordering can be critical to the efficiency or even tractability of the cylindrical algebraic decomposition (CAD) algorithm. We propose new heuristics inspired by complexity analysis of CAD to choose the variable ordering. These heuristics are evaluated against existing heuristics with experiments on the SMT-LIB benchmarks using both existing performance metrics and a new metric we propose for the problem at hand. The best of these new heuristics chooses orderings that lead to timings on average 17% slower than the virtual-best: an improvement compared to the prior state-of-the-art which achieved timings 25% slower.
ProGen2: Exploring the Boundaries of Protein Language Models
Venue: arXiv cs.LG 2022
Year: 2022

Attention-based models trained on protein sequences have demonstrated incredible success at classification and generation tasks relevant for artificial intelligence-driven protein design. However, we lack a sufficient understanding of how very large-scale models and data play a role in effective protein model development. We introduce a suite of protein language models, named ProGen2, that are scaled up to 6.4B parameters and trained on different sequence datasets drawn from over a billion proteins from genomic, metagenomic, and immune repertoire databases. ProGen2 models show state-of-the-art performance in capturing the distribution of observed evolutionary sequences, generating novel viable sequences, and predicting protein fitness without additional finetuning. As large model sizes and raw numbers of protein sequences continue to become more widely accessible, our results suggest that a growing emphasis needs to be placed on the data distribution provided to a protein sequence model. We release the ProGen2 models and code at https://github.com/salesforce/progen.
Reflecting on Recurring Failures in IoT Development
Venue: arXiv cs.SE 2022
Year: 2022

As IoT systems are given more responsibility and autonomy, they offer greater benefits, but also carry greater risks. We believe this trend invigorates an old challenge of software engineering: how to develop high-risk software-intensive systems safely and securely under market pressures? As a first step, we conducted a systematic analysis of recent IoT failures to identify engineering challenges. We collected and analyzed 22 news reports and studied the sources, impacts, and repair strategies of failures in IoT systems. We observed failure trends both within and across application domains. We also observed that failure themes have persisted over time. To alleviate these trends, we outline a research agenda toward a Failure-Aware Software Development Life Cycle for IoT development. We propose an encyclopedia of failures and an empirical basis for system postmortems, complemented by appropriate automated tools.
Cyber Network Resilience against Self-Propagating Malware Attacks
Venue: arXiv cs.CR 2022
Year: 2022

Self-propagating malware (SPM) has led to huge financial losses, major data breaches, and widespread service disruptions in recent years. In this paper, we explore the problem of developing cyber resilient systems capable of mitigating the spread of SPM attacks. We begin with an in-depth study of a well-known self-propagating malware, WannaCry, and present a compartmental model called SIIDR that accurately captures the behavior observed in real-world attack traces. Next, we investigate ten cyber defense techniques, including existing edge and node hardening strategies, as well as newly developed methods based on reconfiguring network communication (NodeSplit) and isolating communities. We evaluate all defense strategies in detail using six real-world communication graphs collected from a large retail network and compare their performance across a wide range of attacks and network topologies. We show that several of these defenses are able to efficiently reduce the spread of SPM attacks modeled with SIIDR. For instance, given a strong attack that infects 97% of nodes when no defense is employed, strategically securing a small number of nodes (0.08%) reduces the infection footprint in one of the networks down to 1%.
Quantitative and Metric Rewriting: Abstract, Non-Expansive, and Graded Systems
Venue: arXiv cs.LO 2022
Year: 2022

We introduce a general theory of quantitative and metric rewriting systems, namely systems with a rewriting relation enriched over quantales modelling abstract quantities. We develop theories of abstract and term-based systems, refining cornerstone results of rewriting theory (such as Newman's Lemma, Church-Rosser Theorem, and critical pair-like lemmas) to a metric and quantitative setting. To avoid distance trivialisation and lack of confluence issues, we introduce non-expansive, linear term rewriting systems, and then generalise the latter to the novel class of graded term rewriting systems. These systems make quantitative rewriting modal and context-sensitive, this endowing rewriting with coeffectful behaviours. We apply the theory developed to several examples coming from the fields of quantitative algebras, programming language semantics, and algorithms.
DF-SCA: Dynamic Frequency Side Channel Attacks are Practical
Venue: arXiv cs.CR 2022
Year: 2022

The arm race between hardware security engineers and side-channel researchers has become more competitive with more sophisticated attacks and defenses in the last decade. While modern hardware features improve the system performance significantly, they may create new attack surfaces for malicious people to extract sensitive information about users without physical access to the victim device. Although many previously exploited hardware and OS features were patched by OS developers and chip vendors, any feature that is accessible from userspace applications can be exploited to perform software-based side-channel attacks. In this paper, we present DF-SCA, which is a software-based dynamic frequency side-channel attack on Linux and Android OS devices. We exploit unprivileged access to cpufreq interface that exposes real-time CPU core frequency values directly correlated with the system utilization, creating a reliable side-channel for attackers. We show that Dynamic Voltage and Frequency Scaling (DVFS) feature in modern systems can be utilized to perform website fingerprinting attacks for Google Chrome and Tor browsers on modern Intel, AMD, and ARM architectures. We further extend our analysis to a wide selection of scaling governors on Intel and AMD CPUs, verifying that all scaling governors provide enough information on the visited web page. Moreover, we extract properties of keystroke patterns on frequency readings, that leads to 95% accuracy to distinguish the keystrokes from other activities on Android phones. We leverage inter-keystroke timings of a user by training a k-th nearest neighbor model, which achieves 88% password recovery rate in the first guess on Bank of America application. Finally, we propose several countermeasures to mask the user activity to mitigate DF-SCA on Linux-based systems.
Towards Global-Scale Crowd+AI Techniques to Map and Assess Sidewalks for People with Disabilities
Venue: arXiv cs.CV 2022
Year: 2022

There is a lack of data on the location, condition, and accessibility of sidewalks across the world, which not only impacts where and how people travel but also fundamentally limits interactive mapping tools and urban analytics. In this paper, we describe initial work in semi-automatically building a sidewalk network topology from satellite imagery using hierarchical multi-scale attention models, inferring surface materials from street-level images using active learning-based semantic segmentation, and assessing sidewalk condition and accessibility features using Crowd+AI. We close with a call to create a database of labeled satellite and streetscape scenes for sidewalks and sidewalk accessibility issues along with standardized benchmarks.
A Combination of Host Overloading Detection and Virtual Machine Selection in Cloud Server Consolidation based on Learning Method
Venue: arXiv cs.DC 2022
Year: 2022

In cloud data center (CDC), reducing energy consumption while maintaining performance has always been a hot issue. In server consolidation, the traditional solution is to divide the problem into multiple small problems such as host overloading detection, virtual machine (VM) selection and VM placement and solve them step by step. However, the design of host overloading detection strategies and VM selection strategies cannot be directly linked to the ultimate goal of reducing energy consumption and ensuring performance. This paper proposes a learning-based VM selection strategy that selects appropriate VMs for migration without direct host overloading detection. Thereby reducing the generation of SLAV, ensuring the performance and reducing the energy consumption of CDC. Simulations driven by real VM workload traces show that our method outperforms the existing methods in reducing SLAV generation and CDC energy consumption.
The Third Place Solution for CVPR2022 AVA Accessibility Vision and Autonomy Challenge
Venue: arXiv cs.CV 2022
Year: 2022

The goal of AVA challenge is to provide vision-based benchmarks and methods relevant to accessibility. In this paper, we introduce the technical details of our submission to the CVPR2022 AVA Challenge. Firstly, we conducted some experiments to help employ proper model and data augmentation strategy for this task. Secondly, an effective training strategy was applied to improve the performance. Thirdly, we integrated the results from two different segmentation frameworks to improve the performance further. Experimental results demonstrate that our approach can achieve a competitive result on the AVA test set. Finally, our approach achieves 63.008\%AP@0.50:0.95 on the test set of CVPR2022 AVA Challenge.
H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture
Venue: arXiv cs.AR 2022
Year: 2022

Graph Neural Networks (GNNs) have drawn tremendous attention due to their unique capability to extend Machine Learning (ML) approaches to applications broadly-defined as having unstructured data, especially graphs. Compared with other Machine Learning (ML) modalities, the acceleration of Graph Neural Networks (GNNs) is more challenging due to the irregularity and heterogeneity derived from graph typologies. Existing efforts, however, have focused mainly on handling graphs' irregularity and have not studied their heterogeneity. To this end we propose H-GCN, a PL (Programmable Logic) and AIE (AI Engine) based hybrid accelerator that leverages the emerging heterogeneity of Xilinx Versal Adaptive Compute Acceleration Platforms (ACAPs) to achieve high-performance GNN inference. In particular, H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity, and processes them using PL and AIE, respectively. To further improve performance, we explore the sparsity support of AIE and develop an efficient density-aware method to automatically map tiles of sparse matrix-matrix multiplication (SpMM) onto the systolic tensor array. Compared with state-of-the-art GCN accelerators, H-GCN achieves, on average, speedups of 1.1~2.3X.
Principal Phrase Mining
Venue: arXiv cs.CL 2022
Year: 2022

Extracting frequent words from a collection of texts is commonly performed in many subjects. However, as useful as it is to obtain a collection of commonly occurring words from texts, there is a need for more specific information to be obtained from texts in the form of most commonly occurring phrases. Despite this need, extracting frequent phrases is not commonly done due to inherent complications, the most significant being double-counting. Double-counting occurs when words or phrases are counted when they appear inside longer phrases that themselves are also counted, resulting in a selection of mostly meaningless phrases that are frequent only because they occur inside frequent super phrases. Several papers have been written on phrase mining that describe solutions to this issue; however, they either require a list of so-called quality phrases to be available to the extracting process, or they require human interaction to identify those quality phrases during the process. We present here a method that eliminates double-counting via a unique rectification process that does not require lists of quality phrases. In the context of a set of texts, we define a principal phrase as a phrase that does not cross punctuation marks, does not start with a stop word, with the exception of the stop words "not" and "no", does not end with a stop word, is frequent within those texts without being double counted, and is meaningful to the user. Our method identifies such principal phrases independently without human input, and enables their extraction from any texts within a reasonable amount of time.
On the Impact of Noises in Crowd-Sourced Data for Speech Translation
Venue: arXiv cs.CL 2022
Year: 2022

Training speech translation (ST) models requires large and high-quality datasets. MuST-C is one of the most widely used ST benchmark datasets. It contains around 400 hours of speech-transcript-translation data for each of the eight translation directions. This dataset passes several quality-control filters during creation. However, we find that MuST-C still suffers from three major quality issues: audio-text misalignment, inaccurate translation, and unnecessary speaker's name. What are the impacts of these data quality issues for model development and evaluation? In this paper, we propose an automatic method to fix or filter the above quality issues, using English-German (En-De) translation as an example. Our experiments show that ST models perform better on clean test sets, and the rank of proposed models remains consistent across different test sets. Besides, simply removing misaligned data points from the training set does not lead to a better ST model.
Classification of ADHD Patients Using Kernel Hierarchical Extreme Learning Machine
Venue: arXiv cs.LG 2022
Year: 2022

Recently, the application of deep learning models to diagnose neuropsychiatric diseases from brain imaging data has received more and more attention. However, in practice, exploring interactions in brain functional connectivity based on operational magnetic resonance imaging data is critical for studying mental illness. Since Attention-Deficit and Hyperactivity Disorder (ADHD) is a type of chronic disease that is very difficult to diagnose in the early stages, it is necessary to improve the diagnosis accuracy of such illness using machine learning models treating patients before the critical condition. In this study, we utilize the dynamics of brain functional connectivity to model features from medical imaging data, which can extract the differences in brain function interactions between Normal Control (NC) and ADHD. To meet that requirement, we employ the Bayesian connectivity change-point model to detect brain dynamics using the local binary encoding approach and kernel hierarchical extreme learning machine for classifying features. To verify our model, we experimented with it on several real-world children's datasets, and our results achieved superior classification rates compared to the state-of-the-art models.
Improving Tese Case Generation for Python Native Libraries Through Constraints on Input Data Structures
Venue: arXiv cs.SE 2022
Year: 2022

Modern Python projects execute computational functions using native libraries and give Python interfaces to boost execution speed; hence, testing these libraries becomes critical to the project's robustness. One challenge is that existing approaches use coverage to guide generation, but native libraries run as black boxes to Python code with no execution information. Another is that dynamic binary instrumentation reduces testing performance as it needs to monitor both native libraries and the Python virtual machine. To address these challenges, in this paper, we propose an automated test case generation approach that works at the Python code layer. Our insight is that many path conditions in native libraries are for processing input data structures through interacting with the VM. In our approach, we instrument the Python Interpreter to monitor the interactions between native libraries and VM, derive constraints on the structures, and then use the constraints to guide test case generation. We implement our approach in a tool named PyCing and apply it to six widely-used Python projects. The experimental results reveal that with the structure constraint guidance, PyCing can cover more execution paths than existing test cases and state-of-the-art tools. Also, with the checkers in the testing framework Pytest, PyCing can identify segmentation faults in 10 Python interfaces and memory leaks in 9. Our instrumentation strategy also has an acceptable influence on testing efficiency.