OpenRaiser/Intern-Atlas huggingface dataset:
A new conversational interaction concept for document creation and editing on mobile devices for visually impaired users
Venue: arXiv cs.HC 2023
Year: 2023

This paper describes the ongoing development of a conversational interaction concept that allows visually impaired users to easily create and edit text documents on mobile devices using mainly voice input. In order to verify the concept, a prototype app was developed and tested for both iOS and Android systems, based on the natural-language understanding (NLU) platform Google Dialogflow. The app and interaction concept were repeatedly tested by users with and without visual impairments. Based on their feedback, the concept was continuously refined, adapted and improved on both mobile platforms. In an iterative user-centred design approach, the following research questions were investigated: Can a visually impaired user rely mainly on speech commands to efficiently create and edit a document on mobile devices? User testing found that an interaction concept based on conversational speech commands was easy and intuitive for visually impaired users. However, it was also found that relying on speech commands alone created its own obstacles, and that a combination of gestures and voice interaction would be more robust. Future research and more extensive useability tests should be carried out among visually impaired users in order to optimize the interaction concept.
Multi-Vehicle Trajectory Prediction at Intersections using State and Intention Information
Venue: arXiv cs.RO 2023
Year: 2023

Traditional approaches to prediction of future trajectory of road agents rely on knowing information about their past trajectory. This work rather relies only on having knowledge of the current state and intended direction to make predictions for multiple vehicles at intersections. Furthermore, message passing of this information between the vehicles provides each one of them a more holistic overview of the environment allowing for a more informed prediction. This is done by training a neural network which takes the state and intent of the multiple vehicles to predict their future trajectory. Using the intention as an input allows our approach to be extended to additionally control the multiple vehicles to drive towards desired paths. Experimental results demonstrate the robustness of our approach both in terms of trajectory prediction and vehicle control at intersections. The complete training and evaluation code for this work is available here: \url{https://github.com/Dekai21/Multi_Agent_Intersection}.
GCS: Generalized Cache Coherence For Efficient Synchronization
Venue: arXiv cs.DC 2023
Year: 2023

We explore the design of scalable synchronization primitives for disaggregated shared memory. Porting existing synchronization primitives to disaggregated shared memory results in poor scalability with the number of application threads because they layer synchronization primitives atop cache-coherence substrates, which engenders redundant inter-core communications. Substantially higher cache-coherence latency ($μ$s) with substantially lower bandwidths in state-of-the-art disaggregated shared memory designs amplifies the impact of such redundant communications and precludes scalability. In this work, we argue for a co-design for the cache-coherence and synchronization layers for better performance scaling of multi-threaded applications on disaggregated memory. This is driven by our observation that synchronization primitives are essentially a generalization of cache-coherence protocols in time and space. We present GCS as an implementation of this co-design. GCS employs wait queues and arbitrarily-sized cache lines directly at the cache-coherence protocol layer for temporal and spatial generalization. We evaluate GCS against the layered approach for synchronization primitives: the pthread implementation of reader-writer lock, and show that GCS improves in-memory key-value store performance at scale by 1 - 2 orders of magnitude.
Contextual Autonomy Evaluation of Unmanned Aerial Vehicles in Subterranean Environments
Venue: arXiv cs.RO 2023
Year: 2023

In this paper we focus on the evaluation of contextual autonomy for robots. More specifically, we propose a fuzzy framework for calculating the autonomy score for a small Unmanned Aerial Systems (sUAS) for performing a task while considering task complexity and environmental factors. Our framework is a cascaded Fuzzy Inference System (cFIS) composed of combination of three FIS which represent different contextual autonomy capabilities. We performed several experiments to test our framework in various contexts, such as endurance time, navigation, take off/land, and room clearing, with seven different sUAS. We introduce a predictive measure which improves upon previous predictive measures, allowing for previous real-world task performance to be used in predicting future mission performance.
Silent Killer: A Stealthy, Clean-Label, Black-Box Backdoor Attack
Venue: arXiv cs.CR 2023
Year: 2023

Backdoor poisoning attacks pose a well-known risk to neural networks. However, most studies have focused on lenient threat models. We introduce Silent Killer, a novel attack that operates in clean-label, black-box settings, uses a stealthy poison and trigger and outperforms existing methods. We investigate the use of universal adversarial perturbations as triggers in clean-label attacks, following the success of such approaches under poison-label settings. We analyze the success of a naive adaptation and find that gradient alignment for crafting the poison is required to ensure high success rates. We conduct thorough experiments on MNIST, CIFAR10, and a reduced version of ImageNet and achieve state-of-the-art results.
A Framework for Large Scale Particle Filters Validated with Data Assimilation for Weather Simulation
Venue: arXiv cs.DC 2023
Year: 2023

Particle filters are a group of algorithms to solve inverse problems through statistical Bayesian methods when the model does not comply with the linear and Gaussian hypothesis. Particle filters are used in domains like data assimilation, probabilistic programming, neural networkoptimization, localization and navigation. Particle filters estimate the probabilitydistribution of model states by running a large number of model instances, the so called particles. The ability to handle a very large number of particles is critical for high dimensional models.This paper proposes a novel paradigm to run very large ensembles of parallel model instances on supercomputers. The approach combines an elastic and fault tolerant runner/server model minimizing data movementswhile enabling dynamic load balancing. Particle weights are computed locally on each runner andtransmitted when available to a server that normalizes them, resamples new particles based on their weight, and redistributes dynamically the work torunners to react to load imbalance. Our approach relies on a an asynchronously manageddistributed particle cache permitting particles to move from one runner to another inthe background while particle propagation goes on. This also enables the number ofrunners to vary during the execution either in reaction to failures and restarts, orto adapt to changing resource availability dictated by external decision processes.The approach is experimented with the Weather Research and Forecasting (WRF) model, toassess its performance for probabilistic weather forecasting. Up to 2555particles on 20442 compute cores are used to assimilate cloud cover observations into short--range weather forecasts over Europe.
Grokking modular arithmetic
Venue: arXiv cs.LG 2023
Year: 2023

We present a simple neural network that can learn modular arithmetic tasks and exhibits a sudden jump in generalization known as ``grokking''. Concretely, we present (i) fully-connected two-layer networks that exhibit grokking on various modular arithmetic tasks under vanilla gradient descent with the MSE loss function in the absence of any regularization; (ii) evidence that grokking modular arithmetic corresponds to learning specific feature maps whose structure is determined by the task; (iii) analytic expressions for the weights -- and thus for the feature maps -- that solve a large class of modular arithmetic tasks; and (iv) evidence that these feature maps are also found by vanilla gradient descent as well as AdamW, thereby establishing complete interpretability of the representations learnt by the network.
CFG2VEC: Hierarchical Graph Neural Network for Cross-Architectural Software Reverse Engineering
Venue: arXiv cs.SE 2023
Year: 2023

Mission-critical embedded software is critical to our society's infrastructure but can be subject to new security vulnerabilities as technology advances. When security issues arise, Reverse Engineers (REs) use Software Reverse Engineering (SRE) tools to analyze vulnerable binaries. However, existing tools have limited support, and REs undergo a time-consuming, costly, and error-prone process that requires experience and expertise to understand the behaviors of software and vulnerabilities. To improve these tools, we propose $\textit{cfg2vec}$, a Hierarchical Graph Neural Network (GNN) based approach. To represent binary, we propose a novel Graph-of-Graph (GoG) representation, combining the information of control-flow and function-call graphs. Our $\textit{cfg2vec}$ learns how to represent each binary function compiled from various CPU architectures, utilizing hierarchical GNN and the siamese network-based supervised learning architecture. We evaluate $\textit{cfg2vec}$'s capability of predicting function names from stripped binaries. Our results show that $\textit{cfg2vec}$ outperforms the state-of-the-art by $24.54\%$ in predicting function names and can even achieve $51.84\%$ better given more training data. Additionally, $\textit{cfg2vec}$ consistently outperforms the state-of-the-art for all CPU architectures, while the baseline requires multiple training to achieve similar performance. More importantly, our results demonstrate that our $\textit{cfg2vec}$ could tackle binaries built from unseen CPU architectures, thus indicating that our approach can generalize the learned knowledge. Lastly, we demonstrate its practicability by implementing it as a Ghidra plugin used during resolving DARPA Assured MicroPatching (AMP) challenges.
Political, economic, and governance attitudes of blockchain users
Venue: arXiv cs.CY 2023
Year: 2023

We present a survey to evaluate crypto-political, crypto-economic, and crypto-governance sentiment in people who are part of a blockchain ecosystem. Based on 3710 survey responses, we describe their beliefs, attitudes, and modes of participation in crypto and investigate how self-reported political affiliation and blockchain ecosystem affiliation are associated with these. We observed polarization in questions on perceptions of the distribution of economic power, personal attitudes towards crypto, normative beliefs about the distribution of power in governance, and external regulation of blockchain technologies. Differences in political self-identification correlated with opinions on economic fairness, gender equity, decision-making power and how to obtain favorable regulation, while blockchain affiliation correlated with opinions on governance and regulation of crypto and respondents' semantic conception of crypto and personal goals for their involvement. We also find that a theory-driven constructed political axis is supported by the data and investigate the possibility of other groupings of respondents or beliefs arising from the data.
Understanding the (In)Effectiveness of Content Moderation: A Case Study of Facebook in the Context of the U.S. Capitol Riot
Venue: arXiv cs.SI 2023
Year: 2023

Social media networks commonly employ content moderation as a tool to limit the spread of harmful content. However, the efficacy of this strategy in limiting the delivery of harmful content to users is not well understood. In this paper, we create a framework to quantify the efficacy of content moderation and use our metrics to analyze content removal on Facebook within the U.S. news ecosystem. In a data set of over 2M posts with 1.6B user engagements collected from 2,551 U.S. news sources before and during the Capitol Riot on January 6, 2021, we identify 10,811 removed posts. We find that the active engagement life cycle of Facebook posts is very short, with 90% of all engagement occurring within the first 30 hours after posting. Thus, even relatively quick intervention allowed significant accrual of engagement before removal, and prevented only 21% of the predicted engagement potential during a baseline period before the U.S. Capitol attack. Nearly a week after the attack, Facebook began removing older content, but these removals occurred so late in these posts' engagement life cycles that they disrupted less than 1% of predicted future engagement, highlighting the limited impact of this intervention. Content moderation likely has limits in its ability to prevent engagement, especially in a crisis, and we recommend that other approaches such as slowing down the rate of content diffusion be investigated.
Mathematical Models and Reinforcement Learning based Evolutionary Algorithm Framework for Satellite Scheduling Problem
Venue: arXiv cs.NE 2023
Year: 2023

For complex combinatorial optimization problems, models and algorithms are at the heart of the solution. The complexity of many types of satellite mission planning problems is NP-hard and places high demands on the solution. In this paper, two types of satellite scheduling problem models are introduced and a reinforcement learning based evolutionary algorithm framework based is proposed.
Visual Story Generation Based on Emotion and Keywords
Venue: arXiv cs.AI 2023
Year: 2023

Automated visual story generation aims to produce stories with corresponding illustrations that exhibit coherence, progression, and adherence to characters' emotional development. This work proposes a story generation pipeline to co-create visual stories with the users. The pipeline allows the user to control events and emotions on the generated content. The pipeline includes two parts: narrative and image generation. For narrative generation, the system generates the next sentence using user-specified keywords and emotion labels. For image generation, diffusion models are used to create a visually appealing image corresponding to each generated sentence. Further, object recognition is applied to the generated images to allow objects in these images to be mentioned in future story development.
Linguistic-style-aware Neural Networks for Fake News Detection
Venue: arXiv cs.CL 2023
Year: 2023

We propose the hierarchical recursive neural network (HERO) to predict fake news by learning its linguistic style, which is distinguishable from the truth, as psychological theories reveal. We first generate the hierarchical linguistic tree of news documents; by doing so, we translate each news document's linguistic style into its writer's usage of words and how these words are recursively structured as phrases, sentences, paragraphs, and, ultimately, the document. By integrating the hierarchical linguistic tree with the neural network, the proposed method learns and classifies the representation of news documents by capturing their locally sequential and globally recursive structures that are linguistically meaningful. It is the first work offering the hierarchical linguistic tree and the neural network preserving the tree information to our best knowledge. Experimental results based on public real-world datasets demonstrate the proposed method's effectiveness, which can outperform state-of-the-art techniques in classifying short and long news documents. We also examine the differential linguistic style of fake news and the truth and observe some patterns of fake news. The code and data have been publicly available.
A Lite Fireworks Algorithm for Optimization
Venue: arXiv cs.NE 2023
Year: 2023

The fireworks algorithm is an optimization algorithm for simulating the explosion phenomenon of fireworks. Because of its fast convergence and high precision, it is widely used in pattern recognition, optimal scheduling, and other fields. However, most of the existing research work on the fireworks algorithm is improved based on its defects, and little consideration is given to reducing the number of parameters of the fireworks algorithm. The original fireworks algorithm has too many parameters, which increases the cost of algorithm adjustment and is not conducive to engineering applications. In addition, in the fireworks population, the unselected individuals are discarded, thus causing a waste of their location information. To reduce the number of parameters of the original Fireworks Algorithm and make full use of the location information of discarded individuals, we propose a simplified version of the Fireworks Algorithm. It reduces the number of algorithm parameters by redesigning the explosion operator of the fireworks algorithm and constructs an adaptive explosion radius by using the historical optimal information to balance the local mining and global exploration capabilities. The comparative experimental results of function optimization show that the overall performance of our proposed LFWA is better than that of comparative algorithms, such as the fireworks algorithm, particle swarm algorithm, and bat algorithm.
MangngalApp -- An integrated package of technology for COVID-19 response and rural development: Acceptability and usability using TAM
Venue: arXiv cs.CY 2023
Year: 2023

The COVID19 pandemic has challenged universities and organizations to devise mechanisms to uplift the well-being and welfare of people and communities. In response, the design and development of an integrated package of technologies, MangngalApp -- A web-based portal and mobile responsive application for rural development served as an opportunity. It showcases different packets of technologies that were outputs of R&D in the field of fisheries and aqua-culture, innovations that were IP-protected, and technologies that harness locally available resources for post-harvest development and aiding in sustaining growth and development in the communities. This paper focused on the usability and acceptability of the MangngalApp implementing a descriptive research design using the Technology Acceptance Model or TAM and ISO 25010 software quality standards. Constrained by government health restrictions due to COVID-19, a Google form-based questionnaire was forwarded to consented participants via an email with the attached consent and evaluation form. Results revealed that the MangngalApp was found to be very acceptable and usable, and compliant to ISO 25010 software quality characteristics to the higher extent. From the results, it is concluded that the developed MangngalApp will be a usable and responsive technology that aids to rural development especially among target users: fishers, gatherers, processors, traders, and farmers. Considering compatibility and usefulness, the MangngalApp is expected to provide greater social development in the community.
How to Allocate your Label Budget? Choosing between Active Learning and Learning to Reject in Anomaly Detection
Venue: arXiv cs.LG 2023
Year: 2023

Anomaly detection attempts at finding examples that deviate from the expected behaviour. Usually, anomaly detection is tackled from an unsupervised perspective because anomalous labels are rare and difficult to acquire. However, the lack of labels makes the anomaly detector have high uncertainty in some regions, which usually results in poor predictive performance or low user trust in the predictions. One can reduce such uncertainty by collecting specific labels using Active Learning (AL), which targets examples close to the detector's decision boundary. Alternatively, one can increase the user trust by allowing the detector to abstain from making highly uncertain predictions, which is called Learning to Reject (LR). One way to do this is by thresholding the detector's uncertainty based on where its performance is low, which requires labels to be evaluated. Although both AL and LR need labels, they work with different types of labels: AL seeks strategic labels, which are evidently biased, while LR requires i.i.d. labels to evaluate the detector's performance and set the rejection threshold. Because one usually has a unique label budget, deciding how to optimally allocate it is challenging. In this paper, we propose a mixed strategy that, given a budget of labels, decides in multiple rounds whether to use the budget to collect AL labels or LR labels. The strategy is based on a reward function that measures the expected gain when allocating the budget to either side. We evaluate our strategy on 18 benchmark datasets and compare it to some baselines.
Learning Symbolic Representations for Reinforcement Learning of Non-Markovian Behavior
Venue: arXiv cs.LG 2023
Year: 2023

Many real-world reinforcement learning (RL) problems necessitate learning complex, temporally extended behavior that may only receive reward signal when the behavior is completed. If the reward-worthy behavior is known, it can be specified in terms of a non-Markovian reward function - a function that depends on aspects of the state-action history, rather than just the current state and action. Such reward functions yield sparse rewards, necessitating an inordinate number of experiences to find a policy that captures the reward-worthy pattern of behavior. Recent work has leveraged Knowledge Representation (KR) to provide a symbolic abstraction of aspects of the state that summarize reward-relevant properties of the state-action history and support learning a Markovian decomposition of the problem in terms of an automaton over the KR. Providing such a decomposition has been shown to vastly improve learning rates, especially when coupled with algorithms that exploit automaton structure. Nevertheless, such techniques rely on a priori knowledge of the KR. In this work, we explore how to automatically discover useful state abstractions that support learning automata over the state-action history. The result is an end-to-end algorithm that can learn optimal policies with significantly fewer environment samples than state-of-the-art RL on simple non-Markovian domains.
Accelerating Voting by Quantum Computation
Venue: arXiv cs.CY 2023
Year: 2023

Studying the computational complexity and designing fast algorithms for determining winners under voting rules are classical and fundamental questions in computational social choice. In this paper, we accelerate voting by leveraging quantum computation: we propose a quantum-accelerated voting algorithm that can be applied to any anonymous voting rule. We show that our algorithm can be quadratically faster than any classical algorithm (based on sampling with replacement) under a wide range of common voting rules, including positional scoring rules, Copeland, and single transferable voting (STV). Precisely, our quantum-accelerated voting algorithm outputs the correct winner with high probability in $Θ\left(\frac{n}{\text{MOV}}\right)$ time, where $n$ is the number of votes and $\text{MOV}$ is {\em margin of victory}, the smallest number of voters to change the winner. In contrast, any classical voting algorithm based on sampling with replacement requires $Ω\left(\frac{n^2}{\text{MOV}^2}\right)$ time under a large class of voting rules. Our theoretical results are supported by experiments under plurality, Borda, Copeland, and STV.